Skip to contentSkip to conversation

ref consistency check infra setup

PATCH Superseded shejialuo 7 participants 282 messages May 30, 2024 – Oct 6, 2024 Documentation Page 3 of 3

lore
Share & tools
lore
author
#241 lore raw

PATCH v14 10/11 files-backend: add unified interface for refs scanning

In reply to shejialuo

For refs and reflogs, we need to scan its corresponding directories to check every regular file or symbolic link which shares the same pattern. Introduce a unified interface for scanning directories for files-backend.

---
 Documentation/fsck-msgids.txt |  3 ++
 fsck.h                        |  1 +
 refs/files-backend.c          | 75 ++++++++++++++++++++++++++++++++++-
 3 files changed, 78 insertions(+), 1 deletion(-)
Documentation/fsck-msgids.txt +3 -0
index f643585a34..7c809fddf1 100644
@@ -19,6 +19,9 @@
 `badParentSha1`::
 	(ERROR) A commit object has a bad parent sha1.
 
+`badRefFiletype`::
+	(ERROR) A ref has a bad file type.
+
 `badTagName`::
 	(INFO) A tag has an invalid format.
 
fsck.h +1 -0
index d551a9fe86..af02174973 100644
@@ -31,6 +31,7 @@ enum fsck_msg_type {
 	FUNC(BAD_NAME, ERROR) \
 	FUNC(BAD_OBJECT_SHA1, ERROR) \
 	FUNC(BAD_PARENT_SHA1, ERROR) \
+	FUNC(BAD_REF_FILETYPE, ERROR) \
 	FUNC(BAD_TIMEZONE, ERROR) \
 	FUNC(BAD_TREE, ERROR) \
 	FUNC(BAD_TREE_SHA1, ERROR) \
refs/files-backend.c +74 -1

93 lines not on this page · raw message

Signature · 1 line
2.45.2
#242 lore raw

Re: [GSoC][PATCH v14 10/11] files-backend: add unified interface for refs scanning

refs/files-backend.c static int files_ref_store_remove_on_disk(struct ref_store *ref_store, · line 3478

Quoted 9 lines · shejialuo
refs/files-backend.c +9 -0
+static int files_fsck_refs(struct ref_store *ref_store,
+			   struct fsck_options *o)
+{
+	files_fsck_refs_fn fsck_refs_fn[]= {
+		NULL,
+	};
+
+	if (o->verbose)
+		fprintf_ln(stderr, "Checking references consistency");

This string should be marked for translation.

Patrick

author
#243 lore raw

PATCH v14 11/11 fsck: add ref name check for files backend

In reply to shejialuo

The git-fsck(1) only implicitly checks the reference, it does not fully check refs with bad format name such as standalone "@".

However, a file ending with ".lock" should not be marked as having a bad ref name. It is expected that concurrent writers may have such lock files. We currently ignore this situation. But for bare ".lock" file, we will report it as error.

In order to provide such checks, add a new fsck message id "badRefName" with default ERROR type. Use existing "check_refname_format" to explicit check the ref name. And add a new unit test to verify the functionality.

---
 Documentation/fsck-msgids.txt |  3 ++
 fsck.h                        |  1 +
 refs/files-backend.c          | 32 ++++++++++++
 t/t0602-reffiles-fsck.sh      | 94 +++++++++++++++++++++++++++++++++++
 4 files changed, 130 insertions(+)
 create mode 100755 t/t0602-reffiles-fsck.sh
Documentation/fsck-msgids.txt +3 -0
index 7c809fddf1..68a2801f15 100644
@@ -22,6 +22,9 @@
 `badRefFiletype`::
 	(ERROR) A ref has a bad file type.
 
+`badRefName`::
+	(ERROR) A ref has an invalid format.
+
 `badTagName`::
 	(INFO) A tag has an invalid format.
 
fsck.h +1 -0
index af02174973..500b4c04d2 100644
@@ -32,6 +32,7 @@ enum fsck_msg_type {
 	FUNC(BAD_OBJECT_SHA1, ERROR) \
 	FUNC(BAD_PARENT_SHA1, ERROR) \
 	FUNC(BAD_REF_FILETYPE, ERROR) \
+	FUNC(BAD_REF_NAME, ERROR) \
 	FUNC(BAD_TIMEZONE, ERROR) \
 	FUNC(BAD_TREE, ERROR) \
 	FUNC(BAD_TREE_SHA1, ERROR) \
refs/files-backend.c +32 -0
index 5574e78656..1186b6cbb1 100644
@@ -3419,6 +3419,37 @@ typedef int (*files_fsck_refs_fn)(struct fsck_options *o,
 				  const char *refs_check_dir,
 				  struct dir_iterator *iter);
 
+static int files_fsck_refs_name(struct fsck_options *o,
+				const char *gitdir UNUSED,
+				const char *refs_check_dir,
+				struct dir_iterator *iter)
+{
+	struct strbuf sb = STRBUF_INIT;
+	size_t len = 0;
+	int ret = 0;
+
+	/*
+	 * Ignore the files ending with ".lock" as they may be lock files
+	 * However, do not allow bare ".lock" files.
+	 */
+	if (strip_suffix(iter->basename, ".lock", &len) && (len != 0))
+		goto clean;
+
+	if (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {
+		struct fsck_ref_report report = { .path = NULL };
+
+		strbuf_addf(&sb, "%s/%s", refs_check_dir, iter->relative_path);
+		report.path = sb.buf;
+		ret = fsck_report_ref(o, &report,
+				      FSCK_MSG_BAD_REF_NAME,
+				      "invalid refname format");
+	}
+
+clean:
+	strbuf_release(&sb);
+	return ret;
+}
+
 static int files_fsck_refs_dir(struct ref_store *ref_store,
 			       struct fsck_options *o,
 			       const char *refs_check_dir,
@@ -3471,6 +3502,7 @@ static int files_fsck_refs(struct ref_store *ref_store,
 			   struct fsck_options *o)
 {
 	files_fsck_refs_fn fsck_refs_fn[]= {
+		files_fsck_refs_name,
 		NULL,
 	};
 
t/t0602-reffiles-fsck.sh new file +94 -0

94 lines not on this page · raw message

Signature · 1 line
2.45.2
#244 lore raw

Re: [GSoC][PATCH v14 11/11] fsck: add ref name check for files backend

Quoted 13 lines · shejialuo
refs/files-backend.c +5 -0
index 5574e78656..1186b6cbb1 100644
@@ -3419,6 +3419,37 @@ typedef int (*files_fsck_refs_fn)(struct fsck_options *o,
 				  const char *refs_check_dir,
 				  struct dir_iterator *iter);
 
+static int files_fsck_refs_name(struct fsck_options *o,
+				const char *gitdir UNUSED,
+				const char *refs_check_dir,
+				struct dir_iterator *iter)
+{

refs/files-backend.c typedef int (*files_fsck_refs_fn)(struct fsck_options *o, · line 3436

Quoted 10 lines
refs/files-backend.c +10 -0
+	struct strbuf sb = STRBUF_INIT;
+	size_t len = 0;
+	int ret = 0;
+
+	/*
+	 * Ignore the files ending with ".lock" as they may be lock files
+	 * However, do not allow bare ".lock" files.
+	 */
+	if (strip_suffix(iter->basename, ".lock", &len) && (len != 0))
+		goto clean;

Better:

    if (ends_with(iter->basename, ".lock))
            goto cleanup;

Patrick

#245 lore raw
In reply to shejialuo
Quoted 24 lines · shejialuo

Hi All,

This version does the following things:

1. By following the advice from Patrick, we should make the callback function be generic by adding only one "void * fsck_report" parameter. Thus the commit sequence will be much more clearer. And it wll be much easier for reviewers to review. And I have split the commit into more commits in this version.
2. Enhance the commit messages to provide more context about why we should do this.
3. Patrick advices that we should initialize the "fsck_options" member when parsing the options. However, because the original "strict" and "verbose" field are defined as the bit field, we cannot take the address of them. So I simply remove the bit field.
4. As Patrick said, ".lock" should not be reported as error. At current, ignore files ending with ".lock".
5. Add a fsck msg type called "badRefFiletype" which indicates that a ref has a bad file type when scanning the directory.
6. Junio advices instead of using "fsck_refs_fns", we should use the singular version "fsck_refs_fn", fix this.
7. Drop the last patch because in this series, we mainly focus on the infra, I will add a series later to add ref content check.

Quoted 7 lines

However, there is one thing holding. Junio advices that I should follow the Patrick's change to change the prototype of "files_fsck_refs_fn"

   https://lore.kernel.org/git/fe0e2c3617c8040c632dbc3de613a1d22e8070f7.1722316795.git.ps@pks.im/

However, at current, this patch is not in the cooking tree. I will handle this later.

This looks very good to me. I've got another set of smallish nits, but all of those should be quick to address, I think. I think that the next version should likely be ready to go.

Thanks!

Patrick

author
#246 lore raw

v15 cover ref consistency check infra setup

In reply to shejialuo

Hi All:

This version handles the following problems:

1. Patrick advices that I should not use va_copy in the changed report function. Actually this is a mistake, this version avoids redundant ap copy.
2. Patrick advices I should rebase [v14 05/11] into [v14 04/11]. I follow this advice in this version.
3. Patrick advices that we should put [v14 06/11] before we introduce ref-related operations. This version reorders the commit sequence. It's a minor change.
4. Patrick suggests at current we should not add git refs verify command into "git-fsck(1)". This is because we should disable this new check by default for the users. Many users use "git-fsck(1)" in their daily workflow. We should not be aggressive. However, if we provide this mechanism in this series, we will again make more complexity. So this version drop patch [v14 09/11]. Also because of dropping, change the test file to use "git refs verify" command instead of "git fsck" command.
5. Patrick suggests that we should use ends_with instead of strip_suffix, fix.

There is another important problem this patch solves:

At v13, Junio has suggested that the files_fsck_refs_fn should be adapted to Patrick's change. Actually, I made a bad design before. I should always pass the ref_store structure. So I change it to

-typedef int (*files_fsck_refs_fn)(struct fsck_options *o,
- const char *gitdir,
+typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,
+ struct fsck_options *o,
const char *refs_check_dir,
struct dir_iterator *iter);

gitdir could be got by using ref_store parameter. By using ref_store parameter, we provide extensibility here. If something else change, we merely need to change "files_fsck_refs_fn" prototype.

Because I drop one patch and rebase one patch. I provide the interdiff for reviewers to make the life easier.

Due to the deadline of the GSoC, I will speed up the review feedback process.

Thanks,
Jialuo

shejialuo (9):
fsck: rename "skiplist" to "skip_oids"
fsck: rename objects-related fsck error functions
fsck: make "fsck_error" callback generic
fsck: add a unified interface for reporting fsck messages
fsck: add refs report function
refs: set up ref consistency check infrastructure builtin/refs: add verify subcommand
files-backend: add unified interface for refs scanning
fsck: add ref name check for files backend

 Documentation/fsck-msgids.txt |   6 ++
 Documentation/git-refs.txt    |  13 ++++
 builtin/fsck.c                |  17 +++--
 builtin/mktag.c               |   3 +-
 builtin/refs.c                |  34 +++++++++
 fsck.c                        | 127 +++++++++++++++++++++++++++-------
 fsck.h                        |  76 +++++++++++++++-----
 object-file.c                 |   9 ++-
 refs.c                        |   5 ++
 refs.h                        |   8 +++
 refs/debug.c                  |  11 +++
 refs/files-backend.c          | 116 ++++++++++++++++++++++++++++++-
 refs/packed-backend.c         |   8 +++
 refs/refs-internal.h          |   6 ++
 refs/reftable-backend.c       |   8 +++
 t/t0602-reffiles-fsck.sh      |  92 ++++++++++++++++++++++++
 16 files changed, 480 insertions(+), 59 deletions(-)
 create mode 100755 t/t0602-reffiles-fsck.sh

Interdiff against v14:

builtin/fsck.c +0 -17
index b6ac878270..766bbd014d 100644
@@ -899,21 +899,6 @@ static int check_pack_rev_indexes(struct repository *r, int show_progress)
 	return res;
 }
 
-static void fsck_refs(void)
-{
-	struct child_process refs_verify = CHILD_PROCESS_INIT;
-	child_process_init(&refs_verify);
-	refs_verify.git_cmd = 1;
-	strvec_pushl(&refs_verify.args, "refs", "verify", NULL);
-	if (verbose)
-		strvec_push(&refs_verify.args, "--verbose");
-	if (check_strict)
-		strvec_push(&refs_verify.args, "--strict");
-
-	if (run_command(&refs_verify))
-		errors_found |= ERROR_REFS;
-}
-
 static char const * const fsck_usage[] = {
 	N_("git fsck [--tags] [--root] [--unreachable] [--cache] [--no-reflogs]\n"
 	   "         [--[no-]full] [--strict] [--verbose] [--lost-found]\n"
@@ -1083,8 +1068,6 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 
 	check_connectivity();
 
-	fsck_refs();
-
 	if (the_repository->settings.core_commit_graph) {
 		struct child_process commit_graph_verify = CHILD_PROCESS_INIT;
 
fsck.c +3 -5
index d5e7c88eab..7eb5cdefdd 100644
@@ -235,7 +235,6 @@ static int fsck_vreport(struct fsck_options *options,
 			void *fsck_report,
 			enum fsck_msg_id msg_id, const char *fmt, va_list ap)
 {
-	va_list ap_copy;
 	struct strbuf sb = STRBUF_INIT;
 	enum fsck_msg_type msg_type = fsck_msg_type(msg_id, options);
 	int result;
@@ -251,12 +250,10 @@ static int fsck_vreport(struct fsck_options *options,
 	prepare_msg_ids();
 	strbuf_addf(&sb, "%s: ", msg_id_info[msg_id].camelcased);
 
-	va_copy(ap_copy, ap);
-	strbuf_vaddf(&sb, fmt, ap_copy);
+	strbuf_vaddf(&sb, fmt, ap);
 	result = options->error_func(options, fsck_report,
 				     msg_type, msg_id, sb.buf);
 	strbuf_release(&sb);
-	va_end(ap);
 
 	return result;
 }
@@ -1391,5 +1388,6 @@ int fsck_objects_error_cb_print_missing_gitmodules(struct fsck_options *o,
 		puts(oid_to_hex(report->oid));
 		return 0;
 	}
-	return fsck_objects_error_function(o, fsck_report, msg_type,msg_id, message);
+	return fsck_objects_error_function(o, fsck_report,
+					   msg_type, msg_id, message);
 }
refs/files-backend.c +10 -12
index 1186b6cbb1..6e6b47251d 100644
@@ -3414,26 +3414,25 @@ static int files_ref_store_remove_on_disk(struct ref_store *ref_store,
  * the whole directory. This function is used as the callback for each
  * regular file or symlink in the directory.
  */
-typedef int (*files_fsck_refs_fn)(struct fsck_options *o,
-				  const char *gitdir,
+typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,
+				  struct fsck_options *o,
 				  const char *refs_check_dir,
 				  struct dir_iterator *iter);
 
-static int files_fsck_refs_name(struct fsck_options *o,
-				const char *gitdir UNUSED,
+static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,
+				struct fsck_options *o,
 				const char *refs_check_dir,
 				struct dir_iterator *iter)
 {
 	struct strbuf sb = STRBUF_INIT;
-	size_t len = 0;
 	int ret = 0;
 
 	/*
 	 * Ignore the files ending with ".lock" as they may be lock files
 	 * However, do not allow bare ".lock" files.
 	 */
-	if (strip_suffix(iter->basename, ".lock", &len) && (len != 0))
-		goto clean;
+	if (iter->basename[0] != '.' && ends_with(iter->basename, ".lock"))
+		goto cleanup;
 
 	if (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {
 		struct fsck_ref_report report = { .path = NULL };
@@ -3445,7 +3444,7 @@ static int files_fsck_refs_name(struct fsck_options *o,
 				      "invalid refname format");
 	}
 
-clean:
+cleanup:
 	strbuf_release(&sb);
 	return ret;
 }
@@ -3455,13 +3454,12 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,
 			       const char *refs_check_dir,
 			       files_fsck_refs_fn *fsck_refs_fn)
 {
-	const char *gitdir = ref_store->gitdir;
 	struct strbuf sb = STRBUF_INIT;
 	struct dir_iterator *iter;
 	int iter_status;
 	int ret = 0;
 
-	strbuf_addf(&sb, "%s/%s", gitdir, refs_check_dir);
+	strbuf_addf(&sb, "%s/%s", ref_store->gitdir, refs_check_dir);
 
 	iter = dir_iterator_begin(sb.buf, 0);
 	if (!iter) {
@@ -3478,7 +3476,7 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,
 				fprintf_ln(stderr, "Checking %s/%s",
 					   refs_check_dir, iter->relative_path);
 			for (size_t i = 0; fsck_refs_fn[i]; i++) {
-				if (fsck_refs_fn[i](o, gitdir, refs_check_dir, iter))
+				if (fsck_refs_fn[i](ref_store, o, refs_check_dir, iter))
 					ret = -1;
 			}
 		} else {
@@ -3507,7 +3505,7 @@ static int files_fsck_refs(struct ref_store *ref_store,
 	};
 
 	if (o->verbose)
-		fprintf_ln(stderr, "Checking references consistency");
+		fprintf_ln(stderr, _("Checking references consistency"));
 	return files_fsck_refs_dir(ref_store, o,  "refs", fsck_refs_fn);
 
 }
t/t0602-reffiles-fsck.sh +7 -9
index 2be28427ab..71a4d1a5ae 100755
@@ -26,7 +26,7 @@ test_expect_success 'ref name should be checked' '
 	git tag multi_hierarchy/tag-2 &&
 
 	cp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&
-	test_must_fail git fsck 2>err &&
+	test_must_fail git refs verify 2>err &&
 	cat >expect <<-EOF &&
 	error: refs/heads/.branch-1: badRefName: invalid refname format
 	EOF
@@ -34,7 +34,7 @@ test_expect_success 'ref name should be checked' '
 	test_cmp expect err &&
 
 	cp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&
-	test_must_fail git fsck 2>err &&
+	test_must_fail git refs verify 2>err &&
 	cat >expect <<-EOF &&
 	error: refs/heads/@: badRefName: invalid refname format
 	EOF
@@ -42,7 +42,7 @@ test_expect_success 'ref name should be checked' '
 	test_cmp expect err &&
 
 	cp $tag_dir_prefix/multi_hierarchy/tag-2 $tag_dir_prefix/multi_hierarchy/@ &&
-	test_must_fail git fsck 2>err &&
+	test_must_fail git refs verify 2>err &&
 	cat >expect <<-EOF &&
 	error: refs/tags/multi_hierarchy/@: badRefName: invalid refname format
 	EOF
@@ -50,12 +50,12 @@ test_expect_success 'ref name should be checked' '
 	test_cmp expect err &&
 
 	cp $tag_dir_prefix/tag-1 $tag_dir_prefix/tag-1.lock &&
-	git fsck 2>err &&
+	git refs verify 2>err &&
 	rm $tag_dir_prefix/tag-1.lock &&
 	test_must_be_empty err &&
 
 	cp $tag_dir_prefix/tag-1 $tag_dir_prefix/.lock &&
-	test_must_fail git fsck 2>err &&
+	test_must_fail git refs verify 2>err &&
 	cat >expect <<-EOF &&
 	error: refs/tags/.lock: badRefName: invalid refname format
 	EOF
@@ -76,18 +76,16 @@ test_expect_success 'ref name check should be adapted into fsck messages' '
 	git checkout -b branch-2 &&
 	git tag tag-2 &&
 
-
 	cp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&
-	git -c fsck.badRefName=warn fsck 2>err &&
+	git -c fsck.badRefName=warn refs verify 2>err &&
 	cat >expect <<-EOF &&
 	warning: refs/heads/.branch-1: badRefName: invalid refname format
 	EOF
 	rm $branch_dir_prefix/.branch-1 &&
 	test_cmp expect err &&
 
-
 	cp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&
-	git -c fsck.badRefName=ignore fsck 2>err &&
+	git -c fsck.badRefName=ignore refs verify 2>err &&
 	test_must_be_empty err
 '
 
Signature · 1 line
2.46.0
author
#247 lore raw

PATCH v15 1/9 fsck: rename "skiplist" to "skip_oids"

The "skiplist" field in "fsck_options" is related to objects. Because we are going to introduce ref consistency check, the "skiplist" name is too general which will make the caller think "skiplist" is related to both the refs and objects.

It may seem that for both refs and objects, we should provide a general "skiplist" here. However, the type for "skiplist" is struct oidset which is totally unsuitable for refs.

To avoid above ambiguity, rename "skiplist" to "skip_oids".

---
 fsck.c | 4 ++--
 fsck.h | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)
fsck.c +2 -2
index eea7145470..3f32441492 100644
@@ -205,7 +205,7 @@ void fsck_set_msg_types(struct fsck_options *options, const char *values)
 		if (!strcmp(buf, "skiplist")) {
 			if (equal == len)
 				die("skiplist requires a path");
-			oidset_parse_file(&options->skiplist, buf + equal + 1,
+			oidset_parse_file(&options->skip_oids, buf + equal + 1,
 					  the_repository->hash_algo);
 			buf += len + 1;
 			continue;
@@ -223,7 +223,7 @@ void fsck_set_msg_types(struct fsck_options *options, const char *values)
 static int object_on_skiplist(struct fsck_options *opts,
 			      const struct object_id *oid)
 {
-	return opts && oid && oidset_contains(&opts->skiplist, oid);
+	return opts && oid && oidset_contains(&opts->skip_oids, oid);
 }
 
 __attribute__((format (printf, 5, 6)))
fsck.h +2 -2
index 6085a384f6..bcfb2e34cd 100644
@@ -136,7 +136,7 @@ struct fsck_options {
 	fsck_error error_func;
 	unsigned strict:1;
 	enum fsck_msg_type *msg_type;
-	struct oidset skiplist;
+	struct oidset skip_oids;
 	struct oidset gitmodules_found;
 	struct oidset gitmodules_done;
 	struct oidset gitattributes_found;
@@ -145,7 +145,7 @@ struct fsck_options {
 };
 
 #define FSCK_OPTIONS_DEFAULT { \
-	.skiplist = OIDSET_INIT, \
+	.skip_oids = OIDSET_INIT, \
 	.gitmodules_found = OIDSET_INIT, \
 	.gitmodules_done = OIDSET_INIT, \
 	.gitattributes_found = OIDSET_INIT, \
Signature · 1 line
2.46.0
author
#248 lore raw

PATCH v15 2/9 fsck: rename objects-related fsck error functions

In reply to shejialuo

The names of objects-related fsck error functions are generic. It's OK when there is only object database check. However, we are going to introduce refs database check report function. To avoid ambiguity, rename object-related fsck error functions to explicitly indicate these functions are used to report objects-related messages.

---
 builtin/fsck.c | 14 +++++++-------
 fsck.c         | 17 +++++++++--------
 fsck.h         | 26 +++++++++++++-------------
 3 files changed, 29 insertions(+), 28 deletions(-)
builtin/fsck.c +7 -7
index d13a226c2e..6d86bbe1e9 100644
@@ -89,12 +89,12 @@ static int objerror(struct object *obj, const char *err)
 	return -1;
 }
 
-static int fsck_error_func(struct fsck_options *o UNUSED,
-			   const struct object_id *oid,
-			   enum object_type object_type,
-			   enum fsck_msg_type msg_type,
-			   enum fsck_msg_id msg_id UNUSED,
-			   const char *message)
+static int fsck_objects_error_func(struct fsck_options *o UNUSED,
+				   const struct object_id *oid,
+				   enum object_type object_type,
+				   enum fsck_msg_type msg_type,
+				   enum fsck_msg_id msg_id UNUSED,
+				   const char *message)
 {
 	switch (msg_type) {
 	case FSCK_WARN:
@@ -938,7 +938,7 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 
 	fsck_walk_options.walk = mark_object;
 	fsck_obj_options.walk = mark_used;
-	fsck_obj_options.error_func = fsck_error_func;
+	fsck_obj_options.error_func = fsck_objects_error_func;
 	if (check_strict)
 		fsck_obj_options.strict = 1;
 
fsck.c +9 -8
index 3f32441492..8347842cfb 100644
@@ -1200,7 +1200,7 @@ int fsck_buffer(const struct object_id *oid, enum object_type type,
 		      type);
 }
 
-int fsck_error_function(struct fsck_options *o,
+int fsck_objects_error_function(struct fsck_options *o,
 			const struct object_id *oid,
 			enum object_type object_type UNUSED,
 			enum fsck_msg_type msg_type,
@@ -1303,16 +1303,17 @@ int git_fsck_config(const char *var, const char *value,
  * Custom error callbacks that are used in more than one place.
  */
 
-int fsck_error_cb_print_missing_gitmodules(struct fsck_options *o,
-					   const struct object_id *oid,
-					   enum object_type object_type,
-					   enum fsck_msg_type msg_type,
-					   enum fsck_msg_id msg_id,
-					   const char *message)
+int fsck_objects_error_cb_print_missing_gitmodules(struct fsck_options *o,
+						   const struct object_id *oid,
+						   enum object_type object_type,
+						   enum fsck_msg_type msg_type,
+						   enum fsck_msg_id msg_id,
+						   const char *message)
 {
 	if (msg_id == FSCK_MSG_GITMODULES_MISSING) {
 		puts(oid_to_hex(oid));
 		return 0;
 	}
-	return fsck_error_function(o, oid, object_type, msg_type, msg_id, message);
+	return fsck_objects_error_function(o, oid, object_type,
+					   msg_type, msg_id, message);
 }
fsck.h +13 -13
index bcfb2e34cd..41ebebbb59 100644
@@ -120,16 +120,16 @@ typedef int (*fsck_error)(struct fsck_options *o,
 			  enum fsck_msg_type msg_type, enum fsck_msg_id msg_id,
 			  const char *message);
 
-int fsck_error_function(struct fsck_options *o,
-			const struct object_id *oid, enum object_type object_type,
-			enum fsck_msg_type msg_type, enum fsck_msg_id msg_id,
-			const char *message);
-int fsck_error_cb_print_missing_gitmodules(struct fsck_options *o,
-					   const struct object_id *oid,
-					   enum object_type object_type,
-					   enum fsck_msg_type msg_type,
-					   enum fsck_msg_id msg_id,
-					   const char *message);
+int fsck_objects_error_function(struct fsck_options *o,
+				const struct object_id *oid, enum object_type object_type,
+				enum fsck_msg_type msg_type, enum fsck_msg_id msg_id,
+				const char *message);
+int fsck_objects_error_cb_print_missing_gitmodules(struct fsck_options *o,
+						   const struct object_id *oid,
+						   enum object_type object_type,
+						   enum fsck_msg_type msg_type,
+						   enum fsck_msg_id msg_id,
+						   const char *message);
 
 struct fsck_options {
 	fsck_walk_func walk;
@@ -150,7 +150,7 @@ struct fsck_options {
 	.gitmodules_done = OIDSET_INIT, \
 	.gitattributes_found = OIDSET_INIT, \
 	.gitattributes_done = OIDSET_INIT, \
-	.error_func = fsck_error_function \
+	.error_func = fsck_objects_error_function \
 }
 #define FSCK_OPTIONS_STRICT { \
 	.strict = 1, \
@@ -158,7 +158,7 @@ struct fsck_options {
 	.gitmodules_done = OIDSET_INIT, \
 	.gitattributes_found = OIDSET_INIT, \
 	.gitattributes_done = OIDSET_INIT, \
-	.error_func = fsck_error_function, \
+	.error_func = fsck_objects_error_function, \
 }
 #define FSCK_OPTIONS_MISSING_GITMODULES { \
 	.strict = 1, \
@@ -166,7 +166,7 @@ struct fsck_options {
 	.gitmodules_done = OIDSET_INIT, \
 	.gitattributes_found = OIDSET_INIT, \
 	.gitattributes_done = OIDSET_INIT, \
-	.error_func = fsck_error_cb_print_missing_gitmodules, \
+	.error_func = fsck_objects_error_cb_print_missing_gitmodules, \
 }
 
 /* descend in all linked child objects
Signature · 1 line
2.46.0
author
#249 lore raw

PATCH v15 3/9 fsck: make "fsck_error" callback generic

In reply to shejialuo

The "fsck_error" callback is designed to report the objects-related error messages. It accepts two parameter "oid" and "object_type" which is not generic. In order to provide a unified callback which can report either objects or refs, remove the objects-related parameters and add the generic parameter "void *fsck_report".

Create a new "fsck_object_report" structure which incorporates the removed parameters "oid" and "object_type". Then change the corresponding references to adapt to new "fsck_error" callback.

---
 builtin/fsck.c  |  7 +++++--
 builtin/mktag.c |  3 +--
 fsck.c          | 26 ++++++++++++++++----------
 fsck.h          | 17 ++++++++++++-----
 object-file.c   |  9 ++++-----
 5 files changed, 38 insertions(+), 24 deletions(-)
builtin/fsck.c +5 -2
index 6d86bbe1e9..766bbd014d 100644
@@ -90,12 +90,15 @@ static int objerror(struct object *obj, const char *err)
 }
 
 static int fsck_objects_error_func(struct fsck_options *o UNUSED,
-				   const struct object_id *oid,
-				   enum object_type object_type,
+				   void *fsck_report,
 				   enum fsck_msg_type msg_type,
 				   enum fsck_msg_id msg_id UNUSED,
 				   const char *message)
 {
+	struct fsck_object_report *report = fsck_report;
+	const struct object_id *oid = report->oid;
+	enum object_type object_type = report->object_type;
+
 	switch (msg_type) {
 	case FSCK_WARN:
 		/* TRANSLATORS: e.g. warning in tree 01bfda: <more explanation> */
builtin/mktag.c +1 -2
index 4767f1a97e..c6b644219f 100644
@@ -18,8 +18,7 @@ static int option_strict = 1;
 static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;
 
 static int mktag_fsck_error_func(struct fsck_options *o UNUSED,
-				 const struct object_id *oid UNUSED,
-				 enum object_type object_type UNUSED,
+				 void *fsck_report UNUSED,
 				 enum fsck_msg_type msg_type,
 				 enum fsck_msg_id msg_id UNUSED,
 				 const char *message)
fsck.c +16 -10
index 8347842cfb..cca6ae144f 100644
@@ -232,6 +232,10 @@ static int report(struct fsck_options *options,
 		  enum fsck_msg_id msg_id, const char *fmt, ...)
 {
 	va_list ap;
+	struct fsck_object_report report = {
+		.oid = oid,
+		.object_type = object_type
+	};
 	struct strbuf sb = STRBUF_INIT;
 	enum fsck_msg_type msg_type = fsck_msg_type(msg_id, options);
 	int result;
@@ -252,7 +256,7 @@ static int report(struct fsck_options *options,
 
 	va_start(ap, fmt);
 	strbuf_vaddf(&sb, fmt, ap);
-	result = options->error_func(options, oid, object_type,
+	result = options->error_func(options, &report,
 				     msg_type, msg_id, sb.buf);
 	strbuf_release(&sb);
 	va_end(ap);
@@ -1201,12 +1205,14 @@ int fsck_buffer(const struct object_id *oid, enum object_type type,
 }
 
 int fsck_objects_error_function(struct fsck_options *o,
-			const struct object_id *oid,
-			enum object_type object_type UNUSED,
-			enum fsck_msg_type msg_type,
-			enum fsck_msg_id msg_id UNUSED,
-			const char *message)
+				void *fsck_report,
+				enum fsck_msg_type msg_type,
+				enum fsck_msg_id msg_id UNUSED,
+				const char *message)
 {
+	struct fsck_object_report *report = fsck_report;
+	const struct object_id *oid = report->oid;
+
 	if (msg_type == FSCK_WARN) {
 		warning("object %s: %s", fsck_describe_object(o, oid), message);
 		return 0;
@@ -1304,16 +1310,16 @@ int git_fsck_config(const char *var, const char *value,
  */
 
 int fsck_objects_error_cb_print_missing_gitmodules(struct fsck_options *o,
-						   const struct object_id *oid,
-						   enum object_type object_type,
+						   void *fsck_report,
 						   enum fsck_msg_type msg_type,
 						   enum fsck_msg_id msg_id,
 						   const char *message)
 {
 	if (msg_id == FSCK_MSG_GITMODULES_MISSING) {
-		puts(oid_to_hex(oid));
+		struct fsck_object_report *report = fsck_report;
+		puts(oid_to_hex(report->oid));
 		return 0;
 	}
-	return fsck_objects_error_function(o, oid, object_type,
+	return fsck_objects_error_function(o, fsck_report,
 					   msg_type, msg_id, message);
 }
fsck.h +12 -5
index 41ebebbb59..3b80d02506 100644
@@ -114,23 +114,30 @@ int is_valid_msg_type(const char *msg_id, const char *msg_type);
 typedef int (*fsck_walk_func)(struct object *obj, enum object_type object_type,
 			      void *data, struct fsck_options *options);
 
-/* callback for fsck_object, type is FSCK_ERROR or FSCK_WARN */
+/*
+ * Callback for reporting errors either for objects or refs. The "fsck_report"
+ * is a generic pointer that can be used to pass any information.
+ */
 typedef int (*fsck_error)(struct fsck_options *o,
-			  const struct object_id *oid, enum object_type object_type,
+			  void *fsck_report,
 			  enum fsck_msg_type msg_type, enum fsck_msg_id msg_id,
 			  const char *message);
 
 int fsck_objects_error_function(struct fsck_options *o,
-				const struct object_id *oid, enum object_type object_type,
+				void *fsck_report,
 				enum fsck_msg_type msg_type, enum fsck_msg_id msg_id,
 				const char *message);
 int fsck_objects_error_cb_print_missing_gitmodules(struct fsck_options *o,
-						   const struct object_id *oid,
-						   enum object_type object_type,
+						   void *fsck_report,
 						   enum fsck_msg_type msg_type,
 						   enum fsck_msg_id msg_id,
 						   const char *message);
 
+struct fsck_object_report {
+	const struct object_id *oid;
+	enum object_type object_type;
+};
+
 struct fsck_options {
 	fsck_walk_func walk;
 	fsck_error error_func;
object-file.c +4 -5
index 065103be3e..05ac6ebed6 100644
@@ -2470,11 +2470,10 @@ int repo_has_object_file(struct repository *r,
  * give more context.
  */
 static int hash_format_check_report(struct fsck_options *opts UNUSED,
-				     const struct object_id *oid UNUSED,
-				     enum object_type object_type UNUSED,
-				     enum fsck_msg_type msg_type UNUSED,
-				     enum fsck_msg_id msg_id UNUSED,
-				     const char *message)
+				    void *fsck_report UNUSED,
+				    enum fsck_msg_type msg_type UNUSED,
+				    enum fsck_msg_id msg_id UNUSED,
+				    const char *message)
 {
 	error(_("object fails fsck: %s"), message);
 	return 1;
Signature · 1 line
2.46.0
#250 lore raw

Re: [GSoC][PATCH v15 3/9] fsck: make "fsck_error" callback generic

Quoted 10 lines · shejialuo

The "fsck_error" callback is designed to report the objects-related error messages. It accepts two parameter "oid" and "object_type" which is not generic. In order to provide a unified callback which can report either objects or refs, remove the objects-related parameters and add the generic parameter "void *fsck_report".

Create a new "fsck_object_report" structure which incorporates the removed parameters "oid" and "object_type". Then change the corresponding references to adapt to new "fsck_error" callback.

So since my last review, we've now changed the function to accept a 'void *' parameter, this makes sense, this way the refs error function and the objects error function can now only receive the data they care about.

[snip]

author
#251 lore raw

PATCH v15 4/9 fsck: add a unified interface for reporting fsck messages

In reply to shejialuo

The static function "report" provided by "fsck.c" aims at checking error type and calling the callback "error_func" to report the message. Both refs and objects need to check the error type of the current fsck message. In order to extract this common behavior, create a new function "fsck_vreport". Instead of using "...", provide "va_list" to allow more flexibility.

Instead of changing "report" prototype to be algin with the "fsck_vreport" function, we leave the "report" prototype unchanged due to the reason that there are nearly 62 references about "report" function. Simply change "report" function to use "fsck_vreport" to report objects related messages.

---
 fsck.c | 44 ++++++++++++++++++++++++++++++--------------
 1 file changed, 30 insertions(+), 14 deletions(-)
fsck.c +30 -14
index cca6ae144f..3614aa56a3 100644
@@ -226,16 +226,15 @@ static int object_on_skiplist(struct fsck_options *opts,
 	return opts && oid && oidset_contains(&opts->skip_oids, oid);
 }
 
-__attribute__((format (printf, 5, 6)))
-static int report(struct fsck_options *options,
-		  const struct object_id *oid, enum object_type object_type,
-		  enum fsck_msg_id msg_id, const char *fmt, ...)
+/*
+ * Provide the common functionality for either fscking refs or objects.
+ * It will get the current msg error type and call the error_func callback
+ * which is registered in the "fsck_options" struct.
+ */
+static int fsck_vreport(struct fsck_options *options,
+			void *fsck_report,
+			enum fsck_msg_id msg_id, const char *fmt, va_list ap)
 {
-	va_list ap;
-	struct fsck_object_report report = {
-		.oid = oid,
-		.object_type = object_type
-	};
 	struct strbuf sb = STRBUF_INIT;
 	enum fsck_msg_type msg_type = fsck_msg_type(msg_id, options);
 	int result;
@@ -243,9 +242,6 @@ static int report(struct fsck_options *options,
 	if (msg_type == FSCK_IGNORE)
 		return 0;
 
-	if (object_on_skiplist(options, oid))
-		return 0;
-
 	if (msg_type == FSCK_FATAL)
 		msg_type = FSCK_ERROR;
 	else if (msg_type == FSCK_INFO)
@@ -254,11 +250,31 @@ static int report(struct fsck_options *options,
 	prepare_msg_ids();
 	strbuf_addf(&sb, "%s: ", msg_id_info[msg_id].camelcased);
 
-	va_start(ap, fmt);
 	strbuf_vaddf(&sb, fmt, ap);
-	result = options->error_func(options, &report,
+	result = options->error_func(options, fsck_report,
 				     msg_type, msg_id, sb.buf);
 	strbuf_release(&sb);
+
+	return result;
+}
+
+__attribute__((format (printf, 5, 6)))
+static int report(struct fsck_options *options,
+		  const struct object_id *oid, enum object_type object_type,
+		  enum fsck_msg_id msg_id, const char *fmt, ...)
+{
+	va_list ap;
+	struct fsck_object_report report = {
+		.oid = oid,
+		.object_type = object_type
+	};
+	int result;
+
+	if (object_on_skiplist(options, oid))
+		return 0;
+
+	va_start(ap, fmt);
+	result = fsck_vreport(options, &report, msg_id, fmt, ap);
 	va_end(ap);
 
 	return result;
Signature · 1 line
2.46.0
#252 lore raw

Re: [GSoC][PATCH v15 4/9] fsck: add a unified interface for reporting fsck messages

Quoted 8 lines · shejialuo

The static function "report" provided by "fsck.c" aims at checking error type and calling the callback "error_func" to report the message. Both refs and objects need to check the error type of the current fsck message. In order to extract this common behavior, create a new function "fsck_vreport". Instead of using "...", provide "va_list" to allow more flexibility.

Instead of changing "report" prototype to be algin with the

Nit: s/be algin/align/

Rest of the patch looks great!

[snip]

author
#253 lore raw

PATCH v15 5/9 fsck: add refs report function

In reply to shejialuo

Introduce a new struct "fsck_ref_report" to contain the information we need when reporting refs-related messages.

With the new "fsck_vreport" function, add a new function "fsck_report_ref" to report refs-related fsck error message. Unlike "report" function uses the exact parameters, we simply pass "struct fsck_ref_report *report" as the parameter. This is because at current we don't know exactly how many fields we need. By passing this parameter, we don't need to change this function prototype when we want to add more information into "fsck_ref_report".

We have introduced "fsck_report_ref" function to report the error message for refs. We still need to add the corresponding callback function. Create refs-specific "error_func" callback "fsck_refs_error_function".

Last, add "FSCK_REFS_OPTIONS_DEFAULT" macro to create default options when checking ref consistency.

---
 fsck.c | 41 +++++++++++++++++++++++++++++++++++++++++
 fsck.h | 25 +++++++++++++++++++++++++
 2 files changed, 66 insertions(+)
fsck.c +41 -0
index 3614aa56a3..38554b626e 100644
@@ -280,6 +280,19 @@ static int report(struct fsck_options *options,
 	return result;
 }
 
+int fsck_report_ref(struct fsck_options *options,
+		    struct fsck_ref_report *report,
+		    enum fsck_msg_id msg_id,
+		    const char *fmt, ...)
+{
+	va_list ap;
+	int result;
+	va_start(ap, fmt);
+	result = fsck_vreport(options, report, msg_id, fmt, ap);
+	va_end(ap);
+	return result;
+}
+
 void fsck_enable_object_names(struct fsck_options *options)
 {
 	if (!options->object_names)
@@ -1237,6 +1250,34 @@ int fsck_objects_error_function(struct fsck_options *o,
 	return 1;
 }
 
+int fsck_refs_error_function(struct fsck_options *options UNUSED,
+			     void *fsck_report,
+			     enum fsck_msg_type msg_type,
+			     enum fsck_msg_id msg_id UNUSED,
+			     const char *message)
+{
+	struct fsck_ref_report *report = fsck_report;
+
+	struct strbuf sb = STRBUF_INIT;
+	int ret = 0;
+
+	strbuf_addstr(&sb, report->path);
+
+	if (report->oid)
+		strbuf_addf(&sb, " -> (%s)", oid_to_hex(report->oid));
+	else if (report->referent)
+		strbuf_addf(&sb, " -> (%s)", report->referent);
+
+	if (msg_type == FSCK_WARN)
+		warning("%s: %s", sb.buf, message);
+	else
+		ret = error("%s: %s", sb.buf, message);
+
+	strbuf_release(&sb);
+	return ret;
+
+}
+
 static int fsck_blobs(struct oidset *blobs_found, struct oidset *blobs_done,
 		      enum fsck_msg_id msg_missing, enum fsck_msg_id msg_type,
 		      struct fsck_options *options, const char *blob_type)
fsck.h +25 -0
index 3b80d02506..2002590f60 100644
@@ -133,11 +133,23 @@ int fsck_objects_error_cb_print_missing_gitmodules(struct fsck_options *o,
 						   enum fsck_msg_id msg_id,
 						   const char *message);
 
+int fsck_refs_error_function(struct fsck_options *options,
+			     void *fsck_report,
+			     enum fsck_msg_type msg_type,
+			     enum fsck_msg_id msg_id,
+			     const char *message);
+
 struct fsck_object_report {
 	const struct object_id *oid;
 	enum object_type object_type;
 };
 
+struct fsck_ref_report {
+	const char *path;
+	const struct object_id *oid;
+	const char *referent;
+};
+
 struct fsck_options {
 	fsck_walk_func walk;
 	fsck_error error_func;
@@ -175,6 +187,9 @@ struct fsck_options {
 	.gitattributes_done = OIDSET_INIT, \
 	.error_func = fsck_objects_error_cb_print_missing_gitmodules, \
 }
+#define FSCK_REFS_OPTIONS_DEFAULT { \
+	.error_func = fsck_refs_error_function, \
+}
 
 /* descend in all linked child objects
  * the return value is:
@@ -216,6 +231,16 @@ int fsck_tag_standalone(const struct object_id *oid, const char *buffer,
  */
 int fsck_finish(struct fsck_options *options);
 
+/*
+ * Report an error or warning for refs.
+ */
+__attribute__((format (printf, 4, 5)))
+int fsck_report_ref(struct fsck_options *options,
+		    struct fsck_ref_report *report,
+		    enum fsck_msg_id msg_id,
+		    const char *fmt, ...);
+
+
 /*
  * Subsystem for storing human-readable names for each object.
  *
Signature · 1 line
2.46.0
#254 lore raw

Re: [GSoC][PATCH v15 5/9] fsck: add refs report function

fsck.c int fsck_objects_error_function(struct fsck_options *o, · line 1260

Quoted 12 lines · shejialuo
fsck.c +8 -0
@@ -1237,6 +1250,34 @@ int fsck_objects_error_function(struct fsck_options *o,
 	return 1;
 }
 
+int fsck_refs_error_function(struct fsck_options *options UNUSED,
+			     void *fsck_report,
+			     enum fsck_msg_type msg_type,
+			     enum fsck_msg_id msg_id UNUSED,
+			     const char *message)
+{
+	struct fsck_ref_report *report = fsck_report;
+

Nit: there's an unneeded empty newline here.

Quoted 8 lines
fsck.c +8 -0
+	struct strbuf sb = STRBUF_INIT;
+	int ret = 0;
+
+	strbuf_addstr(&sb, report->path);
+
+	if (report->oid)
+		strbuf_addf(&sb, " -> (%s)", oid_to_hex(report->oid));
+	else if (report->referent)

fsck.c int fsck_objects_error_function(struct fsck_options *o, · line 1278

Quoted 10 lines
fsck.c +10 -0
+		strbuf_addf(&sb, " -> (%s)", report->referent);
+
+	if (msg_type == FSCK_WARN)
+		warning("%s: %s", sb.buf, message);
+	else
+		ret = error("%s: %s", sb.buf, message);
+
+	strbuf_release(&sb);
+	return ret;
+

Here, as well.

Patrick

author
#255 lore raw

PATCH v15 6/9 refs: set up ref consistency check infrastructure

In reply to shejialuo

The "struct ref_store" is the base class which contains the "be" pointer which provides backend-specific functions whose interfaces are defined in the "ref_storage_be". We could reuse this polymorphism to define only one interface. For every backend, we need to provide its own function pointer.

The interfaces defined in the ref_storage_be are carefully structured in semantic. It's organized as the five parts:

1. The name and the initialization interfaces.
2. The ref transaction interfaces.
3. The ref internal interfaces (pack, rename and copy).
4. The ref filesystem interfaces.
5. The reflog related interfaces.

To keep consistent with the git-fsck(1), add a new interface named "fsck_refs_fn" to the end of "ref_storage_be". This semantic cannot be grouped into any above five categories. Explicitly add blank line to make it different from others.

Last, implement placeholder functions for each ref backends.

---
 refs.c                  |  5 +++++
 refs.h                  |  8 ++++++++
 refs/debug.c            | 11 +++++++++++
 refs/files-backend.c    | 13 ++++++++++++-
 refs/packed-backend.c   |  8 ++++++++
 refs/refs-internal.h    |  6 ++++++
 refs/reftable-backend.c |  8 ++++++++
 7 files changed, 58 insertions(+), 1 deletion(-)
refs.c +5 -0
index 915aeb4d1d..6f642dc681 100644
@@ -318,6 +318,11 @@ int check_refname_format(const char *refname, int flags)
 	return check_or_sanitize_refname(refname, flags, NULL);
 }
 
+int refs_fsck(struct ref_store *refs, struct fsck_options *o)
+{
+	return refs->be->fsck(refs, o);
+}
+
 void sanitize_refname_component(const char *refname, struct strbuf *out)
 {
 	if (check_or_sanitize_refname(refname, REFNAME_ALLOW_ONELEVEL, out))
refs.h +8 -0
index b3e39bc257..405073621a 100644
@@ -4,6 +4,7 @@
 #include "commit.h"
 #include "repository.h"
 
+struct fsck_options;
 struct object_id;
 struct ref_store;
 struct strbuf;
@@ -541,6 +542,13 @@ int refs_for_each_reflog(struct ref_store *refs, each_reflog_fn fn, void *cb_dat
  */
 int check_refname_format(const char *refname, int flags);
 
+/*
+ * Check the reference database for consistency. Return 0 if refs and
+ * reflogs are consistent, and non-zero otherwise. The errors will be
+ * written to stderr.
+ */
+int refs_fsck(struct ref_store *refs, struct fsck_options *o);
+
 /*
  * Apply the rules from check_refname_format, but mutate the result until it
  * is acceptable, and place the result in "out".
refs/debug.c +11 -0
index 547d9245b9..45e2e784a0 100644
@@ -419,6 +419,15 @@ static int debug_reflog_expire(struct ref_store *ref_store, const char *refname,
 	return res;
 }
 
+static int debug_fsck(struct ref_store *ref_store,
+		      struct fsck_options *o)
+{
+	struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;
+	int res = drefs->refs->be->fsck(drefs->refs, o);
+	trace_printf_key(&trace_refs, "fsck: %d\n", res);
+	return res;
+}
+
 struct ref_storage_be refs_be_debug = {
 	.name = "debug",
 	.init = NULL,
@@ -451,4 +460,6 @@ struct ref_storage_be refs_be_debug = {
 	.create_reflog = debug_create_reflog,
 	.delete_reflog = debug_delete_reflog,
 	.reflog_expire = debug_reflog_expire,
+
+	.fsck = debug_fsck,
 };
refs/files-backend.c +12 -1
index aa52d9be7c..4630eb1f80 100644
@@ -3408,6 +3408,15 @@ static int files_ref_store_remove_on_disk(struct ref_store *ref_store,
 	return ret;
 }
 
+static int files_fsck(struct ref_store *ref_store,
+		      struct fsck_options *o)
+{
+	struct files_ref_store *refs =
+		files_downcast(ref_store, REF_STORE_READ, "fsck");
+
+	return refs->packed_ref_store->be->fsck(refs->packed_ref_store, o);
+}
+
 struct ref_storage_be refs_be_files = {
 	.name = "files",
 	.init = files_ref_store_init,
@@ -3434,5 +3443,7 @@ struct ref_storage_be refs_be_files = {
 	.reflog_exists = files_reflog_exists,
 	.create_reflog = files_create_reflog,
 	.delete_reflog = files_delete_reflog,
-	.reflog_expire = files_reflog_expire
+	.reflog_expire = files_reflog_expire,
+
+	.fsck = files_fsck,
 };
refs/packed-backend.c +8 -0
index a0666407cd..5209b0b212 100644
@@ -1735,6 +1735,12 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s
 	return empty_ref_iterator_begin();
 }
 
+static int packed_fsck(struct ref_store *ref_store,
+		       struct fsck_options *o)
+{
+	return 0;
+}
+
 struct ref_storage_be refs_be_packed = {
 	.name = "packed",
 	.init = packed_ref_store_init,
@@ -1762,4 +1768,6 @@ struct ref_storage_be refs_be_packed = {
 	.create_reflog = NULL,
 	.delete_reflog = NULL,
 	.reflog_expire = NULL,
+
+	.fsck = packed_fsck,
 };
refs/refs-internal.h +6 -0
index fa975d69aa..a905e187cd 100644
@@ -4,6 +4,7 @@
 #include "refs.h"
 #include "iterator.h"
 
+struct fsck_options;
 struct ref_transaction;
 
 /*
@@ -650,6 +651,9 @@ typedef int read_raw_ref_fn(struct ref_store *ref_store, const char *refname,
 typedef int read_symbolic_ref_fn(struct ref_store *ref_store, const char *refname,
 				 struct strbuf *referent);
 
+typedef int fsck_fn(struct ref_store *ref_store,
+		    struct fsck_options *o);
+
 struct ref_storage_be {
 	const char *name;
 	ref_store_init_fn *init;
@@ -677,6 +681,8 @@ struct ref_storage_be {
 	create_reflog_fn *create_reflog;
 	delete_reflog_fn *delete_reflog;
 	reflog_expire_fn *reflog_expire;
+
+	fsck_fn *fsck;
 };
 
 extern struct ref_storage_be refs_be_files;
refs/reftable-backend.c +8 -0
index fbe74c239d..b5a1a526df 100644
@@ -2303,6 +2303,12 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store,
 	return ret;
 }
 
+static int reftable_be_fsck(struct ref_store *ref_store,
+			    struct fsck_options *o)
+{
+	return 0;
+}
+
 struct ref_storage_be refs_be_reftable = {
 	.name = "reftable",
 	.init = reftable_be_init,
@@ -2330,4 +2336,6 @@ struct ref_storage_be refs_be_reftable = {
 	.create_reflog = reftable_be_create_reflog,
 	.delete_reflog = reftable_be_delete_reflog,
 	.reflog_expire = reftable_be_reflog_expire,
+
+	.fsck = reftable_be_fsck,
 };
Signature · 1 line
2.46.0
author
#256 lore raw

PATCH v15 7/9 builtin/refs: add verify subcommand

In reply to shejialuo

Introduce a new subcommand "verify" in git-refs(1) to allow the user to check the reference database consistency and also this subcommand will be used as the entry point of checking refs for "git-fsck(1)".

Add "verbose" field into "fsck_options" to indicate whether we should print verbose messages when checking refs and objects consistency.

Remove bit-field for "strict" field, this is because we cannot take address of a bit-field which makes it unhandy to set member variables when parsing the command line options.

The "git-fsck(1)" declares "fsck_options" variable with "static" identifier which avoids complaint by the leak-checker. However, in "git-refs verify", we need to do memory clean manually. Thus add "fsck_options_clear" function in "fsck.c" to provide memory clean operation.

---
 Documentation/git-refs.txt | 13 +++++++++++++
 builtin/refs.c             | 34 ++++++++++++++++++++++++++++++++++
 fsck.c                     | 11 +++++++++++
 fsck.h                     |  8 +++++++-
 4 files changed, 65 insertions(+), 1 deletion(-)
Documentation/git-refs.txt +13 -0
index 5b99e04385..ce31f93061 100644
@@ -10,6 +10,7 @@ SYNOPSIS
 --------
 [verse]
 'git refs migrate' --ref-format=<format> [--dry-run]
+'git refs verify' [--strict] [--verbose]
 
 DESCRIPTION
 -----------
@@ -22,6 +23,9 @@ COMMANDS
 migrate::
 	Migrate ref store between different formats.
 
+verify::
+	Verify reference database consistency.
+
 OPTIONS
 -------
 
@@ -39,6 +43,15 @@ include::ref-storage-format.txt[]
 	can be used to double check that the migration works as expected before
 	performing the actual migration.
 
+The following options are specific to 'git refs verify':
+
+--strict::
+	Enable stricter error checking. This will cause warnings to be
+	reported as errors. See linkgit:git-fsck[1].
+
+--verbose::
+	When verifying the reference database consistency, be chatty.
+
 KNOWN LIMITATIONS
 -----------------
 
builtin/refs.c +34 -0
index 46dcd150d4..131f98be98 100644
@@ -1,4 +1,6 @@
 #include "builtin.h"
+#include "config.h"
+#include "fsck.h"
 #include "parse-options.h"
 #include "refs.h"
 #include "repository.h"
@@ -7,6 +9,9 @@
 #define REFS_MIGRATE_USAGE \
 	N_("git refs migrate --ref-format=<format> [--dry-run]")
 
+#define REFS_VERIFY_USAGE \
+	N_("git refs verify [--strict] [--verbose]")
+
 static int cmd_refs_migrate(int argc, const char **argv, const char *prefix)
 {
 	const char * const migrate_usage[] = {
@@ -58,15 +63,44 @@ static int cmd_refs_migrate(int argc, const char **argv, const char *prefix)
 	return err;
 }
 
+static int cmd_refs_verify(int argc, const char **argv, const char *prefix)
+{
+	struct fsck_options fsck_refs_options = FSCK_REFS_OPTIONS_DEFAULT;
+	const char * const verify_usage[] = {
+		REFS_VERIFY_USAGE,
+		NULL,
+	};
+	struct option options[] = {
+		OPT_BOOL(0, "verbose", &fsck_refs_options.verbose, N_("be verbose")),
+		OPT_BOOL(0, "strict", &fsck_refs_options.strict, N_("enable strict checking")),
+		OPT_END(),
+	};
+	int ret;
+
+	argc = parse_options(argc, argv, prefix, options, verify_usage, 0);
+	if (argc)
+		usage(_("'git refs verify' takes no arguments"));
+
+	git_config(git_fsck_config, &fsck_refs_options);
+	prepare_repo_settings(the_repository);
+
+	ret = refs_fsck(get_main_ref_store(the_repository), &fsck_refs_options);
+
+	fsck_options_clear(&fsck_refs_options);
+	return ret;
+}
+
 int cmd_refs(int argc, const char **argv, const char *prefix)
 {
 	const char * const refs_usage[] = {
 		REFS_MIGRATE_USAGE,
+		REFS_VERIFY_USAGE,
 		NULL,
 	};
 	parse_opt_subcommand_fn *fn = NULL;
 	struct option opts[] = {
 		OPT_SUBCOMMAND("migrate", &fn, cmd_refs_migrate),
+		OPT_SUBCOMMAND("verify", &fn, cmd_refs_verify),
 		OPT_END(),
 	};
 
fsck.c +11 -0
index 38554b626e..7eb5cdefdd 100644
@@ -1333,6 +1333,17 @@ int fsck_finish(struct fsck_options *options)
 	return ret;
 }
 
+void fsck_options_clear(struct fsck_options *options)
+{
+	free(options->msg_type);
+	oidset_clear(&options->skip_oids);
+	oidset_clear(&options->gitmodules_found);
+	oidset_clear(&options->gitmodules_done);
+	oidset_clear(&options->gitattributes_found);
+	oidset_clear(&options->gitattributes_done);
+	kh_clear_oid_map(options->object_names);
+}
+
 int git_fsck_config(const char *var, const char *value,
 		    const struct config_context *ctx, void *cb)
 {
fsck.h +7 -1
index 2002590f60..d551a9fe86 100644
@@ -153,7 +153,8 @@ struct fsck_ref_report {
 struct fsck_options {
 	fsck_walk_func walk;
 	fsck_error error_func;
-	unsigned strict:1;
+	unsigned strict;
+	unsigned verbose;
 	enum fsck_msg_type *msg_type;
 	struct oidset skip_oids;
 	struct oidset gitmodules_found;
@@ -231,6 +232,11 @@ int fsck_tag_standalone(const struct object_id *oid, const char *buffer,
  */
 int fsck_finish(struct fsck_options *options);
 
+/*
+ * Clear the fsck_options struct, freeing any allocated memory.
+ */
+void fsck_options_clear(struct fsck_options *options);
+
 /*
  * Report an error or warning for refs.
  */
Signature · 1 line
2.46.0
#257 lore raw

Re: [GSoC][PATCH v15 7/9] builtin/refs: add verify subcommand

Quoted 11 lines · shejialuo
builtin/refs.c +7 -0
@@ -58,15 +63,44 @@ static int cmd_refs_migrate(int argc, const char **argv, const char *prefix)
 	return err;
 }
 
+static int cmd_refs_verify(int argc, const char **argv, const char *prefix)
+{
+	struct fsck_options fsck_refs_options = FSCK_REFS_OPTIONS_DEFAULT;
+	const char * const verify_usage[] = {
+		REFS_VERIFY_USAGE,
+		NULL,
+	};

builtin/refs.c static int cmd_refs_migrate(int argc, const char **argv, const char *prefix) · line 82

Quoted 10 lines
builtin/refs.c +10 -0
+	struct option options[] = {
+		OPT_BOOL(0, "verbose", &fsck_refs_options.verbose, N_("be verbose")),
+		OPT_BOOL(0, "strict", &fsck_refs_options.strict, N_("enable strict checking")),
+		OPT_END(),
+	};
+	int ret;
+
+	argc = parse_options(argc, argv, prefix, options, verify_usage, 0);
+	if (argc)
+		usage(_("'git refs verify' takes no arguments"));

Junio has posted a patch series [1] where he wants to get rid of messages that simply say "no arguments" or "too many arguments". I guess we can play nice and also move into the same direction here, where we instead tell the user which argument we didn't expect.

So I'd propose to make this:

    argc = parse_options(argc, argv, prefix, options, verify_usage, 0);
    if (argc)
            usage(_("unknown argument: '%s'", argv[0]));

[1]: <[email protected]>

Patrick

#258 lore raw

Re: [GSoC][PATCH v15 7/9] builtin/refs: add verify subcommand

Quoted 11 lines · Patrick Steinhardt

builtin/refs.c static int cmd_refs_migrate(int argc, const char **argv, const char *prefix) · line 82

Quoted 2 lines
builtin/refs.c +2 -0
+	if (argc)
+		usage(_("'git refs verify' takes no arguments"));

Junio has posted a patch series [1] where he wants to get rid of messages that simply say "no arguments" or "too many arguments".
...
So I'd propose to make this:

    argc = parse_options(argc, argv, prefix, options, verify_usage, 0);
    if (argc)
            usage(_("unknown argument: '%s'", argv[0]));

I probably should have said that I am fully behind the intent against "too many arguments", but I am not 100% behind the particular messaging used in the patch series I sent out.

One potential complaint I expected to hear, for example, was that "a is unknown" given when you said "git cmd a a a a a" is not all that clear ;-). To alleviate, you would have to say "git cmd takes only 2 arguments" if 'a' you are complaining about is the third one.

Also, many people would consider that "unexpected argument" is better than "unknown argument".

I personally think the message above is absolutely clear and good.

You say that 'git refs verify' takes no arguments, and for somebody who said "git refs verify a b c d e", there is no doubt that all of these a b c d e are unwanted. And there is no room to misinterpret the message as "'git refs' is ok but 'git refs verify' is already unwelcome with extra argument", either [*].

In short, I think the message in the patch here is good, and it is the other "war on 'too many arguments'" series whose messages need to be thought further.

[Foornote]

* ... which was the problem I was trying to address in the current message "too many arguments" that does not even say which early part of the command line we consider is "command" that was given "arguments"---to uninitiated who said "git refs verify foo", it is unclera if that's "git refs" command whose first argument is "verify", "git" command whose first two arguments are "refs verify", etc.

#259 lore raw

Re: [GSoC][PATCH v15 7/9] builtin/refs: add verify subcommand

Quoted 34 lines · Junio C Hamano
Quoted 11 lines · Patrick Steinhardt

builtin/refs.c static int cmd_refs_migrate(int argc, const char **argv, const char *prefix) · line 82

Quoted 2 lines
builtin/refs.c +2 -0
+	if (argc)
+		usage(_("'git refs verify' takes no arguments"));

Junio has posted a patch series [1] where he wants to get rid of messages that simply say "no arguments" or "too many arguments".
...
So I'd propose to make this:

    argc = parse_options(argc, argv, prefix, options, verify_usage, 0);
    if (argc)
            usage(_("unknown argument: '%s'", argv[0]));

I probably should have said that I am fully behind the intent against "too many arguments", but I am not 100% behind the particular messaging used in the patch series I sent out.

One potential complaint I expected to hear, for example, was that "a is unknown" given when you said "git cmd a a a a a" is not all that clear ;-). To alleviate, you would have to say "git cmd takes only 2 arguments" if 'a' you are complaining about is the third one.

Also, many people would consider that "unexpected argument" is better than "unknown argument".

I personally think the message above is absolutely clear and good.

You say that 'git refs verify' takes no arguments, and for somebody who said "git refs verify a b c d e", there is no doubt that all of these a b c d e are unwanted. And there is no room to misinterpret the message as "'git refs' is ok but 'git refs verify' is already unwelcome with extra argument", either [*].

Quoted 3 lines

In short, I think the message in the patch here is good, and it is the other "war on 'too many arguments'" series whose messages need to be thought further.

Just to clarify: with "the patch" you probably refer to the current version that Jialuo has, right? In other words, keep the current version that he has and adapt the message in the future, when we have decided what to do about those "too many arguments" messages?

If so, then the only two I had were some spurious newlines. I'm not sure whether these really would be worth rerolling the whole patch series.

Patrick

author
#260 lore raw

Re: [GSoC][PATCH v15 7/9] builtin/refs: add verify subcommand

Quoted 3 lines

If so, then the only two I had were some spurious newlines. I'm not sure whether these really would be worth rerolling the whole patch series.

Karthik has given some reviews. I guess I need to reroll because there is one typo error in commit message. It's important to make this fixed.

Quoted 1 line

Patrick

#261 lore raw

Re: [GSoC][PATCH v15 7/9] builtin/refs: add verify subcommand

Quoted 8 lines · shejialuo
Quoted 3 lines

If so, then the only two I had were some spurious newlines. I'm not sure whether these really would be worth rerolling the whole patch series.

Karthik has given some reviews. I guess I need to reroll because there is one typo error in commit message. It's important to make this fixed.

Quoted 1 line

Patrick

I would say, my nits by themselves don't require a re-roll, but if you're re-rolling, it'd be nice to resolve them too :)

#262 lore raw

Re: [GSoC][PATCH v15 7/9] builtin/refs: add verify subcommand

In reply to Patrick Steinhardt
Quoted 35 lines · Patrick Steinhardt
Quoted 34 lines · Junio C Hamano
Quoted 11 lines · Patrick Steinhardt

builtin/refs.c static int cmd_refs_migrate(int argc, const char **argv, const char *prefix) · line 82

Quoted 2 lines
builtin/refs.c +2 -0
+	if (argc)
+		usage(_("'git refs verify' takes no arguments"));

Junio has posted a patch series [1] where he wants to get rid of messages that simply say "no arguments" or "too many arguments".
...
So I'd propose to make this:

    argc = parse_options(argc, argv, prefix, options, verify_usage, 0);
    if (argc)
            usage(_("unknown argument: '%s'", argv[0]));

I probably should have said that I am fully behind the intent against "too many arguments", but I am not 100% behind the particular messaging used in the patch series I sent out.

One potential complaint I expected to hear, for example, was that "a is unknown" given when you said "git cmd a a a a a" is not all that clear ;-). To alleviate, you would have to say "git cmd takes only 2 arguments" if 'a' you are complaining about is the third one.

Also, many people would consider that "unexpected argument" is better than "unknown argument".

I personally think the message above is absolutely clear and good.

You say that 'git refs verify' takes no arguments, and for somebody who said "git refs verify a b c d e", there is no doubt that all of these a b c d e are unwanted. And there is no room to misinterpret the message as "'git refs' is ok but 'git refs verify' is already unwelcome with extra argument", either [*].

Quoted 8 lines
Quoted 3 lines

In short, I think the message in the patch here is good, and it is the other "war on 'too many arguments'" series whose messages need to be thought further.

Just to clarify: with "the patch" you probably refer to the current version that Jialuo has, right? In other words, keep the current version that he has and adapt the message in the future, when we have decided what to do about those "too many arguments" messages?

I meant that I think (1) that "'git refs verify' takes no arguments" is a good message, and (2) that there is no further change needed to the patch that started this review thread, regardless of how we want to deal with "too many arguments" messages.

Quoted 2 lines

If so, then the only two I had were some spurious newlines. I'm not sure whether these really would be worth rerolling the whole patch series.

Yup, those blank lines were annoying while scanning the patches, but they alone would not be something that makes a reroll _required_. A reroll that clearly shows that the incremental change since the last round is only these blank line changes is not too much to process, so "not worth the reviewer time" is not a huge reason to avoid it, either. I'd see that it is up to how perfectionist the patch submitter wants to be ;-)

Thanks.

author
#263 lore raw

PATCH v15 8/9 files-backend: add unified interface for refs scanning

In reply to shejialuo

For refs and reflogs, we need to scan its corresponding directories to check every regular file or symbolic link which shares the same pattern. Introduce a unified interface for scanning directories for files-backend.

---
 Documentation/fsck-msgids.txt |  3 ++
 fsck.h                        |  1 +
 refs/files-backend.c          | 74 ++++++++++++++++++++++++++++++++++-
 3 files changed, 77 insertions(+), 1 deletion(-)
Documentation/fsck-msgids.txt +3 -0
index f643585a34..7c809fddf1 100644
@@ -19,6 +19,9 @@
 `badParentSha1`::
 	(ERROR) A commit object has a bad parent sha1.
 
+`badRefFiletype`::
+	(ERROR) A ref has a bad file type.
+
 `badTagName`::
 	(INFO) A tag has an invalid format.
 
fsck.h +1 -0
index d551a9fe86..af02174973 100644
@@ -31,6 +31,7 @@ enum fsck_msg_type {
 	FUNC(BAD_NAME, ERROR) \
 	FUNC(BAD_OBJECT_SHA1, ERROR) \
 	FUNC(BAD_PARENT_SHA1, ERROR) \
+	FUNC(BAD_REF_FILETYPE, ERROR) \
 	FUNC(BAD_TIMEZONE, ERROR) \
 	FUNC(BAD_TREE, ERROR) \
 	FUNC(BAD_TREE_SHA1, ERROR) \
refs/files-backend.c +73 -1

92 lines not on this page · raw message

Signature · 1 line
2.46.0
#264 lore raw

Re: [GSoC][PATCH v15 8/9] files-backend: add unified interface for refs scanning

On Tue, Aug 06, 2024 at 12:46:24AM +0800, shejialuo wrote:
[snip]

refs/files-backend.c static int files_ref_store_remove_on_disk(struct ref_store *ref_store, · line 3479

Quoted 11 lines
refs/files-backend.c +11 -0
+static int files_fsck_refs(struct ref_store *ref_store,
+			   struct fsck_options *o)
+{
+	files_fsck_refs_fn fsck_refs_fn[]= {
+		NULL,
+	};
+
+	if (o->verbose)
+		fprintf_ln(stderr, _("Checking references consistency"));
+	return files_fsck_refs_dir(ref_store, o,  "refs", fsck_refs_fn);
+

Nit: another empty newline.

Review

Quoted 2 lines
(quoted diff) +2 -0
+}
+

Patrick

author
#265 lore raw

PATCH v15 9/9 fsck: add ref name check for files backend

In reply to shejialuo

The git-fsck(1) only implicitly checks the reference, it does not fully check refs with bad format name such as standalone "@".

However, a file ending with ".lock" should not be marked as having a bad ref name. It is expected that concurrent writers may have such lock files. We currently ignore this situation. But for bare ".lock" file, we will report it as error.

In order to provide such checks, add a new fsck message id "badRefName" with default ERROR type. Use existing "check_refname_format" to explicit check the ref name. And add a new unit test to verify the functionality.

---
 Documentation/fsck-msgids.txt |  3 ++
 fsck.h                        |  1 +
 refs/files-backend.c          | 31 ++++++++++++
 t/t0602-reffiles-fsck.sh      | 92 +++++++++++++++++++++++++++++++++++
 4 files changed, 127 insertions(+)
 create mode 100755 t/t0602-reffiles-fsck.sh
Documentation/fsck-msgids.txt +3 -0
index 7c809fddf1..68a2801f15 100644
@@ -22,6 +22,9 @@
 `badRefFiletype`::
 	(ERROR) A ref has a bad file type.
 
+`badRefName`::
+	(ERROR) A ref has an invalid format.
+
 `badTagName`::
 	(INFO) A tag has an invalid format.
 
fsck.h +1 -0
index af02174973..500b4c04d2 100644
@@ -32,6 +32,7 @@ enum fsck_msg_type {
 	FUNC(BAD_OBJECT_SHA1, ERROR) \
 	FUNC(BAD_PARENT_SHA1, ERROR) \
 	FUNC(BAD_REF_FILETYPE, ERROR) \
+	FUNC(BAD_REF_NAME, ERROR) \
 	FUNC(BAD_TIMEZONE, ERROR) \
 	FUNC(BAD_TREE, ERROR) \
 	FUNC(BAD_TREE_SHA1, ERROR) \
refs/files-backend.c +31 -0
index f337356860..6e6b47251d 100644
@@ -3419,6 +3419,36 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,
 				  const char *refs_check_dir,
 				  struct dir_iterator *iter);
 
+static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,
+				struct fsck_options *o,
+				const char *refs_check_dir,
+				struct dir_iterator *iter)
+{
+	struct strbuf sb = STRBUF_INIT;
+	int ret = 0;
+
+	/*
+	 * Ignore the files ending with ".lock" as they may be lock files
+	 * However, do not allow bare ".lock" files.
+	 */
+	if (iter->basename[0] != '.' && ends_with(iter->basename, ".lock"))
+		goto cleanup;
+
+	if (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {
+		struct fsck_ref_report report = { .path = NULL };
+
+		strbuf_addf(&sb, "%s/%s", refs_check_dir, iter->relative_path);
+		report.path = sb.buf;
+		ret = fsck_report_ref(o, &report,
+				      FSCK_MSG_BAD_REF_NAME,
+				      "invalid refname format");
+	}
+
+cleanup:
+	strbuf_release(&sb);
+	return ret;
+}
+
 static int files_fsck_refs_dir(struct ref_store *ref_store,
 			       struct fsck_options *o,
 			       const char *refs_check_dir,
@@ -3470,6 +3500,7 @@ static int files_fsck_refs(struct ref_store *ref_store,
 			   struct fsck_options *o)
 {
 	files_fsck_refs_fn fsck_refs_fn[]= {
+		files_fsck_refs_name,
 		NULL,
 	};
 
t/t0602-reffiles-fsck.sh new file +92 -0

92 lines not on this page · raw message

Signature · 1 line
2.46.0
#266 lore raw
In reply to shejialuo
Quoted 40 lines · shejialuo

Hi All:

This version handles the following problems:

1. Patrick advices that I should not use va_copy in the changed report function. Actually this is a mistake, this version avoids redundant ap copy.
2. Patrick advices I should rebase [v14 05/11] into [v14 04/11]. I follow this advice in this version.
3. Patrick advices that we should put [v14 06/11] before we introduce ref-related operations. This version reorders the commit sequence. It's a minor change.
4. Patrick suggests at current we should not add git refs verify command into "git-fsck(1)". This is because we should disable this new check by default for the users. Many users use "git-fsck(1)" in their daily workflow. We should not be aggressive. However, if we provide this mechanism in this series, we will again make more complexity. So this version drop patch [v14 09/11]. Also because of dropping, change the test file to use "git refs verify" command instead of "git fsck" command.
5. Patrick suggests that we should use ends_with instead of strip_suffix, fix.

There is another important problem this patch solves:

At v13, Junio has suggested that the files_fsck_refs_fn should be adapted to Patrick's change. Actually, I made a bad design before. I should always pass the ref_store structure. So I change it to

-typedef int (*files_fsck_refs_fn)(struct fsck_options *o,
- const char *gitdir,
+typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,
+ struct fsck_options *o,
const char *refs_check_dir,
struct dir_iterator *iter);

gitdir could be got by using ref_store parameter. By using ref_store parameter, we provide extensibility here. If something else change, we merely need to change "files_fsck_refs_fn" prototype.

Quoted 5 lines

Because I drop one patch and rebase one patch. I provide the interdiff for reviewers to make the life easier.

Due to the deadline of the GSoC, I will speed up the review feedback process.

I've got another small set of nits, almost not worth addressing. I was a bit torn whether to send them or not as the series is in a good shape already, in my opinion. But let's maybe wait one or two more days for additional feedback, and then (hopefully) reroll this a final time.

Thanks for all your work!

Patrick

#267 lore raw
In reply to shejialuo
Quoted 4 lines · shejialuo

Hi All:

This version handles the following problems:

Quoted 16 lines

1. Patrick advices that I should not use va_copy in the changed report function. Actually this is a mistake, this version avoids redundant ap copy.
2. Patrick advices I should rebase [v14 05/11] into [v14 04/11]. I follow this advice in this version.
3. Patrick advices that we should put [v14 06/11] before we introduce ref-related operations. This version reorders the commit sequence. It's a minor change.
4. Patrick suggests at current we should not add git refs verify command into "git-fsck(1)". This is because we should disable this new check by default for the users. Many users use "git-fsck(1)" in their daily workflow. We should not be aggressive. However, if we provide this mechanism in this series, we will again make more complexity. So this version drop patch [v14 09/11]. Also because of dropping, change the test file to use "git refs verify" command instead of "git fsck" command.

This is the biggest change in this version and it makes sense. It can still be added later on, but for now users can use this via git refs verify.

Quoted 3 lines

5. Patrick suggests that we should use ends_with instead of strip_suffix, fix.

Apart from the minor nits, I think this version looks good. Thanks!

author
#268 lore raw

v16 cover ref consistency check infra setup

In reply to shejialuo

Hi All, this version handles some minor changes:

1. Remove redundant newlines.
2. Fix typo in commit message.

At last, I wanna thank every reviewer. As we can see, this series starts from 5.27, it's a very long journey. I have learned a lot.

Thanks
Jialuo

shejialuo (9):
fsck: rename "skiplist" to "skip_oids"
fsck: rename objects-related fsck error functions
fsck: make "fsck_error" callback generic
fsck: add a unified interface for reporting fsck messages
fsck: add refs report function
refs: set up ref consistency check infrastructure builtin/refs: add verify subcommand
files-backend: add unified interface for refs scanning
fsck: add ref name check for files backend

 Documentation/fsck-msgids.txt |   6 ++
 Documentation/git-refs.txt    |  13 ++++
 builtin/fsck.c                |  17 +++--
 builtin/mktag.c               |   3 +-
 builtin/refs.c                |  34 +++++++++
 fsck.c                        | 125 +++++++++++++++++++++++++++-------
 fsck.h                        |  76 ++++++++++++++++-----
 object-file.c                 |   9 ++-
 refs.c                        |   5 ++
 refs.h                        |   8 +++
 refs/debug.c                  |  11 +++
 refs/files-backend.c          | 115 ++++++++++++++++++++++++++++++-
 refs/packed-backend.c         |   8 +++
 refs/refs-internal.h          |   6 ++
 refs/reftable-backend.c       |   8 +++
 t/t0602-reffiles-fsck.sh      |  92 +++++++++++++++++++++++++
 16 files changed, 477 insertions(+), 59 deletions(-)
 create mode 100755 t/t0602-reffiles-fsck.sh

Range-diff against v15:

 1:  9aeaa3211c =  1:  9aeaa3211c fsck: rename "skiplist" to "skip_oids"
 2:  7511340a21 =  2:  7511340a21 fsck: rename objects-related fsck error functions
 3:  ee971d17f4 =  3:  ee971d17f4 fsck: make "fsck_error" callback generic
 4:  59ccdab54d !  4:  f80fa00538 fsck: add a unified interface for reporting fsck messages

@@ Commit message
"fsck_vreport". Instead of using "...", provide "va_list" to allow more flexibility.

- Instead of changing "report" prototype to be algin with the
+ Instead of changing "report" prototype to be align with the
"fsck_vreport" function, we leave the "report" prototype unchanged due to the reason that there are nearly 62 references about "report" function. Simply change "report" function to use "fsck_vreport" to
5: b5607ac61c ! 5: 8c0376cfd5 fsck: add refs report function

    @@ fsck.c: int fsck_objects_error_function(struct fsck_options *o,
     +			     const char *message)
     +{
     +	struct fsck_ref_report *report = fsck_report;
    -+
     +	struct strbuf sb = STRBUF_INIT;
     +	int ret = 0;
     +
    @@ fsck.c: int fsck_objects_error_function(struct fsck_options *o,
     +
     +	strbuf_release(&sb);
     +	return ret;
    -+
     +}
     +
      static int fsck_blobs(struct oidset *blobs_found, struct oidset *blobs_done,
 6:  5819406b40 =  6:  552fe62ff4 refs: set up ref consistency check infrastructure
 7:  6a9c194a05 =  7:  3b357fa89e builtin/refs: add verify subcommand
 8:  76406b1303 !  8:  8d2c7b3aa5 files-backend: add unified interface for refs scanning
    @@ refs/files-backend.c: static int files_ref_store_remove_on_disk(struct ref_store
     +	if (o->verbose)
     +		fprintf_ln(stderr, _("Checking references consistency"));
     +	return files_fsck_refs_dir(ref_store, o,  "refs", fsck_refs_fn);
    -+
     +}
     +
      static int files_fsck(struct ref_store *ref_store,

9: ed02380516 = 9: b39533ae56 fsck: add ref name check for files backend

Signature · 1 line
2.46.0
author
#269 lore raw

PATCH v16 1/9 fsck: rename "skiplist" to "skip_oids"

The "skiplist" field in "fsck_options" is related to objects. Because we are going to introduce ref consistency check, the "skiplist" name is too general which will make the caller think "skiplist" is related to both the refs and objects.

It may seem that for both refs and objects, we should provide a general "skiplist" here. However, the type for "skiplist" is struct oidset which is totally unsuitable for refs.

To avoid above ambiguity, rename "skiplist" to "skip_oids".

---
 fsck.c | 4 ++--
 fsck.h | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)
fsck.c +2 -2
index eea7145470..3f32441492 100644
@@ -205,7 +205,7 @@ void fsck_set_msg_types(struct fsck_options *options, const char *values)
 		if (!strcmp(buf, "skiplist")) {
 			if (equal == len)
 				die("skiplist requires a path");
-			oidset_parse_file(&options->skiplist, buf + equal + 1,
+			oidset_parse_file(&options->skip_oids, buf + equal + 1,
 					  the_repository->hash_algo);
 			buf += len + 1;
 			continue;
@@ -223,7 +223,7 @@ void fsck_set_msg_types(struct fsck_options *options, const char *values)
 static int object_on_skiplist(struct fsck_options *opts,
 			      const struct object_id *oid)
 {
-	return opts && oid && oidset_contains(&opts->skiplist, oid);
+	return opts && oid && oidset_contains(&opts->skip_oids, oid);
 }
 
 __attribute__((format (printf, 5, 6)))
fsck.h +2 -2
index 6085a384f6..bcfb2e34cd 100644
@@ -136,7 +136,7 @@ struct fsck_options {
 	fsck_error error_func;
 	unsigned strict:1;
 	enum fsck_msg_type *msg_type;
-	struct oidset skiplist;
+	struct oidset skip_oids;
 	struct oidset gitmodules_found;
 	struct oidset gitmodules_done;
 	struct oidset gitattributes_found;
@@ -145,7 +145,7 @@ struct fsck_options {
 };
 
 #define FSCK_OPTIONS_DEFAULT { \
-	.skiplist = OIDSET_INIT, \
+	.skip_oids = OIDSET_INIT, \
 	.gitmodules_found = OIDSET_INIT, \
 	.gitmodules_done = OIDSET_INIT, \
 	.gitattributes_found = OIDSET_INIT, \
Signature · 1 line
2.46.0
author
#270 lore raw

PATCH v16 2/9 fsck: rename objects-related fsck error functions

In reply to shejialuo

The names of objects-related fsck error functions are generic. It's OK when there is only object database check. However, we are going to introduce refs database check report function. To avoid ambiguity, rename object-related fsck error functions to explicitly indicate these functions are used to report objects-related messages.

---
 builtin/fsck.c | 14 +++++++-------
 fsck.c         | 17 +++++++++--------
 fsck.h         | 26 +++++++++++++-------------
 3 files changed, 29 insertions(+), 28 deletions(-)
builtin/fsck.c +7 -7
index d13a226c2e..6d86bbe1e9 100644
@@ -89,12 +89,12 @@ static int objerror(struct object *obj, const char *err)
 	return -1;
 }
 
-static int fsck_error_func(struct fsck_options *o UNUSED,
-			   const struct object_id *oid,
-			   enum object_type object_type,
-			   enum fsck_msg_type msg_type,
-			   enum fsck_msg_id msg_id UNUSED,
-			   const char *message)
+static int fsck_objects_error_func(struct fsck_options *o UNUSED,
+				   const struct object_id *oid,
+				   enum object_type object_type,
+				   enum fsck_msg_type msg_type,
+				   enum fsck_msg_id msg_id UNUSED,
+				   const char *message)
 {
 	switch (msg_type) {
 	case FSCK_WARN:
@@ -938,7 +938,7 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 
 	fsck_walk_options.walk = mark_object;
 	fsck_obj_options.walk = mark_used;
-	fsck_obj_options.error_func = fsck_error_func;
+	fsck_obj_options.error_func = fsck_objects_error_func;
 	if (check_strict)
 		fsck_obj_options.strict = 1;
 
fsck.c +9 -8
index 3f32441492..8347842cfb 100644
@@ -1200,7 +1200,7 @@ int fsck_buffer(const struct object_id *oid, enum object_type type,
 		      type);
 }
 
-int fsck_error_function(struct fsck_options *o,
+int fsck_objects_error_function(struct fsck_options *o,
 			const struct object_id *oid,
 			enum object_type object_type UNUSED,
 			enum fsck_msg_type msg_type,
@@ -1303,16 +1303,17 @@ int git_fsck_config(const char *var, const char *value,
  * Custom error callbacks that are used in more than one place.
  */
 
-int fsck_error_cb_print_missing_gitmodules(struct fsck_options *o,
-					   const struct object_id *oid,
-					   enum object_type object_type,
-					   enum fsck_msg_type msg_type,
-					   enum fsck_msg_id msg_id,
-					   const char *message)
+int fsck_objects_error_cb_print_missing_gitmodules(struct fsck_options *o,
+						   const struct object_id *oid,
+						   enum object_type object_type,
+						   enum fsck_msg_type msg_type,
+						   enum fsck_msg_id msg_id,
+						   const char *message)
 {
 	if (msg_id == FSCK_MSG_GITMODULES_MISSING) {
 		puts(oid_to_hex(oid));
 		return 0;
 	}
-	return fsck_error_function(o, oid, object_type, msg_type, msg_id, message);
+	return fsck_objects_error_function(o, oid, object_type,
+					   msg_type, msg_id, message);
 }
fsck.h +13 -13
index bcfb2e34cd..41ebebbb59 100644
@@ -120,16 +120,16 @@ typedef int (*fsck_error)(struct fsck_options *o,
 			  enum fsck_msg_type msg_type, enum fsck_msg_id msg_id,
 			  const char *message);
 
-int fsck_error_function(struct fsck_options *o,
-			const struct object_id *oid, enum object_type object_type,
-			enum fsck_msg_type msg_type, enum fsck_msg_id msg_id,
-			const char *message);
-int fsck_error_cb_print_missing_gitmodules(struct fsck_options *o,
-					   const struct object_id *oid,
-					   enum object_type object_type,
-					   enum fsck_msg_type msg_type,
-					   enum fsck_msg_id msg_id,
-					   const char *message);
+int fsck_objects_error_function(struct fsck_options *o,
+				const struct object_id *oid, enum object_type object_type,
+				enum fsck_msg_type msg_type, enum fsck_msg_id msg_id,
+				const char *message);
+int fsck_objects_error_cb_print_missing_gitmodules(struct fsck_options *o,
+						   const struct object_id *oid,
+						   enum object_type object_type,
+						   enum fsck_msg_type msg_type,
+						   enum fsck_msg_id msg_id,
+						   const char *message);
 
 struct fsck_options {
 	fsck_walk_func walk;
@@ -150,7 +150,7 @@ struct fsck_options {
 	.gitmodules_done = OIDSET_INIT, \
 	.gitattributes_found = OIDSET_INIT, \
 	.gitattributes_done = OIDSET_INIT, \
-	.error_func = fsck_error_function \
+	.error_func = fsck_objects_error_function \
 }
 #define FSCK_OPTIONS_STRICT { \
 	.strict = 1, \
@@ -158,7 +158,7 @@ struct fsck_options {
 	.gitmodules_done = OIDSET_INIT, \
 	.gitattributes_found = OIDSET_INIT, \
 	.gitattributes_done = OIDSET_INIT, \
-	.error_func = fsck_error_function, \
+	.error_func = fsck_objects_error_function, \
 }
 #define FSCK_OPTIONS_MISSING_GITMODULES { \
 	.strict = 1, \
@@ -166,7 +166,7 @@ struct fsck_options {
 	.gitmodules_done = OIDSET_INIT, \
 	.gitattributes_found = OIDSET_INIT, \
 	.gitattributes_done = OIDSET_INIT, \
-	.error_func = fsck_error_cb_print_missing_gitmodules, \
+	.error_func = fsck_objects_error_cb_print_missing_gitmodules, \
 }
 
 /* descend in all linked child objects
Signature · 1 line
2.46.0
author
#271 lore raw

PATCH v16 3/9 fsck: make "fsck_error" callback generic

In reply to shejialuo

The "fsck_error" callback is designed to report the objects-related error messages. It accepts two parameter "oid" and "object_type" which is not generic. In order to provide a unified callback which can report either objects or refs, remove the objects-related parameters and add the generic parameter "void *fsck_report".

Create a new "fsck_object_report" structure which incorporates the removed parameters "oid" and "object_type". Then change the corresponding references to adapt to new "fsck_error" callback.

---
 builtin/fsck.c  |  7 +++++--
 builtin/mktag.c |  3 +--
 fsck.c          | 26 ++++++++++++++++----------
 fsck.h          | 17 ++++++++++++-----
 object-file.c   |  9 ++++-----
 5 files changed, 38 insertions(+), 24 deletions(-)
builtin/fsck.c +5 -2
index 6d86bbe1e9..766bbd014d 100644
@@ -90,12 +90,15 @@ static int objerror(struct object *obj, const char *err)
 }
 
 static int fsck_objects_error_func(struct fsck_options *o UNUSED,
-				   const struct object_id *oid,
-				   enum object_type object_type,
+				   void *fsck_report,
 				   enum fsck_msg_type msg_type,
 				   enum fsck_msg_id msg_id UNUSED,
 				   const char *message)
 {
+	struct fsck_object_report *report = fsck_report;
+	const struct object_id *oid = report->oid;
+	enum object_type object_type = report->object_type;
+
 	switch (msg_type) {
 	case FSCK_WARN:
 		/* TRANSLATORS: e.g. warning in tree 01bfda: <more explanation> */
builtin/mktag.c +1 -2
index 4767f1a97e..c6b644219f 100644
@@ -18,8 +18,7 @@ static int option_strict = 1;
 static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;
 
 static int mktag_fsck_error_func(struct fsck_options *o UNUSED,
-				 const struct object_id *oid UNUSED,
-				 enum object_type object_type UNUSED,
+				 void *fsck_report UNUSED,
 				 enum fsck_msg_type msg_type,
 				 enum fsck_msg_id msg_id UNUSED,
 				 const char *message)
fsck.c +16 -10
index 8347842cfb..cca6ae144f 100644
@@ -232,6 +232,10 @@ static int report(struct fsck_options *options,
 		  enum fsck_msg_id msg_id, const char *fmt, ...)
 {
 	va_list ap;
+	struct fsck_object_report report = {
+		.oid = oid,
+		.object_type = object_type
+	};
 	struct strbuf sb = STRBUF_INIT;
 	enum fsck_msg_type msg_type = fsck_msg_type(msg_id, options);
 	int result;
@@ -252,7 +256,7 @@ static int report(struct fsck_options *options,
 
 	va_start(ap, fmt);
 	strbuf_vaddf(&sb, fmt, ap);
-	result = options->error_func(options, oid, object_type,
+	result = options->error_func(options, &report,
 				     msg_type, msg_id, sb.buf);
 	strbuf_release(&sb);
 	va_end(ap);
@@ -1201,12 +1205,14 @@ int fsck_buffer(const struct object_id *oid, enum object_type type,
 }
 
 int fsck_objects_error_function(struct fsck_options *o,
-			const struct object_id *oid,
-			enum object_type object_type UNUSED,
-			enum fsck_msg_type msg_type,
-			enum fsck_msg_id msg_id UNUSED,
-			const char *message)
+				void *fsck_report,
+				enum fsck_msg_type msg_type,
+				enum fsck_msg_id msg_id UNUSED,
+				const char *message)
 {
+	struct fsck_object_report *report = fsck_report;
+	const struct object_id *oid = report->oid;
+
 	if (msg_type == FSCK_WARN) {
 		warning("object %s: %s", fsck_describe_object(o, oid), message);
 		return 0;
@@ -1304,16 +1310,16 @@ int git_fsck_config(const char *var, const char *value,
  */
 
 int fsck_objects_error_cb_print_missing_gitmodules(struct fsck_options *o,
-						   const struct object_id *oid,
-						   enum object_type object_type,
+						   void *fsck_report,
 						   enum fsck_msg_type msg_type,
 						   enum fsck_msg_id msg_id,
 						   const char *message)
 {
 	if (msg_id == FSCK_MSG_GITMODULES_MISSING) {
-		puts(oid_to_hex(oid));
+		struct fsck_object_report *report = fsck_report;
+		puts(oid_to_hex(report->oid));
 		return 0;
 	}
-	return fsck_objects_error_function(o, oid, object_type,
+	return fsck_objects_error_function(o, fsck_report,
 					   msg_type, msg_id, message);
 }
fsck.h +12 -5
index 41ebebbb59..3b80d02506 100644
@@ -114,23 +114,30 @@ int is_valid_msg_type(const char *msg_id, const char *msg_type);
 typedef int (*fsck_walk_func)(struct object *obj, enum object_type object_type,
 			      void *data, struct fsck_options *options);
 
-/* callback for fsck_object, type is FSCK_ERROR or FSCK_WARN */
+/*
+ * Callback for reporting errors either for objects or refs. The "fsck_report"
+ * is a generic pointer that can be used to pass any information.
+ */
 typedef int (*fsck_error)(struct fsck_options *o,
-			  const struct object_id *oid, enum object_type object_type,
+			  void *fsck_report,
 			  enum fsck_msg_type msg_type, enum fsck_msg_id msg_id,
 			  const char *message);
 
 int fsck_objects_error_function(struct fsck_options *o,
-				const struct object_id *oid, enum object_type object_type,
+				void *fsck_report,
 				enum fsck_msg_type msg_type, enum fsck_msg_id msg_id,
 				const char *message);
 int fsck_objects_error_cb_print_missing_gitmodules(struct fsck_options *o,
-						   const struct object_id *oid,
-						   enum object_type object_type,
+						   void *fsck_report,
 						   enum fsck_msg_type msg_type,
 						   enum fsck_msg_id msg_id,
 						   const char *message);
 
+struct fsck_object_report {
+	const struct object_id *oid;
+	enum object_type object_type;
+};
+
 struct fsck_options {
 	fsck_walk_func walk;
 	fsck_error error_func;
object-file.c +4 -5
index 065103be3e..05ac6ebed6 100644
@@ -2470,11 +2470,10 @@ int repo_has_object_file(struct repository *r,
  * give more context.
  */
 static int hash_format_check_report(struct fsck_options *opts UNUSED,
-				     const struct object_id *oid UNUSED,
-				     enum object_type object_type UNUSED,
-				     enum fsck_msg_type msg_type UNUSED,
-				     enum fsck_msg_id msg_id UNUSED,
-				     const char *message)
+				    void *fsck_report UNUSED,
+				    enum fsck_msg_type msg_type UNUSED,
+				    enum fsck_msg_id msg_id UNUSED,
+				    const char *message)
 {
 	error(_("object fails fsck: %s"), message);
 	return 1;
Signature · 1 line
2.46.0
author
#272 lore raw

PATCH v16 4/9 fsck: add a unified interface for reporting fsck messages

In reply to shejialuo

The static function "report" provided by "fsck.c" aims at checking error type and calling the callback "error_func" to report the message. Both refs and objects need to check the error type of the current fsck message. In order to extract this common behavior, create a new function "fsck_vreport". Instead of using "...", provide "va_list" to allow more flexibility.

Instead of changing "report" prototype to be align with the "fsck_vreport" function, we leave the "report" prototype unchanged due to the reason that there are nearly 62 references about "report" function. Simply change "report" function to use "fsck_vreport" to report objects related messages.

---
 fsck.c | 44 ++++++++++++++++++++++++++++++--------------
 1 file changed, 30 insertions(+), 14 deletions(-)
fsck.c +30 -14
index cca6ae144f..3614aa56a3 100644
@@ -226,16 +226,15 @@ static int object_on_skiplist(struct fsck_options *opts,
 	return opts && oid && oidset_contains(&opts->skip_oids, oid);
 }
 
-__attribute__((format (printf, 5, 6)))
-static int report(struct fsck_options *options,
-		  const struct object_id *oid, enum object_type object_type,
-		  enum fsck_msg_id msg_id, const char *fmt, ...)
+/*
+ * Provide the common functionality for either fscking refs or objects.
+ * It will get the current msg error type and call the error_func callback
+ * which is registered in the "fsck_options" struct.
+ */
+static int fsck_vreport(struct fsck_options *options,
+			void *fsck_report,
+			enum fsck_msg_id msg_id, const char *fmt, va_list ap)
 {
-	va_list ap;
-	struct fsck_object_report report = {
-		.oid = oid,
-		.object_type = object_type
-	};
 	struct strbuf sb = STRBUF_INIT;
 	enum fsck_msg_type msg_type = fsck_msg_type(msg_id, options);
 	int result;
@@ -243,9 +242,6 @@ static int report(struct fsck_options *options,
 	if (msg_type == FSCK_IGNORE)
 		return 0;
 
-	if (object_on_skiplist(options, oid))
-		return 0;
-
 	if (msg_type == FSCK_FATAL)
 		msg_type = FSCK_ERROR;
 	else if (msg_type == FSCK_INFO)
@@ -254,11 +250,31 @@ static int report(struct fsck_options *options,
 	prepare_msg_ids();
 	strbuf_addf(&sb, "%s: ", msg_id_info[msg_id].camelcased);
 
-	va_start(ap, fmt);
 	strbuf_vaddf(&sb, fmt, ap);
-	result = options->error_func(options, &report,
+	result = options->error_func(options, fsck_report,
 				     msg_type, msg_id, sb.buf);
 	strbuf_release(&sb);
+
+	return result;
+}
+
+__attribute__((format (printf, 5, 6)))
+static int report(struct fsck_options *options,
+		  const struct object_id *oid, enum object_type object_type,
+		  enum fsck_msg_id msg_id, const char *fmt, ...)
+{
+	va_list ap;
+	struct fsck_object_report report = {
+		.oid = oid,
+		.object_type = object_type
+	};
+	int result;
+
+	if (object_on_skiplist(options, oid))
+		return 0;
+
+	va_start(ap, fmt);
+	result = fsck_vreport(options, &report, msg_id, fmt, ap);
 	va_end(ap);
 
 	return result;
Signature · 1 line
2.46.0
author
#273 lore raw

PATCH v16 5/9 fsck: add refs report function

In reply to shejialuo

Introduce a new struct "fsck_ref_report" to contain the information we need when reporting refs-related messages.

With the new "fsck_vreport" function, add a new function "fsck_report_ref" to report refs-related fsck error message. Unlike "report" function uses the exact parameters, we simply pass "struct fsck_ref_report *report" as the parameter. This is because at current we don't know exactly how many fields we need. By passing this parameter, we don't need to change this function prototype when we want to add more information into "fsck_ref_report".

We have introduced "fsck_report_ref" function to report the error message for refs. We still need to add the corresponding callback function. Create refs-specific "error_func" callback "fsck_refs_error_function".

Last, add "FSCK_REFS_OPTIONS_DEFAULT" macro to create default options when checking ref consistency.

---
 fsck.c | 39 +++++++++++++++++++++++++++++++++++++++
 fsck.h | 25 +++++++++++++++++++++++++
 2 files changed, 64 insertions(+)
fsck.c +39 -0
index 3614aa56a3..e16c892f6a 100644
@@ -280,6 +280,19 @@ static int report(struct fsck_options *options,
 	return result;
 }
 
+int fsck_report_ref(struct fsck_options *options,
+		    struct fsck_ref_report *report,
+		    enum fsck_msg_id msg_id,
+		    const char *fmt, ...)
+{
+	va_list ap;
+	int result;
+	va_start(ap, fmt);
+	result = fsck_vreport(options, report, msg_id, fmt, ap);
+	va_end(ap);
+	return result;
+}
+
 void fsck_enable_object_names(struct fsck_options *options)
 {
 	if (!options->object_names)
@@ -1237,6 +1250,32 @@ int fsck_objects_error_function(struct fsck_options *o,
 	return 1;
 }
 
+int fsck_refs_error_function(struct fsck_options *options UNUSED,
+			     void *fsck_report,
+			     enum fsck_msg_type msg_type,
+			     enum fsck_msg_id msg_id UNUSED,
+			     const char *message)
+{
+	struct fsck_ref_report *report = fsck_report;
+	struct strbuf sb = STRBUF_INIT;
+	int ret = 0;
+
+	strbuf_addstr(&sb, report->path);
+
+	if (report->oid)
+		strbuf_addf(&sb, " -> (%s)", oid_to_hex(report->oid));
+	else if (report->referent)
+		strbuf_addf(&sb, " -> (%s)", report->referent);
+
+	if (msg_type == FSCK_WARN)
+		warning("%s: %s", sb.buf, message);
+	else
+		ret = error("%s: %s", sb.buf, message);
+
+	strbuf_release(&sb);
+	return ret;
+}
+
 static int fsck_blobs(struct oidset *blobs_found, struct oidset *blobs_done,
 		      enum fsck_msg_id msg_missing, enum fsck_msg_id msg_type,
 		      struct fsck_options *options, const char *blob_type)
fsck.h +25 -0
index 3b80d02506..2002590f60 100644
@@ -133,11 +133,23 @@ int fsck_objects_error_cb_print_missing_gitmodules(struct fsck_options *o,
 						   enum fsck_msg_id msg_id,
 						   const char *message);
 
+int fsck_refs_error_function(struct fsck_options *options,
+			     void *fsck_report,
+			     enum fsck_msg_type msg_type,
+			     enum fsck_msg_id msg_id,
+			     const char *message);
+
 struct fsck_object_report {
 	const struct object_id *oid;
 	enum object_type object_type;
 };
 
+struct fsck_ref_report {
+	const char *path;
+	const struct object_id *oid;
+	const char *referent;
+};
+
 struct fsck_options {
 	fsck_walk_func walk;
 	fsck_error error_func;
@@ -175,6 +187,9 @@ struct fsck_options {
 	.gitattributes_done = OIDSET_INIT, \
 	.error_func = fsck_objects_error_cb_print_missing_gitmodules, \
 }
+#define FSCK_REFS_OPTIONS_DEFAULT { \
+	.error_func = fsck_refs_error_function, \
+}
 
 /* descend in all linked child objects
  * the return value is:
@@ -216,6 +231,16 @@ int fsck_tag_standalone(const struct object_id *oid, const char *buffer,
  */
 int fsck_finish(struct fsck_options *options);
 
+/*
+ * Report an error or warning for refs.
+ */
+__attribute__((format (printf, 4, 5)))
+int fsck_report_ref(struct fsck_options *options,
+		    struct fsck_ref_report *report,
+		    enum fsck_msg_id msg_id,
+		    const char *fmt, ...);
+
+
 /*
  * Subsystem for storing human-readable names for each object.
  *
Signature · 1 line
2.46.0
author
#274 lore raw

PATCH v16 6/9 refs: set up ref consistency check infrastructure

In reply to shejialuo

The "struct ref_store" is the base class which contains the "be" pointer which provides backend-specific functions whose interfaces are defined in the "ref_storage_be". We could reuse this polymorphism to define only one interface. For every backend, we need to provide its own function pointer.

The interfaces defined in the ref_storage_be are carefully structured in semantic. It's organized as the five parts:

1. The name and the initialization interfaces.
2. The ref transaction interfaces.
3. The ref internal interfaces (pack, rename and copy).
4. The ref filesystem interfaces.
5. The reflog related interfaces.

To keep consistent with the git-fsck(1), add a new interface named "fsck_refs_fn" to the end of "ref_storage_be". This semantic cannot be grouped into any above five categories. Explicitly add blank line to make it different from others.

Last, implement placeholder functions for each ref backends.

---
 refs.c                  |  5 +++++
 refs.h                  |  8 ++++++++
 refs/debug.c            | 11 +++++++++++
 refs/files-backend.c    | 13 ++++++++++++-
 refs/packed-backend.c   |  8 ++++++++
 refs/refs-internal.h    |  6 ++++++
 refs/reftable-backend.c |  8 ++++++++
 7 files changed, 58 insertions(+), 1 deletion(-)
refs.c +5 -0
index 915aeb4d1d..6f642dc681 100644
@@ -318,6 +318,11 @@ int check_refname_format(const char *refname, int flags)
 	return check_or_sanitize_refname(refname, flags, NULL);
 }
 
+int refs_fsck(struct ref_store *refs, struct fsck_options *o)
+{
+	return refs->be->fsck(refs, o);
+}
+
 void sanitize_refname_component(const char *refname, struct strbuf *out)
 {
 	if (check_or_sanitize_refname(refname, REFNAME_ALLOW_ONELEVEL, out))
refs.h +8 -0
index b3e39bc257..405073621a 100644
@@ -4,6 +4,7 @@
 #include "commit.h"
 #include "repository.h"
 
+struct fsck_options;
 struct object_id;
 struct ref_store;
 struct strbuf;
@@ -541,6 +542,13 @@ int refs_for_each_reflog(struct ref_store *refs, each_reflog_fn fn, void *cb_dat
  */
 int check_refname_format(const char *refname, int flags);
 
+/*
+ * Check the reference database for consistency. Return 0 if refs and
+ * reflogs are consistent, and non-zero otherwise. The errors will be
+ * written to stderr.
+ */
+int refs_fsck(struct ref_store *refs, struct fsck_options *o);
+
 /*
  * Apply the rules from check_refname_format, but mutate the result until it
  * is acceptable, and place the result in "out".
refs/debug.c +11 -0
index 547d9245b9..45e2e784a0 100644
@@ -419,6 +419,15 @@ static int debug_reflog_expire(struct ref_store *ref_store, const char *refname,
 	return res;
 }
 
+static int debug_fsck(struct ref_store *ref_store,
+		      struct fsck_options *o)
+{
+	struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;
+	int res = drefs->refs->be->fsck(drefs->refs, o);
+	trace_printf_key(&trace_refs, "fsck: %d\n", res);
+	return res;
+}
+
 struct ref_storage_be refs_be_debug = {
 	.name = "debug",
 	.init = NULL,
@@ -451,4 +460,6 @@ struct ref_storage_be refs_be_debug = {
 	.create_reflog = debug_create_reflog,
 	.delete_reflog = debug_delete_reflog,
 	.reflog_expire = debug_reflog_expire,
+
+	.fsck = debug_fsck,
 };
refs/files-backend.c +12 -1
index aa52d9be7c..4630eb1f80 100644
@@ -3408,6 +3408,15 @@ static int files_ref_store_remove_on_disk(struct ref_store *ref_store,
 	return ret;
 }
 
+static int files_fsck(struct ref_store *ref_store,
+		      struct fsck_options *o)
+{
+	struct files_ref_store *refs =
+		files_downcast(ref_store, REF_STORE_READ, "fsck");
+
+	return refs->packed_ref_store->be->fsck(refs->packed_ref_store, o);
+}
+
 struct ref_storage_be refs_be_files = {
 	.name = "files",
 	.init = files_ref_store_init,
@@ -3434,5 +3443,7 @@ struct ref_storage_be refs_be_files = {
 	.reflog_exists = files_reflog_exists,
 	.create_reflog = files_create_reflog,
 	.delete_reflog = files_delete_reflog,
-	.reflog_expire = files_reflog_expire
+	.reflog_expire = files_reflog_expire,
+
+	.fsck = files_fsck,
 };
refs/packed-backend.c +8 -0
index a0666407cd..5209b0b212 100644
@@ -1735,6 +1735,12 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s
 	return empty_ref_iterator_begin();
 }
 
+static int packed_fsck(struct ref_store *ref_store,
+		       struct fsck_options *o)
+{
+	return 0;
+}
+
 struct ref_storage_be refs_be_packed = {
 	.name = "packed",
 	.init = packed_ref_store_init,
@@ -1762,4 +1768,6 @@ struct ref_storage_be refs_be_packed = {
 	.create_reflog = NULL,
 	.delete_reflog = NULL,
 	.reflog_expire = NULL,
+
+	.fsck = packed_fsck,
 };
refs/refs-internal.h +6 -0
index fa975d69aa..a905e187cd 100644
@@ -4,6 +4,7 @@
 #include "refs.h"
 #include "iterator.h"
 
+struct fsck_options;
 struct ref_transaction;
 
 /*
@@ -650,6 +651,9 @@ typedef int read_raw_ref_fn(struct ref_store *ref_store, const char *refname,
 typedef int read_symbolic_ref_fn(struct ref_store *ref_store, const char *refname,
 				 struct strbuf *referent);
 
+typedef int fsck_fn(struct ref_store *ref_store,
+		    struct fsck_options *o);
+
 struct ref_storage_be {
 	const char *name;
 	ref_store_init_fn *init;
@@ -677,6 +681,8 @@ struct ref_storage_be {
 	create_reflog_fn *create_reflog;
 	delete_reflog_fn *delete_reflog;
 	reflog_expire_fn *reflog_expire;
+
+	fsck_fn *fsck;
 };
 
 extern struct ref_storage_be refs_be_files;
refs/reftable-backend.c +8 -0
index fbe74c239d..b5a1a526df 100644
@@ -2303,6 +2303,12 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store,
 	return ret;
 }
 
+static int reftable_be_fsck(struct ref_store *ref_store,
+			    struct fsck_options *o)
+{
+	return 0;
+}
+
 struct ref_storage_be refs_be_reftable = {
 	.name = "reftable",
 	.init = reftable_be_init,
@@ -2330,4 +2336,6 @@ struct ref_storage_be refs_be_reftable = {
 	.create_reflog = reftable_be_create_reflog,
 	.delete_reflog = reftable_be_delete_reflog,
 	.reflog_expire = reftable_be_reflog_expire,
+
+	.fsck = reftable_be_fsck,
 };
Signature · 1 line
2.46.0
author
#275 lore raw

PATCH v16 7/9 builtin/refs: add verify subcommand

In reply to shejialuo

Introduce a new subcommand "verify" in git-refs(1) to allow the user to check the reference database consistency and also this subcommand will be used as the entry point of checking refs for "git-fsck(1)".

Add "verbose" field into "fsck_options" to indicate whether we should print verbose messages when checking refs and objects consistency.

Remove bit-field for "strict" field, this is because we cannot take address of a bit-field which makes it unhandy to set member variables when parsing the command line options.

The "git-fsck(1)" declares "fsck_options" variable with "static" identifier which avoids complaint by the leak-checker. However, in "git-refs verify", we need to do memory clean manually. Thus add "fsck_options_clear" function in "fsck.c" to provide memory clean operation.

---
 Documentation/git-refs.txt | 13 +++++++++++++
 builtin/refs.c             | 34 ++++++++++++++++++++++++++++++++++
 fsck.c                     | 11 +++++++++++
 fsck.h                     |  8 +++++++-
 4 files changed, 65 insertions(+), 1 deletion(-)
Documentation/git-refs.txt +13 -0
index 5b99e04385..ce31f93061 100644
@@ -10,6 +10,7 @@ SYNOPSIS
 --------
 [verse]
 'git refs migrate' --ref-format=<format> [--dry-run]
+'git refs verify' [--strict] [--verbose]
 
 DESCRIPTION
 -----------
@@ -22,6 +23,9 @@ COMMANDS
 migrate::
 	Migrate ref store between different formats.
 
+verify::
+	Verify reference database consistency.
+
 OPTIONS
 -------
 
@@ -39,6 +43,15 @@ include::ref-storage-format.txt[]
 	can be used to double check that the migration works as expected before
 	performing the actual migration.
 
+The following options are specific to 'git refs verify':
+
+--strict::
+	Enable stricter error checking. This will cause warnings to be
+	reported as errors. See linkgit:git-fsck[1].
+
+--verbose::
+	When verifying the reference database consistency, be chatty.
+
 KNOWN LIMITATIONS
 -----------------
 
builtin/refs.c +34 -0
index 46dcd150d4..131f98be98 100644
@@ -1,4 +1,6 @@
 #include "builtin.h"
+#include "config.h"
+#include "fsck.h"
 #include "parse-options.h"
 #include "refs.h"
 #include "repository.h"
@@ -7,6 +9,9 @@
 #define REFS_MIGRATE_USAGE \
 	N_("git refs migrate --ref-format=<format> [--dry-run]")
 
+#define REFS_VERIFY_USAGE \
+	N_("git refs verify [--strict] [--verbose]")
+
 static int cmd_refs_migrate(int argc, const char **argv, const char *prefix)
 {
 	const char * const migrate_usage[] = {
@@ -58,15 +63,44 @@ static int cmd_refs_migrate(int argc, const char **argv, const char *prefix)
 	return err;
 }
 
+static int cmd_refs_verify(int argc, const char **argv, const char *prefix)
+{
+	struct fsck_options fsck_refs_options = FSCK_REFS_OPTIONS_DEFAULT;
+	const char * const verify_usage[] = {
+		REFS_VERIFY_USAGE,
+		NULL,
+	};
+	struct option options[] = {
+		OPT_BOOL(0, "verbose", &fsck_refs_options.verbose, N_("be verbose")),
+		OPT_BOOL(0, "strict", &fsck_refs_options.strict, N_("enable strict checking")),
+		OPT_END(),
+	};
+	int ret;
+
+	argc = parse_options(argc, argv, prefix, options, verify_usage, 0);
+	if (argc)
+		usage(_("'git refs verify' takes no arguments"));
+
+	git_config(git_fsck_config, &fsck_refs_options);
+	prepare_repo_settings(the_repository);
+
+	ret = refs_fsck(get_main_ref_store(the_repository), &fsck_refs_options);
+
+	fsck_options_clear(&fsck_refs_options);
+	return ret;
+}
+
 int cmd_refs(int argc, const char **argv, const char *prefix)
 {
 	const char * const refs_usage[] = {
 		REFS_MIGRATE_USAGE,
+		REFS_VERIFY_USAGE,
 		NULL,
 	};
 	parse_opt_subcommand_fn *fn = NULL;
 	struct option opts[] = {
 		OPT_SUBCOMMAND("migrate", &fn, cmd_refs_migrate),
+		OPT_SUBCOMMAND("verify", &fn, cmd_refs_verify),
 		OPT_END(),
 	};
 
fsck.c +11 -0
index e16c892f6a..3756f52459 100644
@@ -1331,6 +1331,17 @@ int fsck_finish(struct fsck_options *options)
 	return ret;
 }
 
+void fsck_options_clear(struct fsck_options *options)
+{
+	free(options->msg_type);
+	oidset_clear(&options->skip_oids);
+	oidset_clear(&options->gitmodules_found);
+	oidset_clear(&options->gitmodules_done);
+	oidset_clear(&options->gitattributes_found);
+	oidset_clear(&options->gitattributes_done);
+	kh_clear_oid_map(options->object_names);
+}
+
 int git_fsck_config(const char *var, const char *value,
 		    const struct config_context *ctx, void *cb)
 {
fsck.h +7 -1
index 2002590f60..d551a9fe86 100644
@@ -153,7 +153,8 @@ struct fsck_ref_report {
 struct fsck_options {
 	fsck_walk_func walk;
 	fsck_error error_func;
-	unsigned strict:1;
+	unsigned strict;
+	unsigned verbose;
 	enum fsck_msg_type *msg_type;
 	struct oidset skip_oids;
 	struct oidset gitmodules_found;
@@ -231,6 +232,11 @@ int fsck_tag_standalone(const struct object_id *oid, const char *buffer,
  */
 int fsck_finish(struct fsck_options *options);
 
+/*
+ * Clear the fsck_options struct, freeing any allocated memory.
+ */
+void fsck_options_clear(struct fsck_options *options);
+
 /*
  * Report an error or warning for refs.
  */
Signature · 1 line
2.46.0
#276 lore raw

PATCH 1/2 refs.c: remove redundant translation markers

From: Teng Long <[email protected]>

Quoted 13 lines · shejialuo
Quoted 13 lines
builtin/refs.c +2 -0
@@ -1,4 +1,6 @@
 #include "builtin.h"
+#include "config.h"
+#include "fsck.h"
 #include "parse-options.h"
 #include "refs.h"
 #include "repository.h"
@@ -7,6 +9,9 @@
 #define REFS_MIGRATE_USAGE \
 	N_("git refs migrate --ref-format=<format> [--dry-run]")
 
Quoted 2 lines

builtin/refs.c line 13

Quoted 2 lines
builtin/refs.c +2 -0
+#define REFS_VERIFY_USAGE \
+	N_("git refs verify [--strict] [--verbose]")

This define may not require translation, then maybe it should look like:

---
 builtin/refs.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
builtin/refs.c +1 -1
index 24978a7b..3a799aa6 100644
@@ -10,7 +10,7 @@
 	N_("git refs migrate --ref-format=<format> [--dry-run]")
 
 #define REFS_VERIFY_USAGE \
-	N_("git refs verify [--strict] [--verbose]")
+	"git refs verify [--strict] [--verbose]"
 
 static int cmd_refs_migrate(int argc, const char **argv, const char *prefix)
 {
Signature · 4 lines
2.43.0.rc1.4.g79da3975.dirty


Thanks.
author
#277 lore raw

Re: [PATCH 1/2] refs.c: remove redundant translation markers

Quoted 1 line · Teng Long

From: Teng Long <[email protected]>

[snip]

Quoted 20 lines
Quoted 6 lines

Review line 13

Quoted 6 lines
(quoted diff) +2 -0
@@ -7,6 +9,9 @@
 #define REFS_MIGRATE_USAGE \
 	N_("git refs migrate --ref-format=<format> [--dry-run]")
 
+#define REFS_VERIFY_USAGE \
+	N_("git refs verify [--strict] [--verbose]")

This define may not require translation, then maybe it should look like:

---
 builtin/refs.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
builtin/refs.c +0 -0
index 24978a7b..3a799aa6 100644

builtin/refs.c line 13

Quoted 5 lines
builtin/refs.c +1 -1
@@ -10,7 +10,7 @@
 	N_("git refs migrate --ref-format=<format> [--dry-run]")
 
 #define REFS_VERIFY_USAGE \
-	N_("git refs verify [--strict] [--verbose]")
+	"git refs verify [--strict] [--verbose]"

I am OK with this change, actually I just followed the way what Patrick does for "git refs migrate" subcommand. So at first glance, I wonder why the "git refs migrate" subcommand usage should not be changed.

After searching the mailing list for information, I found that Junio has explained why in

	https://lore.kernel.org/git/[email protected]/

Because --ref-format could be used as the placeholder, it will be much more helpful when translating in a different locale.

Thanks,
Jialuo

author
#278 lore raw

PATCH v16 8/9 files-backend: add unified interface for refs scanning

In reply to shejialuo

For refs and reflogs, we need to scan its corresponding directories to check every regular file or symbolic link which shares the same pattern. Introduce a unified interface for scanning directories for files-backend.

---
 Documentation/fsck-msgids.txt |  3 ++
 fsck.h                        |  1 +
 refs/files-backend.c          | 73 ++++++++++++++++++++++++++++++++++-
 3 files changed, 76 insertions(+), 1 deletion(-)
Documentation/fsck-msgids.txt +3 -0
index f643585a34..7c809fddf1 100644
@@ -19,6 +19,9 @@
 `badParentSha1`::
 	(ERROR) A commit object has a bad parent sha1.
 
+`badRefFiletype`::
+	(ERROR) A ref has a bad file type.
+
 `badTagName`::
 	(INFO) A tag has an invalid format.
 
fsck.h +1 -0
index d551a9fe86..af02174973 100644
@@ -31,6 +31,7 @@ enum fsck_msg_type {
 	FUNC(BAD_NAME, ERROR) \
 	FUNC(BAD_OBJECT_SHA1, ERROR) \
 	FUNC(BAD_PARENT_SHA1, ERROR) \
+	FUNC(BAD_REF_FILETYPE, ERROR) \
 	FUNC(BAD_TIMEZONE, ERROR) \
 	FUNC(BAD_TREE, ERROR) \
 	FUNC(BAD_TREE_SHA1, ERROR) \
refs/files-backend.c +72 -1

91 lines not on this page · raw message

Signature · 1 line
2.46.0
author
#279 lore raw

PATCH v16 9/9 fsck: add ref name check for files backend

In reply to shejialuo

The git-fsck(1) only implicitly checks the reference, it does not fully check refs with bad format name such as standalone "@".

However, a file ending with ".lock" should not be marked as having a bad ref name. It is expected that concurrent writers may have such lock files. We currently ignore this situation. But for bare ".lock" file, we will report it as error.

In order to provide such checks, add a new fsck message id "badRefName" with default ERROR type. Use existing "check_refname_format" to explicit check the ref name. And add a new unit test to verify the functionality.

---
 Documentation/fsck-msgids.txt |  3 ++
 fsck.h                        |  1 +
 refs/files-backend.c          | 31 ++++++++++++
 t/t0602-reffiles-fsck.sh      | 92 +++++++++++++++++++++++++++++++++++
 4 files changed, 127 insertions(+)
 create mode 100755 t/t0602-reffiles-fsck.sh
Documentation/fsck-msgids.txt +3 -0
index 7c809fddf1..68a2801f15 100644
@@ -22,6 +22,9 @@
 `badRefFiletype`::
 	(ERROR) A ref has a bad file type.
 
+`badRefName`::
+	(ERROR) A ref has an invalid format.
+
 `badTagName`::
 	(INFO) A tag has an invalid format.
 
fsck.h +1 -0
index af02174973..500b4c04d2 100644
@@ -32,6 +32,7 @@ enum fsck_msg_type {
 	FUNC(BAD_OBJECT_SHA1, ERROR) \
 	FUNC(BAD_PARENT_SHA1, ERROR) \
 	FUNC(BAD_REF_FILETYPE, ERROR) \
+	FUNC(BAD_REF_NAME, ERROR) \
 	FUNC(BAD_TIMEZONE, ERROR) \
 	FUNC(BAD_TREE, ERROR) \
 	FUNC(BAD_TREE_SHA1, ERROR) \
refs/files-backend.c +31 -0
index e511e1dcce..7f6eefa960 100644
@@ -3419,6 +3419,36 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,
 				  const char *refs_check_dir,
 				  struct dir_iterator *iter);
 
+static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,
+				struct fsck_options *o,
+				const char *refs_check_dir,
+				struct dir_iterator *iter)
+{
+	struct strbuf sb = STRBUF_INIT;
+	int ret = 0;
+
+	/*
+	 * Ignore the files ending with ".lock" as they may be lock files
+	 * However, do not allow bare ".lock" files.
+	 */
+	if (iter->basename[0] != '.' && ends_with(iter->basename, ".lock"))
+		goto cleanup;
+
+	if (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {
+		struct fsck_ref_report report = { .path = NULL };
+
+		strbuf_addf(&sb, "%s/%s", refs_check_dir, iter->relative_path);
+		report.path = sb.buf;
+		ret = fsck_report_ref(o, &report,
+				      FSCK_MSG_BAD_REF_NAME,
+				      "invalid refname format");
+	}
+
+cleanup:
+	strbuf_release(&sb);
+	return ret;
+}
+
 static int files_fsck_refs_dir(struct ref_store *ref_store,
 			       struct fsck_options *o,
 			       const char *refs_check_dir,
@@ -3470,6 +3500,7 @@ static int files_fsck_refs(struct ref_store *ref_store,
 			   struct fsck_options *o)
 {
 	files_fsck_refs_fn fsck_refs_fn[]= {
+		files_fsck_refs_name,
 		NULL,
 	};
 
t/t0602-reffiles-fsck.sh new file +92 -0

92 lines not on this page · raw message

Signature · 1 line
2.46.0
#280 lore raw
In reply to shejialuo
Quoted 10 lines · shejialuo

Hi All, this version handles some minor changes:

1. Remove redundant newlines.
2. Fix typo in commit message.

At last, I wanna thank every reviewer. As we can see, this series starts from 5.27, it's a very long journey. I have learned a lot.

Thanks
Jialuo

The range diff looks as expected to me, so this series is ready to be merged from my point of view. Thanks!

Patrick

#281 lore raw
Quoted 9 lines · Patrick Steinhardt
Quoted 8 lines · shejialuo

Hi All, this version handles some minor changes:

1. Remove redundant newlines.
2. Fix typo in commit message.

At last, I wanna thank every reviewer. As we can see, this series starts from 5.27, it's a very long journey. I have learned a lot.

Quoted 7 lines
Quoted 2 lines

Thanks
Jialuo

The range diff looks as expected to me, so this series is ready to be merged from my point of view. Thanks!

Patrick

Yeah, looking good here, too.

Will replace. Let me mark this topic for 'next'.

Thanks.