Skip to contentSkip to conversation

repository: move fetch_if_missing into struct repository

PATCH Superseded Tian Yuchen 3 participants 26 messages Jul 15 – Aug 16 Documentation

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 repository: move fetch_if_missing into struct repository

The global variable 'fetch_if_missing' controls whether a missing object check should prompt a lazy fetch from a promisor remote. In order to continue the libification effort, move it into 'struct repository' and initialize it to 1 by default to keep the previous behavior.

Subsystems that already pass around a repository pointer, are updated to read this flag directly from their respective 'repo' instances. For the rest, we access 'the_repository'.

Note that in builtin/fsck.c and builtin/index-pack.c, when running related commands with the '-h' parameter, the 'repo' pointer is not passed in. To prevent null pointer dereferences, we defer operations on the repo in until after parameter parsing is complete.

Additionally, update the partial clone documentation to reflect that this is now a per-repository flag.

---
 Documentation/technical/partial-clone.adoc |  2 +-
 builtin/fetch-pack.c                       |  2 +-
 builtin/fsck.c                             |  6 +++---
 builtin/index-pack.c                       |  9 +++++----
 builtin/pack-objects.c                     | 14 +++++++-------
 builtin/prune.c                            |  2 +-
 builtin/rev-list.c                         | 10 +++++-----
 git.c                                      |  2 +-
 midx-write.c                               |  2 +-
 odb.c                                      |  4 +---
 odb.h                                      |  8 --------
 repository.c                               |  1 +
 repository.h                               |  6 ++++++
 revision.c                                 |  2 +-
 setup.c                                    |  2 +-
 15 files changed, 35 insertions(+), 37 deletions(-)
Documentation/technical/partial-clone.adoc +1 -1
index e513e391ea..18718a3840 100644
@@ -159,7 +159,7 @@ and prefetch those objects in bulk.
 - `repack` in GC has been updated to not touch promisor packfiles at all,
   and to only repack other objects.
 
-- The global variable "fetch_if_missing" is used to control whether an
+- The per-repository flag "fetch_if_missing" is used to control whether an
   object lookup will attempt to dynamically fetch a missing object or
   report an error.
 +
builtin/fetch-pack.c +1 -1
index 316badd969..c5edd7b80f 100644
@@ -67,7 +67,7 @@ int cmd_fetch_pack(int argc,
 	struct packet_reader reader;
 	enum protocol_version version;
 
-	fetch_if_missing = 0;
+	the_repository->fetch_if_missing = 0;
 
 	packet_trace_identity("fetch-pack");
 
builtin/fsck.c +3 -3
index 248f8ff5a0..aa31c69486 100644
@@ -1017,15 +1017,15 @@ int cmd_fsck(int argc,
 		.ref = NULL
 	};
 
-	/* fsck knows how to handle missing promisor objects */
-	fetch_if_missing = 0;
-
 	errors_found = 0;
 	disable_replace_refs();
 	save_commit_buffer = 0;
 
 	argc = parse_options(argc, argv, prefix, fsck_opts, fsck_usage, 0);
 
+	/* fsck knows how to handle missing promisor objects */
+	repo->fetch_if_missing = 0;
+
 	fsck_options_init(&fsck_walk_options, repo, FSCK_OPTIONS_DEFAULT);
 	fsck_walk_options.walk = mark_object;
 
builtin/index-pack.c +5 -4
index 0793dc595c..721d576938 100644
@@ -1881,7 +1881,7 @@ static void repack_local_links(void)
 int cmd_index_pack(int argc,
 		   const char **argv,
 		   const char *prefix,
-		   struct repository *repo UNUSED)
+		   struct repository *repo)
 {
 	int i, fix_thin_pack = 0, verify = 0, stat_only = 0, rev_index;
 	const char *curr_index;
@@ -1898,15 +1898,16 @@ int cmd_index_pack(int argc,
 	int report_end_of_input = 0;
 	int hash_algo = 0;
 
+	show_usage_if_asked(argc, argv, index_pack_usage);
+
 	/*
 	 * index-pack never needs to fetch missing objects except when
 	 * REF_DELTA bases are missing (which are explicitly handled). It only
 	 * accesses the repo to do hash collision checks and to check which
 	 * REF_DELTA bases need to be fetched.
 	 */
-	fetch_if_missing = 0;
-
-	show_usage_if_asked(argc, argv, index_pack_usage);
+	if (repo)
+		repo->fetch_if_missing = 0;
 
 	disable_replace_refs();
 
builtin/pack-objects.c +7 -7
index 8a1709a1ab..c6536b1f65 100644
@@ -4059,7 +4059,7 @@ static void add_unreachable_loose_objects(struct rev_info *revs);
 
 static void read_stdin_packs(enum stdin_packs_mode mode, int rev_list_unpacked)
 {
-	int prev_fetch_if_missing = fetch_if_missing;
+	int prev_fetch_if_missing = the_repository->fetch_if_missing;
 	struct rev_info revs;
 
 	/*
@@ -4067,7 +4067,7 @@ static void read_stdin_packs(enum stdin_packs_mode mode, int rev_list_unpacked)
 	 * walk is best-effort though we don't want to perform backfill fetches
 	 * for them.
 	 */
-	fetch_if_missing = 0;
+	the_repository->fetch_if_missing = 0;
 
 	repo_init_revisions(the_repository, &revs, NULL);
 	/*
@@ -4115,7 +4115,7 @@ static void read_stdin_packs(enum stdin_packs_mode mode, int rev_list_unpacked)
 	trace2_data_intmax("pack-objects", the_repository, "stdin_packs_hints",
 			   stdin_packs_hints_nr);
 
-	fetch_if_missing = prev_fetch_if_missing;
+	the_repository->fetch_if_missing = prev_fetch_if_missing;
 }
 
 static void add_cruft_object_entry(const struct object_id *oid, enum object_type type,
@@ -4451,14 +4451,14 @@ static int option_parse_missing_action(const struct option *opt UNUSED,
 
 	if (!strcmp(arg, "allow-any")) {
 		arg_missing_action = MA_ALLOW_ANY;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 		fn_show_object = show_object__ma_allow_any;
 		return 0;
 	}
 
 	if (!strcmp(arg, "allow-promisor")) {
 		arg_missing_action = MA_ALLOW_PROMISOR;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 		fn_show_object = show_object__ma_allow_promisor;
 		return 0;
 	}
@@ -5247,7 +5247,7 @@ int cmd_pack_objects(int argc,
 				  exclude_promisor_objects_best_effort,
 				  "--exclude-promisor-objects-best-effort");
 	if (exclude_promisor_objects) {
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 
 		/* --stdin-packs handles promisor objects separately. */
 		if (!stdin_packs) {
@@ -5256,7 +5256,7 @@ int cmd_pack_objects(int argc,
 		}
 	} else if (exclude_promisor_objects_best_effort) {
 		use_internal_rev_list = 1;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 		option_parse_missing_action(NULL, "allow-any", 0);
 		/* revs configured below */
 	}
builtin/prune.c +1 -1
index 55635a891f..a7e4678d11 100644
@@ -194,7 +194,7 @@ int cmd_prune(int argc,
 	if (show_progress == -1)
 		show_progress = isatty(2);
 	if (exclude_promisor_objects) {
-		fetch_if_missing = 0;
+		repo->fetch_if_missing = 0;
 		revs.exclude_promisor_objects = 1;
 	}
 
builtin/rev-list.c +5 -5
index 8f63003709..a6a0c5559e 100644
@@ -509,25 +509,25 @@ static inline int parse_missing_action_value(const char *value)
 
 	if (!strcmp(value, "allow-any")) {
 		arg_missing_action = MA_ALLOW_ANY;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 		return 1;
 	}
 
 	if (!strcmp(value, "print")) {
 		arg_missing_action = MA_PRINT;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 		return 1;
 	}
 
 	if (!strcmp(value, "print-info")) {
 		arg_missing_action = MA_PRINT_INFO;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 		return 1;
 	}
 
 	if (!strcmp(value, "allow-promisor")) {
 		arg_missing_action = MA_ALLOW_PROMISOR;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 		return 1;
 	}
 
@@ -745,7 +745,7 @@ int cmd_rev_list(int argc,
 	for (i = 1; i < argc; i++) {
 		const char *arg = argv[i];
 		if (!strcmp(arg, "--exclude-promisor-objects")) {
-			fetch_if_missing = 0;
+			the_repository->fetch_if_missing = 0;
 			revs.exclude_promisor_objects = 1;
 		} else if (skip_prefix(arg, "--missing=", &arg)) {
 			parse_missing_action_value(arg);
git.c +1 -1
index 36f08891ef..315d2e160e 100644
@@ -202,7 +202,7 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 			if (envchanged)
 				*envchanged = 1;
 		} else if (!strcmp(cmd, "--no-lazy-fetch")) {
-			fetch_if_missing = 0;
+			the_repository->fetch_if_missing = 0;
 			setenv(NO_LAZY_FETCH_ENVIRONMENT, "1", 1);
 			if (envchanged)
 				*envchanged = 1;
midx-write.c +1 -1
index 19e1cd10b7..e7313c9d2c 100644
@@ -865,7 +865,7 @@ static void find_commits_for_midx_bitmap(struct commit_stack *commits,
 	 * complain later that we don't have reachability closure (and fail
 	 * appropriately).
 	 */
-	fetch_if_missing = 0;
+	ctx->repo->fetch_if_missing = 0;
 	revs.exclude_promisor_objects = 1;
 
 	if (prepare_revision_walk(&revs))
odb.c +1 -3
index 965ef68e4e..664256e1a4 100644
@@ -528,8 +528,6 @@ void disable_obj_read_lock(void)
 	pthread_mutex_destroy(&obj_read_mutex);
 }
 
-int fetch_if_missing = 1;
-
 static int register_all_submodule_sources(struct object_database *odb)
 {
 	int ret = odb->submodule_source_paths.nr;
@@ -595,7 +593,7 @@ static int do_oid_object_info_extended(struct object_database *odb,
 			continue;
 
 		/* Check if it is a missing object */
-		if (fetch_if_missing && repo_has_promisor_remote(odb->repo) &&
+		if (odb->repo->fetch_if_missing && repo_has_promisor_remote(odb->repo) &&
 		    !already_retried &&
 		    !(flags & OBJECT_INFO_SKIP_FETCH_OBJECT)) {
 			promisor_remote_get_direct(odb->repo, real, 1);
odb.h +0 -8
index 0030467a52..1dca583fcb 100644
@@ -14,14 +14,6 @@ struct repository;
 struct strbuf;
 struct strvec;
 
-/*
- * Set this to 0 to prevent odb_read_object_info_extended() from fetching missing
- * blobs. This has a difference only if extensions.partialClone is set.
- *
- * Its default value is 1.
- */
-extern int fetch_if_missing;
-
 /*
  * Compute the exact path an alternate is at and returns it. In case of
  * error NULL is returned and the human readable error is added to `err`
repository.c +1 -0
index 187dd471c4..b959f7a028 100644
@@ -73,6 +73,7 @@ void initialize_repository(struct repository *repo)
 	ALLOC_ARRAY(repo->index, 1);
 	index_state_init(repo->index, repo);
 	repo->check_deprecated_config = true;
+	repo->fetch_if_missing = 1;
 	repo_config_values_init(&repo->config_values_private_);
 
 	/*
repository.h +6 -0
index 36e2db2633..e8bd6ef0e7 100644
@@ -169,6 +169,12 @@ struct repository {
 	/* True if commit-graph has been disabled within this process. */
 	int commit_graph_disabled;
 
+	/*
+	 * Controls whether the repository should lazily fetch missing
+	 * objects from promisor remotes. Defaults to 1.
+	 */
+	int fetch_if_missing;
+
 	/*
 	 * Lazily-populated cache mapping hook event names to configured hooks.
 	 * NULL until first hook use.
revision.c +1 -1
index e91d7e1f11..bb645654c3 100644
@@ -2714,7 +2714,7 @@ static int handle_revision_opt(struct rev_info *revs, int argc, const char **arg
 		revs->ignore_missing = 1;
 	} else if (opt && opt->allow_exclude_promisor_objects &&
 		   !strcmp(arg, "--exclude-promisor-objects")) {
-		if (fetch_if_missing)
+		if (revs->repo->fetch_if_missing)
 			BUG("exclude_promisor_objects can only be used when fetch_if_missing is 0");
 		revs->exclude_promisor_objects = 1;
 	} else {
setup.c +1 -1
index b4652651df..ce2a80ac31 100644
@@ -1064,7 +1064,7 @@ static void setup_git_env_internal(struct repository *repo,
 		set_alternate_shallow_file(repo, shallow_file, 0);
 
 	if (git_env_bool(NO_LAZY_FETCH_ENVIRONMENT, 0))
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 }
 
 static void set_git_dir_1(struct repository *repo, const char *path)
Signature · 1 line
2.43.0
#1 lore raw
Quoted 11 lines · Tian Yuchen

The global variable 'fetch_if_missing' controls whether a missing object check should prompt a lazy fetch from a promisor remote. In order to continue the libification effort, move it into 'struct repository' and initialize it to 1 by default to keep the previous behavior.
...

setup.c +0 -0
index b4652651df..ce2a80ac31 100644

setup.c static void setup_git_env_internal(struct repository *repo, · line 1068

Quoted 6 lines
setup.c +1 -1
@@ -1064,7 +1064,7 @@ static void setup_git_env_internal(struct repository *repo,
 		set_alternate_shallow_file(repo, shallow_file, 0);
 
 	if (git_env_bool(NO_LAZY_FETCH_ENVIRONMENT, 0))
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 }

Could a caller pass a "repo" that is not the_repository? In other words, shouldn't this be

		repo->fetch_if_missing = 0;

instead?

#2 lore raw
Quoted 23 lines · Junio C Hamano
Quoted 11 lines · Tian Yuchen

The global variable 'fetch_if_missing' controls whether a missing object check should prompt a lazy fetch from a promisor remote. In order to continue the libification effort, move it into 'struct repository' and initialize it to 1 by default to keep the previous behavior.
...

setup.c +0 -0
index b4652651df..ce2a80ac31 100644

setup.c static void setup_git_env_internal(struct repository *repo, · line 1068

Quoted 6 lines
setup.c +1 -1
@@ -1064,7 +1064,7 @@ static void setup_git_env_internal(struct repository *repo,
  		set_alternate_shallow_file(repo, shallow_file, 0);
 
  	if (git_env_bool(NO_LAZY_FETCH_ENVIRONMENT, 0))
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
  }

Could a caller pass a "repo" that is not the_repository? In other words, shouldn't this be

Quoted 3 lines
		repo->fetch_if_missing = 0;

instead?

Thanks, will change in the next reroll!

Regards, yuchen

#3 lore raw
In reply to Tian Yuchen
Quoted 5 lines · Tian Yuchen

The global variable 'fetch_if_missing' controls whether a missing object check should prompt a lazy fetch from a promisor remote. In order to continue the libification effort, move it into 'struct repository' and initialize it to 1 by default to keep the previous behavior.

Right. I was also thinking about moving this into a non-global scope multiple times. I was approaching this a bit differently though: it's ultimately a property of the object database whether or not we want to accept missing objects, so I moved it in there instead.

I don't really think there's a downside with your version, though. Quite on the contrary: we can really only perform the backfill fetches with a whole repository at hand anyway. So conceptually your version might even be more sensible.

Quoted 8 lines

Subsystems that already pass around a repository pointer, are updated to read this flag directly from their respective 'repo' instances. For the rest, we access 'the_repository'.

Note that in builtin/fsck.c and builtin/index-pack.c, when running related commands with the '-h' parameter, the 'repo' pointer is not passed in. To prevent null pointer dereferences, we defer operations on the repo in until after parameter parsing is complete.

s/on the repo in/on the repo/

Quoted 14 lines
builtin/index-pack.c +2 -0
index 0793dc595c..721d576938 100644
@@ -1898,15 +1898,16 @@ int cmd_index_pack(int argc,
 	int report_end_of_input = 0;
 	int hash_algo = 0;
 
+	show_usage_if_asked(argc, argv, index_pack_usage);
+
 	/*
 	 * index-pack never needs to fetch missing objects except when
 	 * REF_DELTA bases are missing (which are explicitly handled). It only
 	 * accesses the repo to do hash collision checks and to check which

builtin/index-pack.c int cmd_index_pack(int argc, · line 1913

Quoted 10 lines
builtin/index-pack.c +2 -3
 	 * REF_DELTA bases need to be fetched.
 	 */
-	fetch_if_missing = 0;
-
-	show_usage_if_asked(argc, argv, index_pack_usage);
+	if (repo)
+		repo->fetch_if_missing = 0;
 
 	disable_replace_refs();
 

Okay. This command can run without a repository, in which case we'll end up just indexing the pack. My assumption is that we'll probably end up using the_repository if so, as we still use the_repository in this file. So could this here cause a change in behaviour?

If the answer is "maybe" I'd propose that we simply continue to use the_repository here.

revision.c static int handle_revision_opt(struct rev_info *revs, int argc, const char **arg · line 2720

Quoted 13 lines
revision.c +1 -1
index e91d7e1f11..bb645654c3 100644
@@ -2714,7 +2714,7 @@ static int handle_revision_opt(struct rev_info *revs, int argc, const char **arg
 		revs->ignore_missing = 1;
 	} else if (opt && opt->allow_exclude_promisor_objects &&
 		   !strcmp(arg, "--exclude-promisor-objects")) {
-		if (fetch_if_missing)
+		if (revs->repo->fetch_if_missing)
 			BUG("exclude_promisor_objects can only be used when fetch_if_missing is 0");
 		revs->exclude_promisor_objects = 1;
 	} else {

This one here also makes me wonder whether it could cause weird interactions in case a caller passes a repository other than the_repository. It ideally _shouldn't_, but it's hard to tell because we still use the_repository in lots of places here.

Thanks!

Patrick

#4 lore raw
Quoted 50 lines · Patrick Steinhardt
Quoted 5 lines · Tian Yuchen

The global variable 'fetch_if_missing' controls whether a missing object check should prompt a lazy fetch from a promisor remote. In order to continue the libification effort, move it into 'struct repository' and initialize it to 1 by default to keep the previous behavior.

Right. I was also thinking about moving this into a non-global scope multiple times. I was approaching this a bit differently though: it's ultimately a property of the object database whether or not we want to accept missing objects, so I moved it in there instead.

I don't really think there's a downside with your version, though. Quite on the contrary: we can really only perform the backfill fetches with a whole repository at hand anyway. So conceptually your version might even be more sensible.

Quoted 8 lines

Subsystems that already pass around a repository pointer, are updated to read this flag directly from their respective 'repo' instances. For the rest, we access 'the_repository'.

Note that in builtin/fsck.c and builtin/index-pack.c, when running related commands with the '-h' parameter, the 'repo' pointer is not passed in. To prevent null pointer dereferences, we defer operations on the repo in until after parameter parsing is complete.

s/on the repo in/on the repo/

Quoted 22 lines
builtin/index-pack.c +4 -3
index 0793dc595c..721d576938 100644
@@ -1898,15 +1898,16 @@ int cmd_index_pack(int argc,
  	int report_end_of_input = 0;
  	int hash_algo = 0;
 
+	show_usage_if_asked(argc, argv, index_pack_usage);
+
  	/*
  	 * index-pack never needs to fetch missing objects except when
  	 * REF_DELTA bases are missing (which are explicitly handled). It only
  	 * accesses the repo to do hash collision checks and to check which
  	 * REF_DELTA bases need to be fetched.
  	 */
-	fetch_if_missing = 0;
-
-	show_usage_if_asked(argc, argv, index_pack_usage);
+	if (repo)
+		repo->fetch_if_missing = 0;
 
Quoted 7 lines

builtin/index-pack.c int cmd_index_pack(int argc, · line 1913

Quoted 2 lines
builtin/index-pack.c +0 -0
  	disable_replace_refs();
 

Okay. This command can run without a repository, in which case we'll end up just indexing the pack. My assumption is that we'll probably end up using the_repository if so, as we still use the_repository in this file. So could this here cause a change in behaviour?

Hummm...

Quoted 14 lines

If the answer is "maybe" I'd propose that we simply continue to use the_repository here.

Quoted 10 lines
revision.c +1 -1
index e91d7e1f11..bb645654c3 100644
@@ -2714,7 +2714,7 @@ static int handle_revision_opt(struct rev_info *revs, int argc, const char **arg
  		revs->ignore_missing = 1;
  	} else if (opt && opt->allow_exclude_promisor_objects &&
  		   !strcmp(arg, "--exclude-promisor-objects")) {
-		if (fetch_if_missing)
+		if (revs->repo->fetch_if_missing)
Quoted 9 lines

revision.c static int handle_revision_opt(struct rev_info *revs, int argc, const char **arg · line 2720

Quoted 3 lines
revision.c +0 -0
  			BUG("exclude_promisor_objects can only be used when fetch_if_missing is 0");
  		revs->exclude_promisor_objects = 1;
  	} else {

This one here also makes me wonder whether it could cause weird interactions in case a caller passes a repository other than the_repository. It ideally _shouldn't_, but it's hard to tell because we still use the_repository in lots of places here.

This makes sense to me. Let's use the_repository then.

Quoted 3 lines

Thanks!

Patrick

Regards, yuchen

#5 lore raw
In reply to Patrick Steinhardt
Quoted 12 lines · Patrick Steinhardt
Quoted 12 lines
builtin/index-pack.c +2 -3
 	/*
 	 * index-pack never needs to fetch missing objects except when
 	 * REF_DELTA bases are missing (which are explicitly handled). It only
 	 * accesses the repo to do hash collision checks and to check which
 	 * REF_DELTA bases need to be fetched.
 	 */
-	fetch_if_missing = 0;
-
-	show_usage_if_asked(argc, argv, index_pack_usage);
+	if (repo)
+		repo->fetch_if_missing = 0;
Quoted 7 lines
Quoted 2 lines

disable_replace_refs();

Okay. This command can run without a repository, in which case we'll end up just indexing the pack. My assumption is that we'll probably end up using the_repository if so, as we still use the_repository in this file. So could this here cause a change in behaviour?

Meaning that even outside a repository, we could have read the setting from ~/.gitconfig or some other places other than the per-repository .git/config file?

#6 lore raw

PATCH v2 1/1 repository: move fetch_if_missing into struct repository

In reply to Tian Yuchen

The global variable 'fetch_if_missing' controls whether a missing object check should prompt a lazy fetch from a promisor remote. In order to continue the libification effort, move it into 'struct repository' and initialize it to 1 by default to keep the previous behavior.

Note that in builtin/fsck.c and builtin/index-pack.c, when running related commands with the '-h' parameter, the 'repo' pointer is not passed in. To prevent null pointer dereferences, we defer operations on the repo until after parameter parsing is complete.

Additionally, update the partial clone documentation to reflect that this is now a per-repository flag.

---

Change since V1:

- Following Patrick's advice, use the_repository whenever possible without re-introducing #define USE_THE_REPOSITORY_VARIABLE.

 Documentation/technical/partial-clone.adoc |  2 +-
 builtin/fetch-pack.c                       |  2 +-
 builtin/fsck.c                             |  6 +++---
 builtin/index-pack.c                       |  7 ++++---
 builtin/pack-objects.c                     | 14 +++++++-------
 builtin/prune.c                            |  2 +-
 builtin/rev-list.c                         | 10 +++++-----
 git.c                                      |  2 +-
 midx-write.c                               |  2 +-
 odb.c                                      |  4 +---
 odb.h                                      |  8 --------
 repository.c                               |  1 +
 repository.h                               |  6 ++++++
 revision.c                                 |  2 +-
 setup.c                                    |  2 +-
 15 files changed, 34 insertions(+), 36 deletions(-)
Documentation/technical/partial-clone.adoc +1 -1
index e513e391ea..18718a3840 100644
@@ -159,7 +159,7 @@ and prefetch those objects in bulk.
 - `repack` in GC has been updated to not touch promisor packfiles at all,
   and to only repack other objects.
 
-- The global variable "fetch_if_missing" is used to control whether an
+- The per-repository flag "fetch_if_missing" is used to control whether an
   object lookup will attempt to dynamically fetch a missing object or
   report an error.
 +
builtin/fetch-pack.c +1 -1
index 316badd969..c5edd7b80f 100644
@@ -67,7 +67,7 @@ int cmd_fetch_pack(int argc,
 	struct packet_reader reader;
 	enum protocol_version version;
 
-	fetch_if_missing = 0;
+	the_repository->fetch_if_missing = 0;
 
 	packet_trace_identity("fetch-pack");
 
builtin/fsck.c +3 -3
index 248f8ff5a0..aa31c69486 100644
@@ -1017,15 +1017,15 @@ int cmd_fsck(int argc,
 		.ref = NULL
 	};
 
-	/* fsck knows how to handle missing promisor objects */
-	fetch_if_missing = 0;
-
 	errors_found = 0;
 	disable_replace_refs();
 	save_commit_buffer = 0;
 
 	argc = parse_options(argc, argv, prefix, fsck_opts, fsck_usage, 0);
 
+	/* fsck knows how to handle missing promisor objects */
+	repo->fetch_if_missing = 0;
+
 	fsck_options_init(&fsck_walk_options, repo, FSCK_OPTIONS_DEFAULT);
 	fsck_walk_options.walk = mark_object;
 
builtin/index-pack.c +4 -3
index 0793dc595c..74f9694662 100644
@@ -1898,15 +1898,16 @@ int cmd_index_pack(int argc,
 	int report_end_of_input = 0;
 	int hash_algo = 0;
 
+	show_usage_if_asked(argc, argv, index_pack_usage);
+
 	/*
 	 * index-pack never needs to fetch missing objects except when
 	 * REF_DELTA bases are missing (which are explicitly handled). It only
 	 * accesses the repo to do hash collision checks and to check which
 	 * REF_DELTA bases need to be fetched.
 	 */
-	fetch_if_missing = 0;
-
-	show_usage_if_asked(argc, argv, index_pack_usage);
+	if (repo)
+		the_repository->fetch_if_missing = 0;
 
 	disable_replace_refs();
 
builtin/pack-objects.c +7 -7
index 8a1709a1ab..c6536b1f65 100644
@@ -4059,7 +4059,7 @@ static void add_unreachable_loose_objects(struct rev_info *revs);
 
 static void read_stdin_packs(enum stdin_packs_mode mode, int rev_list_unpacked)
 {
-	int prev_fetch_if_missing = fetch_if_missing;
+	int prev_fetch_if_missing = the_repository->fetch_if_missing;
 	struct rev_info revs;
 
 	/*
@@ -4067,7 +4067,7 @@ static void read_stdin_packs(enum stdin_packs_mode mode, int rev_list_unpacked)
 	 * walk is best-effort though we don't want to perform backfill fetches
 	 * for them.
 	 */
-	fetch_if_missing = 0;
+	the_repository->fetch_if_missing = 0;
 
 	repo_init_revisions(the_repository, &revs, NULL);
 	/*
@@ -4115,7 +4115,7 @@ static void read_stdin_packs(enum stdin_packs_mode mode, int rev_list_unpacked)
 	trace2_data_intmax("pack-objects", the_repository, "stdin_packs_hints",
 			   stdin_packs_hints_nr);
 
-	fetch_if_missing = prev_fetch_if_missing;
+	the_repository->fetch_if_missing = prev_fetch_if_missing;
 }
 
 static void add_cruft_object_entry(const struct object_id *oid, enum object_type type,
@@ -4451,14 +4451,14 @@ static int option_parse_missing_action(const struct option *opt UNUSED,
 
 	if (!strcmp(arg, "allow-any")) {
 		arg_missing_action = MA_ALLOW_ANY;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 		fn_show_object = show_object__ma_allow_any;
 		return 0;
 	}
 
 	if (!strcmp(arg, "allow-promisor")) {
 		arg_missing_action = MA_ALLOW_PROMISOR;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 		fn_show_object = show_object__ma_allow_promisor;
 		return 0;
 	}
@@ -5247,7 +5247,7 @@ int cmd_pack_objects(int argc,
 				  exclude_promisor_objects_best_effort,
 				  "--exclude-promisor-objects-best-effort");
 	if (exclude_promisor_objects) {
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 
 		/* --stdin-packs handles promisor objects separately. */
 		if (!stdin_packs) {
@@ -5256,7 +5256,7 @@ int cmd_pack_objects(int argc,
 		}
 	} else if (exclude_promisor_objects_best_effort) {
 		use_internal_rev_list = 1;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 		option_parse_missing_action(NULL, "allow-any", 0);
 		/* revs configured below */
 	}
builtin/prune.c +1 -1
index 55635a891f..a7e4678d11 100644
@@ -194,7 +194,7 @@ int cmd_prune(int argc,
 	if (show_progress == -1)
 		show_progress = isatty(2);
 	if (exclude_promisor_objects) {
-		fetch_if_missing = 0;
+		repo->fetch_if_missing = 0;
 		revs.exclude_promisor_objects = 1;
 	}
 
builtin/rev-list.c +5 -5
index 8f63003709..a6a0c5559e 100644
@@ -509,25 +509,25 @@ static inline int parse_missing_action_value(const char *value)
 
 	if (!strcmp(value, "allow-any")) {
 		arg_missing_action = MA_ALLOW_ANY;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 		return 1;
 	}
 
 	if (!strcmp(value, "print")) {
 		arg_missing_action = MA_PRINT;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 		return 1;
 	}
 
 	if (!strcmp(value, "print-info")) {
 		arg_missing_action = MA_PRINT_INFO;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 		return 1;
 	}
 
 	if (!strcmp(value, "allow-promisor")) {
 		arg_missing_action = MA_ALLOW_PROMISOR;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 		return 1;
 	}
 
@@ -745,7 +745,7 @@ int cmd_rev_list(int argc,
 	for (i = 1; i < argc; i++) {
 		const char *arg = argv[i];
 		if (!strcmp(arg, "--exclude-promisor-objects")) {
-			fetch_if_missing = 0;
+			the_repository->fetch_if_missing = 0;
 			revs.exclude_promisor_objects = 1;
 		} else if (skip_prefix(arg, "--missing=", &arg)) {
 			parse_missing_action_value(arg);
git.c +1 -1
index 36f08891ef..315d2e160e 100644
@@ -202,7 +202,7 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 			if (envchanged)
 				*envchanged = 1;
 		} else if (!strcmp(cmd, "--no-lazy-fetch")) {
-			fetch_if_missing = 0;
+			the_repository->fetch_if_missing = 0;
 			setenv(NO_LAZY_FETCH_ENVIRONMENT, "1", 1);
 			if (envchanged)
 				*envchanged = 1;
midx-write.c +1 -1
index 19e1cd10b7..e7313c9d2c 100644
@@ -865,7 +865,7 @@ static void find_commits_for_midx_bitmap(struct commit_stack *commits,
 	 * complain later that we don't have reachability closure (and fail
 	 * appropriately).
 	 */
-	fetch_if_missing = 0;
+	ctx->repo->fetch_if_missing = 0;
 	revs.exclude_promisor_objects = 1;
 
 	if (prepare_revision_walk(&revs))
odb.c +1 -3
index 965ef68e4e..664256e1a4 100644
@@ -528,8 +528,6 @@ void disable_obj_read_lock(void)
 	pthread_mutex_destroy(&obj_read_mutex);
 }
 
-int fetch_if_missing = 1;
-
 static int register_all_submodule_sources(struct object_database *odb)
 {
 	int ret = odb->submodule_source_paths.nr;
@@ -595,7 +593,7 @@ static int do_oid_object_info_extended(struct object_database *odb,
 			continue;
 
 		/* Check if it is a missing object */
-		if (fetch_if_missing && repo_has_promisor_remote(odb->repo) &&
+		if (odb->repo->fetch_if_missing && repo_has_promisor_remote(odb->repo) &&
 		    !already_retried &&
 		    !(flags & OBJECT_INFO_SKIP_FETCH_OBJECT)) {
 			promisor_remote_get_direct(odb->repo, real, 1);
odb.h +0 -8
index 0030467a52..1dca583fcb 100644
@@ -14,14 +14,6 @@ struct repository;
 struct strbuf;
 struct strvec;
 
-/*
- * Set this to 0 to prevent odb_read_object_info_extended() from fetching missing
- * blobs. This has a difference only if extensions.partialClone is set.
- *
- * Its default value is 1.
- */
-extern int fetch_if_missing;
-
 /*
  * Compute the exact path an alternate is at and returns it. In case of
  * error NULL is returned and the human readable error is added to `err`
repository.c +1 -0
index 187dd471c4..b959f7a028 100644
@@ -73,6 +73,7 @@ void initialize_repository(struct repository *repo)
 	ALLOC_ARRAY(repo->index, 1);
 	index_state_init(repo->index, repo);
 	repo->check_deprecated_config = true;
+	repo->fetch_if_missing = 1;
 	repo_config_values_init(&repo->config_values_private_);
 
 	/*
repository.h +6 -0
index 36e2db2633..e8bd6ef0e7 100644
@@ -169,6 +169,12 @@ struct repository {
 	/* True if commit-graph has been disabled within this process. */
 	int commit_graph_disabled;
 
+	/*
+	 * Controls whether the repository should lazily fetch missing
+	 * objects from promisor remotes. Defaults to 1.
+	 */
+	int fetch_if_missing;
+
 	/*
 	 * Lazily-populated cache mapping hook event names to configured hooks.
 	 * NULL until first hook use.
revision.c +1 -1
index e91d7e1f11..5f70aa81e6 100644
@@ -2714,7 +2714,7 @@ static int handle_revision_opt(struct rev_info *revs, int argc, const char **arg
 		revs->ignore_missing = 1;
 	} else if (opt && opt->allow_exclude_promisor_objects &&
 		   !strcmp(arg, "--exclude-promisor-objects")) {
-		if (fetch_if_missing)
+		if (the_repository->fetch_if_missing)
 			BUG("exclude_promisor_objects can only be used when fetch_if_missing is 0");
 		revs->exclude_promisor_objects = 1;
 	} else {
setup.c +1 -1
index b4652651df..ce2a80ac31 100644
@@ -1064,7 +1064,7 @@ static void setup_git_env_internal(struct repository *repo,
 		set_alternate_shallow_file(repo, shallow_file, 0);
 
 	if (git_env_bool(NO_LAZY_FETCH_ENVIRONMENT, 0))
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 }
 
 static void set_git_dir_1(struct repository *repo, const char *path)
Signature · 1 line
2.43.0
#7 lore raw
Quoted 351 lines · Tian Yuchen

The global variable 'fetch_if_missing' controls whether a missing object check should prompt a lazy fetch from a promisor remote. In order to continue the libification effort, move it into 'struct repository' and initialize it to 1 by default to keep the previous behavior.

Note that in builtin/fsck.c and builtin/index-pack.c, when running related commands with the '-h' parameter, the 'repo' pointer is not passed in. To prevent null pointer dereferences, we defer operations on the repo until after parameter parsing is complete.

Additionally, update the partial clone documentation to reflect that this is now a per-repository flag.

---

Change since V1:

- Following Patrick's advice, use the_repository whenever possible without re-introducing #define USE_THE_REPOSITORY_VARIABLE.

  Documentation/technical/partial-clone.adoc |  2 +-
  builtin/fetch-pack.c                       |  2 +-
  builtin/fsck.c                             |  6 +++---
  builtin/index-pack.c                       |  7 ++++---
  builtin/pack-objects.c                     | 14 +++++++-------
  builtin/prune.c                            |  2 +-
  builtin/rev-list.c                         | 10 +++++-----
  git.c                                      |  2 +-
  midx-write.c                               |  2 +-
  odb.c                                      |  4 +---
  odb.h                                      |  8 --------
  repository.c                               |  1 +
  repository.h                               |  6 ++++++
  revision.c                                 |  2 +-
  setup.c                                    |  2 +-
  15 files changed, 34 insertions(+), 36 deletions(-)
Documentation/technical/partial-clone.adoc +1 -1
index e513e391ea..18718a3840 100644
@@ -159,7 +159,7 @@ and prefetch those objects in bulk.
  - `repack` in GC has been updated to not touch promisor packfiles at all,
    and to only repack other objects.
 
-- The global variable "fetch_if_missing" is used to control whether an
+- The per-repository flag "fetch_if_missing" is used to control whether an
    object lookup will attempt to dynamically fetch a missing object or
    report an error.
  +
builtin/fetch-pack.c +1 -1
index 316badd969..c5edd7b80f 100644
@@ -67,7 +67,7 @@ int cmd_fetch_pack(int argc,
  	struct packet_reader reader;
  	enum protocol_version version;
 
-	fetch_if_missing = 0;
+	the_repository->fetch_if_missing = 0;
 
  	packet_trace_identity("fetch-pack");
 
builtin/fsck.c +3 -3
index 248f8ff5a0..aa31c69486 100644
@@ -1017,15 +1017,15 @@ int cmd_fsck(int argc,
  		.ref = NULL
  	};
 
-	/* fsck knows how to handle missing promisor objects */
-	fetch_if_missing = 0;
-
  	errors_found = 0;
  	disable_replace_refs();
  	save_commit_buffer = 0;
 
  	argc = parse_options(argc, argv, prefix, fsck_opts, fsck_usage, 0);
 
+	/* fsck knows how to handle missing promisor objects */
+	repo->fetch_if_missing = 0;
+
  	fsck_options_init(&fsck_walk_options, repo, FSCK_OPTIONS_DEFAULT);
  	fsck_walk_options.walk = mark_object;
 
builtin/index-pack.c +4 -3
index 0793dc595c..74f9694662 100644
@@ -1898,15 +1898,16 @@ int cmd_index_pack(int argc,
  	int report_end_of_input = 0;
  	int hash_algo = 0;
 
+	show_usage_if_asked(argc, argv, index_pack_usage);
+
  	/*
  	 * index-pack never needs to fetch missing objects except when
  	 * REF_DELTA bases are missing (which are explicitly handled). It only
  	 * accesses the repo to do hash collision checks and to check which
  	 * REF_DELTA bases need to be fetched.
  	 */
-	fetch_if_missing = 0;
-
-	show_usage_if_asked(argc, argv, index_pack_usage);
+	if (repo)
+		the_repository->fetch_if_missing = 0;
 
  	disable_replace_refs();
 
builtin/pack-objects.c +7 -7
index 8a1709a1ab..c6536b1f65 100644
@@ -4059,7 +4059,7 @@ static void add_unreachable_loose_objects(struct rev_info *revs);
 
  static void read_stdin_packs(enum stdin_packs_mode mode, int rev_list_unpacked)
  {
-	int prev_fetch_if_missing = fetch_if_missing;
+	int prev_fetch_if_missing = the_repository->fetch_if_missing;
  	struct rev_info revs;
 
  	/*
@@ -4067,7 +4067,7 @@ static void read_stdin_packs(enum stdin_packs_mode mode, int rev_list_unpacked)
  	 * walk is best-effort though we don't want to perform backfill fetches
  	 * for them.
  	 */
-	fetch_if_missing = 0;
+	the_repository->fetch_if_missing = 0;
 
  	repo_init_revisions(the_repository, &revs, NULL);
  	/*
@@ -4115,7 +4115,7 @@ static void read_stdin_packs(enum stdin_packs_mode mode, int rev_list_unpacked)
  	trace2_data_intmax("pack-objects", the_repository, "stdin_packs_hints",
  			   stdin_packs_hints_nr);
 
-	fetch_if_missing = prev_fetch_if_missing;
+	the_repository->fetch_if_missing = prev_fetch_if_missing;
  }
 
  static void add_cruft_object_entry(const struct object_id *oid, enum object_type type,
@@ -4451,14 +4451,14 @@ static int option_parse_missing_action(const struct option *opt UNUSED,
 
  	if (!strcmp(arg, "allow-any")) {
  		arg_missing_action = MA_ALLOW_ANY;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
  		fn_show_object = show_object__ma_allow_any;
  		return 0;
  	}
 
  	if (!strcmp(arg, "allow-promisor")) {
  		arg_missing_action = MA_ALLOW_PROMISOR;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
  		fn_show_object = show_object__ma_allow_promisor;
  		return 0;
  	}
@@ -5247,7 +5247,7 @@ int cmd_pack_objects(int argc,
  				  exclude_promisor_objects_best_effort,
  				  "--exclude-promisor-objects-best-effort");
  	if (exclude_promisor_objects) {
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 
  		/* --stdin-packs handles promisor objects separately. */
  		if (!stdin_packs) {
@@ -5256,7 +5256,7 @@ int cmd_pack_objects(int argc,
  		}
  	} else if (exclude_promisor_objects_best_effort) {
  		use_internal_rev_list = 1;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
  		option_parse_missing_action(NULL, "allow-any", 0);
  		/* revs configured below */
  	}
builtin/prune.c +1 -1
index 55635a891f..a7e4678d11 100644
@@ -194,7 +194,7 @@ int cmd_prune(int argc,
  	if (show_progress == -1)
  		show_progress = isatty(2);
  	if (exclude_promisor_objects) {
-		fetch_if_missing = 0;
+		repo->fetch_if_missing = 0;
  		revs.exclude_promisor_objects = 1;
  	}
 
builtin/rev-list.c +5 -5
index 8f63003709..a6a0c5559e 100644
@@ -509,25 +509,25 @@ static inline int parse_missing_action_value(const char *value)
 
  	if (!strcmp(value, "allow-any")) {
  		arg_missing_action = MA_ALLOW_ANY;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
  		return 1;
  	}
 
  	if (!strcmp(value, "print")) {
  		arg_missing_action = MA_PRINT;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
  		return 1;
  	}
 
  	if (!strcmp(value, "print-info")) {
  		arg_missing_action = MA_PRINT_INFO;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
  		return 1;
  	}
 
  	if (!strcmp(value, "allow-promisor")) {
  		arg_missing_action = MA_ALLOW_PROMISOR;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
  		return 1;
  	}
 
@@ -745,7 +745,7 @@ int cmd_rev_list(int argc,
  	for (i = 1; i < argc; i++) {
  		const char *arg = argv[i];
  		if (!strcmp(arg, "--exclude-promisor-objects")) {
-			fetch_if_missing = 0;
+			the_repository->fetch_if_missing = 0;
  			revs.exclude_promisor_objects = 1;
  		} else if (skip_prefix(arg, "--missing=", &arg)) {
  			parse_missing_action_value(arg);
git.c +1 -1
index 36f08891ef..315d2e160e 100644
@@ -202,7 +202,7 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
  			if (envchanged)
  				*envchanged = 1;
  		} else if (!strcmp(cmd, "--no-lazy-fetch")) {
-			fetch_if_missing = 0;
+			the_repository->fetch_if_missing = 0;
  			setenv(NO_LAZY_FETCH_ENVIRONMENT, "1", 1);
  			if (envchanged)
  				*envchanged = 1;
midx-write.c +1 -1
index 19e1cd10b7..e7313c9d2c 100644
@@ -865,7 +865,7 @@ static void find_commits_for_midx_bitmap(struct commit_stack *commits,
  	 * complain later that we don't have reachability closure (and fail
  	 * appropriately).
  	 */
-	fetch_if_missing = 0;
+	ctx->repo->fetch_if_missing = 0;
  	revs.exclude_promisor_objects = 1;
 
  	if (prepare_revision_walk(&revs))
odb.c +1 -3
index 965ef68e4e..664256e1a4 100644
@@ -528,8 +528,6 @@ void disable_obj_read_lock(void)
  	pthread_mutex_destroy(&obj_read_mutex);
  }
 
-int fetch_if_missing = 1;
-
  static int register_all_submodule_sources(struct object_database *odb)
  {
  	int ret = odb->submodule_source_paths.nr;
@@ -595,7 +593,7 @@ static int do_oid_object_info_extended(struct object_database *odb,
  			continue;
 
  		/* Check if it is a missing object */
-		if (fetch_if_missing && repo_has_promisor_remote(odb->repo) &&
+		if (odb->repo->fetch_if_missing && repo_has_promisor_remote(odb->repo) &&
  		    !already_retried &&
  		    !(flags & OBJECT_INFO_SKIP_FETCH_OBJECT)) {
  			promisor_remote_get_direct(odb->repo, real, 1);
odb.h +0 -8
index 0030467a52..1dca583fcb 100644
@@ -14,14 +14,6 @@ struct repository;
  struct strbuf;
  struct strvec;
 
-/*
- * Set this to 0 to prevent odb_read_object_info_extended() from fetching missing
- * blobs. This has a difference only if extensions.partialClone is set.
- *
- * Its default value is 1.
- */
-extern int fetch_if_missing;
-
  /*
   * Compute the exact path an alternate is at and returns it. In case of
   * error NULL is returned and the human readable error is added to `err`
repository.c +1 -0
index 187dd471c4..b959f7a028 100644
@@ -73,6 +73,7 @@ void initialize_repository(struct repository *repo)
  	ALLOC_ARRAY(repo->index, 1);
  	index_state_init(repo->index, repo);
  	repo->check_deprecated_config = true;
+	repo->fetch_if_missing = 1;
  	repo_config_values_init(&repo->config_values_private_);
 
  	/*
repository.h +6 -0
index 36e2db2633..e8bd6ef0e7 100644
@@ -169,6 +169,12 @@ struct repository {
  	/* True if commit-graph has been disabled within this process. */
  	int commit_graph_disabled;
 
+	/*
+	 * Controls whether the repository should lazily fetch missing
+	 * objects from promisor remotes. Defaults to 1.
+	 */
+	int fetch_if_missing;
+
  	/*
  	 * Lazily-populated cache mapping hook event names to configured hooks.
  	 * NULL until first hook use.
revision.c +1 -1
index e91d7e1f11..5f70aa81e6 100644
@@ -2714,7 +2714,7 @@ static int handle_revision_opt(struct rev_info *revs, int argc, const char **arg
  		revs->ignore_missing = 1;
  	} else if (opt && opt->allow_exclude_promisor_objects &&
  		   !strcmp(arg, "--exclude-promisor-objects")) {
-		if (fetch_if_missing)
+		if (the_repository->fetch_if_missing)
  			BUG("exclude_promisor_objects can only be used when fetch_if_missing is 0");
  		revs->exclude_promisor_objects = 1;
  	} else {
setup.c +0 -0
index b4652651df..ce2a80ac31 100644

setup.c static void setup_git_env_internal(struct repository *repo, · line 1070

Quoted 8 lines
setup.c +1 -1
@@ -1064,7 +1064,7 @@ static void setup_git_env_internal(struct repository *repo,
  		set_alternate_shallow_file(repo, shallow_file, 0);
 
  	if (git_env_bool(NO_LAZY_FETCH_ENVIRONMENT, 0))
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
  }
 
  static void set_git_dir_1(struct repository *repo, const char *path)

What are everyone's thoughts on this patch? It seems like it's been a while since it received responses. ^^

Regards, yuchen

#8 lore raw
In reply to Tian Yuchen
Quoted 18 lines · Tian Yuchen

The global variable 'fetch_if_missing' controls whether a missing object check should prompt a lazy fetch from a promisor remote. In order to continue the libification effort, move it into 'struct repository' and initialize it to 1 by default to keep the previous behavior.

Note that in builtin/fsck.c and builtin/index-pack.c, when running related commands with the '-h' parameter, the 'repo' pointer is not passed in. To prevent null pointer dereferences, we defer operations on the repo until after parameter parsing is complete.

Additionally, update the partial clone documentation to reflect that this is now a per-repository flag.

Quoted 6 lines

---

Change since V1:

- Following Patrick's advice, use the_repository whenever possible without re-introducing #define USE_THE_REPOSITORY_VARIABLE.

It would be great to include the range-diff compared to the previous version so that it's easier for the reviewer to spot what's changed. Tools like b4 automate this for you :)

Quoted 14 lines
builtin/index-pack.c +2 -0
index 0793dc595c..74f9694662 100644
@@ -1898,15 +1898,16 @@ int cmd_index_pack(int argc,
 	int report_end_of_input = 0;
 	int hash_algo = 0;
 
+	show_usage_if_asked(argc, argv, index_pack_usage);
+
 	/*
 	 * index-pack never needs to fetch missing objects except when
 	 * REF_DELTA bases are missing (which are explicitly handled). It only
 	 * accesses the repo to do hash collision checks and to check which

builtin/index-pack.c int cmd_index_pack(int argc, · line 1913

Quoted 10 lines
builtin/index-pack.c +2 -3
 	 * REF_DELTA bases need to be fetched.
 	 */
-	fetch_if_missing = 0;
-
-	show_usage_if_asked(argc, argv, index_pack_usage);
+	if (repo)
+		the_repository->fetch_if_missing = 0;
 
 	disable_replace_refs();
 

This one looks a bit weird -- we check for repo, but then set the_repository->fetch_if_missing. We can probably just loose the conditional completely, and furthermore we don't need to reorder any code here at all anymore.

Other than that this patch looks good to me, thanks!

Patrick

#9 lore raw
Quoted 22 lines · Patrick Steinhardt
Quoted 22 lines
builtin/index-pack.c +4 -3
index 0793dc595c..74f9694662 100644
@@ -1898,15 +1898,16 @@ int cmd_index_pack(int argc,
 	int report_end_of_input = 0;
 	int hash_algo = 0;
 
+	show_usage_if_asked(argc, argv, index_pack_usage);
+
 	/*
 	 * index-pack never needs to fetch missing objects except when
 	 * REF_DELTA bases are missing (which are explicitly handled). It only
 	 * accesses the repo to do hash collision checks and to check which
 	 * REF_DELTA bases need to be fetched.
 	 */
-	fetch_if_missing = 0;
-
-	show_usage_if_asked(argc, argv, index_pack_usage);
+	if (repo)
+		the_repository->fetch_if_missing = 0;
 
Quoted 7 lines

builtin/index-pack.c int cmd_index_pack(int argc, · line 1913

Quoted 2 lines
builtin/index-pack.c +0 -0
 	disable_replace_refs();
 

This one looks a bit weird -- we check for repo, but then set the_repository->fetch_if_missing. We can probably just loose the conditional completely, and furthermore we don't need to reorder any code here at all anymore.

The 4-line comment is about disabling fetch-if-missing, so the code movement is not even unnecessary, but it is harmful, I think. If the command can work without repository, incoming "repo" might be NULL, and unconditionally doing

	repo->fetch_if_missing = 0;

may cause a crash. But that is not an excuse to blindly add

if (repo)

in front of such an assignment.

It gives you a chance to rethink what you are doing.

If a command can work without a repository, yet it cares about how fetch_if_missing bit is set, it hints that it may be a mistake in the first place to try associating fetch_if_missing bit with a particular struct repository instance, as you must be prepared to work with repo==NULL.

There could be at least three approaches you may have to think about at that point.

* Perhaps the command may not have to work outside a repository at all. If so, then it is a bug for the caller to call this function with repo==NULL. So we should just say

	repo->fetch_if_missing = 0;

without "if (repo)" check at all here. After all, the situation we might want to enable fetch_if_missing is where we have a place to fetch into, so by definition, we _should_ have a repository in such a case.

* Perhaps the command may want to work outside a repository but it may be acceptable to operate in a degraded way. By definition, when we are outside a repository, we have no object store to fetch objects lazily into, so fetch_if_missing MUST BE off.

Because Git is primarily about working inside a repository, perhaps it may be acceptable, even when you are outside a repository, to assume that the_repository can be used as a back-up "fake repository" object, and fetch_if_missing and its friends that are necessary to have their meaning to be in that fake repository object. If that the stance we are going to take, this part should probably say:

	(repo ? repo : the_repository)->fetch_if_missing = 0;

We need to make sure that everybody who passes the code paths that ever reference fetch_if_missing would pass the_repository down when the command is running outside a repository, though.

* Or perhaps there are some settings that really need to be available whether you are in a repository or not. I think fetch_if_missing is a borderline case, but more generally, things like user.name should conceptually be available even outside a repository, with in-repository configuration files overriding them. And it may be a mistake to force such settings to be stored in an instance of "struct repository" (or repo_settings that is part of it). We would need a framework to represent a structure in which a basic setting, which does not belong to any repository (whose members may be the same as those in "struct repo_settings", so I think it is OK to use an instance of that struct to represnt this "basic settings that is global"), exists globally, and it is overriden by per repository setting, which is in "struct repo_settings" embedded in "struct repository".

The earlier choices require fewer changes than the later choices, but the later choices are more concepturely pure, I think.

#10 lore raw
Quoted 37 lines · Junio C Hamano
Quoted 22 lines · Patrick Steinhardt
Quoted 22 lines
builtin/index-pack.c +4 -3
index 0793dc595c..74f9694662 100644
@@ -1898,15 +1898,16 @@ int cmd_index_pack(int argc,
  	int report_end_of_input = 0;
  	int hash_algo = 0;
 
+	show_usage_if_asked(argc, argv, index_pack_usage);
+
  	/*
  	 * index-pack never needs to fetch missing objects except when
  	 * REF_DELTA bases are missing (which are explicitly handled). It only
  	 * accesses the repo to do hash collision checks and to check which
  	 * REF_DELTA bases need to be fetched.
  	 */
-	fetch_if_missing = 0;
-
-	show_usage_if_asked(argc, argv, index_pack_usage);
+	if (repo)
+		the_repository->fetch_if_missing = 0;
 
Quoted 7 lines

builtin/index-pack.c int cmd_index_pack(int argc, · line 1913

Quoted 2 lines
builtin/index-pack.c +0 -0
  	disable_replace_refs();
 

This one looks a bit weird -- we check for repo, but then set the_repository->fetch_if_missing. We can probably just loose the conditional completely, and furthermore we don't need to reorder any code here at all anymore.

The 4-line comment is about disabling fetch-if-missing, so the code movement is not even unnecessary, but it is harmful, I think. If the command can work without repository, incoming "repo" might be NULL, and unconditionally doing

Quoted 7 lines
	repo->fetch_if_missing = 0;

may cause a crash. But that is not an excuse to blindly add

if (repo)

in front of such an assignment.

Yes, this is a mistake. :(

Quoted 16 lines

It gives you a chance to rethink what you are doing.

If a command can work without a repository, yet it cares about how fetch_if_missing bit is set, it hints that it may be a mistake in the first place to try associating fetch_if_missing bit with a particular struct repository instance, as you must be prepared to work with repo==NULL.

There could be at least three approaches you may have to think about at that point.

* Perhaps the command may not have to work outside a repository at all. If so, then it is a bug for the caller to call this function with repo==NULL. So we should just say

Quoted 7 lines

repo->fetch_if_missing = 0;

without "if (repo)" check at all here. After all, the situation we might want to enable fetch_if_missing is where we have a place to fetch into, so by definition, we _should_ have a repository in such a case.

Yep, this approach looks the most reasonable to me. I think I will apply this approach and update the commit message.

Quoted 19 lines

* Perhaps the command may want to work outside a repository but it may be acceptable to operate in a degraded way. By definition, when we are outside a repository, we have no object store to fetch objects lazily into, so fetch_if_missing MUST BE off.

Because Git is primarily about working inside a repository, perhaps it may be acceptable, even when you are outside a repository, to assume that the_repository can be used as a back-up "fake repository" object, and fetch_if_missing and its friends that are necessary to have their meaning to be in that fake repository object. If that the stance we are going to take, this part should probably say:

(repo ? repo : the_repository)->fetch_if_missing = 0;

We need to make sure that everybody who passes the code paths that ever reference fetch_if_missing would pass the_repository down when the command is running outside a repository, though.

Quoted 15 lines

* Or perhaps there are some settings that really need to be available whether you are in a repository or not. I think fetch_if_missing is a borderline case, but more generally, things like user.name should conceptually be available even outside a repository, with in-repository configuration files overriding them. And it may be a mistake to force such settings to be stored in an instance of "struct repository" (or repo_settings that is part of it). We would need a framework to represent a structure in which a basic setting, which does not belong to any repository (whose members may be the same as those in "struct repo_settings", so I think it is OK to use an instance of that struct to represnt this "basic settings that is global"), exists globally, and it is overriden by per repository setting, which is in "struct repo_settings" embedded in "struct repository".

I'm not sure if I'm right, but I feel that 'fetch_if_missing' essentially carries two meanings simultaneously: "Does this repository allow lazy fetching?" and "Does this object lookup allow network behavior to be triggered?" In other words, suppose we consider 'fetch_if_missing' to meet the third case, allowing it to exist in an ambiguous situation, i.e. it makes sense both within and outside the repository...Can we then consider the existence of this variable itself to have some semantic ambiguity? If so, do we really need to invent another mechanism to accommodate this ambiguity? Or should we temporarily apply minimal changes and discuss the ownership issue when the time is right (i.e. subsystems ready for 'repo' rather than 'the_repository' only)? Perhaps this is what you mean by "a borderline case".

Furthermore: when we do libification, is the goal to eliminate global state, or to "package global state more reasonably"? The third approach seems to be the latter one.

Quoted 3 lines

The earlier choices require fewer changes than the later choices, but the later choices are more concepturely pure, I think.

All in all, I think your core point is that we shouldn't blindly put all global variables in a struct repository. I totally agree with that.

Quoted 1 line

Thanks! yuchen

#11 lore raw
In reply to Patrick Steinhardt

Hi Patrick,

Quoted 21 lines · Patrick Steinhardt
Quoted 20 lines · Tian Yuchen

The global variable 'fetch_if_missing' controls whether a missing object check should prompt a lazy fetch from a promisor remote. In order to continue the libification effort, move it into 'struct repository' and initialize it to 1 by default to keep the previous behavior.

Note that in builtin/fsck.c and builtin/index-pack.c, when running related commands with the '-h' parameter, the 'repo' pointer is not passed in. To prevent null pointer dereferences, we defer operations on the repo until after parameter parsing is complete.

Additionally, update the partial clone documentation to reflect that this is now a per-repository flag.

---

Quoted 9 lines
Quoted 4 lines

Change since V1:

- Following Patrick's advice, use the_repository whenever possible without re-introducing #define USE_THE_REPOSITORY_VARIABLE.

It would be great to include the range-diff compared to the previous version so that it's easier for the reviewer to spot what's changed. Tools like b4 automate this for you :)

I see, I will include it in the next reroll!

Thanks! yuchen

#12 lore raw

PATCH v3 1/1 repository: move fetch_if_missing into struct repository

In reply to Tian Yuchen

The global variable 'fetch_if_missing' controls whether a missing object check should prompt a lazy fetch from a promisor remote. In order to continue the libification effort, move it into 'struct repository' and initialize it to 1 by default to keep the previous behavior.

Note that in builtin/fsck.c and builtin/index-pack.c, when running related commands with the '-h' parameter, the 'repo' pointer is not passed in. To prevent null pointer dereferences, we defer operations on the repo until after parameter parsing is complete.

Additionally, update the partial clone documentation to reflect that this is now a per-repository flag.

---
Range-diff since v2:

@@ builtin/fsck.c: int cmd_fsck(int argc,

## builtin/index-pack.c ##
+@@ builtin/index-pack.c: static void repack_local_links(void)
+ int cmd_index_pack(int argc,
+ const char **argv,
+ const char *prefix,
+- struct repository *repo UNUSED)
++ struct repository *repo)
+ {
+ int i, fix_thin_pack = 0, verify = 0, stat_only = 0, rev_index;
+ const char *curr_index;
@@ builtin/index-pack.c: int cmd_index_pack(int argc,
int report_end_of_input = 0;
int hash_algo = 0;
@@ builtin/index-pack.c: int cmd_index_pack(int argc,
- fetch_if_missing = 0;
-
- show_usage_if_asked(argc, argv, index_pack_usage);
-+ if (repo)
-+ the_repository->fetch_if_missing = 0;
++ (repo ? repo : the_repository)->fetch_if_missing = 0;

disable_replace_refs();

 Documentation/technical/partial-clone.adoc |  2 +-
 builtin/fetch-pack.c                       |  2 +-
 builtin/fsck.c                             |  6 +++---
 builtin/index-pack.c                       |  8 ++++----
 builtin/pack-objects.c                     | 14 +++++++-------
 builtin/prune.c                            |  2 +-
 builtin/rev-list.c                         | 10 +++++-----
 git.c                                      |  2 +-
 midx-write.c                               |  2 +-
 odb.c                                      |  4 +---
 odb.h                                      |  8 --------
 repository.c                               |  1 +
 repository.h                               |  6 ++++++
 revision.c                                 |  2 +-
 setup.c                                    |  2 +-
 15 files changed, 34 insertions(+), 37 deletions(-)
Documentation/technical/partial-clone.adoc +1 -1
index e513e391ea..18718a3840 100644
@@ -159,7 +159,7 @@ and prefetch those objects in bulk.
 - `repack` in GC has been updated to not touch promisor packfiles at all,
   and to only repack other objects.
 
-- The global variable "fetch_if_missing" is used to control whether an
+- The per-repository flag "fetch_if_missing" is used to control whether an
   object lookup will attempt to dynamically fetch a missing object or
   report an error.
 +
builtin/fetch-pack.c +1 -1
index 316badd969..c5edd7b80f 100644
@@ -67,7 +67,7 @@ int cmd_fetch_pack(int argc,
 	struct packet_reader reader;
 	enum protocol_version version;
 
-	fetch_if_missing = 0;
+	the_repository->fetch_if_missing = 0;
 
 	packet_trace_identity("fetch-pack");
 
builtin/fsck.c +3 -3
index 248f8ff5a0..aa31c69486 100644
@@ -1017,15 +1017,15 @@ int cmd_fsck(int argc,
 		.ref = NULL
 	};
 
-	/* fsck knows how to handle missing promisor objects */
-	fetch_if_missing = 0;
-
 	errors_found = 0;
 	disable_replace_refs();
 	save_commit_buffer = 0;
 
 	argc = parse_options(argc, argv, prefix, fsck_opts, fsck_usage, 0);
 
+	/* fsck knows how to handle missing promisor objects */
+	repo->fetch_if_missing = 0;
+
 	fsck_options_init(&fsck_walk_options, repo, FSCK_OPTIONS_DEFAULT);
 	fsck_walk_options.walk = mark_object;
 
builtin/index-pack.c +4 -4
index 0793dc595c..1be374806d 100644
@@ -1881,7 +1881,7 @@ static void repack_local_links(void)
 int cmd_index_pack(int argc,
 		   const char **argv,
 		   const char *prefix,
-		   struct repository *repo UNUSED)
+		   struct repository *repo)
 {
 	int i, fix_thin_pack = 0, verify = 0, stat_only = 0, rev_index;
 	const char *curr_index;
@@ -1898,15 +1898,15 @@ int cmd_index_pack(int argc,
 	int report_end_of_input = 0;
 	int hash_algo = 0;
 
+	show_usage_if_asked(argc, argv, index_pack_usage);
+
 	/*
 	 * index-pack never needs to fetch missing objects except when
 	 * REF_DELTA bases are missing (which are explicitly handled). It only
 	 * accesses the repo to do hash collision checks and to check which
 	 * REF_DELTA bases need to be fetched.
 	 */
-	fetch_if_missing = 0;
-
-	show_usage_if_asked(argc, argv, index_pack_usage);
+	(repo ? repo : the_repository)->fetch_if_missing = 0;
 
 	disable_replace_refs();
 
builtin/pack-objects.c +7 -7
index 8a1709a1ab..c6536b1f65 100644
@@ -4059,7 +4059,7 @@ static void add_unreachable_loose_objects(struct rev_info *revs);
 
 static void read_stdin_packs(enum stdin_packs_mode mode, int rev_list_unpacked)
 {
-	int prev_fetch_if_missing = fetch_if_missing;
+	int prev_fetch_if_missing = the_repository->fetch_if_missing;
 	struct rev_info revs;
 
 	/*
@@ -4067,7 +4067,7 @@ static void read_stdin_packs(enum stdin_packs_mode mode, int rev_list_unpacked)
 	 * walk is best-effort though we don't want to perform backfill fetches
 	 * for them.
 	 */
-	fetch_if_missing = 0;
+	the_repository->fetch_if_missing = 0;
 
 	repo_init_revisions(the_repository, &revs, NULL);
 	/*
@@ -4115,7 +4115,7 @@ static void read_stdin_packs(enum stdin_packs_mode mode, int rev_list_unpacked)
 	trace2_data_intmax("pack-objects", the_repository, "stdin_packs_hints",
 			   stdin_packs_hints_nr);
 
-	fetch_if_missing = prev_fetch_if_missing;
+	the_repository->fetch_if_missing = prev_fetch_if_missing;
 }
 
 static void add_cruft_object_entry(const struct object_id *oid, enum object_type type,
@@ -4451,14 +4451,14 @@ static int option_parse_missing_action(const struct option *opt UNUSED,
 
 	if (!strcmp(arg, "allow-any")) {
 		arg_missing_action = MA_ALLOW_ANY;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 		fn_show_object = show_object__ma_allow_any;
 		return 0;
 	}
 
 	if (!strcmp(arg, "allow-promisor")) {
 		arg_missing_action = MA_ALLOW_PROMISOR;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 		fn_show_object = show_object__ma_allow_promisor;
 		return 0;
 	}
@@ -5247,7 +5247,7 @@ int cmd_pack_objects(int argc,
 				  exclude_promisor_objects_best_effort,
 				  "--exclude-promisor-objects-best-effort");
 	if (exclude_promisor_objects) {
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 
 		/* --stdin-packs handles promisor objects separately. */
 		if (!stdin_packs) {
@@ -5256,7 +5256,7 @@ int cmd_pack_objects(int argc,
 		}
 	} else if (exclude_promisor_objects_best_effort) {
 		use_internal_rev_list = 1;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 		option_parse_missing_action(NULL, "allow-any", 0);
 		/* revs configured below */
 	}
builtin/prune.c +1 -1
index 55635a891f..a7e4678d11 100644
@@ -194,7 +194,7 @@ int cmd_prune(int argc,
 	if (show_progress == -1)
 		show_progress = isatty(2);
 	if (exclude_promisor_objects) {
-		fetch_if_missing = 0;
+		repo->fetch_if_missing = 0;
 		revs.exclude_promisor_objects = 1;
 	}
 
builtin/rev-list.c +5 -5
index 8f63003709..a6a0c5559e 100644
@@ -509,25 +509,25 @@ static inline int parse_missing_action_value(const char *value)
 
 	if (!strcmp(value, "allow-any")) {
 		arg_missing_action = MA_ALLOW_ANY;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 		return 1;
 	}
 
 	if (!strcmp(value, "print")) {
 		arg_missing_action = MA_PRINT;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 		return 1;
 	}
 
 	if (!strcmp(value, "print-info")) {
 		arg_missing_action = MA_PRINT_INFO;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 		return 1;
 	}
 
 	if (!strcmp(value, "allow-promisor")) {
 		arg_missing_action = MA_ALLOW_PROMISOR;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 		return 1;
 	}
 
@@ -745,7 +745,7 @@ int cmd_rev_list(int argc,
 	for (i = 1; i < argc; i++) {
 		const char *arg = argv[i];
 		if (!strcmp(arg, "--exclude-promisor-objects")) {
-			fetch_if_missing = 0;
+			the_repository->fetch_if_missing = 0;
 			revs.exclude_promisor_objects = 1;
 		} else if (skip_prefix(arg, "--missing=", &arg)) {
 			parse_missing_action_value(arg);
git.c +1 -1
index 36f08891ef..315d2e160e 100644
@@ -202,7 +202,7 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 			if (envchanged)
 				*envchanged = 1;
 		} else if (!strcmp(cmd, "--no-lazy-fetch")) {
-			fetch_if_missing = 0;
+			the_repository->fetch_if_missing = 0;
 			setenv(NO_LAZY_FETCH_ENVIRONMENT, "1", 1);
 			if (envchanged)
 				*envchanged = 1;
midx-write.c +1 -1
index 19e1cd10b7..e7313c9d2c 100644
@@ -865,7 +865,7 @@ static void find_commits_for_midx_bitmap(struct commit_stack *commits,
 	 * complain later that we don't have reachability closure (and fail
 	 * appropriately).
 	 */
-	fetch_if_missing = 0;
+	ctx->repo->fetch_if_missing = 0;
 	revs.exclude_promisor_objects = 1;
 
 	if (prepare_revision_walk(&revs))
odb.c +1 -3
index 965ef68e4e..664256e1a4 100644
@@ -528,8 +528,6 @@ void disable_obj_read_lock(void)
 	pthread_mutex_destroy(&obj_read_mutex);
 }
 
-int fetch_if_missing = 1;
-
 static int register_all_submodule_sources(struct object_database *odb)
 {
 	int ret = odb->submodule_source_paths.nr;
@@ -595,7 +593,7 @@ static int do_oid_object_info_extended(struct object_database *odb,
 			continue;
 
 		/* Check if it is a missing object */
-		if (fetch_if_missing && repo_has_promisor_remote(odb->repo) &&
+		if (odb->repo->fetch_if_missing && repo_has_promisor_remote(odb->repo) &&
 		    !already_retried &&
 		    !(flags & OBJECT_INFO_SKIP_FETCH_OBJECT)) {
 			promisor_remote_get_direct(odb->repo, real, 1);
odb.h +0 -8
index 0030467a52..1dca583fcb 100644
@@ -14,14 +14,6 @@ struct repository;
 struct strbuf;
 struct strvec;
 
-/*
- * Set this to 0 to prevent odb_read_object_info_extended() from fetching missing
- * blobs. This has a difference only if extensions.partialClone is set.
- *
- * Its default value is 1.
- */
-extern int fetch_if_missing;
-
 /*
  * Compute the exact path an alternate is at and returns it. In case of
  * error NULL is returned and the human readable error is added to `err`
repository.c +1 -0
index 187dd471c4..b959f7a028 100644
@@ -73,6 +73,7 @@ void initialize_repository(struct repository *repo)
 	ALLOC_ARRAY(repo->index, 1);
 	index_state_init(repo->index, repo);
 	repo->check_deprecated_config = true;
+	repo->fetch_if_missing = 1;
 	repo_config_values_init(&repo->config_values_private_);
 
 	/*
repository.h +6 -0
index 36e2db2633..e8bd6ef0e7 100644
@@ -169,6 +169,12 @@ struct repository {
 	/* True if commit-graph has been disabled within this process. */
 	int commit_graph_disabled;
 
+	/*
+	 * Controls whether the repository should lazily fetch missing
+	 * objects from promisor remotes. Defaults to 1.
+	 */
+	int fetch_if_missing;
+
 	/*
 	 * Lazily-populated cache mapping hook event names to configured hooks.
 	 * NULL until first hook use.
revision.c +1 -1
index e91d7e1f11..5f70aa81e6 100644
@@ -2714,7 +2714,7 @@ static int handle_revision_opt(struct rev_info *revs, int argc, const char **arg
 		revs->ignore_missing = 1;
 	} else if (opt && opt->allow_exclude_promisor_objects &&
 		   !strcmp(arg, "--exclude-promisor-objects")) {
-		if (fetch_if_missing)
+		if (the_repository->fetch_if_missing)
 			BUG("exclude_promisor_objects can only be used when fetch_if_missing is 0");
 		revs->exclude_promisor_objects = 1;
 	} else {
setup.c +1 -1
index b4652651df..ce2a80ac31 100644
@@ -1064,7 +1064,7 @@ static void setup_git_env_internal(struct repository *repo,
 		set_alternate_shallow_file(repo, shallow_file, 0);
 
 	if (git_env_bool(NO_LAZY_FETCH_ENVIRONMENT, 0))
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 }
 
 static void set_git_dir_1(struct repository *repo, const char *path)
Signature · 1 line
2.43.0
#13 lore raw
Quoted 11 lines · Tian Yuchen

The global variable 'fetch_if_missing' controls whether a missing object check should prompt a lazy fetch from a promisor remote. In order to continue the libification effort, move it into 'struct repository' and initialize it to 1 by default to keep the previous behavior.

Note that in builtin/fsck.c and builtin/index-pack.c, when running related commands with the '-h' parameter, the 'repo' pointer is not passed in. To prevent null pointer dereferences, we defer operations on the repo until after parameter parsing is complete.

Quoted 8 lines

Additionally, update the partial clone documentation to reflect that this is now a per-repository flag.

---

Have you made a trial merge to 'seen' and 'next' to see how well this change plays with other topics in flight?

setup.c static void setup_git_env_internal(struct repository *repo, · line 1068

Quoted 11 lines
setup.c +1 -1
index b4652651df..ce2a80ac31 100644
@@ -1064,7 +1064,7 @@ static void setup_git_env_internal(struct repository *repo,
 		set_alternate_shallow_file(repo, shallow_file, 0);
 
 	if (git_env_bool(NO_LAZY_FETCH_ENVIRONMENT, 0))
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 }

Pelease remind me the reason why we use the_repository here, even though a few lines above we are already using the caller-supplied repo.

Also setup_git_env_internal() no longer exists in 'seen', if this topic wants to play well with cf1687a41c (setup: split up concerns of setup_git_env_internal(), 2026-07-07), you'd need to coordinate the other topics and their author(s) before relying on the presence of what they are removing.

#14 lore raw
Quoted 26 lines · Junio C Hamano
Quoted 11 lines · Tian Yuchen

The global variable 'fetch_if_missing' controls whether a missing object check should prompt a lazy fetch from a promisor remote. In order to continue the libification effort, move it into 'struct repository' and initialize it to 1 by default to keep the previous behavior.

Note that in builtin/fsck.c and builtin/index-pack.c, when running related commands with the '-h' parameter, the 'repo' pointer is not passed in. To prevent null pointer dereferences, we defer operations on the repo until after parameter parsing is complete.

Quoted 8 lines

Additionally, update the partial clone documentation to reflect that this is now a per-repository flag.

---

Have you made a trial merge to 'seen' and 'next' to see how well this change plays with other topics in flight?

Quoted 5 lines

Also setup_git_env_internal() no longer exists in 'seen', if this topic wants to play well with cf1687a41c (setup: split up concerns of setup_git_env_internal(), 2026-07-07), you'd need to coordinate the other topics and their author(s) before relying on the presence of what they are removing.

Thanks for reminding. I did a trial merge and the dependency does exist. Since that topic is still in flight, I'd prefer to wait for it to settle before rerolling this patch. What do you think? ;-)

Quoted 7 lines
Quoted 7 lines
setup.c +0 -0
index b4652651df..ce2a80ac31 100644
@@ -1064,7 +1064,7 @@ static void setup_git_env_internal(struct repository *repo,
  		set_alternate_shallow_file(repo, shallow_file, 0);
 
Quoted 9 lines

setup.c static void setup_git_env_internal(struct repository *repo, · line 1068

Quoted 4 lines
setup.c +1 -1
  	if (git_env_bool(NO_LAZY_FETCH_ENVIRONMENT, 0))
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
  }

Pelease remind me the reason why we use the_repository here, even though a few lines above we are already using the caller-supplied repo.

Interestingly, in Patrick's patch, setup_environment() doesn't have a 'repo' passed in and is only called once by init_git(). So I think we'll have to use the_repository, though I never thought of this... Anyway, I'll pay more attention to it in the next reroll.

Regards, yuchen

#15 lore raw

PATCH v4 1/1 repository: move fetch_if_missing into struct repository

In reply to Tian Yuchen

The global variable 'fetch_if_missing' controls whether a missing object check should prompt a lazy fetch from a promisor remote. In order to continue the libification effort, move it into 'struct repository' and initialize it to 1 by default to keep the previous behavior.

Note that in builtin/fsck.c and builtin/index-pack.c, when running related commands with the '-h' parameter, the 'repo' pointer is not passed in. To prevent null pointer dereferences, we defer operations on the repo until after parameter parsing is complete.

Additionally, update the partial clone documentation to reflect that this is now a per-repository flag.

---

Changes since v3:

- Use revs->repo in revision.c instead of the_repository.

- Coordinate the other topics. Specifically, for common-init.c, use the_repository->fetch_if_missing in setup_environment(), etc. This patch currently does not conflict with seen or next.

 Documentation/technical/partial-clone.adoc |  2 +-
 builtin/fetch-pack.c                       |  2 +-
 builtin/fsck.c                             |  6 +++---
 builtin/index-pack.c                       |  8 ++++----
 builtin/pack-objects.c                     | 14 +++++++-------
 builtin/prune.c                            |  2 +-
 builtin/rev-list.c                         | 10 +++++-----
 common-init.c                              |  2 +-
 git.c                                      |  2 +-
 midx-write.c                               |  2 +-
 odb.c                                      |  4 +---
 odb.h                                      |  8 --------
 repository.c                               |  1 +
 repository.h                               |  6 ++++++
 revision.c                                 |  2 +-
 15 files changed, 34 insertions(+), 37 deletions(-)
Documentation/technical/partial-clone.adoc +1 -1
index e513e391ea..18718a3840 100644
@@ -159,7 +159,7 @@ and prefetch those objects in bulk.
 - `repack` in GC has been updated to not touch promisor packfiles at all,
   and to only repack other objects.
 
-- The global variable "fetch_if_missing" is used to control whether an
+- The per-repository flag "fetch_if_missing" is used to control whether an
   object lookup will attempt to dynamically fetch a missing object or
   report an error.
 +
builtin/fetch-pack.c +1 -1
index 316badd969..c5edd7b80f 100644
@@ -67,7 +67,7 @@ int cmd_fetch_pack(int argc,
 	struct packet_reader reader;
 	enum protocol_version version;
 
-	fetch_if_missing = 0;
+	the_repository->fetch_if_missing = 0;
 
 	packet_trace_identity("fetch-pack");
 
builtin/fsck.c +3 -3
index a6c054e45b..8cfc0e8b26 100644
@@ -1017,15 +1017,15 @@ int cmd_fsck(int argc,
 		.ref = NULL
 	};
 
-	/* fsck knows how to handle missing promisor objects */
-	fetch_if_missing = 0;
-
 	errors_found = 0;
 	disable_replace_refs();
 	save_commit_buffer = 0;
 
 	argc = parse_options(argc, argv, prefix, fsck_opts, fsck_usage, 0);
 
+	/* fsck knows how to handle missing promisor objects */
+	repo->fetch_if_missing = 0;
+
 	fsck_options_init(&fsck_walk_options, repo, FSCK_OPTIONS_DEFAULT);
 	fsck_walk_options.walk = mark_object;
 
builtin/index-pack.c +4 -4
index bc86925ad0..28f8d01e04 100644
@@ -1886,7 +1886,7 @@ static void repack_local_links(void)
 int cmd_index_pack(int argc,
 		   const char **argv,
 		   const char *prefix,
-		   struct repository *repo UNUSED)
+		   struct repository *repo)
 {
 	int i, fix_thin_pack = 0, verify = 0, stat_only = 0, rev_index;
 	const char *curr_index;
@@ -1903,15 +1903,15 @@ int cmd_index_pack(int argc,
 	int report_end_of_input = 0;
 	int hash_algo = 0;
 
+	show_usage_if_asked(argc, argv, index_pack_usage);
+
 	/*
 	 * index-pack never needs to fetch missing objects except when
 	 * REF_DELTA bases are missing (which are explicitly handled). It only
 	 * accesses the repo to do hash collision checks and to check which
 	 * REF_DELTA bases need to be fetched.
 	 */
-	fetch_if_missing = 0;
-
-	show_usage_if_asked(argc, argv, index_pack_usage);
+	(repo ? repo : the_repository)->fetch_if_missing = 0;
 
 	disable_replace_refs();
 
builtin/pack-objects.c +7 -7
index 1ec5b6f206..9732b7947a 100644
@@ -4089,7 +4089,7 @@ static void add_unreachable_loose_objects(struct rev_info *revs);
 
 static void read_stdin_packs(enum stdin_packs_mode mode, int rev_list_unpacked)
 {
-	int prev_fetch_if_missing = fetch_if_missing;
+	int prev_fetch_if_missing = the_repository->fetch_if_missing;
 	struct rev_info revs;
 
 	/*
@@ -4097,7 +4097,7 @@ static void read_stdin_packs(enum stdin_packs_mode mode, int rev_list_unpacked)
 	 * walk is best-effort though we don't want to perform backfill fetches
 	 * for them.
 	 */
-	fetch_if_missing = 0;
+	the_repository->fetch_if_missing = 0;
 
 	repo_init_revisions(the_repository, &revs, NULL);
 	/*
@@ -4145,7 +4145,7 @@ static void read_stdin_packs(enum stdin_packs_mode mode, int rev_list_unpacked)
 	trace2_data_intmax("pack-objects", the_repository, "stdin_packs_hints",
 			   stdin_packs_hints_nr);
 
-	fetch_if_missing = prev_fetch_if_missing;
+	the_repository->fetch_if_missing = prev_fetch_if_missing;
 }
 
 static void add_cruft_object_entry(const struct object_id *oid, enum object_type type,
@@ -4469,14 +4469,14 @@ static int option_parse_missing_action(const struct option *opt UNUSED,
 
 	if (!strcmp(arg, "allow-any")) {
 		arg_missing_action = MA_ALLOW_ANY;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 		fn_show_object = show_object__ma_allow_any;
 		return 0;
 	}
 
 	if (!strcmp(arg, "allow-promisor")) {
 		arg_missing_action = MA_ALLOW_PROMISOR;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 		fn_show_object = show_object__ma_allow_promisor;
 		return 0;
 	}
@@ -5345,7 +5345,7 @@ int cmd_pack_objects(int argc,
 				  exclude_promisor_objects_best_effort,
 				  "--exclude-promisor-objects-best-effort");
 	if (exclude_promisor_objects) {
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 
 		/* --stdin-packs handles promisor objects separately. */
 		if (!stdin_packs) {
@@ -5354,7 +5354,7 @@ int cmd_pack_objects(int argc,
 		}
 	} else if (exclude_promisor_objects_best_effort) {
 		use_internal_rev_list = 1;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 		option_parse_missing_action(NULL, "allow-any", 0);
 		/* revs configured below */
 	}
builtin/prune.c +1 -1
index 55635a891f..a7e4678d11 100644
@@ -194,7 +194,7 @@ int cmd_prune(int argc,
 	if (show_progress == -1)
 		show_progress = isatty(2);
 	if (exclude_promisor_objects) {
-		fetch_if_missing = 0;
+		repo->fetch_if_missing = 0;
 		revs.exclude_promisor_objects = 1;
 	}
 
builtin/rev-list.c +5 -5
index 02818b81c6..40e62baa9f 100644
@@ -509,25 +509,25 @@ static inline int parse_missing_action_value(const char *value)
 
 	if (!strcmp(value, "allow-any")) {
 		arg_missing_action = MA_ALLOW_ANY;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 		return 1;
 	}
 
 	if (!strcmp(value, "print")) {
 		arg_missing_action = MA_PRINT;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 		return 1;
 	}
 
 	if (!strcmp(value, "print-info")) {
 		arg_missing_action = MA_PRINT_INFO;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 		return 1;
 	}
 
 	if (!strcmp(value, "allow-promisor")) {
 		arg_missing_action = MA_ALLOW_PROMISOR;
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 		return 1;
 	}
 
@@ -745,7 +745,7 @@ int cmd_rev_list(int argc,
 	for (i = 1; i < argc; i++) {
 		const char *arg = argv[i];
 		if (!strcmp(arg, "--exclude-promisor-objects")) {
-			fetch_if_missing = 0;
+			the_repository->fetch_if_missing = 0;
 			revs.exclude_promisor_objects = 1;
 		} else if (skip_prefix(arg, "--missing=", &arg)) {
 			parse_missing_action_value(arg);
common-init.c +1 -1
index d26c9c1f20..4a3fa4d7be 100644
@@ -47,7 +47,7 @@ static void setup_environment(void)
 	update_ref_namespace(NAMESPACE_REPLACE, git_replace_ref_base);
 
 	if (git_env_bool(NO_LAZY_FETCH_ENVIRONMENT, 0))
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 }
 
 void init_git(const char **argv)
git.c +1 -1
index e5f1811b6b..f3ad3aad96 100644
@@ -202,7 +202,7 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 			if (envchanged)
 				*envchanged = 1;
 		} else if (!strcmp(cmd, "--no-lazy-fetch")) {
-			fetch_if_missing = 0;
+			the_repository->fetch_if_missing = 0;
 			setenv(NO_LAZY_FETCH_ENVIRONMENT, "1", 1);
 			if (envchanged)
 				*envchanged = 1;
midx-write.c +1 -1
index 580724d21a..8537102254 100644
@@ -865,7 +865,7 @@ static void find_commits_for_midx_bitmap(struct commit_stack *commits,
 	 * complain later that we don't have reachability closure (and fail
 	 * appropriately).
 	 */
-	fetch_if_missing = 0;
+	ctx->repo->fetch_if_missing = 0;
 	revs.exclude_promisor_objects = 1;
 
 	if (prepare_revision_walk(&revs))
odb.c +1 -3
index dabd481f57..110326f063 100644
@@ -528,8 +528,6 @@ void disable_obj_read_lock(void)
 	pthread_mutex_destroy(&obj_read_mutex);
 }
 
-int fetch_if_missing = 1;
-
 static int register_all_submodule_sources(struct object_database *odb)
 {
 	int ret = odb->submodule_source_paths.nr;
@@ -595,7 +593,7 @@ static int do_oid_object_info_extended(struct object_database *odb,
 			continue;
 
 		/* Check if it is a missing object */
-		if (fetch_if_missing && repo_has_promisor_remote(odb->repo) &&
+		if (odb->repo->fetch_if_missing && repo_has_promisor_remote(odb->repo) &&
 		    !already_retried &&
 		    !(flags & OBJECT_INFO_SKIP_FETCH_OBJECT)) {
 			promisor_remote_get_direct(odb->repo, real, 1);
odb.h +0 -8
index cbc2f9ced4..d3a1e378b6 100644
@@ -15,14 +15,6 @@ struct repository;
 struct strbuf;
 struct strvec;
 
-/*
- * Set this to 0 to prevent odb_read_object_info_extended() from fetching missing
- * blobs. This has a difference only if extensions.partialClone is set.
- *
- * Its default value is 1.
- */
-extern int fetch_if_missing;
-
 /*
  * Compute the exact path an alternate is at and returns it. In case of
  * error NULL is returned and the human readable error is added to `err`
repository.c +1 -0
index 651b0f6933..c2d954cf83 100644
@@ -74,6 +74,7 @@ void initialize_repository(struct repository *repo)
 	index_state_init(repo->index, repo);
 	repo->check_deprecated_config = true;
 	repo->bare_cfg = -1;
+	repo->fetch_if_missing = 1;
 	repo_config_values_init(&repo->config_values_private_);
 
 	/*
repository.h +6 -0
index 3b467a2513..11f5c2ed10 100644
@@ -184,6 +184,12 @@ struct repository {
 	/* True if commit-graph has been disabled within this process. */
 	int commit_graph_disabled;
 
+	/*
+	 * Controls whether the repository should lazily fetch missing
+	 * objects from promisor remotes. Defaults to 1.
+	 */
+	int fetch_if_missing;
+
 	/*
 	 * Lazily-populated cache mapping hook event names to configured hooks.
 	 * NULL until first hook use.
revision.c +1 -1
index 526bcf3fb5..40cb1cc828 100644
@@ -2732,7 +2732,7 @@ static int handle_revision_opt(struct rev_info *revs, int argc, const char **arg
 		revs->ignore_missing = 1;
 	} else if (opt && opt->allow_exclude_promisor_objects &&
 		   !strcmp(arg, "--exclude-promisor-objects")) {
-		if (fetch_if_missing)
+		if (revs->repo->fetch_if_missing)
 			BUG("exclude_promisor_objects can only be used when fetch_if_missing is 0");
 		revs->exclude_promisor_objects = 1;
 	} else {
Signature · 1 line
2.43.0
#16 lore raw
Quoted 28 lines · Tian Yuchen

The global variable 'fetch_if_missing' controls whether a missing object check should prompt a lazy fetch from a promisor remote. In order to continue the libification effort, move it into 'struct repository' and initialize it to 1 by default to keep the previous behavior.

Note that in builtin/fsck.c and builtin/index-pack.c, when running related commands with the '-h' parameter, the 'repo' pointer is not passed in. To prevent null pointer dereferences, we defer operations on the repo until after parameter parsing is complete.

Additionally, update the partial clone documentation to reflect that this is now a per-repository flag.

---

Changes since v3:

- Use revs->repo in revision.c instead of the_repository.

- Coordinate the other topics. Specifically, for common-init.c, use the_repository->fetch_if_missing in setup_environment(), etc. This patch currently does not conflict with seen or next.

Quoted 16 lines
 Documentation/technical/partial-clone.adoc |  2 +-
 builtin/fetch-pack.c                       |  2 +-
 builtin/fsck.c                             |  6 +++---
 builtin/index-pack.c                       |  8 ++++----
 builtin/pack-objects.c                     | 14 +++++++-------
 builtin/prune.c                            |  2 +-
 builtin/rev-list.c                         | 10 +++++-----
 common-init.c                              |  2 +-
 git.c                                      |  2 +-
 midx-write.c                               |  2 +-
 odb.c                                      |  4 +---
 odb.h                                      |  8 --------
 repository.c                               |  1 +
 repository.h                               |  6 ++++++
 revision.c                                 |  2 +-
 15 files changed, 34 insertions(+), 37 deletions(-)

There still are references to the_repository->fetch_if_missing remaining in the codebase with this change.

        $ git grep -l -e 'the_repository->fetch_if_missing'
        builtin/fetch-pack.c
        builtin/pack-objects.c
        builtin/rev-list.c
        common-init.c
        git.c

Some of them I suspect should just use the caller supplied 'repo', possibly after removing the UNUSED marker. For example:

        int cmd_fetch_pack(int argc,
                           const char **argv,
                           const char *prefix UNUSED,
                           struct repository *repo UNUSED)
        {
                int i, ret;
        ...
                enum protocol_version version;

                the_repository->fetch_if_missing = 0;

                packet_trace_identity("fetch-pack");

                memset(&args, 0, sizeof(args));
                list_objects_filter_init(&args.filter_options);
                args.uploadpack = "git-upload-pack";

                show_usage_if_asked(argc, argv, fetch_pack_usage);

As { "fetch-pack", cmd_fetch_pack } in the git.c:commands[] array is marked as RUN_SETUP, repo will not be NULL unless "git fetch-pack -h" is requested, and when repo is NULL, show_usage_if_asked() will give the short help text and never return.

So I think it makes sense to set 'fetch_if_missing' *after* the call to show_usage_if_asked() and set it in 'repo', not in 'the_repository'.

Other hits in the above "git grep" output looked similar. The code paths in pack-objects.c may need a preliminary clean-up patch (or two) before moving fetch_if_missing to the repository instance. I.e., pass repo through the call graph from cmd_pack_objects() to read_stdin_packs(), and then update assignments to fetch_if_missing variable to instead assign to repo->fetch_if_missing in a second step. There are other code paths that want similar clean-up.

HTH.

#17 lore raw

v5 cover repository: move fetch_if_missing into struct repository

In reply to Tian Yuchen

Hi all,

To continue the libification effort, this commit series aim to migrate the global 'fetch_if_missing' into struct repository, making it a per-repository configuration.

Changes since v4:

- Use repo instead of the_repository in more call sites.

Tian Yuchen (2):
pack-objects: give fetch_if_missing call sites access to 'repo'
repository: move fetch_if_missing into struct repository

 Documentation/technical/partial-clone.adoc |  2 +-
 builtin/fetch-pack.c                       |  6 +++---
 builtin/fsck.c                             |  6 +++---
 builtin/index-pack.c                       |  8 +++----
 builtin/pack-objects.c                     | 25 ++++++++++++----------
 builtin/prune.c                            |  2 +-
 builtin/rev-list.c                         | 17 ++++++++-------
 common-init.c                              |  2 +-
 git.c                                      |  2 +-
 midx-write.c                               |  2 +-
 odb.c                                      |  4 +---
 odb.h                                      |  8 -------
 repository.c                               |  1 +
 repository.h                               |  6 ++++++
 revision.c                                 |  2 +-
 15 files changed, 47 insertions(+), 46 deletions(-)
Signature · 1 line
2.43.0
#18 lore raw

PATCH v5 1/2 pack-objects: give fetch_if_missing call sites access to 'repo'

In order to move the global 'fetch_if_missing' variable into 'struct repository' in a follow-up commit, first make sure every call site in builtin/pack-objects.c has a 'struct repository *'.

'cmd_pack_objects()' already receives a 'repo' parameter. Drop the UNUSED mark.

'option_parse_missing_action()' is registered as an OPT_CALLBACK, so its signature is fixed and cannot easily gain a 'repo' parameter of its own. Let it only record 'arg_missing_action'. Instead, apply the side effect right after 'parse_options()' returns in 'cmd_pack_objects()', where 'repo' is available.

---
 builtin/pack-objects.c | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)
builtin/pack-objects.c +9 -6
index 1ec5b6f206..2b14dd2f31 100644
@@ -4087,7 +4087,8 @@ static void stdin_packs_read_input(struct rev_info *revs,
 
 static void add_unreachable_loose_objects(struct rev_info *revs);
 
-static void read_stdin_packs(enum stdin_packs_mode mode, int rev_list_unpacked)
+static void read_stdin_packs(struct repository *repo,
+			     enum stdin_packs_mode mode, int rev_list_unpacked)
 {
 	int prev_fetch_if_missing = fetch_if_missing;
 	struct rev_info revs;
@@ -4099,7 +4100,7 @@ static void read_stdin_packs(enum stdin_packs_mode mode, int rev_list_unpacked)
 	 */
 	fetch_if_missing = 0;
 
-	repo_init_revisions(the_repository, &revs, NULL);
+	repo_init_revisions(repo, &revs, NULL);
 	/*
 	 * Use a revision walk to fill in the namehash of objects in the include
 	 * packs. To save time, we'll avoid traversing through objects that are
@@ -4469,14 +4470,12 @@ static int option_parse_missing_action(const struct option *opt UNUSED,
 
 	if (!strcmp(arg, "allow-any")) {
 		arg_missing_action = MA_ALLOW_ANY;
-		fetch_if_missing = 0;
 		fn_show_object = show_object__ma_allow_any;
 		return 0;
 	}
 
 	if (!strcmp(arg, "allow-promisor")) {
 		arg_missing_action = MA_ALLOW_PROMISOR;
-		fetch_if_missing = 0;
 		fn_show_object = show_object__ma_allow_promisor;
 		return 0;
 	}
@@ -5118,7 +5117,7 @@ static int parse_stdin_packs_mode(const struct option *opt, const char *arg,
 int cmd_pack_objects(int argc,
 		     const char **argv,
 		     const char *prefix,
-		     struct repository *repo UNUSED)
+		     struct repository *repo)
 {
 	int use_internal_rev_list = 0;
 	int all_progress_implied = 0;
@@ -5267,6 +5266,10 @@ int cmd_pack_objects(int argc,
 	argc = parse_options(argc, argv, prefix, pack_objects_options,
 			     pack_usage, 0);
 
+	if (arg_missing_action == MA_ALLOW_ANY ||
+	    arg_missing_action == MA_ALLOW_PROMISOR)
+		fetch_if_missing = 0;
+
 	if (argc) {
 		base_name = argv[0];
 		argc--;
@@ -5471,7 +5474,7 @@ int cmd_pack_objects(int argc,
 		progress_state = start_progress(the_repository,
 						_("Enumerating objects"), 0);
 	if (stdin_packs) {
-		read_stdin_packs(stdin_packs, rev_list_unpacked);
+		read_stdin_packs(repo, stdin_packs, rev_list_unpacked);
 	} else if (cruft) {
 		read_cruft_objects();
 	} else if (!use_internal_rev_list) {
Signature · 1 line
2.43.0
#19 lore raw

Re: [PATCH v5 1/2] pack-objects: give fetch_if_missing call sites access to 'repo'

Quoted 3 lines · Tian Yuchen

'option_parse_missing_action()' is registered as an OPT_CALLBACK, so its signature is fixed and cannot easily gain a 'repo' parameter of its own.

Is that true? OPT_CALLBACK takes (short, long, value, arghelp, help, callback), but the value parameter, which typically receives the location to store the parsed value in, is NULL for "--missing=<...>" option, like so:

		OPT_CALLBACK_F(0, "missing", NULL, N_("action"),
		  N_("handling for missing objects"), PARSE_OPT_NONEG,
		  option_parse_missing_action),

And "value" does not have to be the exact location; we can pass the address of the repository instance there, and the callback can dereference it to reach repo->fetch_if_missing member to make an assignment, if it wants to.

Am I missing something?

Quoted 3 lines

Let it only record 'arg_missing_action'. Instead, apply the side effect right after 'parse_options()' returns in 'cmd_pack_objects()', where 'repo' is available.

When I review a new iteration, I first apply and look at the difference from the previous iteration before reading the new iteration of patches or their proposed commit log messages, and I did notice this business about "arg_missing_action" and wondering what it was about (which by the way is precisely I read the patch before the log message to ensure that the log message explains what I find unclear in the code). Thanks for describing this. But if we use OPT_CALLBACK() to pass the location of the variable, this may no longer be necessary, no?

#20 lore raw

Re: [PATCH v5 1/2] pack-objects: give fetch_if_missing call sites access to 'repo'

Quoted 15 lines · Junio C Hamano
Quoted 3 lines · Tian Yuchen

'option_parse_missing_action()' is registered as an OPT_CALLBACK, so its signature is fixed and cannot easily gain a 'repo' parameter of its own.

Is that true? OPT_CALLBACK takes (short, long, value, arghelp, help, callback), but the value parameter, which typically receives the location to store the parsed value in, is NULL for "--missing=<...>" option, like so:

		OPT_CALLBACK_F(0, "missing", NULL, N_("action"),
		  N_("handling for missing objects"), PARSE_OPT_NONEG,
		  option_parse_missing_action),
Quoted 7 lines

And "value" does not have to be the exact location; we can pass the address of the repository instance there, and the callback can dereference it to reach repo->fetch_if_missing member to make an assignment, if it wants to.

Am I missing something?

Oh yeah indeed, 'value' can be used for passing repo. My mind went blank at that moment :(

Quoted 13 lines
Quoted 3 lines

Let it only record 'arg_missing_action'. Instead, apply the side effect right after 'parse_options()' returns in 'cmd_pack_objects()', where 'repo' is available.

When I review a new iteration, I first apply and look at the difference from the previous iteration before reading the new iteration of patches or their proposed commit log messages, and I did notice this business about "arg_missing_action" and wondering what it was about (which by the way is precisely I read the patch before the log message to ensure that the log message explains what

Quoted 4 lines

I find unclear in the code). Thanks for describing this. But if we use OPT_CALLBACK() to pass the location of the variable, this may no longer be necessary, no?

Will change, thanks!

Regards, yuchen

#21 lore raw

PATCH v5 2/2 repository: move fetch_if_missing into struct repository

In reply to Tian Yuchen

The global variable 'fetch_if_missing' controls whether a missing object check should prompt a lazy fetch from a promisor remote. In order to continue the libification effort, move it into 'struct repository' and initialize it to 1 by default to keep the previous behavior.

builtin/fetch-pack.c, builtin/fsck.c, and builtin/rev-list.c are entered via commands marked RUN_SETUP in git.c:commands[]. Their 'repo' parameter is only NULL when '-h' is given outside of a repository, in which case either show_usage_if_asked() or parse_options()'s own '-h' handling exits the process before returning. We can therefore drop their UNUSED markers and assign to 'repo' directly.

builtin/index-pack.c is entered via RUN_SETUP_GENTLY, so its 'repo' pointer can be NULL any time it is run outside of a repository, not only with '-h'. We keep a NULL check there and fall back to 'the_repository'.

builtin/pack-objects.c's call sites were prepared in the preceding commit to have 'repo' in scope. This commit performs the replacement of 'fetch_if_missing' with 'repo->fetch_if_missing' there.

Additionally, update the partial clone documentation to reflect that this is now a per-repository flag.

---
 Documentation/technical/partial-clone.adoc |  2 +-
 builtin/fetch-pack.c                       |  6 +++---
 builtin/fsck.c                             |  6 +++---
 builtin/index-pack.c                       |  8 ++++----
 builtin/pack-objects.c                     | 12 ++++++------
 builtin/prune.c                            |  2 +-
 builtin/rev-list.c                         | 17 +++++++++--------
 common-init.c                              |  2 +-
 git.c                                      |  2 +-
 midx-write.c                               |  2 +-
 odb.c                                      |  4 +---
 odb.h                                      |  8 --------
 repository.c                               |  1 +
 repository.h                               |  6 ++++++
 revision.c                                 |  2 +-
 15 files changed, 39 insertions(+), 41 deletions(-)
Documentation/technical/partial-clone.adoc +1 -1
index e513e391ea..18718a3840 100644
@@ -159,7 +159,7 @@ and prefetch those objects in bulk.
 - `repack` in GC has been updated to not touch promisor packfiles at all,
   and to only repack other objects.
 
-- The global variable "fetch_if_missing" is used to control whether an
+- The per-repository flag "fetch_if_missing" is used to control whether an
   object lookup will attempt to dynamically fetch a missing object or
   report an error.
 +
builtin/fetch-pack.c +3 -3
index 316badd969..86754296fa 100644
@@ -49,7 +49,7 @@ static void add_sought_entry(struct ref ***sought, int *nr, int *alloc,
 int cmd_fetch_pack(int argc,
 		   const char **argv,
 		   const char *prefix UNUSED,
-		   struct repository *repo UNUSED)
+		   struct repository *repo)
 {
 	int i, ret;
 	struct ref *fetched_refs = NULL, *remote_refs = NULL;
@@ -67,8 +67,6 @@ int cmd_fetch_pack(int argc,
 	struct packet_reader reader;
 	enum protocol_version version;
 
-	fetch_if_missing = 0;
-
 	packet_trace_identity("fetch-pack");
 
 	memset(&args, 0, sizeof(args));
@@ -77,6 +75,8 @@ int cmd_fetch_pack(int argc,
 
 	show_usage_if_asked(argc, argv, fetch_pack_usage);
 
+	repo->fetch_if_missing = 0;
+
 	for (i = 1; i < argc && *argv[i] == '-'; i++) {
 		const char *arg = argv[i];
 
builtin/fsck.c +3 -3
index a6c054e45b..8cfc0e8b26 100644
@@ -1017,15 +1017,15 @@ int cmd_fsck(int argc,
 		.ref = NULL
 	};
 
-	/* fsck knows how to handle missing promisor objects */
-	fetch_if_missing = 0;
-
 	errors_found = 0;
 	disable_replace_refs();
 	save_commit_buffer = 0;
 
 	argc = parse_options(argc, argv, prefix, fsck_opts, fsck_usage, 0);
 
+	/* fsck knows how to handle missing promisor objects */
+	repo->fetch_if_missing = 0;
+
 	fsck_options_init(&fsck_walk_options, repo, FSCK_OPTIONS_DEFAULT);
 	fsck_walk_options.walk = mark_object;
 
builtin/index-pack.c +4 -4
index bc86925ad0..28f8d01e04 100644
@@ -1886,7 +1886,7 @@ static void repack_local_links(void)
 int cmd_index_pack(int argc,
 		   const char **argv,
 		   const char *prefix,
-		   struct repository *repo UNUSED)
+		   struct repository *repo)
 {
 	int i, fix_thin_pack = 0, verify = 0, stat_only = 0, rev_index;
 	const char *curr_index;
@@ -1903,15 +1903,15 @@ int cmd_index_pack(int argc,
 	int report_end_of_input = 0;
 	int hash_algo = 0;
 
+	show_usage_if_asked(argc, argv, index_pack_usage);
+
 	/*
 	 * index-pack never needs to fetch missing objects except when
 	 * REF_DELTA bases are missing (which are explicitly handled). It only
 	 * accesses the repo to do hash collision checks and to check which
 	 * REF_DELTA bases need to be fetched.
 	 */
-	fetch_if_missing = 0;
-
-	show_usage_if_asked(argc, argv, index_pack_usage);
+	(repo ? repo : the_repository)->fetch_if_missing = 0;
 
 	disable_replace_refs();
 
builtin/pack-objects.c +6 -6
index 2b14dd2f31..7dea8940a0 100644
@@ -4090,7 +4090,7 @@ static void add_unreachable_loose_objects(struct rev_info *revs);
 static void read_stdin_packs(struct repository *repo,
 			     enum stdin_packs_mode mode, int rev_list_unpacked)
 {
-	int prev_fetch_if_missing = fetch_if_missing;
+	int prev_fetch_if_missing = repo->fetch_if_missing;
 	struct rev_info revs;
 
 	/*
@@ -4098,7 +4098,7 @@ static void read_stdin_packs(struct repository *repo,
 	 * walk is best-effort though we don't want to perform backfill fetches
 	 * for them.
 	 */
-	fetch_if_missing = 0;
+	repo->fetch_if_missing = 0;
 
 	repo_init_revisions(repo, &revs, NULL);
 	/*
@@ -4146,7 +4146,7 @@ static void read_stdin_packs(struct repository *repo,
 	trace2_data_intmax("pack-objects", the_repository, "stdin_packs_hints",
 			   stdin_packs_hints_nr);
 
-	fetch_if_missing = prev_fetch_if_missing;
+	repo->fetch_if_missing = prev_fetch_if_missing;
 }
 
 static void add_cruft_object_entry(const struct object_id *oid, enum object_type type,
@@ -5268,7 +5268,7 @@ int cmd_pack_objects(int argc,
 
 	if (arg_missing_action == MA_ALLOW_ANY ||
 	    arg_missing_action == MA_ALLOW_PROMISOR)
-		fetch_if_missing = 0;
+		repo->fetch_if_missing = 0;
 
 	if (argc) {
 		base_name = argv[0];
@@ -5348,7 +5348,7 @@ int cmd_pack_objects(int argc,
 				  exclude_promisor_objects_best_effort,
 				  "--exclude-promisor-objects-best-effort");
 	if (exclude_promisor_objects) {
-		fetch_if_missing = 0;
+		repo->fetch_if_missing = 0;
 
 		/* --stdin-packs handles promisor objects separately. */
 		if (!stdin_packs) {
@@ -5357,7 +5357,7 @@ int cmd_pack_objects(int argc,
 		}
 	} else if (exclude_promisor_objects_best_effort) {
 		use_internal_rev_list = 1;
-		fetch_if_missing = 0;
+		repo->fetch_if_missing = 0;
 		option_parse_missing_action(NULL, "allow-any", 0);
 		/* revs configured below */
 	}
builtin/prune.c +1 -1
index 55635a891f..a7e4678d11 100644
@@ -194,7 +194,7 @@ int cmd_prune(int argc,
 	if (show_progress == -1)
 		show_progress = isatty(2);
 	if (exclude_promisor_objects) {
-		fetch_if_missing = 0;
+		repo->fetch_if_missing = 0;
 		revs.exclude_promisor_objects = 1;
 	}
 
builtin/rev-list.c +9 -8
index 02818b81c6..6b596231ab 100644
@@ -500,7 +500,8 @@ static void print_disk_usage(off_t size)
 	strbuf_release(&sb);
 }
 
-static inline int parse_missing_action_value(const char *value)
+static inline int parse_missing_action_value(struct repository *repo,
+					     const char *value)
 {
 	if (!strcmp(value, "error")) {
 		arg_missing_action = MA_ERROR;
@@ -509,25 +510,25 @@ static inline int parse_missing_action_value(const char *value)
 
 	if (!strcmp(value, "allow-any")) {
 		arg_missing_action = MA_ALLOW_ANY;
-		fetch_if_missing = 0;
+		repo->fetch_if_missing = 0;
 		return 1;
 	}
 
 	if (!strcmp(value, "print")) {
 		arg_missing_action = MA_PRINT;
-		fetch_if_missing = 0;
+		repo->fetch_if_missing = 0;
 		return 1;
 	}
 
 	if (!strcmp(value, "print-info")) {
 		arg_missing_action = MA_PRINT_INFO;
-		fetch_if_missing = 0;
+		repo->fetch_if_missing = 0;
 		return 1;
 	}
 
 	if (!strcmp(value, "allow-promisor")) {
 		arg_missing_action = MA_ALLOW_PROMISOR;
-		fetch_if_missing = 0;
+		repo->fetch_if_missing = 0;
 		return 1;
 	}
 
@@ -692,7 +693,7 @@ static void prepare_maximal_independent(struct rev_info *revs)
 int cmd_rev_list(int argc,
 		 const char **argv,
 		 const char *prefix,
-		 struct repository *repo UNUSED)
+		 struct repository *repo)
 {
 	struct rev_info revs;
 	struct rev_list_info info;
@@ -745,10 +746,10 @@ int cmd_rev_list(int argc,
 	for (i = 1; i < argc; i++) {
 		const char *arg = argv[i];
 		if (!strcmp(arg, "--exclude-promisor-objects")) {
-			fetch_if_missing = 0;
+			repo->fetch_if_missing = 0;
 			revs.exclude_promisor_objects = 1;
 		} else if (skip_prefix(arg, "--missing=", &arg)) {
-			parse_missing_action_value(arg);
+			parse_missing_action_value(repo, arg);
 		} else if (!strcmp(arg, "-z")) {
 			line_term = '\0';
 			info_term = '\0';
common-init.c +1 -1
index d26c9c1f20..4a3fa4d7be 100644
@@ -47,7 +47,7 @@ static void setup_environment(void)
 	update_ref_namespace(NAMESPACE_REPLACE, git_replace_ref_base);
 
 	if (git_env_bool(NO_LAZY_FETCH_ENVIRONMENT, 0))
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 }
 
 void init_git(const char **argv)
git.c +1 -1
index e5f1811b6b..f3ad3aad96 100644
@@ -202,7 +202,7 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 			if (envchanged)
 				*envchanged = 1;
 		} else if (!strcmp(cmd, "--no-lazy-fetch")) {
-			fetch_if_missing = 0;
+			the_repository->fetch_if_missing = 0;
 			setenv(NO_LAZY_FETCH_ENVIRONMENT, "1", 1);
 			if (envchanged)
 				*envchanged = 1;
midx-write.c +1 -1
index 580724d21a..8537102254 100644
@@ -865,7 +865,7 @@ static void find_commits_for_midx_bitmap(struct commit_stack *commits,
 	 * complain later that we don't have reachability closure (and fail
 	 * appropriately).
 	 */
-	fetch_if_missing = 0;
+	ctx->repo->fetch_if_missing = 0;
 	revs.exclude_promisor_objects = 1;
 
 	if (prepare_revision_walk(&revs))
odb.c +1 -3
index dabd481f57..110326f063 100644
@@ -528,8 +528,6 @@ void disable_obj_read_lock(void)
 	pthread_mutex_destroy(&obj_read_mutex);
 }
 
-int fetch_if_missing = 1;
-
 static int register_all_submodule_sources(struct object_database *odb)
 {
 	int ret = odb->submodule_source_paths.nr;
@@ -595,7 +593,7 @@ static int do_oid_object_info_extended(struct object_database *odb,
 			continue;
 
 		/* Check if it is a missing object */
-		if (fetch_if_missing && repo_has_promisor_remote(odb->repo) &&
+		if (odb->repo->fetch_if_missing && repo_has_promisor_remote(odb->repo) &&
 		    !already_retried &&
 		    !(flags & OBJECT_INFO_SKIP_FETCH_OBJECT)) {
 			promisor_remote_get_direct(odb->repo, real, 1);
odb.h +0 -8
index cbc2f9ced4..d3a1e378b6 100644
@@ -15,14 +15,6 @@ struct repository;
 struct strbuf;
 struct strvec;
 
-/*
- * Set this to 0 to prevent odb_read_object_info_extended() from fetching missing
- * blobs. This has a difference only if extensions.partialClone is set.
- *
- * Its default value is 1.
- */
-extern int fetch_if_missing;
-
 /*
  * Compute the exact path an alternate is at and returns it. In case of
  * error NULL is returned and the human readable error is added to `err`
repository.c +1 -0
index 651b0f6933..c2d954cf83 100644
@@ -74,6 +74,7 @@ void initialize_repository(struct repository *repo)
 	index_state_init(repo->index, repo);
 	repo->check_deprecated_config = true;
 	repo->bare_cfg = -1;
+	repo->fetch_if_missing = 1;
 	repo_config_values_init(&repo->config_values_private_);
 
 	/*
repository.h +6 -0
index 3b467a2513..11f5c2ed10 100644
@@ -184,6 +184,12 @@ struct repository {
 	/* True if commit-graph has been disabled within this process. */
 	int commit_graph_disabled;
 
+	/*
+	 * Controls whether the repository should lazily fetch missing
+	 * objects from promisor remotes. Defaults to 1.
+	 */
+	int fetch_if_missing;
+
 	/*
 	 * Lazily-populated cache mapping hook event names to configured hooks.
 	 * NULL until first hook use.
revision.c +1 -1
index 526bcf3fb5..40cb1cc828 100644
@@ -2732,7 +2732,7 @@ static int handle_revision_opt(struct rev_info *revs, int argc, const char **arg
 		revs->ignore_missing = 1;
 	} else if (opt && opt->allow_exclude_promisor_objects &&
 		   !strcmp(arg, "--exclude-promisor-objects")) {
-		if (fetch_if_missing)
+		if (revs->repo->fetch_if_missing)
 			BUG("exclude_promisor_objects can only be used when fetch_if_missing is 0");
 		revs->exclude_promisor_objects = 1;
 	} else {
Signature · 1 line
2.43.0
#22 lore raw
In reply to Tian Yuchen
Quoted 8 lines · Tian Yuchen
 Documentation/technical/partial-clone.adoc |  2 +-
 builtin/fetch-pack.c                       |  6 +++---
 builtin/fsck.c                             |  6 +++---
 builtin/index-pack.c                       |  8 +++----
 builtin/pack-objects.c                     | 25 ++++++++++++----------
 builtin/prune.c                            |  2 +-
 builtin/rev-list.c                         | 17 ++++++++-------
 common-init.c                              |  2 +-
Quoted 8 lines
 git.c                                      |  2 +-
 midx-write.c                               |  2 +-
 odb.c                                      |  4 +---
 odb.h                                      |  8 -------
 repository.c                               |  1 +
 repository.h                               |  6 ++++++
 revision.c                                 |  2 +-
 15 files changed, 47 insertions(+), 46 deletions(-)
$ git grep -e 'the_repository->fetch_if_missing' \
    builtin/{fetch-pack,fsck,index-pack,pack-objects,prune,rev-list}.c \
    common-init.c midx-write.c odb.c repository.c revision.c
common-init.c:          the_repository->fetch_if_missing = 0;

Compared to the previous round, there is no remaining reference to the_repository's '.fetch_if_missing' member. There is a reference in common-init.c:setup_environment() that reacts to the GIT_NO_LAZY_FETCH environment variable, but that does not count.

Much better.

Thanks.

#23 lore raw

PATCH v6 1/1 repository: move fetch_if_missing into struct repository

In reply to Tian Yuchen

The global variable 'fetch_if_missing' controls whether a missing object check should prompt a lazy fetch from a promisor remote. In order to continue the libification effort, move it into 'struct repository' and initialize it to 1 by default to keep the previous behavior.

builtin/fetch-pack.c, builtin/fsck.c, and builtin/rev-list.c are entered via commands marked RUN_SETUP in git.c:commands[]. Their 'repo' parameter is only NULL when '-h' is given outside of a repository, in which case either show_usage_if_asked() or parse_options()'s own '-h' handling exits the process before returning. We can therefore drop their UNUSED markers and assign to 'repo' directly.

builtin/index-pack.c is entered via RUN_SETUP_GENTLY, so its 'repo' pointer can be NULL any time it is run outside of a repository, not only with '-h'. We keep a NULL check there and fall back to 'the_repository'.

builtin/pack-objects.c needs two adjustments to make 'repo' reach every 'fetch_if_missing' call site: 'read_stdin_packs()' now takes a 'struct repository *'; 'option_parse_missing_action()', which is registered as an OPT_CALLBACK, receives a 'repo' through the option's 'value' field now.

Additionally, update the partial clone documentation to reflect that this is now a per-repository flag.

---
Changes since v5:

- for 'option_parse_missing_action' in builtin/pack-objects.c, pass in a repo instance via the option's 'value' field. Hence, the commit 1/2 of the v5 iteration (pack-objects: give fetch_if_missing call sites access to 'repo') is no longer needed. Drop it.

 Documentation/technical/partial-clone.adoc |  2 +-
 builtin/fetch-pack.c                       |  6 ++--
 builtin/fsck.c                             |  6 ++--
 builtin/index-pack.c                       |  8 +++---
 builtin/pack-objects.c                     | 32 ++++++++++++----------
 builtin/prune.c                            |  2 +-
 builtin/rev-list.c                         | 17 ++++++------
 common-init.c                              |  2 +-
 git.c                                      |  2 +-
 midx-write.c                               |  2 +-
 odb.c                                      |  4 +--
 odb.h                                      |  8 ------
 repository.c                               |  1 +
 repository.h                               |  6 ++++
 revision.c                                 |  2 +-
 15 files changed, 51 insertions(+), 49 deletions(-)
Documentation/technical/partial-clone.adoc +1 -1
index e513e391ea..18718a3840 100644
@@ -159,7 +159,7 @@ and prefetch those objects in bulk.
 - `repack` in GC has been updated to not touch promisor packfiles at all,
   and to only repack other objects.
 
-- The global variable "fetch_if_missing" is used to control whether an
+- The per-repository flag "fetch_if_missing" is used to control whether an
   object lookup will attempt to dynamically fetch a missing object or
   report an error.
 +
builtin/fetch-pack.c +3 -3
index 316badd969..86754296fa 100644
@@ -49,7 +49,7 @@ static void add_sought_entry(struct ref ***sought, int *nr, int *alloc,
 int cmd_fetch_pack(int argc,
 		   const char **argv,
 		   const char *prefix UNUSED,
-		   struct repository *repo UNUSED)
+		   struct repository *repo)
 {
 	int i, ret;
 	struct ref *fetched_refs = NULL, *remote_refs = NULL;
@@ -67,8 +67,6 @@ int cmd_fetch_pack(int argc,
 	struct packet_reader reader;
 	enum protocol_version version;
 
-	fetch_if_missing = 0;
-
 	packet_trace_identity("fetch-pack");
 
 	memset(&args, 0, sizeof(args));
@@ -77,6 +75,8 @@ int cmd_fetch_pack(int argc,
 
 	show_usage_if_asked(argc, argv, fetch_pack_usage);
 
+	repo->fetch_if_missing = 0;
+
 	for (i = 1; i < argc && *argv[i] == '-'; i++) {
 		const char *arg = argv[i];
 
builtin/fsck.c +3 -3
index a6c054e45b..8cfc0e8b26 100644
@@ -1017,15 +1017,15 @@ int cmd_fsck(int argc,
 		.ref = NULL
 	};
 
-	/* fsck knows how to handle missing promisor objects */
-	fetch_if_missing = 0;
-
 	errors_found = 0;
 	disable_replace_refs();
 	save_commit_buffer = 0;
 
 	argc = parse_options(argc, argv, prefix, fsck_opts, fsck_usage, 0);
 
+	/* fsck knows how to handle missing promisor objects */
+	repo->fetch_if_missing = 0;
+
 	fsck_options_init(&fsck_walk_options, repo, FSCK_OPTIONS_DEFAULT);
 	fsck_walk_options.walk = mark_object;
 
builtin/index-pack.c +4 -4
index bc86925ad0..28f8d01e04 100644
@@ -1886,7 +1886,7 @@ static void repack_local_links(void)
 int cmd_index_pack(int argc,
 		   const char **argv,
 		   const char *prefix,
-		   struct repository *repo UNUSED)
+		   struct repository *repo)
 {
 	int i, fix_thin_pack = 0, verify = 0, stat_only = 0, rev_index;
 	const char *curr_index;
@@ -1903,15 +1903,15 @@ int cmd_index_pack(int argc,
 	int report_end_of_input = 0;
 	int hash_algo = 0;
 
+	show_usage_if_asked(argc, argv, index_pack_usage);
+
 	/*
 	 * index-pack never needs to fetch missing objects except when
 	 * REF_DELTA bases are missing (which are explicitly handled). It only
 	 * accesses the repo to do hash collision checks and to check which
 	 * REF_DELTA bases need to be fetched.
 	 */
-	fetch_if_missing = 0;
-
-	show_usage_if_asked(argc, argv, index_pack_usage);
+	(repo ? repo : the_repository)->fetch_if_missing = 0;
 
 	disable_replace_refs();
 
builtin/pack-objects.c +18 -14
index 1ec5b6f206..cb1bd39aee 100644
@@ -4087,9 +4087,10 @@ static void stdin_packs_read_input(struct rev_info *revs,
 
 static void add_unreachable_loose_objects(struct rev_info *revs);
 
-static void read_stdin_packs(enum stdin_packs_mode mode, int rev_list_unpacked)
+static void read_stdin_packs(struct repository *repo,
+			     enum stdin_packs_mode mode, int rev_list_unpacked)
 {
-	int prev_fetch_if_missing = fetch_if_missing;
+	int prev_fetch_if_missing = repo->fetch_if_missing;
 	struct rev_info revs;
 
 	/*
@@ -4097,9 +4098,9 @@ static void read_stdin_packs(enum stdin_packs_mode mode, int rev_list_unpacked)
 	 * walk is best-effort though we don't want to perform backfill fetches
 	 * for them.
 	 */
-	fetch_if_missing = 0;
+	repo->fetch_if_missing = 0;
 
-	repo_init_revisions(the_repository, &revs, NULL);
+	repo_init_revisions(repo, &revs, NULL);
 	/*
 	 * Use a revision walk to fill in the namehash of objects in the include
 	 * packs. To save time, we'll avoid traversing through objects that are
@@ -4145,7 +4146,7 @@ static void read_stdin_packs(enum stdin_packs_mode mode, int rev_list_unpacked)
 	trace2_data_intmax("pack-objects", the_repository, "stdin_packs_hints",
 			   stdin_packs_hints_nr);
 
-	fetch_if_missing = prev_fetch_if_missing;
+	repo->fetch_if_missing = prev_fetch_if_missing;
 }
 
 static void add_cruft_object_entry(const struct object_id *oid, enum object_type type,
@@ -4455,9 +4456,11 @@ static void show_object__ma_allow_promisor(struct object *obj, const char *name,
 	show_object(obj, name, data);
 }
 
-static int option_parse_missing_action(const struct option *opt UNUSED,
+static int option_parse_missing_action(const struct option *opt,
 				       const char *arg, int unset)
 {
+	struct repository *repo = opt->value;
+
 	assert(arg);
 	assert(!unset);
 
@@ -4469,14 +4472,14 @@ static int option_parse_missing_action(const struct option *opt UNUSED,
 
 	if (!strcmp(arg, "allow-any")) {
 		arg_missing_action = MA_ALLOW_ANY;
-		fetch_if_missing = 0;
+		repo->fetch_if_missing = 0;
 		fn_show_object = show_object__ma_allow_any;
 		return 0;
 	}
 
 	if (!strcmp(arg, "allow-promisor")) {
 		arg_missing_action = MA_ALLOW_PROMISOR;
-		fetch_if_missing = 0;
+		repo->fetch_if_missing = 0;
 		fn_show_object = show_object__ma_allow_promisor;
 		return 0;
 	}
@@ -5118,7 +5121,7 @@ static int parse_stdin_packs_mode(const struct option *opt, const char *arg,
 int cmd_pack_objects(int argc,
 		     const char **argv,
 		     const char *prefix,
-		     struct repository *repo UNUSED)
+		     struct repository *repo)
 {
 	int use_internal_rev_list = 0;
 	int all_progress_implied = 0;
@@ -5225,7 +5228,7 @@ int cmd_pack_objects(int argc,
 			      N_("write a bitmap index if possible"),
 			      WRITE_BITMAP_QUIET, PARSE_OPT_HIDDEN),
 		OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options),
-		OPT_CALLBACK_F(0, "missing", NULL, N_("action"),
+		OPT_CALLBACK_F(0, "missing", repo, N_("action"),
 		  N_("handling for missing objects"), PARSE_OPT_NONEG,
 		  option_parse_missing_action),
 		OPT_BOOL(0, "exclude-promisor-objects", &exclude_promisor_objects,
@@ -5345,7 +5348,7 @@ int cmd_pack_objects(int argc,
 				  exclude_promisor_objects_best_effort,
 				  "--exclude-promisor-objects-best-effort");
 	if (exclude_promisor_objects) {
-		fetch_if_missing = 0;
+		repo->fetch_if_missing = 0;
 
 		/* --stdin-packs handles promisor objects separately. */
 		if (!stdin_packs) {
@@ -5354,8 +5357,9 @@ int cmd_pack_objects(int argc,
 		}
 	} else if (exclude_promisor_objects_best_effort) {
 		use_internal_rev_list = 1;
-		fetch_if_missing = 0;
-		option_parse_missing_action(NULL, "allow-any", 0);
+		arg_missing_action = MA_ALLOW_ANY;
+		repo->fetch_if_missing = 0;
+		fn_show_object = show_object__ma_allow_any;
 		/* revs configured below */
 	}
 	if (unpack_unreachable || keep_unreachable || pack_loose_unreachable)
@@ -5471,7 +5475,7 @@ int cmd_pack_objects(int argc,
 		progress_state = start_progress(the_repository,
 						_("Enumerating objects"), 0);
 	if (stdin_packs) {
-		read_stdin_packs(stdin_packs, rev_list_unpacked);
+		read_stdin_packs(repo, stdin_packs, rev_list_unpacked);
 	} else if (cruft) {
 		read_cruft_objects();
 	} else if (!use_internal_rev_list) {
builtin/prune.c +1 -1
index 55635a891f..a7e4678d11 100644
@@ -194,7 +194,7 @@ int cmd_prune(int argc,
 	if (show_progress == -1)
 		show_progress = isatty(2);
 	if (exclude_promisor_objects) {
-		fetch_if_missing = 0;
+		repo->fetch_if_missing = 0;
 		revs.exclude_promisor_objects = 1;
 	}
 
builtin/rev-list.c +9 -8
index 02818b81c6..6b596231ab 100644
@@ -500,7 +500,8 @@ static void print_disk_usage(off_t size)
 	strbuf_release(&sb);
 }
 
-static inline int parse_missing_action_value(const char *value)
+static inline int parse_missing_action_value(struct repository *repo,
+					     const char *value)
 {
 	if (!strcmp(value, "error")) {
 		arg_missing_action = MA_ERROR;
@@ -509,25 +510,25 @@ static inline int parse_missing_action_value(const char *value)
 
 	if (!strcmp(value, "allow-any")) {
 		arg_missing_action = MA_ALLOW_ANY;
-		fetch_if_missing = 0;
+		repo->fetch_if_missing = 0;
 		return 1;
 	}
 
 	if (!strcmp(value, "print")) {
 		arg_missing_action = MA_PRINT;
-		fetch_if_missing = 0;
+		repo->fetch_if_missing = 0;
 		return 1;
 	}
 
 	if (!strcmp(value, "print-info")) {
 		arg_missing_action = MA_PRINT_INFO;
-		fetch_if_missing = 0;
+		repo->fetch_if_missing = 0;
 		return 1;
 	}
 
 	if (!strcmp(value, "allow-promisor")) {
 		arg_missing_action = MA_ALLOW_PROMISOR;
-		fetch_if_missing = 0;
+		repo->fetch_if_missing = 0;
 		return 1;
 	}
 
@@ -692,7 +693,7 @@ static void prepare_maximal_independent(struct rev_info *revs)
 int cmd_rev_list(int argc,
 		 const char **argv,
 		 const char *prefix,
-		 struct repository *repo UNUSED)
+		 struct repository *repo)
 {
 	struct rev_info revs;
 	struct rev_list_info info;
@@ -745,10 +746,10 @@ int cmd_rev_list(int argc,
 	for (i = 1; i < argc; i++) {
 		const char *arg = argv[i];
 		if (!strcmp(arg, "--exclude-promisor-objects")) {
-			fetch_if_missing = 0;
+			repo->fetch_if_missing = 0;
 			revs.exclude_promisor_objects = 1;
 		} else if (skip_prefix(arg, "--missing=", &arg)) {
-			parse_missing_action_value(arg);
+			parse_missing_action_value(repo, arg);
 		} else if (!strcmp(arg, "-z")) {
 			line_term = '\0';
 			info_term = '\0';
common-init.c +1 -1
index d26c9c1f20..4a3fa4d7be 100644
@@ -47,7 +47,7 @@ static void setup_environment(void)
 	update_ref_namespace(NAMESPACE_REPLACE, git_replace_ref_base);
 
 	if (git_env_bool(NO_LAZY_FETCH_ENVIRONMENT, 0))
-		fetch_if_missing = 0;
+		the_repository->fetch_if_missing = 0;
 }
 
 void init_git(const char **argv)
git.c +1 -1
index e5f1811b6b..f3ad3aad96 100644
@@ -202,7 +202,7 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 			if (envchanged)
 				*envchanged = 1;
 		} else if (!strcmp(cmd, "--no-lazy-fetch")) {
-			fetch_if_missing = 0;
+			the_repository->fetch_if_missing = 0;
 			setenv(NO_LAZY_FETCH_ENVIRONMENT, "1", 1);
 			if (envchanged)
 				*envchanged = 1;
midx-write.c +1 -1
index 580724d21a..8537102254 100644
@@ -865,7 +865,7 @@ static void find_commits_for_midx_bitmap(struct commit_stack *commits,
 	 * complain later that we don't have reachability closure (and fail
 	 * appropriately).
 	 */
-	fetch_if_missing = 0;
+	ctx->repo->fetch_if_missing = 0;
 	revs.exclude_promisor_objects = 1;
 
 	if (prepare_revision_walk(&revs))
odb.c +1 -3
index dabd481f57..110326f063 100644
@@ -528,8 +528,6 @@ void disable_obj_read_lock(void)
 	pthread_mutex_destroy(&obj_read_mutex);
 }
 
-int fetch_if_missing = 1;
-
 static int register_all_submodule_sources(struct object_database *odb)
 {
 	int ret = odb->submodule_source_paths.nr;
@@ -595,7 +593,7 @@ static int do_oid_object_info_extended(struct object_database *odb,
 			continue;
 
 		/* Check if it is a missing object */
-		if (fetch_if_missing && repo_has_promisor_remote(odb->repo) &&
+		if (odb->repo->fetch_if_missing && repo_has_promisor_remote(odb->repo) &&
 		    !already_retried &&
 		    !(flags & OBJECT_INFO_SKIP_FETCH_OBJECT)) {
 			promisor_remote_get_direct(odb->repo, real, 1);
odb.h +0 -8
index cbc2f9ced4..d3a1e378b6 100644
@@ -15,14 +15,6 @@ struct repository;
 struct strbuf;
 struct strvec;
 
-/*
- * Set this to 0 to prevent odb_read_object_info_extended() from fetching missing
- * blobs. This has a difference only if extensions.partialClone is set.
- *
- * Its default value is 1.
- */
-extern int fetch_if_missing;
-
 /*
  * Compute the exact path an alternate is at and returns it. In case of
  * error NULL is returned and the human readable error is added to `err`
repository.c +1 -0
index 651b0f6933..c2d954cf83 100644
@@ -74,6 +74,7 @@ void initialize_repository(struct repository *repo)
 	index_state_init(repo->index, repo);
 	repo->check_deprecated_config = true;
 	repo->bare_cfg = -1;
+	repo->fetch_if_missing = 1;
 	repo_config_values_init(&repo->config_values_private_);
 
 	/*
repository.h +6 -0
index 3b467a2513..11f5c2ed10 100644
@@ -184,6 +184,12 @@ struct repository {
 	/* True if commit-graph has been disabled within this process. */
 	int commit_graph_disabled;
 
+	/*
+	 * Controls whether the repository should lazily fetch missing
+	 * objects from promisor remotes. Defaults to 1.
+	 */
+	int fetch_if_missing;
+
 	/*
 	 * Lazily-populated cache mapping hook event names to configured hooks.
 	 * NULL until first hook use.
revision.c +1 -1
index 526bcf3fb5..40cb1cc828 100644
@@ -2732,7 +2732,7 @@ static int handle_revision_opt(struct rev_info *revs, int argc, const char **arg
 		revs->ignore_missing = 1;
 	} else if (opt && opt->allow_exclude_promisor_objects &&
 		   !strcmp(arg, "--exclude-promisor-objects")) {
-		if (fetch_if_missing)
+		if (revs->repo->fetch_if_missing)
 			BUG("exclude_promisor_objects can only be used when fetch_if_missing is 0");
 		revs->exclude_promisor_objects = 1;
 	} else {
Signature · 1 line
2.43.0
#24 lore raw
Quoted 14 lines · Tian Yuchen

The global variable 'fetch_if_missing' controls whether a missing object check should prompt a lazy fetch from a promisor remote. In order to continue the libification effort, move it into 'struct repository' and initialize it to 1 by default to keep the previous behavior.

builtin/fetch-pack.c, builtin/fsck.c, and builtin/rev-list.c are entered via commands marked RUN_SETUP in git.c:commands[]. Their 'repo' parameter is only NULL when '-h' is given outside of a repository, in which case either show_usage_if_asked() or parse_options()'s own '-h' handling exits the process before returning. We can therefore drop their UNUSED markers and assign to 'repo' directly.

Quoted 4 lines

builtin/index-pack.c is entered via RUN_SETUP_GENTLY, so its 'repo' pointer can be NULL any time it is run outside of a repository, not only with '-h'. We keep a NULL check there and fall back to 'the_repository'.

Hmph, are there legitimate situations where we run "git index-pack" outside a repository, or is it a user error? Just being curious and not a suggestion to change the established behaviour (in other words, even if it turns out that it should be diagnosed as a user error, we shouldn't do so in the context of changing where the fetch_if_missing bit lives).

Other than that, this iteration looks good to me.

#25 lore raw
Quoted 21 lines · Junio C Hamano
Quoted 14 lines · Tian Yuchen

The global variable 'fetch_if_missing' controls whether a missing object check should prompt a lazy fetch from a promisor remote. In order to continue the libification effort, move it into 'struct repository' and initialize it to 1 by default to keep the previous behavior.

builtin/fetch-pack.c, builtin/fsck.c, and builtin/rev-list.c are entered via commands marked RUN_SETUP in git.c:commands[]. Their 'repo' parameter is only NULL when '-h' is given outside of a repository, in which case either show_usage_if_asked() or parse_options()'s own '-h' handling exits the process before returning. We can therefore drop their UNUSED markers and assign to 'repo' directly.

Quoted 4 lines

builtin/index-pack.c is entered via RUN_SETUP_GENTLY, so its 'repo' pointer can be NULL any time it is run outside of a repository, not only with '-h'. We keep a NULL check there and fall back to 'the_repository'.

Quoted 9 lines

Hmph, are there legitimate situations where we run "git index-pack" outside a repository, or is it a user error? Just being curious and not a suggestion to change the established behaviour (in other words, even if it turns out that it should be diagnosed as a user error, we shouldn't do so in the context of changing where the fetch_if_missing bit lives).

Other than that, this iteration looks good to me.

Interesting topic. Refer to the git-index-pack documentation:

...Reads a packed archive (.pack) from the specified file, builds a pack index file (.idx) for it, and optionally writes a reverse-index (.rev) for the specified pack. The packed archive, together with the pack index, *can then be placed in the objects/pack/ directory of a Git repository*...

Does this clarify that the design intent is to operate a standalone .pack file, and whether or not to put it into a repository are two different things?

Thanks! yuchen