Skip to contentSkip to conversation

Moving global state into the repository object (part 2)

PATCH In review Duy Nguyen 6 participants 135 messages Mar 3, 2018 – Mar 26, 2018 Page 1 of 2

lore
Share & tools
lore

Duy Nguyen

author
#0 lore raw
Quoted 12 lines · Junio C Hamano
Quoted 10 lines · Stefan Beller
Quoted 8 lines · Junio C Hamano
Quoted 4 lines · Duy Nguyen

Looking at the full-series diff though, it makes me wonder if we should keep prepare_packed_git() private (i.e. how we initialize the object store, packfile included, is a private matter). How about something like this on top?

Yup, that looks cleaner.

Quoted 7 lines
Quoted 2 lines

I agree that it looks cleaner. So we plan on just putting it on top of that series?

We tend to avoid "oops, that was wrong and here is a band aid on top" for things that are still mushy, so it would be preferrable to get it fixed inline, especially if there are more changes to the other parts of the series coming.

I agree with this. Stefan, if you're getting bored with rerolling refactor patches, I can update this series and send out v2.

Signature · 1 line
Duy
#1 lore raw

cover reroll nd/remove-ignore-env.. sb/object-store and sb/packfiles..

Quoted 16 lines · Duy Nguyen
Quoted 15 lines · Junio C Hamano
Quoted 12 lines · Stefan Beller
Quoted 8 lines · Junio C Hamano
Quoted 4 lines · Duy Nguyen

Looking at the full-series diff though, it makes me wonder if we should keep prepare_packed_git() private (i.e. how we initialize the object store, packfile included, is a private matter). How about something like this on top?

Yup, that looks cleaner.

I agree that it looks cleaner. So we plan on just putting it on top of that series?

Quoted 7 lines
Quoted 4 lines

We tend to avoid "oops, that was wrong and here is a band aid on top" for things that are still mushy, so it would be preferrable to get it fixed inline, especially if there are more changes to the other parts of the series coming.

I agree with this. Stefan, if you're getting bored with rerolling refactor patches, I can update this series and send out v2.

Since Stefan is traveling, I take this opportunity to reroll it. Unfortunately, I think the fix should go in 46cd557bd9 (object-store:
move packed_git and packed_git_mru to object store - 2018-02-23) where we start removing the global "packed_git". But that's in sb/object-store, so.. I'm rerolling all three

01/44 - 05/44: nd/remove-ignore-env-field

This series is moved up top. After this the patch that touch alternate-db in sha1_file.c looks natural because no env is involved anymore

I also take this opportunity to introduce a new patch 01/44 to avoid struct initialization that makes it hard to read and update. Later patches are also simplified thanks to this.

06/44 - 32/44: sb/object-store

06/44 is updated to introduce raw_object_store_init() instead of RAW_OBJECT_STORE_INIT macro. This function is now used to initialize both main repo and submodule ones.

10/44 (moving "packed_git") also introduces two new access wrapper get_packed_git() and get_packed_git_mru()

33/44 - 44/44: sb/packfiles-in-repository

The only thing new here is 44/44 which makes prepare_packed_git() internal. get_packed_git() and get_packed_git_mru() introduced earlier will call prepare_packed_git() automatically.

The whole thing is also available at

https://github.com/pclouds/git/tree/ignore-env-object-store-packfiles

And interdiff of all three, compared to what is currently in 'pu'. Looks pretty good in my opinon:

builtin/count-objects.c +2 -3
index d480301763..ee6ae35244 100644
@@ -121,9 +121,8 @@ int cmd_count_objects(int argc, const char **argv, const char *prefix)
 		struct strbuf loose_buf = STRBUF_INIT;
 		struct strbuf pack_buf = STRBUF_INIT;
 		struct strbuf garbage_buf = STRBUF_INIT;
-		if (!the_repository->objects.packed_git)
-			prepare_packed_git(the_repository);
-		for (p = the_repository->objects.packed_git; p; p = p->next) {
+
+		for (p = get_packed_git(the_repository); p; p = p->next) {
 			if (!p->pack_local)
 				continue;
 			if (open_pack_index(p))
builtin/fsck.c +2 -4
index 030c7fb7a0..9911c52bc8 100644
@@ -726,10 +726,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 			uint32_t total = 0, count = 0;
 			struct progress *progress = NULL;
 
-			prepare_packed_git(the_repository);
-
 			if (show_progress) {
-				for (p = the_repository->objects.packed_git; p;
+				for (p = get_packed_git(the_repository); p;
 				     p = p->next) {
 					if (open_pack_index(p))
 						continue;
@@ -738,7 +736,7 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 
 				progress = start_progress(_("Checking objects"), total);
 			}
-			for (p = the_repository->objects.packed_git; p;
+			for (p = get_packed_git(the_repository); p;
 			     p = p->next) {
 				/* verify gives error messages itself */
 				if (verify_pack(p, fsck_obj_buffer,
builtin/gc.c +1 -2
index 80d19c54d5..be63bec09c 100644
@@ -173,8 +173,7 @@ static int too_many_packs(void)
 	if (gc_auto_pack_limit <= 0)
 		return 0;
 
-	prepare_packed_git(the_repository);
-	for (cnt = 0, p = the_repository->objects.packed_git; p; p = p->next) {
+	for (cnt = 0, p = get_packed_git(the_repository); p; p = p->next) {
 		if (!p->pack_local)
 			continue;
 		if (p->pack_keep)
builtin/pack-objects.c +8 -9
index 142ec223cb..435f091a69 100644
@@ -1026,7 +1026,7 @@ static int want_object_in_pack(const struct object_id *oid,
 		if (want != -1)
 			return want;
 	}
-	list_for_each(pos, &the_repository->objects.packed_git_mru) {
+	list_for_each(pos, get_packed_git_mru(the_repository)) {
 		struct packed_git *p = list_entry(pos, struct packed_git, mru);
 		off_t offset;
 
@@ -1045,7 +1045,7 @@ static int want_object_in_pack(const struct object_id *oid,
 			want = want_found_object(exclude, p);
 			if (!exclude && want > 0)
 				list_move(&p->mru,
-					  &the_repository->objects.packed_git_mru);
+					  get_packed_git_mru(the_repository));
 			if (want != -1)
 				return want;
 		}
@@ -2674,7 +2674,7 @@ static void add_objects_in_unpacked_packs(struct rev_info *revs)
 
 	memset(&in_pack, 0, sizeof(in_pack));
 
-	for (p = the_repository->objects.packed_git; p; p = p->next) {
+	for (p = get_packed_git(the_repository); p; p = p->next) {
 		struct object_id oid;
 		struct object *o;
 
@@ -2738,7 +2738,7 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 	struct packed_git *p;
 
 	p = (last_found != (void *)1) ? last_found :
-					the_repository->objects.packed_git;
+					get_packed_git(the_repository);
 
 	while (p) {
 		if ((!p->pack_local || p->pack_keep) &&
@@ -2747,7 +2747,7 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 			return 1;
 		}
 		if (p == last_found)
-			p = the_repository->objects.packed_git;
+			p = get_packed_git(the_repository);
 		else
 			p = p->next;
 		if (p == last_found)
@@ -2783,7 +2783,7 @@ static void loosen_unused_packed_objects(struct rev_info *revs)
 	uint32_t i;
 	struct object_id oid;
 
-	for (p = the_repository->objects.packed_git; p; p = p->next) {
+	for (p = get_packed_git(the_repository); p; p = p->next) {
 		if (!p->pack_local || p->pack_keep)
 			continue;
 
@@ -3151,10 +3151,9 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 	if (progress && all_progress_implied)
 		progress = 2;
 
-	prepare_packed_git(the_repository);
 	if (ignore_packed_keep) {
 		struct packed_git *p;
-		for (p = the_repository->objects.packed_git; p; p = p->next)
+		for (p = get_packed_git(the_repository); p; p = p->next)
 			if (p->pack_local && p->pack_keep)
 				break;
 		if (!p) /* no keep-able packs found */
@@ -3167,7 +3166,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 		 * also covers non-local objects
 		 */
 		struct packed_git *p;
-		for (p = the_repository->objects.packed_git; p; p = p->next) {
+		for (p = get_packed_git(the_repository); p; p = p->next) {
 			if (!p->pack_local) {
 				have_non_local_packs = 1;
 				break;
builtin/pack-redundant.c +2 -4
index 3b02f94248..02b5f0becc 100644
@@ -572,7 +572,7 @@ static struct pack_list * add_pack(struct packed_git *p)
 
 static struct pack_list * add_pack_file(const char *filename)
 {
-	struct packed_git *p = the_repository->objects.packed_git;
+	struct packed_git *p = get_packed_git(the_repository);
 
 	if (strlen(filename) < 40)
 		die("Bad pack filename: %s", filename);
@@ -587,7 +587,7 @@ static struct pack_list * add_pack_file(const char *filename)
 
 static void load_all(void)
 {
-	struct packed_git *p = the_repository->objects.packed_git;
+	struct packed_git *p = get_packed_git(the_repository);
 
 	while (p) {
 		add_pack(p);
@@ -630,8 +630,6 @@ int cmd_pack_redundant(int argc, const char **argv, const char *prefix)
 			break;
 	}
 
-	prepare_packed_git(the_repository);
-
 	if (load_all_packs)
 		load_all();
 	else
common-main.c +2 -0
index 6a689007e7..7d716d5a54 100644
@@ -34,6 +34,8 @@ int main(int argc, const char **argv)
 
 	git_setup_gettext();
 
+	initialize_the_repository();
+
 	attr_start();
 
 	git_extract_argv0_path(argv[0]);
environment.c +1 -1
index b2128c1188..c05705e384 100644
@@ -13,8 +13,8 @@
 #include "refs.h"
 #include "fmt-merge-msg.h"
 #include "commit.h"
-#include "object-store.h"
 #include "argv-array.h"
+#include "object-store.h"
 
 int trust_executable_bit = 1;
 int trust_ctime = 1;
fast-import.c +2 -3
index d2032e4ade..2298bfcdfd 100644
@@ -1112,7 +1112,7 @@ static int store_object(
 		duplicate_count_by_type[type]++;
 		return 1;
 	} else if (find_sha1_pack(oid.hash,
-				  the_repository->objects.packed_git)) {
+				  get_packed_git(the_repository))) {
 		e->type = type;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
@@ -1308,7 +1308,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
 		truncate_pack(&checkpoint);
 
 	} else if (find_sha1_pack(oid.hash,
-				  the_repository->objects.packed_git)) {
+				  get_packed_git(the_repository))) {
 		e->type = OBJ_BLOB;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
@@ -3472,7 +3472,6 @@ int cmd_main(int argc, const char **argv)
 		rc_free[i].next = &rc_free[i + 1];
 	rc_free[cmd_save - 1].next = NULL;
 
-	prepare_packed_git(the_repository);
 	start_packfile();
 	set_die_routine(die_nicely);
 	set_checkpoint_signal();
http-backend.c +2 -3
index defa6ba350..22d2e1668e 100644
@@ -518,14 +518,13 @@ static void get_info_packs(struct strbuf *hdr, char *arg)
 	size_t cnt = 0;
 
 	select_getanyfile(hdr);
-	prepare_packed_git(the_repository);
-	for (p = the_repository->objects.packed_git; p; p = p->next) {
+	for (p = get_packed_git(the_repository); p; p = p->next) {
 		if (p->pack_local)
 			cnt++;
 	}
 
 	strbuf_grow(&buf, cnt * 53 + 2);
-	for (p = the_repository->objects.packed_git; p; p = p->next) {
+	for (p = get_packed_git(the_repository); p; p = p->next) {
 		if (p->pack_local)
 			strbuf_addf(&buf, "P %s\n", p->pack_name + objdirlen + 6);
 	}
object-store.h +3 -4
index 9b1303549b..521f5a1755 100644
@@ -90,9 +90,9 @@ struct raw_object_store {
 	/* Path to extra alternate object database if not NULL */
 	char *alternate_db;
 
-	struct packed_git *packed_git;
+	struct packed_git *packed_git; /* private */
 	/* A most-recently-used ordered version of the packed_git list. */
-	struct list_head packed_git_mru;
+	struct list_head packed_git_mru; /* private */
 
 	struct alternate_object_database *alt_odb_list;
 	struct alternate_object_database **alt_odb_tail;
@@ -112,8 +112,7 @@ struct raw_object_store {
 	unsigned packed_git_initialized : 1;
 };
 
-#define RAW_OBJECT_STORE_INIT(o) { NULL, NULL, NULL, LIST_HEAD_INIT(o.packed_git_mru), NULL, NULL, 0, 0, 0 }
-
+void raw_object_store_init(struct raw_object_store *o);
 void raw_object_store_clear(struct raw_object_store *o);
 
 /*
object.c +6 -0
index 5317cfc390..60ca76b285 100644
@@ -447,6 +447,12 @@ void clear_commit_marks_all(unsigned int flags)
 	}
 }
 
+void raw_object_store_init(struct raw_object_store *o)
+{
+	memset(o, 0, sizeof(*o));
+	INIT_LIST_HEAD(&o->packed_git_mru);
+}
+
 static void free_alt_odb(struct alternate_object_database *alt)
 {
 	strbuf_release(&alt->scratch);
pack-bitmap.c +1 -2
index bcc04bc45e..2a007b5539 100644
@@ -335,8 +335,7 @@ static int open_pack_bitmap(void)
 
 	assert(!bitmap_git.map && !bitmap_git.loaded);
 
-	prepare_packed_git(the_repository);
-	for (p = the_repository->objects.packed_git; p; p = p->next) {
+	for (p = get_packed_git(the_repository); p; p = p->next) {
 		if (open_pack_bitmap_1(p) == 0)
 			ret = 0;
 	}
packfile.c +14 -1
index 1a045157d0..773cd99a13 100644
@@ -802,6 +802,7 @@ static void prepare_packed_git_one(struct repository *r, char *objdir, int local
 	strbuf_release(&path);
 }
 
+static void prepare_packed_git(struct repository *r);
 /*
  * Give a fast, rough count of the number of objects in the repository. This
  * ignores loose objects completely. If you have a lot of them, then either
@@ -882,7 +883,7 @@ static void prepare_packed_git_mru(struct repository *r)
 		list_add_tail(&p->mru, &r->objects.packed_git_mru);
 }
 
-void prepare_packed_git(struct repository *r)
+static void prepare_packed_git(struct repository *r)
 {
 	struct alternate_object_database *alt;
 
@@ -904,6 +905,18 @@ void reprepare_packed_git(struct repository *r)
 	prepare_packed_git(r);
 }
 
+struct packed_git *get_packed_git(struct repository *r)
+{
+	prepare_packed_git(r);
+	return r->objects.packed_git;
+}
+
+struct list_head *get_packed_git_mru(struct repository *r)
+{
+	prepare_packed_git(r);
+	return &r->objects.packed_git_mru;
+}
+
 unsigned long unpack_object_header_buffer(const unsigned char *buf,
 		unsigned long len, enum object_type *type, unsigned long *sizep)
 {
packfile.h +3 -1
index 6f7b9e91d6..efda10329c 100644
@@ -34,10 +34,12 @@ extern struct packed_git *parse_pack_index(unsigned char *sha1, const char *idx_
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-extern void prepare_packed_git(struct repository *r);
 extern void reprepare_packed_git(struct repository *r);
 extern void install_packed_git(struct repository *r, struct packed_git *pack);
 
+struct packed_git *get_packed_git(struct repository *r);
+struct list_head *get_packed_git_mru(struct repository *r);
+
 /*
  * Give a rough count of objects in the repository. This sacrifices accuracy
  * for speed.
repository.c +15 -12
index 4f44384dde..34c0a7f180 100644
@@ -5,16 +5,17 @@
 #include "submodule-config.h"
 
 /* The main repository */
-static struct repository the_repo = {
-	NULL, NULL,
-	RAW_OBJECT_STORE_INIT(the_repo.objects),
-	NULL, NULL, NULL,
-	NULL, NULL, NULL,
-	&the_index,
-	&hash_algos[GIT_HASH_SHA1],
-	0
-};
-struct repository *the_repository = &the_repo;
+static struct repository the_repo;
+struct repository *the_repository;
+
+void initialize_the_repository(void)
+{
+	the_repository = &the_repo;
+
+	the_repo.index = &the_index;
+	raw_object_store_init(&the_repo.objects);
+	repo_set_hash_algo(&the_repo, GIT_HASH_SHA1);
+}
 
 static void expand_base_dir(char **out, const char *in,
 			    const char *base_dir, const char *def_in)
@@ -134,12 +135,14 @@ static int read_and_verify_repository_format(struct repository_format *format,
  * Initialize 'repo' based on the provided 'gitdir'.
  * Return 0 upon success and a non-zero value upon failure.
  */
-int repo_init(struct repository *repo, const char *gitdir, const char *worktree)
+static int repo_init(struct repository *repo,
+		     const char *gitdir,
+		     const char *worktree)
 {
 	struct repository_format format;
 	memset(repo, 0, sizeof(*repo));
 
-	INIT_LIST_HEAD(&repo->objects.packed_git_mru);
+	raw_object_store_init(&repo->objects);
 
 	if (repo_init_gitdir(repo, gitdir))
 		goto error;
repository.h +1 -1
index e05a77a099..6c383c05c6 100644
@@ -93,7 +93,7 @@ extern void repo_set_gitdir(struct repository *repo,
 			    const struct set_gitdir_args *optional);
 extern void repo_set_worktree(struct repository *repo, const char *path);
 extern void repo_set_hash_algo(struct repository *repo, int algo);
-extern int repo_init(struct repository *repo, const char *gitdir, const char *worktree);
+extern void initialize_the_repository(void);
 extern int repo_submodule_init(struct repository *submodule,
 			       struct repository *superproject,
 			       const char *path);
server-info.c +2 -3
index f5e4d1cc89..af737acd24 100644
@@ -200,8 +200,7 @@ static void init_pack_info(const char *infofile, int force)
 	objdir = get_object_directory();
 	objdirlen = strlen(objdir);
 
-	prepare_packed_git(the_repository);
-	for (p = the_repository->objects.packed_git; p; p = p->next) {
+	for (p = get_packed_git(the_repository); p; p = p->next) {
 		/* we ignore things on alternate path since they are
 		 * not available to the pullers in general.
 		 */
@@ -211,7 +210,7 @@ static void init_pack_info(const char *infofile, int force)
 	}
 	num_pack = i;
 	info = xcalloc(num_pack, sizeof(struct pack_info *));
-	for (i = 0, p = the_repository->objects.packed_git; p; p = p->next) {
+	for (i = 0, p = get_packed_git(the_repository); p; p = p->next) {
 		if (!p->pack_local)
 			continue;
 		info[i] = xcalloc(1, sizeof(struct pack_info));
sha1_name.c +2 -4
index 773322ac81..f25a0970ca 100644
@@ -195,8 +195,7 @@ static void find_short_packed_object(struct disambiguate_state *ds)
 {
 	struct packed_git *p;
 
-	prepare_packed_git(the_repository);
-	for (p = the_repository->objects.packed_git; p && !ds->ambiguous;
+	for (p = get_packed_git(the_repository); p && !ds->ambiguous;
 	     p = p->next)
 		unique_in_pack(p, ds);
 }
@@ -566,8 +565,7 @@ static void find_abbrev_len_packed(struct min_abbrev_data *mad)
 {
 	struct packed_git *p;
 
-	prepare_packed_git(the_repository);
-	for (p = the_repository->objects.packed_git; p; p = p->next)
+	for (p = get_packed_git(the_repository); p; p = p->next)
 		find_abbrev_len_for_pack(p, mad);
 }
 
Signature · 1 line
2.16.1.435.g8f24da2e1a
#2 lore raw

PATCH 1/44 repository: initialize the_repository in main()

This simplifies initialization of struct repository and anything inside. Easier to read. Easier to add/remove fields.

Everything will go through main() common-main.c so this should cover all programs, including t/helper.

---
 common-main.c |  2 ++
 repository.c  | 18 +++++++++++++-----
 repository.h  |  2 +-
 3 files changed, 16 insertions(+), 6 deletions(-)
common-main.c +2 -0
index 6a689007e7..7d716d5a54 100644
@@ -34,6 +34,8 @@ int main(int argc, const char **argv)
 
 	git_setup_gettext();
 
+	initialize_the_repository();
+
 	attr_start();
 
 	git_extract_argv0_path(argv[0]);
repository.c +13 -5
index 4ffbe9bc94..0eddf28fcd 100644
@@ -4,10 +4,16 @@
 #include "submodule-config.h"
 
 /* The main repository */
-static struct repository the_repo = {
-	NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &the_index, &hash_algos[GIT_HASH_SHA1], 0, 0
-};
-struct repository *the_repository = &the_repo;
+static struct repository the_repo;
+struct repository *the_repository;
+
+void initialize_the_repository(void)
+{
+	the_repository = &the_repo;
+
+	the_repo.index = &the_index;
+	repo_set_hash_algo(&the_repo, GIT_HASH_SHA1);
+}
 
 static char *git_path_from_env(const char *envvar, const char *git_dir,
 			       const char *path, int fromenv)
@@ -128,7 +134,9 @@ static int read_and_verify_repository_format(struct repository_format *format,
  * Initialize 'repo' based on the provided 'gitdir'.
  * Return 0 upon success and a non-zero value upon failure.
  */
-int repo_init(struct repository *repo, const char *gitdir, const char *worktree)
+static int repo_init(struct repository *repo,
+		     const char *gitdir,
+		     const char *worktree)
 {
 	struct repository_format format;
 	memset(repo, 0, sizeof(*repo));
repository.h +1 -1
index 0329e40c7f..40c1c81bdc 100644
@@ -91,7 +91,7 @@ extern struct repository *the_repository;
 extern void repo_set_gitdir(struct repository *repo, const char *path);
 extern void repo_set_worktree(struct repository *repo, const char *path);
 extern void repo_set_hash_algo(struct repository *repo, int algo);
-extern int repo_init(struct repository *repo, const char *gitdir, const char *worktree);
+extern void initialize_the_repository(void);
 extern int repo_submodule_init(struct repository *submodule,
 			       struct repository *superproject,
 			       const char *path);
Signature · 1 line
2.16.1.435.g8f24da2e1a
#3 lore raw

Re: [PATCH 01/44] repository: initialize the_repository in main()

Quoted 38 lines · Nguyễn Thái Ngọc Duy

This simplifies initialization of struct repository and anything inside. Easier to read. Easier to add/remove fields.

Everything will go through main() common-main.c so this should cover all programs, including t/helper.

---
 common-main.c |  2 ++
 repository.c  | 18 +++++++++++++-----
 repository.h  |  2 +-
 3 files changed, 16 insertions(+), 6 deletions(-)
common-main.c +2 -0
index 6a689007e7..7d716d5a54 100644
@@ -34,6 +34,8 @@ int main(int argc, const char **argv)
 
 	git_setup_gettext();
 
+	initialize_the_repository();
+
 	attr_start();
 
 	git_extract_argv0_path(argv[0]);
repository.c +0 -4
index 4ffbe9bc94..0eddf28fcd 100644
@@ -4,10 +4,16 @@
 #include "submodule-config.h"
 
 /* The main repository */
-static struct repository the_repo = {
-	NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &the_index, &hash_algos[GIT_HASH_SHA1], 0, 0
-};
-struct repository *the_repository = &the_repo;

repository.c line 16

Quoted 10 lines
repository.c +10 -0
+static struct repository the_repo;
+struct repository *the_repository;
+
+void initialize_the_repository(void)
+{
+	the_repository = &the_repo;
+
+	the_repo.index = &the_index;
+	repo_set_hash_algo(&the_repo, GIT_HASH_SHA1);
+}

Nice, one place to do all the crazy initialization for the_repo. I wanted to do something like this when I introduced the repository but I shied away from it because it didn't seem like the right thing to do at the time. Now that its grown a bit, its definitely the right move until using the repository object is ubiquitous enough that the setup code can produce a repository struct instead of relying on a global one.

Quoted 19 lines
repository.c +3 -1
 static char *git_path_from_env(const char *envvar, const char *git_dir,
 			       const char *path, int fromenv)
@@ -128,7 +134,9 @@ static int read_and_verify_repository_format(struct repository_format *format,
  * Initialize 'repo' based on the provided 'gitdir'.
  * Return 0 upon success and a non-zero value upon failure.
  */
-int repo_init(struct repository *repo, const char *gitdir, const char *worktree)
+static int repo_init(struct repository *repo,
+		     const char *gitdir,
+		     const char *worktree)
 {
 	struct repository_format format;
 	memset(repo, 0, sizeof(*repo));
repository.h +0 -0
index 0329e40c7f..40c1c81bdc 100644

repository.h extern struct repository *the_repository; · line 94

Quoted 4 lines
repository.h +0 -1
@@ -91,7 +91,7 @@ extern struct repository *the_repository;
 extern void repo_set_gitdir(struct repository *repo, const char *path);
 extern void repo_set_worktree(struct repository *repo, const char *path);
 extern void repo_set_hash_algo(struct repository *repo, int algo);
-extern int repo_init(struct repository *repo, const char *gitdir, const char *worktree);

I had to double check that this was ok, it indeed is as repo_init is only used in repository.c.

Quoted 7 lines
repository.h +1 -0
+extern void initialize_the_repository(void);
 extern int repo_submodule_init(struct repository *submodule,
 			       struct repository *superproject,
 			       const char *path);
Signature · 1 line
2.16.1.435.g8f24da2e1a
Signature · 1 line
Brandon Williams
#4 lore raw

PATCH 2/44 repository.c: move env-related setup code back to environment.c

In reply to Nguyễn Thái Ngọc Duy

It does not make sense that generic repository code contains handling of environment variables, which are specific for the main repository only. Refactor repo_set_gitdir() function to take $GIT_DIR and optionally _all_ other customizable paths. These optional paths can be NULL and will be calculated according to the default directory layout.

Note that some dead functions are left behind to reduce diff noise. They will be deleted in the next patch.

---
 cache.h       |  2 +-
 environment.c | 30 +++++++++++++++++++++++---
 repository.c  | 58 +++++++++++++++++++++++++++++++++++----------------
 repository.h  | 11 +++++++++-
 setup.c       |  3 +--
 5 files changed, 79 insertions(+), 25 deletions(-)
cache.h +1 -1
index 9cac7bb518..41ba67cc16 100644
@@ -484,7 +484,7 @@ static inline enum object_type object_type(unsigned int mode)
  */
 extern const char * const local_repo_env[];
 
-extern void setup_git_env(void);
+extern void setup_git_env(const char *git_dir);
 
 /*
  * Returns true iff we have a configured git repository (either via
environment.c +27 -3
index de8431e01e..da3f7daa09 100644
@@ -13,6 +13,7 @@
 #include "refs.h"
 #include "fmt-merge-msg.h"
 #include "commit.h"
+#include "argv-array.h"
 
 int trust_executable_bit = 1;
 int trust_ctime = 1;
@@ -147,10 +148,34 @@ static char *expand_namespace(const char *raw_namespace)
 	return strbuf_detach(&buf, NULL);
 }
 
-void setup_git_env(void)
+/*
+ * Wrapper of getenv() that returns a strdup value. This value is kept
+ * in argv to be freed later.
+ */
+static const char *getenv_safe(struct argv_array *argv, const char *name)
+{
+	const char *value = getenv(name);
+
+	if (!value)
+		return NULL;
+
+	argv_array_push(argv, value);
+	return argv->argv[argv->argc - 1];
+}
+
+void setup_git_env(const char *git_dir)
 {
 	const char *shallow_file;
 	const char *replace_ref_base;
+	struct set_gitdir_args args = { NULL };
+	struct argv_array to_free = ARGV_ARRAY_INIT;
+
+	args.commondir = getenv_safe(&to_free, GIT_COMMON_DIR_ENVIRONMENT);
+	args.object_dir = getenv_safe(&to_free, DB_ENVIRONMENT);
+	args.graft_file = getenv_safe(&to_free, GRAFT_ENVIRONMENT);
+	args.index_file = getenv_safe(&to_free, INDEX_ENVIRONMENT);
+	repo_set_gitdir(the_repository, git_dir, &args);
+	argv_array_clear(&to_free);
 
 	if (getenv(NO_REPLACE_OBJECTS_ENVIRONMENT))
 		check_replace_refs = 0;
@@ -300,8 +325,7 @@ int set_git_dir(const char *path)
 {
 	if (setenv(GIT_DIR_ENVIRONMENT, path, 1))
 		return error("Could not set GIT_DIR to '%s'", path);
-	repo_set_gitdir(the_repository, path);
-	setup_git_env();
+	setup_git_env(path);
 	return 0;
 }
 
repository.c +40 -18

87 lines not on this page · raw message

repository.h +10 -1
index 40c1c81bdc..84aeac2825 100644
@@ -88,7 +88,16 @@ struct repository {
 
 extern struct repository *the_repository;
 
-extern void repo_set_gitdir(struct repository *repo, const char *path);
+struct set_gitdir_args {
+	const char *commondir;
+	const char *object_dir;
+	const char *graft_file;
+	const char *index_file;
+};
+
+extern void repo_set_gitdir(struct repository *repo,
+			    const char *root,
+			    const struct set_gitdir_args *optional);
 extern void repo_set_worktree(struct repository *repo, const char *path);
 extern void repo_set_hash_algo(struct repository *repo, int algo);
 extern void initialize_the_repository(void);
setup.c +1 -2
index c5d55dcee4..6fac1bb58a 100644
@@ -1116,8 +1116,7 @@ const char *setup_git_directory_gently(int *nongit_ok)
 			const char *gitdir = getenv(GIT_DIR_ENVIRONMENT);
 			if (!gitdir)
 				gitdir = DEFAULT_GIT_DIR_ENVIRONMENT;
-			repo_set_gitdir(the_repository, gitdir);
-			setup_git_env();
+			setup_git_env(gitdir);
 		}
 		if (startup_info->have_repository)
 			repo_set_hash_algo(the_repository, repo_fmt.hash_algo);
Signature · 1 line
2.16.1.435.g8f24da2e1a
#5 lore raw

Re: [PATCH 02/44] repository.c: move env-related setup code back to environment.c

Quoted 10 lines · On Sat, 3 Mar 2018 18:35:55 +0700 Nguyễn Thái Ngọc Duy <[email protected]>…

It does not make sense that generic repository code contains handling of environment variables, which are specific for the main repository only. Refactor repo_set_gitdir() function to take $GIT_DIR and optionally _all_ other customizable paths. These optional paths can be NULL and will be calculated according to the default directory layout.

Note that some dead functions are left behind to reduce diff noise. They will be deleted in the next patch.

Thanks - I've reviewed up to this patch, and patches 1 and 2 look good.

repository.h struct repository { · line 100

Quoted 11 lines
repository.h +10 -1
-extern void repo_set_gitdir(struct repository *repo, const char *path);
+struct set_gitdir_args {
+	const char *commondir;
+	const char *object_dir;
+	const char *graft_file;
+	const char *index_file;
+};
+
+extern void repo_set_gitdir(struct repository *repo,
+			    const char *root,
+			    const struct set_gitdir_args *optional);

Optional: Reading this header file alone makes me think that the 3rd argument can be NULL, but it actually can't. I would name it "extra_args" and add a comment inside "struct set_gitdir_args" explaining (e.g.):

/*
* Any of these fields may be NULL. If so, the name of that directory
* is instead derived from the root path of the repository. */

author
#6 lore raw

Re: [PATCH 02/44] repository.c: move env-related setup code back to environment.c

Quoted 17 lines · Jonathan Tan

repository.h struct repository { · line 100

Quoted 11 lines
repository.h +10 -1
-extern void repo_set_gitdir(struct repository *repo, const char *path);
+struct set_gitdir_args {
+     const char *commondir;
+     const char *object_dir;
+     const char *graft_file;
+     const char *index_file;
+};
+
+extern void repo_set_gitdir(struct repository *repo,
+                         const char *root,
+                         const struct set_gitdir_args *optional);

Optional: Reading this header file alone makes me think that the 3rd argument can be NULL, but it actually can't. I would name it "extra_args" and add a comment inside "struct set_gitdir_args" explaining (e.g.):

Quoted 4 lines

/*
* Any of these fields may be NULL. If so, the name of that directory
* is instead derived from the root path of the repository. */

Yeah. I think Eric made the same comment. I'm still (very slowly) in the process of unifying the repo setup for the main repo and submodules, which hopefully may kill this function or replace it with something better. But it's too early to tell. Since this part of the series has landed in 'next', I'll post a fixup patch soon with your suggestion.

Signature · 1 line
Duy
#7 lore raw

Re: [PATCH 02/44] repository.c: move env-related setup code back to environment.c

Quoted 23 lines · Duy Nguyen
Quoted 17 lines · Jonathan Tan

repository.h struct repository { · line 100

Quoted 11 lines
repository.h +10 -1
-extern void repo_set_gitdir(struct repository *repo, const char *path);
+struct set_gitdir_args {
+     const char *commondir;
+     const char *object_dir;
+     const char *graft_file;
+     const char *index_file;
+};
+
+extern void repo_set_gitdir(struct repository *repo,
+                         const char *root,
+                         const struct set_gitdir_args *optional);

Optional: Reading this header file alone makes me think that the 3rd argument can be NULL, but it actually can't. I would name it "extra_args" and add a comment inside "struct set_gitdir_args" explaining (e.g.):

Quoted 4 lines

/*
* Any of these fields may be NULL. If so, the name of that directory
* is instead derived from the root path of the repository. */

Quoted 8 lines

Yeah. I think Eric made the same comment. I'm still (very slowly) in the process of unifying the repo setup for the main repo and submodules, which hopefully may kill this function or replace it with something better. But it's too early to tell. Since this part of the series has landed in 'next', I'll post a fixup patch soon with your suggestion.

Signature · 1 line
Duy

Yeah looking at the patch this is probably my only complaint. The rest looked good.

Signature · 1 line
Brandon Williams
#8 lore raw

PATCH 3/44 repository.c: delete dead functions

In reply to Nguyễn Thái Ngọc Duy
---
 repository.c | 25 -------------------------
 1 file changed, 25 deletions(-)
repository.c +0 -25
index bb53b54b6d..e65f4138a7 100644
@@ -15,31 +15,6 @@ void initialize_the_repository(void)
 	repo_set_hash_algo(&the_repo, GIT_HASH_SHA1);
 }
 
-static char *git_path_from_env(const char *envvar, const char *git_dir,
-			       const char *path, int fromenv)
-{
-	if (fromenv) {
-		const char *value = getenv(envvar);
-		if (value)
-			return xstrdup(value);
-	}
-
-	return xstrfmt("%s/%s", git_dir, path);
-}
-
-static int find_common_dir(struct strbuf *sb, const char *gitdir, int fromenv)
-{
-	if (fromenv) {
-		const char *value = getenv(GIT_COMMON_DIR_ENVIRONMENT);
-		if (value) {
-			strbuf_addstr(sb, value);
-			return 1;
-		}
-	}
-
-	return get_common_dir_noenv(sb, gitdir);
-}
-
 static void expand_base_dir(char **out, const char *in,
 			    const char *base_dir, const char *def_in)
 {
Signature · 1 line
2.16.1.435.g8f24da2e1a
#9 lore raw

Re: [PATCH 03/44] repository.c: delete dead functions

Quoted 45 lines · Nguyễn Thái Ngọc Duy
---
 repository.c | 25 -------------------------
 1 file changed, 25 deletions(-)
repository.c +0 -25
index bb53b54b6d..e65f4138a7 100644
@@ -15,31 +15,6 @@ void initialize_the_repository(void)
 	repo_set_hash_algo(&the_repo, GIT_HASH_SHA1);
 }
 
-static char *git_path_from_env(const char *envvar, const char *git_dir,
-			       const char *path, int fromenv)
-{
-	if (fromenv) {
-		const char *value = getenv(envvar);
-		if (value)
-			return xstrdup(value);
-	}
-
-	return xstrfmt("%s/%s", git_dir, path);
-}
-
-static int find_common_dir(struct strbuf *sb, const char *gitdir, int fromenv)
-{
-	if (fromenv) {
-		const char *value = getenv(GIT_COMMON_DIR_ENVIRONMENT);
-		if (value) {
-			strbuf_addstr(sb, value);
-			return 1;
-		}
-	}
-
-	return get_common_dir_noenv(sb, gitdir);
-}
-
 static void expand_base_dir(char **out, const char *in,
 			    const char *base_dir, const char *def_in)
 {
Signature · 1 line
2.16.1.435.g8f24da2e1a

This patch needs to be squashed into the previous one, the build breaks otherwise (unless built with -Werror=unused-function).

Signature · 1 line
Brandon Williams
#10 lore raw

PATCH 4/44 sha1_file.c: move delayed getenv(altdb) back to setup_git_env()

In reply to Nguyễn Thái Ngọc Duy

getenv() is supposed to work on the main repository only. This delayed getenv() code in sha1_file.c makes it more difficult to convert sha1_file.c to a generic object store that could be used by both submodule and main repositories.

Move the getenv() back in setup_git_env() where other env vars are also fetched.

---
 environment.c | 1 +
 repository.c  | 3 +++
 repository.h  | 4 ++++
 sha1_file.c   | 6 +-----
 4 files changed, 9 insertions(+), 5 deletions(-)
environment.c +1 -0
index da3f7daa09..a5eaa97fb1 100644
@@ -174,6 +174,7 @@ void setup_git_env(const char *git_dir)
 	args.object_dir = getenv_safe(&to_free, DB_ENVIRONMENT);
 	args.graft_file = getenv_safe(&to_free, GRAFT_ENVIRONMENT);
 	args.index_file = getenv_safe(&to_free, INDEX_ENVIRONMENT);
+	args.alternate_db = getenv_safe(&to_free, ALTERNATE_DB_ENVIRONMENT);
 	repo_set_gitdir(the_repository, git_dir, &args);
 	argv_array_clear(&to_free);
 
repository.c +3 -0
index e65f4138a7..04d85a2869 100644
@@ -60,6 +60,8 @@ void repo_set_gitdir(struct repository *repo,
 	repo_set_commondir(repo, o->commondir);
 	expand_base_dir(&repo->objectdir, o->object_dir,
 			repo->commondir, "objects");
+	free(repo->alternate_db);
+	repo->alternate_db = xstrdup_or_null(o->alternate_db);
 	expand_base_dir(&repo->graft_file, o->graft_file,
 			repo->commondir, "info/grafts");
 	expand_base_dir(&repo->index_file, o->index_file,
@@ -215,6 +217,7 @@ void repo_clear(struct repository *repo)
 	FREE_AND_NULL(repo->gitdir);
 	FREE_AND_NULL(repo->commondir);
 	FREE_AND_NULL(repo->objectdir);
+	FREE_AND_NULL(repo->alternate_db);
 	FREE_AND_NULL(repo->graft_file);
 	FREE_AND_NULL(repo->index_file);
 	FREE_AND_NULL(repo->worktree);
repository.h +4 -0
index 84aeac2825..2bfbf762f3 100644
@@ -26,6 +26,9 @@ struct repository {
 	 */
 	char *objectdir;
 
+	/* Path to extra alternate object database if not NULL */
+	char *alternate_db;
+
 	/*
 	 * Path to the repository's graft file.
 	 * Cannot be NULL after initialization.
@@ -93,6 +96,7 @@ struct set_gitdir_args {
 	const char *object_dir;
 	const char *graft_file;
 	const char *index_file;
+	const char *alternate_db;
 };
 
 extern void repo_set_gitdir(struct repository *repo,
sha1_file.c +1 -5
index 831d9e7343..4af422e3cf 100644
@@ -667,15 +667,11 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 
 void prepare_alt_odb(void)
 {
-	const char *alt;
-
 	if (alt_odb_tail)
 		return;
 
-	alt = getenv(ALTERNATE_DB_ENVIRONMENT);
-
 	alt_odb_tail = &alt_odb_list;
-	link_alt_odb_entries(alt, PATH_SEP, NULL, 0);
+	link_alt_odb_entries(the_repository->alternate_db, PATH_SEP, NULL, 0);
 
 	read_info_alternates(get_object_directory(), 0);
 }
Signature · 1 line
2.16.1.435.g8f24da2e1a
#11 lore raw

Re: [PATCH 04/44] sha1_file.c: move delayed getenv(altdb) back to setup_git_env()

Quoted 9 lines · Nguyễn Thái Ngọc Duy

getenv() is supposed to work on the main repository only. This delayed getenv() code in sha1_file.c makes it more difficult to convert sha1_file.c to a generic object store that could be used by both submodule and main repositories.

Move the getenv() back in setup_git_env() where other env vars are also fetched.

Looks good

Quoted 86 lines
---
 environment.c | 1 +
 repository.c  | 3 +++
 repository.h  | 4 ++++
 sha1_file.c   | 6 +-----
 4 files changed, 9 insertions(+), 5 deletions(-)
environment.c +1 -0
index da3f7daa09..a5eaa97fb1 100644
@@ -174,6 +174,7 @@ void setup_git_env(const char *git_dir)
 	args.object_dir = getenv_safe(&to_free, DB_ENVIRONMENT);
 	args.graft_file = getenv_safe(&to_free, GRAFT_ENVIRONMENT);
 	args.index_file = getenv_safe(&to_free, INDEX_ENVIRONMENT);
+	args.alternate_db = getenv_safe(&to_free, ALTERNATE_DB_ENVIRONMENT);
 	repo_set_gitdir(the_repository, git_dir, &args);
 	argv_array_clear(&to_free);
 
repository.c +3 -0
index e65f4138a7..04d85a2869 100644
@@ -60,6 +60,8 @@ void repo_set_gitdir(struct repository *repo,
 	repo_set_commondir(repo, o->commondir);
 	expand_base_dir(&repo->objectdir, o->object_dir,
 			repo->commondir, "objects");
+	free(repo->alternate_db);
+	repo->alternate_db = xstrdup_or_null(o->alternate_db);
 	expand_base_dir(&repo->graft_file, o->graft_file,
 			repo->commondir, "info/grafts");
 	expand_base_dir(&repo->index_file, o->index_file,
@@ -215,6 +217,7 @@ void repo_clear(struct repository *repo)
 	FREE_AND_NULL(repo->gitdir);
 	FREE_AND_NULL(repo->commondir);
 	FREE_AND_NULL(repo->objectdir);
+	FREE_AND_NULL(repo->alternate_db);
 	FREE_AND_NULL(repo->graft_file);
 	FREE_AND_NULL(repo->index_file);
 	FREE_AND_NULL(repo->worktree);
repository.h +4 -0
index 84aeac2825..2bfbf762f3 100644
@@ -26,6 +26,9 @@ struct repository {
 	 */
 	char *objectdir;
 
+	/* Path to extra alternate object database if not NULL */
+	char *alternate_db;
+
 	/*
 	 * Path to the repository's graft file.
 	 * Cannot be NULL after initialization.
@@ -93,6 +96,7 @@ struct set_gitdir_args {
 	const char *object_dir;
 	const char *graft_file;
 	const char *index_file;
+	const char *alternate_db;
 };
 
 extern void repo_set_gitdir(struct repository *repo,
sha1_file.c +1 -5
index 831d9e7343..4af422e3cf 100644
@@ -667,15 +667,11 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 
 void prepare_alt_odb(void)
 {
-	const char *alt;
-
 	if (alt_odb_tail)
 		return;
 
-	alt = getenv(ALTERNATE_DB_ENVIRONMENT);
-
 	alt_odb_tail = &alt_odb_list;
-	link_alt_odb_entries(alt, PATH_SEP, NULL, 0);
+	link_alt_odb_entries(the_repository->alternate_db, PATH_SEP, NULL, 0);
 
 	read_info_alternates(get_object_directory(), 0);
 }
Signature · 1 line
2.16.1.435.g8f24da2e1a
Signature · 1 line
Brandon Williams
#12 lore raw

PATCH 5/44 repository: delete ignore_env member

In reply to Nguyễn Thái Ngọc Duy

This variable was added because the repo_set_gitdir() was created to cover both submodule and main repos, but these two are initialized a bit differently so ignore_env == 0 means main repo, while ignore_env != 0 is submodules.

Since the difference part (env variables) has been moved out of repo_set_gitdir(), this function works the same way for both repo types and ignore_env is not needed anymore.

---
 repository.c | 2 --
 repository.h | 9 ---------
 2 files changed, 11 deletions(-)
repository.c +0 -2
index 04d85a2869..62f52f47fc 100644
@@ -140,8 +140,6 @@ static int repo_init(struct repository *repo,
 	struct repository_format format;
 	memset(repo, 0, sizeof(*repo));
 
-	repo->ignore_env = 1;
-
 	if (repo_init_gitdir(repo, gitdir))
 		goto error;
 
repository.h +0 -9
index 2bfbf762f3..e7127baffb 100644
@@ -75,15 +75,6 @@ struct repository {
 	const struct git_hash_algo *hash_algo;
 
 	/* Configurations */
-	/*
-	 * Bit used during initialization to indicate if repository state (like
-	 * the location of the 'objectdir') should be read from the
-	 * environment.  By default this bit will be set at the begining of
-	 * 'repo_init()' so that all repositories will ignore the environment.
-	 * The exception to this is 'the_repository', which doesn't go through
-	 * the normal 'repo_init()' process.
-	 */
-	unsigned ignore_env:1;
 
 	/* Indicate if a repository has a different 'commondir' from 'gitdir' */
 	unsigned different_commondir:1;
Signature · 1 line
2.16.1.435.g8f24da2e1a
#13 lore raw

Re: [PATCH 05/44] repository: delete ignore_env member

Quoted 10 lines · Nguyễn Thái Ngọc Duy

This variable was added because the repo_set_gitdir() was created to cover both submodule and main repos, but these two are initialized a bit differently so ignore_env == 0 means main repo, while ignore_env != 0 is submodules.

Since the difference part (env variables) has been moved out of repo_set_gitdir(), this function works the same way for both repo types and ignore_env is not needed anymore.

Sweet! Thanks for getting rid of this.

Quoted 41 lines
---
 repository.c | 2 --
 repository.h | 9 ---------
 2 files changed, 11 deletions(-)
repository.c +0 -2
index 04d85a2869..62f52f47fc 100644
@@ -140,8 +140,6 @@ static int repo_init(struct repository *repo,
 	struct repository_format format;
 	memset(repo, 0, sizeof(*repo));
 
-	repo->ignore_env = 1;
-
 	if (repo_init_gitdir(repo, gitdir))
 		goto error;
 
repository.h +0 -9
index 2bfbf762f3..e7127baffb 100644
@@ -75,15 +75,6 @@ struct repository {
 	const struct git_hash_algo *hash_algo;
 
 	/* Configurations */
-	/*
-	 * Bit used during initialization to indicate if repository state (like
-	 * the location of the 'objectdir') should be read from the
-	 * environment.  By default this bit will be set at the begining of
-	 * 'repo_init()' so that all repositories will ignore the environment.
-	 * The exception to this is 'the_repository', which doesn't go through
-	 * the normal 'repo_init()' process.
-	 */
-	unsigned ignore_env:1;
 
 	/* Indicate if a repository has a different 'commondir' from 'gitdir' */
 	unsigned different_commondir:1;
Signature · 1 line
2.16.1.435.g8f24da2e1a
Signature · 1 line
Brandon Williams
#14 lore raw

PATCH 6/44 repository: introduce raw object store field

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

The raw object store field will contain any objects needed for access to objects in a given repository.

This patch introduces the raw object store and populates it with the objectdir, which used to be part of the repository struct.

As the struct gains members, we'll also populate the function to clear the memory for these members.

In a later we'll introduce a struct object_parser, that will complement the object parsing in a repository struct: The raw object parser is the layer that will provide access to raw object content, while the higher level object parser code will parse raw objects and keeps track of parenthood and other object relationships using 'struct object'. For now only add the lower level to the repository struct.

---
 builtin/grep.c |  2 +-
 environment.c  |  5 +++--
 object-store.h | 18 ++++++++++++++++++
 object.c       | 10 ++++++++++
 path.c         |  2 +-
 repository.c   | 14 +++++++++-----
 repository.h   | 10 ++++------
 sha1_file.c    |  3 ++-
 8 files changed, 48 insertions(+), 16 deletions(-)
 create mode 100644 object-store.h
builtin/grep.c +1 -1
index 3ca4ac80d8..0f0c195705 100644
@@ -432,7 +432,7 @@ static int grep_submodule(struct grep_opt *opt, struct repository *superproject,
 	 * object.
 	 */
 	grep_read_lock();
-	add_to_alternates_memory(submodule.objectdir);
+	add_to_alternates_memory(submodule.objects.objectdir);
 	grep_read_unlock();
 
 	if (oid) {
environment.c +3 -2
index a5eaa97fb1..c05705e384 100644
@@ -14,6 +14,7 @@
 #include "fmt-merge-msg.h"
 #include "commit.h"
 #include "argv-array.h"
+#include "object-store.h"
 
 int trust_executable_bit = 1;
 int trust_ctime = 1;
@@ -270,9 +271,9 @@ const char *get_git_work_tree(void)
 
 char *get_object_directory(void)
 {
-	if (!the_repository->objectdir)
+	if (!the_repository->objects.objectdir)
 		BUG("git environment hasn't been setup");
-	return the_repository->objectdir;
+	return the_repository->objects.objectdir;
 }
 
 int odb_mkstemp(struct strbuf *template, const char *pattern)
object-store.h new file +18 -0
new file mode 100644
index 0000000000..69bb5ac065
@@ -0,0 +1,18 @@
+#ifndef OBJECT_STORE_H
+#define OBJECT_STORE_H
+
+struct raw_object_store {
+	/*
+	 * Path to the repository's object store.
+	 * Cannot be NULL after initialization.
+	 */
+	char *objectdir;
+
+	/* Path to extra alternate object database if not NULL */
+	char *alternate_db;
+};
+
+void raw_object_store_init(struct raw_object_store *o);
+void raw_object_store_clear(struct raw_object_store *o);
+
+#endif /* OBJECT_STORE_H */
object.c +10 -0
index 9e6f9ff20b..e91711dd56 100644
@@ -445,3 +445,13 @@ void clear_commit_marks_all(unsigned int flags)
 			obj->flags &= ~flags;
 	}
 }
+
+void raw_object_store_init(struct raw_object_store *o)
+{
+	memset(o, 0, sizeof(*o));
+}
+void raw_object_store_clear(struct raw_object_store *o)
+{
+	FREE_AND_NULL(o->objectdir);
+	FREE_AND_NULL(o->alternate_db);
+}
path.c +1 -1
index da8b655730..81a42d9115 100644
@@ -382,7 +382,7 @@ static void adjust_git_path(const struct repository *repo,
 		strbuf_splice(buf, 0, buf->len,
 			      repo->index_file, strlen(repo->index_file));
 	else if (dir_prefix(base, "objects"))
-		replace_dir(buf, git_dir_len + 7, repo->objectdir);
+		replace_dir(buf, git_dir_len + 7, repo->objects.objectdir);
 	else if (git_hooks_path && dir_prefix(base, "hooks"))
 		replace_dir(buf, git_dir_len + 5, git_hooks_path);
 	else if (repo->different_commondir)
repository.c +9 -5
index 62f52f47fc..34c0a7f180 100644
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "repository.h"
+#include "object-store.h"
 #include "config.h"
 #include "submodule-config.h"
 
@@ -12,6 +13,7 @@ void initialize_the_repository(void)
 	the_repository = &the_repo;
 
 	the_repo.index = &the_index;
+	raw_object_store_init(&the_repo.objects);
 	repo_set_hash_algo(&the_repo, GIT_HASH_SHA1);
 }
 
@@ -58,10 +60,10 @@ void repo_set_gitdir(struct repository *repo,
 	free(old_gitdir);
 
 	repo_set_commondir(repo, o->commondir);
-	expand_base_dir(&repo->objectdir, o->object_dir,
+	expand_base_dir(&repo->objects.objectdir, o->object_dir,
 			repo->commondir, "objects");
-	free(repo->alternate_db);
-	repo->alternate_db = xstrdup_or_null(o->alternate_db);
+	free(repo->objects.alternate_db);
+	repo->objects.alternate_db = xstrdup_or_null(o->alternate_db);
 	expand_base_dir(&repo->graft_file, o->graft_file,
 			repo->commondir, "info/grafts");
 	expand_base_dir(&repo->index_file, o->index_file,
@@ -140,6 +142,8 @@ static int repo_init(struct repository *repo,
 	struct repository_format format;
 	memset(repo, 0, sizeof(*repo));
 
+	raw_object_store_init(&repo->objects);
+
 	if (repo_init_gitdir(repo, gitdir))
 		goto error;
 
@@ -214,13 +218,13 @@ void repo_clear(struct repository *repo)
 {
 	FREE_AND_NULL(repo->gitdir);
 	FREE_AND_NULL(repo->commondir);
-	FREE_AND_NULL(repo->objectdir);
-	FREE_AND_NULL(repo->alternate_db);
 	FREE_AND_NULL(repo->graft_file);
 	FREE_AND_NULL(repo->index_file);
 	FREE_AND_NULL(repo->worktree);
 	FREE_AND_NULL(repo->submodule_prefix);
 
+	raw_object_store_clear(&repo->objects);
+
 	if (repo->config) {
 		git_configset_clear(repo->config);
 		FREE_AND_NULL(repo->config);
repository.h +4 -6
index e7127baffb..6c383c05c6 100644
@@ -1,6 +1,8 @@
 #ifndef REPOSITORY_H
 #define REPOSITORY_H
 
+#include "object-store.h"
+
 struct config_set;
 struct index_state;
 struct submodule_cache;
@@ -21,13 +23,9 @@ struct repository {
 	char *commondir;
 
 	/*
-	 * Path to the repository's object store.
-	 * Cannot be NULL after initialization.
+	 * Holds any information related to accessing the raw object content.
 	 */
-	char *objectdir;
-
-	/* Path to extra alternate object database if not NULL */
-	char *alternate_db;
+	struct raw_object_store objects;
 
 	/*
 	 * Path to the repository's graft file.
sha1_file.c +2 -1
index 4af422e3cf..792bb21c15 100644
@@ -671,7 +671,8 @@ void prepare_alt_odb(void)
 		return;
 
 	alt_odb_tail = &alt_odb_list;
-	link_alt_odb_entries(the_repository->alternate_db, PATH_SEP, NULL, 0);
+	link_alt_odb_entries(the_repository->objects.alternate_db,
+			     PATH_SEP, NULL, 0);
 
 	read_info_alternates(get_object_directory(), 0);
 }
Signature · 1 line
2.16.1.435.g8f24da2e1a
#15 lore raw

Re: [PATCH 06/44] repository: introduce raw object store field

Quoted 10 lines · Nguyễn Thái Ngọc Duy

The raw object store field will contain any objects needed for access to objects in a given repository.

This patch introduces the raw object store and populates it with the objectdir, which used to be part of the repository struct.

As the struct gains members, we'll also populate the function to clear the memory for these members.

In a later we'll introduce a struct object_parser, that will complement

s/In a later/Later,/

Quoted 9 lines

the object parsing in a repository struct: The raw object parser is the layer that will provide access to raw object content, while the higher level object parser code will parse raw objects and keeps track of parenthood and other object relationships using 'struct object'. For now only add the lower level to the repository struct.

#16 lore raw

Re: [PATCH 06/44] repository: introduce raw object store field

In reply to Nguyễn Thái Ngọc Duy
Quoted 11 lines · Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

The raw object store field will contain any objects needed for access to objects in a given repository.

This patch introduces the raw object store and populates it with the objectdir, which used to be part of the repository struct.

As the struct gains members, we'll also populate the function to clear the memory for these members.

Quoted 6 lines

In a later we'll introduce a struct object_parser, that will complement the object parsing in a repository struct: The raw object parser is the layer that will provide access to raw object content, while the higher level object parser code will parse raw objects and keeps track of parenthood and other object relationships using 'struct object'. For now only add the lower level to the repository struct.

Patch looks good. Only nit would be that I prefer the embedded struct to be a pointer simply because I don't have to worry about putting '&' at some points. I know others probably feel the opposite so its not worth changing for me :)

Quoted 212 lines
---
 builtin/grep.c |  2 +-
 environment.c  |  5 +++--
 object-store.h | 18 ++++++++++++++++++
 object.c       | 10 ++++++++++
 path.c         |  2 +-
 repository.c   | 14 +++++++++-----
 repository.h   | 10 ++++------
 sha1_file.c    |  3 ++-
 8 files changed, 48 insertions(+), 16 deletions(-)
 create mode 100644 object-store.h
builtin/grep.c +1 -1
index 3ca4ac80d8..0f0c195705 100644
@@ -432,7 +432,7 @@ static int grep_submodule(struct grep_opt *opt, struct repository *superproject,
 	 * object.
 	 */
 	grep_read_lock();
-	add_to_alternates_memory(submodule.objectdir);
+	add_to_alternates_memory(submodule.objects.objectdir);
 	grep_read_unlock();
 
 	if (oid) {
environment.c +3 -2
index a5eaa97fb1..c05705e384 100644
@@ -14,6 +14,7 @@
 #include "fmt-merge-msg.h"
 #include "commit.h"
 #include "argv-array.h"
+#include "object-store.h"
 
 int trust_executable_bit = 1;
 int trust_ctime = 1;
@@ -270,9 +271,9 @@ const char *get_git_work_tree(void)
 
 char *get_object_directory(void)
 {
-	if (!the_repository->objectdir)
+	if (!the_repository->objects.objectdir)
 		BUG("git environment hasn't been setup");
-	return the_repository->objectdir;
+	return the_repository->objects.objectdir;
 }
 
 int odb_mkstemp(struct strbuf *template, const char *pattern)
object-store.h new file +18 -0
new file mode 100644
index 0000000000..69bb5ac065
@@ -0,0 +1,18 @@
+#ifndef OBJECT_STORE_H
+#define OBJECT_STORE_H
+
+struct raw_object_store {
+	/*
+	 * Path to the repository's object store.
+	 * Cannot be NULL after initialization.
+	 */
+	char *objectdir;
+
+	/* Path to extra alternate object database if not NULL */
+	char *alternate_db;
+};
+
+void raw_object_store_init(struct raw_object_store *o);
+void raw_object_store_clear(struct raw_object_store *o);
+
+#endif /* OBJECT_STORE_H */
object.c +10 -0
index 9e6f9ff20b..e91711dd56 100644
@@ -445,3 +445,13 @@ void clear_commit_marks_all(unsigned int flags)
 			obj->flags &= ~flags;
 	}
 }
+
+void raw_object_store_init(struct raw_object_store *o)
+{
+	memset(o, 0, sizeof(*o));
+}
+void raw_object_store_clear(struct raw_object_store *o)
+{
+	FREE_AND_NULL(o->objectdir);
+	FREE_AND_NULL(o->alternate_db);
+}
path.c +1 -1
index da8b655730..81a42d9115 100644
@@ -382,7 +382,7 @@ static void adjust_git_path(const struct repository *repo,
 		strbuf_splice(buf, 0, buf->len,
 			      repo->index_file, strlen(repo->index_file));
 	else if (dir_prefix(base, "objects"))
-		replace_dir(buf, git_dir_len + 7, repo->objectdir);
+		replace_dir(buf, git_dir_len + 7, repo->objects.objectdir);
 	else if (git_hooks_path && dir_prefix(base, "hooks"))
 		replace_dir(buf, git_dir_len + 5, git_hooks_path);
 	else if (repo->different_commondir)
repository.c +9 -5
index 62f52f47fc..34c0a7f180 100644
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "repository.h"
+#include "object-store.h"
 #include "config.h"
 #include "submodule-config.h"
 
@@ -12,6 +13,7 @@ void initialize_the_repository(void)
 	the_repository = &the_repo;
 
 	the_repo.index = &the_index;
+	raw_object_store_init(&the_repo.objects);
 	repo_set_hash_algo(&the_repo, GIT_HASH_SHA1);
 }
 
@@ -58,10 +60,10 @@ void repo_set_gitdir(struct repository *repo,
 	free(old_gitdir);
 
 	repo_set_commondir(repo, o->commondir);
-	expand_base_dir(&repo->objectdir, o->object_dir,
+	expand_base_dir(&repo->objects.objectdir, o->object_dir,
 			repo->commondir, "objects");
-	free(repo->alternate_db);
-	repo->alternate_db = xstrdup_or_null(o->alternate_db);
+	free(repo->objects.alternate_db);
+	repo->objects.alternate_db = xstrdup_or_null(o->alternate_db);
 	expand_base_dir(&repo->graft_file, o->graft_file,
 			repo->commondir, "info/grafts");
 	expand_base_dir(&repo->index_file, o->index_file,
@@ -140,6 +142,8 @@ static int repo_init(struct repository *repo,
 	struct repository_format format;
 	memset(repo, 0, sizeof(*repo));
 
+	raw_object_store_init(&repo->objects);
+
 	if (repo_init_gitdir(repo, gitdir))
 		goto error;
 
@@ -214,13 +218,13 @@ void repo_clear(struct repository *repo)
 {
 	FREE_AND_NULL(repo->gitdir);
 	FREE_AND_NULL(repo->commondir);
-	FREE_AND_NULL(repo->objectdir);
-	FREE_AND_NULL(repo->alternate_db);
 	FREE_AND_NULL(repo->graft_file);
 	FREE_AND_NULL(repo->index_file);
 	FREE_AND_NULL(repo->worktree);
 	FREE_AND_NULL(repo->submodule_prefix);
 
+	raw_object_store_clear(&repo->objects);
+
 	if (repo->config) {
 		git_configset_clear(repo->config);
 		FREE_AND_NULL(repo->config);
repository.h +4 -6
index e7127baffb..6c383c05c6 100644
@@ -1,6 +1,8 @@
 #ifndef REPOSITORY_H
 #define REPOSITORY_H
 
+#include "object-store.h"
+
 struct config_set;
 struct index_state;
 struct submodule_cache;
@@ -21,13 +23,9 @@ struct repository {
 	char *commondir;
 
 	/*
-	 * Path to the repository's object store.
-	 * Cannot be NULL after initialization.
+	 * Holds any information related to accessing the raw object content.
 	 */
-	char *objectdir;
-
-	/* Path to extra alternate object database if not NULL */
-	char *alternate_db;
+	struct raw_object_store objects;
 
 	/*
 	 * Path to the repository's graft file.
sha1_file.c +2 -1
index 4af422e3cf..792bb21c15 100644
@@ -671,7 +671,8 @@ void prepare_alt_odb(void)
 		return;
 
 	alt_odb_tail = &alt_odb_list;
-	link_alt_odb_entries(the_repository->alternate_db, PATH_SEP, NULL, 0);
+	link_alt_odb_entries(the_repository->objects.alternate_db,
+			     PATH_SEP, NULL, 0);
 
 	read_info_alternates(get_object_directory(), 0);
 }
Signature · 1 line
2.16.1.435.g8f24da2e1a
Signature · 1 line
Brandon Williams
#17 lore raw

PATCH 7/44 object-store: migrate alternates struct and functions from cache.h

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Migrate the struct alternate_object_database and all its related functions to the object store as these functions are easier found in that header. The migration is just a verbatim copy, no need to include the object store header at any C file, because cache.h includes repository.h which in turn includes the object-store.h

---
 cache.h        | 51 --------------------------------------------------
 object-store.h | 51 ++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 51 insertions(+), 51 deletions(-)
cache.h +0 -51
index 41ba67cc16..41530d5d16 100644
@@ -1576,57 +1576,6 @@ extern int has_dirs_only_path(const char *name, int len, int prefix_len);
 extern void schedule_dir_for_removal(const char *name, int len);
 extern void remove_scheduled_dirs(void);
 
-extern struct alternate_object_database {
-	struct alternate_object_database *next;
-
-	/* see alt_scratch_buf() */
-	struct strbuf scratch;
-	size_t base_len;
-
-	/*
-	 * Used to store the results of readdir(3) calls when searching
-	 * for unique abbreviated hashes.  This cache is never
-	 * invalidated, thus it's racy and not necessarily accurate.
-	 * That's fine for its purpose; don't use it for tasks requiring
-	 * greater accuracy!
-	 */
-	char loose_objects_subdir_seen[256];
-	struct oid_array loose_objects_cache;
-
-	char path[FLEX_ARRAY];
-} *alt_odb_list;
-extern void prepare_alt_odb(void);
-extern char *compute_alternate_path(const char *path, struct strbuf *err);
-typedef int alt_odb_fn(struct alternate_object_database *, void *);
-extern int foreach_alt_odb(alt_odb_fn, void*);
-
-/*
- * Allocate a "struct alternate_object_database" but do _not_ actually
- * add it to the list of alternates.
- */
-struct alternate_object_database *alloc_alt_odb(const char *dir);
-
-/*
- * Add the directory to the on-disk alternates file; the new entry will also
- * take effect in the current process.
- */
-extern void add_to_alternates_file(const char *dir);
-
-/*
- * Add the directory to the in-memory list of alternates (along with any
- * recursive alternates it points to), but do not modify the on-disk alternates
- * file.
- */
-extern void add_to_alternates_memory(const char *dir);
-
-/*
- * Returns a scratch strbuf pre-filled with the alternate object directory,
- * including a trailing slash, which can be used to access paths in the
- * alternate. Always use this over direct access to alt->scratch, as it
- * cleans up any previous use of the scratch buffer.
- */
-extern struct strbuf *alt_scratch_buf(struct alternate_object_database *alt);
-
 struct pack_window {
 	struct pack_window *next;
 	unsigned char *base;
object-store.h +51 -0

57 lines not on this page · raw message

Signature · 1 line
2.16.1.435.g8f24da2e1a
#18 lore raw

PATCH 8/44 object-store: move alt_odb_list and alt_odb_tail to object store

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

In a process with multiple repositories open, alternates should be associated to a single repository and not shared globally. Move alt_odb_list and alt_odb_tail into the_repository and adjust callers to reflect this.

Now that the alternative object data base is per repository, we're leaking its memory upon freeing a repository. The next patch plugs this hole.

No functional change intended.

---
 builtin/fsck.c |  4 +++-
 object-store.h |  7 +++++--
 packfile.c     |  3 ++-
 sha1_file.c    | 25 ++++++++++++-------------
 sha1_name.c    |  3 ++-
 5 files changed, 24 insertions(+), 18 deletions(-)
builtin/fsck.c +3 -1
index 7a8a679d4f..b284a3a74e 100644
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "commit.h"
 #include "tree.h"
@@ -716,7 +717,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 		fsck_object_dir(get_object_directory());
 
 		prepare_alt_odb();
-		for (alt = alt_odb_list; alt; alt = alt->next)
+		for (alt = the_repository->objects.alt_odb_list;
+		     alt; alt = alt->next)
 			fsck_object_dir(alt->path);
 
 		if (check_full) {
object-store.h +5 -2
index 505f123976..0ca9233a85 100644
@@ -1,7 +1,7 @@
 #ifndef OBJECT_STORE_H
 #define OBJECT_STORE_H
 
-extern struct alternate_object_database {
+struct alternate_object_database {
 	struct alternate_object_database *next;
 
 	/* see alt_scratch_buf() */
@@ -19,7 +19,7 @@ extern struct alternate_object_database {
 	struct oid_array loose_objects_cache;
 
 	char path[FLEX_ARRAY];
-} *alt_odb_list;
+};
 void prepare_alt_odb(void);
 char *compute_alternate_path(const char *path, struct strbuf *err);
 typedef int alt_odb_fn(struct alternate_object_database *, void *);
@@ -61,6 +61,9 @@ struct raw_object_store {
 
 	/* Path to extra alternate object database if not NULL */
 	char *alternate_db;
+
+	struct alternate_object_database *alt_odb_list;
+	struct alternate_object_database **alt_odb_tail;
 };
 
 void raw_object_store_init(struct raw_object_store *o);
packfile.c +2 -1
index 7dbe8739d1..216ea836ee 100644
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "list.h"
 #include "pack.h"
+#include "repository.h"
 #include "dir.h"
 #include "mergesort.h"
 #include "packfile.h"
@@ -891,7 +892,7 @@ void prepare_packed_git(void)
 		return;
 	prepare_packed_git_one(get_object_directory(), 1);
 	prepare_alt_odb();
-	for (alt = alt_odb_list; alt; alt = alt->next)
+	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next)
 		prepare_packed_git_one(alt->path, 0);
 	rearrange_packed_git();
 	prepare_packed_git_mru();
sha1_file.c +12 -13
index 792bb21c15..4c9635dada 100644
@@ -22,6 +22,7 @@
 #include "pack-revindex.h"
 #include "sha1-lookup.h"
 #include "bulk-checkin.h"
+#include "repository.h"
 #include "streaming.h"
 #include "dir.h"
 #include "list.h"
@@ -342,9 +343,6 @@ static const char *alt_sha1_path(struct alternate_object_database *alt,
 	return buf->buf;
 }
 
-struct alternate_object_database *alt_odb_list;
-static struct alternate_object_database **alt_odb_tail;
-
 /*
  * Return non-zero iff the path is usable as an alternate object database.
  */
@@ -364,7 +362,7 @@ static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)
 	 * Prevent the common mistake of listing the same
 	 * thing twice, or object directory itself.
 	 */
-	for (alt = alt_odb_list; alt; alt = alt->next) {
+	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		if (!fspathcmp(path->buf, alt->path))
 			return 0;
 	}
@@ -424,8 +422,8 @@ static int link_alt_odb_entry(const char *entry, const char *relative_base,
 	ent = alloc_alt_odb(pathbuf.buf);
 
 	/* add the alternate entry */
-	*alt_odb_tail = ent;
-	alt_odb_tail = &(ent->next);
+	*the_repository->objects.alt_odb_tail = ent;
+	the_repository->objects.alt_odb_tail = &(ent->next);
 	ent->next = NULL;
 
 	/* recursively add alternates */
@@ -559,7 +557,7 @@ void add_to_alternates_file(const char *reference)
 		fprintf_or_die(out, "%s\n", reference);
 		if (commit_lock_file(&lock))
 			die_errno("unable to move new alternates file into place");
-		if (alt_odb_tail)
+		if (the_repository->objects.alt_odb_tail)
 			link_alt_odb_entries(reference, '\n', NULL, 0);
 	}
 	free(alts);
@@ -657,7 +655,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 	int r = 0;
 
 	prepare_alt_odb();
-	for (ent = alt_odb_list; ent; ent = ent->next) {
+	for (ent = the_repository->objects.alt_odb_list; ent; ent = ent->next) {
 		r = fn(ent, cb);
 		if (r)
 			break;
@@ -667,10 +665,11 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 
 void prepare_alt_odb(void)
 {
-	if (alt_odb_tail)
+	if (the_repository->objects.alt_odb_tail)
 		return;
 
-	alt_odb_tail = &alt_odb_list;
+	the_repository->objects.alt_odb_tail =
+			&the_repository->objects.alt_odb_list;
 	link_alt_odb_entries(the_repository->objects.alternate_db,
 			     PATH_SEP, NULL, 0);
 
@@ -715,7 +714,7 @@ static int check_and_freshen_nonlocal(const unsigned char *sha1, int freshen)
 {
 	struct alternate_object_database *alt;
 	prepare_alt_odb();
-	for (alt = alt_odb_list; alt; alt = alt->next) {
+	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		const char *path = alt_sha1_path(alt, sha1);
 		if (check_and_freshen_file(path, freshen))
 			return 1;
@@ -875,7 +874,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,
 
 	prepare_alt_odb();
 	errno = ENOENT;
-	for (alt = alt_odb_list; alt; alt = alt->next) {
+	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		if (!lstat(*path, st))
 			return 0;
@@ -905,7 +904,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)
 	most_interesting_errno = errno;
 
 	prepare_alt_odb();
-	for (alt = alt_odb_list; alt; alt = alt->next) {
+	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		fd = git_open(*path);
 		if (fd >= 0)
sha1_name.c +2 -1
index 611c7d24dd..957ce25680 100644
@@ -10,6 +10,7 @@
 #include "dir.h"
 #include "sha1-array.h"
 #include "packfile.h"
+#include "repository.h"
 
 static int get_oid_oneline(const char *, struct object_id *, struct commit_list *);
 
@@ -104,7 +105,7 @@ static void find_short_object_filename(struct disambiguate_state *ds)
 		 */
 		fakeent = alloc_alt_odb(get_object_directory());
 	}
-	fakeent->next = alt_odb_list;
+	fakeent->next = the_repository->objects.alt_odb_list;
 
 	for (alt = fakeent; alt && !ds->ambiguous; alt = alt->next) {
 		int pos;
Signature · 1 line
2.16.1.435.g8f24da2e1a
#19 lore raw

PATCH 9/44 object-store: free alt_odb_list

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Free the memory and reset alt_odb_{list, tail} to NULL.

---
 object.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
object.c +18 -0
index e91711dd56..a71ab34e69 100644
@@ -450,8 +450,26 @@ void raw_object_store_init(struct raw_object_store *o)
 {
 	memset(o, 0, sizeof(*o));
 }
+
+static void free_alt_odb(struct alternate_object_database *alt)
+{
+	strbuf_release(&alt->scratch);
+	oid_array_clear(&alt->loose_objects_cache);
+}
+
+static void free_alt_odbs(struct raw_object_store *o)
+{
+	while (o->alt_odb_list) {
+		free_alt_odb(o->alt_odb_list);
+		o->alt_odb_list = o->alt_odb_list->next;
+	}
+}
+
 void raw_object_store_clear(struct raw_object_store *o)
 {
 	FREE_AND_NULL(o->objectdir);
 	FREE_AND_NULL(o->alternate_db);
+
+	free_alt_odbs(o);
+	o->alt_odb_tail = NULL;
 }
Signature · 1 line
2.16.1.435.g8f24da2e1a
#20 lore raw

Re: [PATCH 09/44] object-store: free alt_odb_list

object.c void raw_object_store_init(struct raw_object_store *o) · line 454

Quoted 12 lines · Nguyễn Thái Ngọc Duy

Free the memory and reset alt_odb_{list, tail} to NULL.

---

object.c +5 -0
@@ -450,8 +450,26 @@ void raw_object_store_init(struct raw_object_store *o)
+static void free_alt_odb(struct alternate_object_database *alt)
+{
+       strbuf_release(&alt->scratch);
+       oid_array_clear(&alt->loose_objects_cache);
+}

This doesn't free the 'struct alternate_object_database' entry itself, right?

Is that intentional? Isn't the idea that this should free the entries too?

object.c void raw_object_store_init(struct raw_object_store *o) · line 466

Quoted 7 lines
object.c +7 -0
+static void free_alt_odbs(struct raw_object_store *o)
+{
+       while (o->alt_odb_list) {
+               free_alt_odb(o->alt_odb_list);
+               o->alt_odb_list = o->alt_odb_list->next;
+       }
+}

Accessing an entry's 'next' member after invoking free_alt_odb() works because the entry itself hasn't been freed (as noted above).

Is leaking the entries themselves intentional?

object.c void raw_object_store_init(struct raw_object_store *o) · line 475

Quoted 8 lines
object.c +3 -0
 void raw_object_store_clear(struct raw_object_store *o)
 {
        FREE_AND_NULL(o->objectdir);
        FREE_AND_NULL(o->alternate_db);
+
+       free_alt_odbs(o);
+       o->alt_odb_tail = NULL;
 }

The commit message talks about freeing memory and resetting alt_odb_list and alt_odb_tail, but this code only resets alt_odb_tail.

author
#21 lore raw

Re: [PATCH 09/44] object-store: free alt_odb_list

Quoted 16 lines · Eric Sunshine

object.c void raw_object_store_init(struct raw_object_store *o) · line 454

Quoted 12 lines · Nguyễn Thái Ngọc Duy

Free the memory and reset alt_odb_{list, tail} to NULL.

---

object.c +5 -0
@@ -450,8 +450,26 @@ void raw_object_store_init(struct raw_object_store *o)
+static void free_alt_odb(struct alternate_object_database *alt)
+{
+       strbuf_release(&alt->scratch);
+       oid_array_clear(&alt->loose_objects_cache);
+}

This doesn't free the 'struct alternate_object_database' entry itself, right?

Quoted 1 line

Is that intentional? Isn't the idea that this should free the entries too?

I think "alt" should be freed too. But since this series is more about refactoring, and we're still in the middle of converting to struct repository/raw_object_store, I don't mind "leaking a bit" (the current state is leaking freely when most things are global). Once the conversion is done, I think we could have some nice test that creates and destroys the object store gracefully, then tools like valgrind. asan... can help clean up these leaks.

Signature · 1 line
Duy
#22 lore raw

Re: [PATCH 09/44] object-store: free alt_odb_list

In reply to Eric Sunshine
Quoted 16 lines · Eric Sunshine

object.c void raw_object_store_init(struct raw_object_store *o) · line 454

Quoted 12 lines · Nguyễn Thái Ngọc Duy

Free the memory and reset alt_odb_{list, tail} to NULL.

---

object.c +5 -0
@@ -450,8 +450,26 @@ void raw_object_store_init(struct raw_object_store *o)
+static void free_alt_odb(struct alternate_object_database *alt)
+{
+       strbuf_release(&alt->scratch);
+       oid_array_clear(&alt->loose_objects_cache);
+}

This doesn't free the 'struct alternate_object_database' entry itself, right?

Quoted 1 line

Is that intentional? Isn't the idea that this should free the entries too?

Yep it definitely should free the entry itself. Should be fixed easy enough by freeing the list entry after grabbing the next entry

Quoted 4 lines

object.c void raw_object_store_init(struct raw_object_store *o) · line 462

Quoted 4 lines
object.c +3 -0
+static void free_alt_odbs(struct raw_object_store *o)
+{
+       while (o->alt_odb_list) {
                    struct alternate_object_database old = o->alt_odb_list;
Quoted 2 lines

object.c void raw_object_store_init(struct raw_object_store *o) · line 464

Quoted 2 lines
object.c +2 -0
+               free_alt_odb(o->alt_odb_list);
+               o->alt_odb_list = o->alt_odb_list->next;
                    free(old);
Quoted 19 lines

object.c void raw_object_store_init(struct raw_object_store *o) · line 461

Quoted 2 lines
object.c +2 -0
+       }
+}

Accessing an entry's 'next' member after invoking free_alt_odb() works because the entry itself hasn't been freed (as noted above).

Is leaking the entries themselves intentional?

object.c void raw_object_store_init(struct raw_object_store *o) · line 475

Quoted 8 lines
object.c +3 -0
 void raw_object_store_clear(struct raw_object_store *o)
 {
        FREE_AND_NULL(o->objectdir);
        FREE_AND_NULL(o->alternate_db);
+
+       free_alt_odbs(o);
+       o->alt_odb_tail = NULL;
 }

The commit message talks about freeing memory and resetting alt_odb_list and alt_odb_tail, but this code only resets alt_odb_tail.

Signature · 1 line
Brandon Williams
#23 lore raw

PATCH 10/44 object-store: move packed_git and packed_git_mru to object store

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

In a process with multiple repositories open, packfile accessors should be associated to a single repository and not shared globally. Move packed_git and packed_git_mru into the_repository and adjust callers to reflect this.

[nd: while at there, wrap access to these two fields in get_packed_git() and get_packed_git_mru(). This allows us to lazily initialize these fields without caller doing that explicitly]

Patch generated by

1. Moving the struct packed_git declaration to object-store.h and packed_git, packed_git_mru globals to struct object_store.

2. Apply the following semantic patch to adjust callers:
@@ @@
- packed_git
+ the_repository->objects.packed_git

@@ @@
- packed_git_mru
+ the_repository->objects.packed_git_mru

3. Applying line wrapping fixes from "make style" to break the resulting long lines.

4. Adding missing #includes of repository.h where needed.

5. As the packfiles are now owned by an objectstore, which is ephemeral unlike globals, we introduce memory leaks. So address them in raw_object_store_clear(). Defer freeing packed_git to the next patch due to the size of this patch.

---
 builtin/count-objects.c  |  5 ++--
 builtin/fsck.c           |  6 +++--
 builtin/gc.c             |  3 ++-
 builtin/pack-objects.c   | 20 ++++++++--------
 builtin/pack-redundant.c |  5 ++--
 cache.h                  | 29 ------------------------
 fast-import.c            |  7 ++++--
 http-backend.c           |  5 ++--
 object-store.h           | 28 +++++++++++++++++++++++
 object.c                 |  7 ++++++
 pack-bitmap.c            |  3 ++-
 packfile.c               | 49 ++++++++++++++++++++++++----------------
 packfile.h               |  3 +++
 server-info.c            |  5 ++--
 sha1_name.c              |  5 ++--
 15 files changed, 107 insertions(+), 73 deletions(-)
builtin/count-objects.c +3 -2
index 33343818c8..5c7c3c6ae3 100644
@@ -7,6 +7,7 @@
 #include "cache.h"
 #include "config.h"
 #include "dir.h"
+#include "repository.h"
 #include "builtin.h"
 #include "parse-options.h"
 #include "quote.h"
@@ -120,9 +121,9 @@ int cmd_count_objects(int argc, const char **argv, const char *prefix)
 		struct strbuf loose_buf = STRBUF_INIT;
 		struct strbuf pack_buf = STRBUF_INIT;
 		struct strbuf garbage_buf = STRBUF_INIT;
-		if (!packed_git)
+		if (!get_packed_git(the_repository))
 			prepare_packed_git();
-		for (p = packed_git; p; p = p->next) {
+		for (p = get_packed_git(the_repository); p; p = p->next) {
 			if (!p->pack_local)
 				continue;
 			if (open_pack_index(p))
builtin/fsck.c +4 -2
index b284a3a74e..7aca9699f6 100644
@@ -729,7 +729,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 			prepare_packed_git();
 
 			if (show_progress) {
-				for (p = packed_git; p; p = p->next) {
+				for (p = get_packed_git(the_repository); p;
+				     p = p->next) {
 					if (open_pack_index(p))
 						continue;
 					total += p->num_objects;
@@ -737,7 +738,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 
 				progress = start_progress(_("Checking objects"), total);
 			}
-			for (p = packed_git; p; p = p->next) {
+			for (p = get_packed_git(the_repository); p;
+			     p = p->next) {
 				/* verify gives error messages itself */
 				if (verify_pack(p, fsck_obj_buffer,
 						progress, count))
builtin/gc.c +2 -1
index 77fa720bd0..dd30067ac1 100644
@@ -11,6 +11,7 @@
  */
 
 #include "builtin.h"
+#include "repository.h"
 #include "config.h"
 #include "tempfile.h"
 #include "lockfile.h"
@@ -173,7 +174,7 @@ static int too_many_packs(void)
 		return 0;
 
 	prepare_packed_git();
-	for (cnt = 0, p = packed_git; p; p = p->next) {
+	for (cnt = 0, p = get_packed_git(the_repository); p; p = p->next) {
 		if (!p->pack_local)
 			continue;
 		if (p->pack_keep)
builtin/pack-objects.c +11 -9
index 83dcbc9773..0e5fde1d6b 100644
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "attr.h"
 #include "object.h"
@@ -1025,8 +1026,7 @@ static int want_object_in_pack(const struct object_id *oid,
 		if (want != -1)
 			return want;
 	}
-
-	list_for_each(pos, &packed_git_mru) {
+	list_for_each(pos, get_packed_git_mru(the_repository)) {
 		struct packed_git *p = list_entry(pos, struct packed_git, mru);
 		off_t offset;
 
@@ -1044,7 +1044,8 @@ static int want_object_in_pack(const struct object_id *oid,
 			}
 			want = want_found_object(exclude, p);
 			if (!exclude && want > 0)
-				list_move(&p->mru, &packed_git_mru);
+				list_move(&p->mru,
+					  get_packed_git_mru(the_repository));
 			if (want != -1)
 				return want;
 		}
@@ -2673,7 +2674,7 @@ static void add_objects_in_unpacked_packs(struct rev_info *revs)
 
 	memset(&in_pack, 0, sizeof(in_pack));
 
-	for (p = packed_git; p; p = p->next) {
+	for (p = get_packed_git(the_repository); p; p = p->next) {
 		struct object_id oid;
 		struct object *o;
 
@@ -2736,7 +2737,8 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 	static struct packed_git *last_found = (void *)1;
 	struct packed_git *p;
 
-	p = (last_found != (void *)1) ? last_found : packed_git;
+	p = (last_found != (void *)1) ? last_found :
+					get_packed_git(the_repository);
 
 	while (p) {
 		if ((!p->pack_local || p->pack_keep) &&
@@ -2745,7 +2747,7 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 			return 1;
 		}
 		if (p == last_found)
-			p = packed_git;
+			p = get_packed_git(the_repository);
 		else
 			p = p->next;
 		if (p == last_found)
@@ -2781,7 +2783,7 @@ static void loosen_unused_packed_objects(struct rev_info *revs)
 	uint32_t i;
 	struct object_id oid;
 
-	for (p = packed_git; p; p = p->next) {
+	for (p = get_packed_git(the_repository); p; p = p->next) {
 		if (!p->pack_local || p->pack_keep)
 			continue;
 
@@ -3152,7 +3154,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 	prepare_packed_git();
 	if (ignore_packed_keep) {
 		struct packed_git *p;
-		for (p = packed_git; p; p = p->next)
+		for (p = get_packed_git(the_repository); p; p = p->next)
 			if (p->pack_local && p->pack_keep)
 				break;
 		if (!p) /* no keep-able packs found */
@@ -3165,7 +3167,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 		 * also covers non-local objects
 		 */
 		struct packed_git *p;
-		for (p = packed_git; p; p = p->next) {
+		for (p = get_packed_git(the_repository); p; p = p->next) {
 			if (!p->pack_local) {
 				have_non_local_packs = 1;
 				break;
builtin/pack-redundant.c +3 -2
index aaa8136322..d6d8a44959 100644
@@ -7,6 +7,7 @@
 */
 
 #include "builtin.h"
+#include "repository.h"
 #include "packfile.h"
 
 #define BLKSIZE 512
@@ -571,7 +572,7 @@ static struct pack_list * add_pack(struct packed_git *p)
 
 static struct pack_list * add_pack_file(const char *filename)
 {
-	struct packed_git *p = packed_git;
+	struct packed_git *p = get_packed_git(the_repository);
 
 	if (strlen(filename) < 40)
 		die("Bad pack filename: %s", filename);
@@ -586,7 +587,7 @@ static struct pack_list * add_pack_file(const char *filename)
 
 static void load_all(void)
 {
-	struct packed_git *p = packed_git;
+	struct packed_git *p = get_packed_git(the_repository);
 
 	while (p) {
 		add_pack(p);
cache.h +0 -29
index 41530d5d16..d3429a0d48 100644
@@ -1585,35 +1585,6 @@ struct pack_window {
 	unsigned int inuse_cnt;
 };
 
-extern struct packed_git {
-	struct packed_git *next;
-	struct list_head mru;
-	struct pack_window *windows;
-	off_t pack_size;
-	const void *index_data;
-	size_t index_size;
-	uint32_t num_objects;
-	uint32_t num_bad_objects;
-	unsigned char *bad_object_sha1;
-	int index_version;
-	time_t mtime;
-	int pack_fd;
-	unsigned pack_local:1,
-		 pack_keep:1,
-		 freshened:1,
-		 do_not_close:1,
-		 pack_promisor:1;
-	unsigned char sha1[20];
-	struct revindex_entry *revindex;
-	/* something like ".git/objects/pack/xxxxx.pack" */
-	char pack_name[FLEX_ARRAY]; /* more */
-} *packed_git;
-
-/*
- * A most-recently-used ordered version of the packed_git list.
- */
-extern struct list_head packed_git_mru;
-
 struct pack_entry {
 	off_t offset;
 	unsigned char sha1[20];
fast-import.c +5 -2
index b70ac025e0..0dba555478 100644
@@ -154,6 +154,7 @@ Format of STDIN stream:
 
 #include "builtin.h"
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "lockfile.h"
 #include "object.h"
@@ -1110,7 +1111,8 @@ static int store_object(
 	if (e->idx.offset) {
 		duplicate_count_by_type[type]++;
 		return 1;
-	} else if (find_sha1_pack(oid.hash, packed_git)) {
+	} else if (find_sha1_pack(oid.hash,
+				  get_packed_git(the_repository))) {
 		e->type = type;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
@@ -1305,7 +1307,8 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
 		duplicate_count_by_type[OBJ_BLOB]++;
 		truncate_pack(&checkpoint);
 
-	} else if (find_sha1_pack(oid.hash, packed_git)) {
+	} else if (find_sha1_pack(oid.hash,
+				  get_packed_git(the_repository))) {
 		e->type = OBJ_BLOB;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
http-backend.c +3 -2
index f3dc218b2a..22921d169a 100644
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "config.h"
+#include "repository.h"
 #include "refs.h"
 #include "pkt-line.h"
 #include "object.h"
@@ -518,13 +519,13 @@ static void get_info_packs(struct strbuf *hdr, char *arg)
 
 	select_getanyfile(hdr);
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = get_packed_git(the_repository); p; p = p->next) {
 		if (p->pack_local)
 			cnt++;
 	}
 
 	strbuf_grow(&buf, cnt * 53 + 2);
-	for (p = packed_git; p; p = p->next) {
+	for (p = get_packed_git(the_repository); p; p = p->next) {
 		if (p->pack_local)
 			strbuf_addf(&buf, "P %s\n", p->pack_name + objdirlen + 6);
 	}
object-store.h +28 -0
index 0ca9233a85..1f3e66a3b8 100644
@@ -52,6 +52,30 @@ void add_to_alternates_memory(const char *dir);
  */
 struct strbuf *alt_scratch_buf(struct alternate_object_database *alt);
 
+struct packed_git {
+	struct packed_git *next;
+	struct list_head mru;
+	struct pack_window *windows;
+	off_t pack_size;
+	const void *index_data;
+	size_t index_size;
+	uint32_t num_objects;
+	uint32_t num_bad_objects;
+	unsigned char *bad_object_sha1;
+	int index_version;
+	time_t mtime;
+	int pack_fd;
+	unsigned pack_local:1,
+		 pack_keep:1,
+		 freshened:1,
+		 do_not_close:1,
+		 pack_promisor:1;
+	unsigned char sha1[20];
+	struct revindex_entry *revindex;
+	/* something like ".git/objects/pack/xxxxx.pack" */
+	char pack_name[FLEX_ARRAY]; /* more */
+};
+
 struct raw_object_store {
 	/*
 	 * Path to the repository's object store.
@@ -62,6 +86,10 @@ struct raw_object_store {
 	/* Path to extra alternate object database if not NULL */
 	char *alternate_db;
 
+	struct packed_git *packed_git; /* private */
+	/* A most-recently-used ordered version of the packed_git list. */
+	struct list_head packed_git_mru; /* private */
+
 	struct alternate_object_database *alt_odb_list;
 	struct alternate_object_database **alt_odb_tail;
 };
object.c +7 -0
index a71ab34e69..83be6b1ecb 100644
@@ -449,6 +449,7 @@ void clear_commit_marks_all(unsigned int flags)
 void raw_object_store_init(struct raw_object_store *o)
 {
 	memset(o, 0, sizeof(*o));
+	INIT_LIST_HEAD(&o->packed_git_mru);
 }
 
 static void free_alt_odb(struct alternate_object_database *alt)
@@ -472,4 +473,10 @@ void raw_object_store_clear(struct raw_object_store *o)
 
 	free_alt_odbs(o);
 	o->alt_odb_tail = NULL;
+
+	INIT_LIST_HEAD(&o->packed_git_mru);
+	/*
+	 * TODO: call close_all_packs once migrated to
+	 * take an object store argument
+	 */
 }
pack-bitmap.c +2 -1
index 9270983e5f..abed43cdb5 100644
@@ -10,6 +10,7 @@
 #include "pack-revindex.h"
 #include "pack-objects.h"
 #include "packfile.h"
+#include "repository.h"
 
 /*
  * An entry on the bitmap index, representing the bitmap for a given
@@ -335,7 +336,7 @@ static int open_pack_bitmap(void)
 	assert(!bitmap_git.map && !bitmap_git.loaded);
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = get_packed_git(the_repository); p; p = p->next) {
 		if (open_pack_bitmap_1(p) == 0)
 			ret = 0;
 	}
packfile.c +30 -19

132 lines not on this page · raw message

packfile.h +3 -0
index a7fca598d6..76496226bb 100644
@@ -38,6 +38,9 @@ extern void prepare_packed_git(void);
 extern void reprepare_packed_git(void);
 extern void install_packed_git(struct packed_git *pack);
 
+struct packed_git *get_packed_git(struct repository *r);
+struct list_head *get_packed_git_mru(struct repository *r);
+
 /*
  * Give a rough count of objects in the repository. This sacrifices accuracy
  * for speed.
server-info.c +3 -2
index 26a6c20b7d..6fe64ede17 100644
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "repository.h"
 #include "refs.h"
 #include "object.h"
 #include "commit.h"
@@ -200,7 +201,7 @@ static void init_pack_info(const char *infofile, int force)
 	objdirlen = strlen(objdir);
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = get_packed_git(the_repository); p; p = p->next) {
 		/* we ignore things on alternate path since they are
 		 * not available to the pullers in general.
 		 */
@@ -210,7 +211,7 @@ static void init_pack_info(const char *infofile, int force)
 	}
 	num_pack = i;
 	info = xcalloc(num_pack, sizeof(struct pack_info *));
-	for (i = 0, p = packed_git; p; p = p->next) {
+	for (i = 0, p = get_packed_git(the_repository); p; p = p->next) {
 		if (!p->pack_local)
 			continue;
 		info[i] = xcalloc(1, sizeof(struct pack_info));
sha1_name.c +3 -2
index 957ce25680..bd4d7352ce 100644
@@ -196,7 +196,8 @@ static void find_short_packed_object(struct disambiguate_state *ds)
 	struct packed_git *p;
 
 	prepare_packed_git();
-	for (p = packed_git; p && !ds->ambiguous; p = p->next)
+	for (p = get_packed_git(the_repository); p && !ds->ambiguous;
+	     p = p->next)
 		unique_in_pack(p, ds);
 }
 
@@ -566,7 +567,7 @@ static void find_abbrev_len_packed(struct min_abbrev_data *mad)
 	struct packed_git *p;
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next)
+	for (p = get_packed_git(the_repository); p; p = p->next)
 		find_abbrev_len_for_pack(p, mad);
 }
 
Signature · 1 line
2.16.1.435.g8f24da2e1a
#24 lore raw

Re: [PATCH 10/44] object-store: move packed_git and packed_git_mru to object store

Quoted 10 lines · On Sat, 3 Mar 2018 18:36:03 +0700 Nguyễn Thái Ngọc Duy <[email protected]>…

From: Stefan Beller <[email protected]>

In a process with multiple repositories open, packfile accessors should be associated to a single repository and not shared globally. Move packed_git and packed_git_mru into the_repository and adjust callers to reflect this.

[nd: while at there, wrap access to these two fields in get_packed_git() and get_packed_git_mru(). This allows us to lazily initialize these fields without caller doing that explicitly]

The patches up to this one look good. (I didn't reply for each individual one to avoid unnecessarily sending messages to the list.)

About this patch: will lazily initializing these fields be done in a later patch?

Quoted 5 lines

Patch generated by

1. Moving the struct packed_git declaration to object-store.h and packed_git, packed_git_mru globals to struct object_store.

Quoted 8 lines

2. Apply the following semantic patch to adjust callers:
@@ @@
- packed_git
+ the_repository->objects.packed_git

@@ @@
- packed_git_mru
+ the_repository->objects.packed_git_mru

This doesn't seem up-to-date - they are being replaced with a function call, not a field access. I would be OK with just removing this "Patch generated by" section.

[snip remainder of "Patch generated by" section]

packfile.c static int unuse_one_window(struct packed_git *current) · line 250

Quoted 9 lines
packfile.c +1 -1
@@ -246,7 +244,7 @@ static int unuse_one_window(struct packed_git *current)
 
 	if (current)
 		scan_windows(current, &lru_p, &lru_w, &lru_l);
-	for (p = packed_git; p; p = p->next)
+	for (p = the_repository->objects.packed_git; p; p = p->next)
 		scan_windows(p, &lru_p, &lru_w, &lru_l);
 	if (lru_p) {
 		munmap(lru_w->base, lru_w->len);

Here (and elsewhere), "the_repository->objects.packed_git" instead of "get_packed_git" is still used.

author
#25 lore raw

Re: [PATCH 10/44] object-store: move packed_git and packed_git_mru to object store

Quoted 14 lines · Jonathan Tan
Quoted 10 lines · On Sat, 3 Mar 2018 18:36:03 +0700 Nguyễn Thái Ngọc Duy <[email protected]>…

From: Stefan Beller <[email protected]>

In a process with multiple repositories open, packfile accessors should be associated to a single repository and not shared globally. Move packed_git and packed_git_mru into the_repository and adjust callers to reflect this.

[nd: while at there, wrap access to these two fields in get_packed_git() and get_packed_git_mru(). This allows us to lazily initialize these fields without caller doing that explicitly]

Quoted 5 lines

The patches up to this one look good. (I didn't reply for each individual one to avoid unnecessarily sending messages to the list.)

About this patch: will lazily initializing these fields be done in a later patch?

Yes.

Quoted 10 lines
Quoted 10 lines

Patch generated by

1. Moving the struct packed_git declaration to object-store.h and packed_git, packed_git_mru globals to struct object_store.

2. Apply the following semantic patch to adjust callers:
@@ @@
- packed_git
+ the_repository->objects.packed_git

Quoted 7 lines
Quoted 3 lines

@@ @@
- packed_git_mru
+ the_repository->objects.packed_git_mru

This doesn't seem up-to-date - they are being replaced with a function call, not a field access. I would be OK with just removing this "Patch generated by" section.

I'll just drop this part. I'm manually updating the series anyway.

Quoted 4 lines

[snip remainder of "Patch generated by" section]

Quoted 2 lines
(quoted diff) +0 -0
@@ -246,7 +244,7 @@ static int unuse_one_window(struct packed_git *current)
 
Quoted 10 lines

packfile.c static int unuse_one_window(struct packed_git *current) · line 250

Quoted 7 lines
packfile.c +1 -1
      if (current)
              scan_windows(current, &lru_p, &lru_w, &lru_l);
-     for (p = packed_git; p; p = p->next)
+     for (p = the_repository->objects.packed_git; p; p = p->next)
              scan_windows(p, &lru_p, &lru_w, &lru_l);
      if (lru_p) {
              munmap(lru_w->base, lru_w->len);

Here (and elsewhere), "the_repository->objects.packed_git" instead of "get_packed_git" is still used.

Yeah. I saw your other comment about the not converting in packfile.c too. My view is, packfile has intimate knowledge about pack manipulation and we don't need to try to "protect" from misuse of packed_git pointer without prepare_packed_git. People how modify this code probably know that by heart at this point. And we can't avoid it completely anyway because it will have access to the static function prepare_packed_git.

Signature · 1 line
Duy
#26 lore raw

Re: [PATCH 10/44] object-store: move packed_git and packed_git_mru to object store

In reply to Nguyễn Thái Ngọc Duy
Quoted 16 lines · Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

In a process with multiple repositories open, packfile accessors should be associated to a single repository and not shared globally. Move packed_git and packed_git_mru into the_repository and adjust callers to reflect this.

[nd: while at there, wrap access to these two fields in get_packed_git() and get_packed_git_mru(). This allows us to lazily initialize these fields without caller doing that explicitly]

Patch generated by

1. Moving the struct packed_git declaration to object-store.h and packed_git, packed_git_mru globals to struct object_store.

Quoted 8 lines

2. Apply the following semantic patch to adjust callers:
@@ @@
- packed_git
+ the_repository->objects.packed_git

@@ @@
- packed_git_mru
+ the_repository->objects.packed_git_mru

As Jonathan mentioned, this should probably be taken out of the commit message because it doesn't reflect what the code actually does. What it actually does took me a second to figure out. You're marking packed_git as "private"...well C has no notion of private vs public fields in a struct so it might be difficult to keep that convention, it also took me a second to realize that it was only in the scope of packfile.c where it was ok to reference it directly. Maybe it'll be ok? If we really wanted something to be private we'd need it to be an opaque type instead, which may be out of the scope of this code refactor.

Quoted 622 lines

3. Applying line wrapping fixes from "make style" to break the resulting long lines.

4. Adding missing #includes of repository.h where needed.

5. As the packfiles are now owned by an objectstore, which is ephemeral unlike globals, we introduce memory leaks. So address them in raw_object_store_clear(). Defer freeing packed_git to the next patch due to the size of this patch.

---
 builtin/count-objects.c  |  5 ++--
 builtin/fsck.c           |  6 +++--
 builtin/gc.c             |  3 ++-
 builtin/pack-objects.c   | 20 ++++++++--------
 builtin/pack-redundant.c |  5 ++--
 cache.h                  | 29 ------------------------
 fast-import.c            |  7 ++++--
 http-backend.c           |  5 ++--
 object-store.h           | 28 +++++++++++++++++++++++
 object.c                 |  7 ++++++
 pack-bitmap.c            |  3 ++-
 packfile.c               | 49 ++++++++++++++++++++++++----------------
 packfile.h               |  3 +++
 server-info.c            |  5 ++--
 sha1_name.c              |  5 ++--
 15 files changed, 107 insertions(+), 73 deletions(-)
builtin/count-objects.c +3 -2
index 33343818c8..5c7c3c6ae3 100644
@@ -7,6 +7,7 @@
 #include "cache.h"
 #include "config.h"
 #include "dir.h"
+#include "repository.h"
 #include "builtin.h"
 #include "parse-options.h"
 #include "quote.h"
@@ -120,9 +121,9 @@ int cmd_count_objects(int argc, const char **argv, const char *prefix)
 		struct strbuf loose_buf = STRBUF_INIT;
 		struct strbuf pack_buf = STRBUF_INIT;
 		struct strbuf garbage_buf = STRBUF_INIT;
-		if (!packed_git)
+		if (!get_packed_git(the_repository))
 			prepare_packed_git();
-		for (p = packed_git; p; p = p->next) {
+		for (p = get_packed_git(the_repository); p; p = p->next) {
 			if (!p->pack_local)
 				continue;
 			if (open_pack_index(p))
builtin/fsck.c +4 -2
index b284a3a74e..7aca9699f6 100644
@@ -729,7 +729,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 			prepare_packed_git();
 
 			if (show_progress) {
-				for (p = packed_git; p; p = p->next) {
+				for (p = get_packed_git(the_repository); p;
+				     p = p->next) {
 					if (open_pack_index(p))
 						continue;
 					total += p->num_objects;
@@ -737,7 +738,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 
 				progress = start_progress(_("Checking objects"), total);
 			}
-			for (p = packed_git; p; p = p->next) {
+			for (p = get_packed_git(the_repository); p;
+			     p = p->next) {
 				/* verify gives error messages itself */
 				if (verify_pack(p, fsck_obj_buffer,
 						progress, count))
builtin/gc.c +2 -1
index 77fa720bd0..dd30067ac1 100644
@@ -11,6 +11,7 @@
  */
 
 #include "builtin.h"
+#include "repository.h"
 #include "config.h"
 #include "tempfile.h"
 #include "lockfile.h"
@@ -173,7 +174,7 @@ static int too_many_packs(void)
 		return 0;
 
 	prepare_packed_git();
-	for (cnt = 0, p = packed_git; p; p = p->next) {
+	for (cnt = 0, p = get_packed_git(the_repository); p; p = p->next) {
 		if (!p->pack_local)
 			continue;
 		if (p->pack_keep)
builtin/pack-objects.c +11 -9
index 83dcbc9773..0e5fde1d6b 100644
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "attr.h"
 #include "object.h"
@@ -1025,8 +1026,7 @@ static int want_object_in_pack(const struct object_id *oid,
 		if (want != -1)
 			return want;
 	}
-
-	list_for_each(pos, &packed_git_mru) {
+	list_for_each(pos, get_packed_git_mru(the_repository)) {
 		struct packed_git *p = list_entry(pos, struct packed_git, mru);
 		off_t offset;
 
@@ -1044,7 +1044,8 @@ static int want_object_in_pack(const struct object_id *oid,
 			}
 			want = want_found_object(exclude, p);
 			if (!exclude && want > 0)
-				list_move(&p->mru, &packed_git_mru);
+				list_move(&p->mru,
+					  get_packed_git_mru(the_repository));
 			if (want != -1)
 				return want;
 		}
@@ -2673,7 +2674,7 @@ static void add_objects_in_unpacked_packs(struct rev_info *revs)
 
 	memset(&in_pack, 0, sizeof(in_pack));
 
-	for (p = packed_git; p; p = p->next) {
+	for (p = get_packed_git(the_repository); p; p = p->next) {
 		struct object_id oid;
 		struct object *o;
 
@@ -2736,7 +2737,8 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 	static struct packed_git *last_found = (void *)1;
 	struct packed_git *p;
 
-	p = (last_found != (void *)1) ? last_found : packed_git;
+	p = (last_found != (void *)1) ? last_found :
+					get_packed_git(the_repository);
 
 	while (p) {
 		if ((!p->pack_local || p->pack_keep) &&
@@ -2745,7 +2747,7 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 			return 1;
 		}
 		if (p == last_found)
-			p = packed_git;
+			p = get_packed_git(the_repository);
 		else
 			p = p->next;
 		if (p == last_found)
@@ -2781,7 +2783,7 @@ static void loosen_unused_packed_objects(struct rev_info *revs)
 	uint32_t i;
 	struct object_id oid;
 
-	for (p = packed_git; p; p = p->next) {
+	for (p = get_packed_git(the_repository); p; p = p->next) {
 		if (!p->pack_local || p->pack_keep)
 			continue;
 
@@ -3152,7 +3154,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 	prepare_packed_git();
 	if (ignore_packed_keep) {
 		struct packed_git *p;
-		for (p = packed_git; p; p = p->next)
+		for (p = get_packed_git(the_repository); p; p = p->next)
 			if (p->pack_local && p->pack_keep)
 				break;
 		if (!p) /* no keep-able packs found */
@@ -3165,7 +3167,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 		 * also covers non-local objects
 		 */
 		struct packed_git *p;
-		for (p = packed_git; p; p = p->next) {
+		for (p = get_packed_git(the_repository); p; p = p->next) {
 			if (!p->pack_local) {
 				have_non_local_packs = 1;
 				break;
builtin/pack-redundant.c +3 -2
index aaa8136322..d6d8a44959 100644
@@ -7,6 +7,7 @@
 */
 
 #include "builtin.h"
+#include "repository.h"
 #include "packfile.h"
 
 #define BLKSIZE 512
@@ -571,7 +572,7 @@ static struct pack_list * add_pack(struct packed_git *p)
 
 static struct pack_list * add_pack_file(const char *filename)
 {
-	struct packed_git *p = packed_git;
+	struct packed_git *p = get_packed_git(the_repository);
 
 	if (strlen(filename) < 40)
 		die("Bad pack filename: %s", filename);
@@ -586,7 +587,7 @@ static struct pack_list * add_pack_file(const char *filename)
 
 static void load_all(void)
 {
-	struct packed_git *p = packed_git;
+	struct packed_git *p = get_packed_git(the_repository);
 
 	while (p) {
 		add_pack(p);
cache.h +0 -29
index 41530d5d16..d3429a0d48 100644
@@ -1585,35 +1585,6 @@ struct pack_window {
 	unsigned int inuse_cnt;
 };
 
-extern struct packed_git {
-	struct packed_git *next;
-	struct list_head mru;
-	struct pack_window *windows;
-	off_t pack_size;
-	const void *index_data;
-	size_t index_size;
-	uint32_t num_objects;
-	uint32_t num_bad_objects;
-	unsigned char *bad_object_sha1;
-	int index_version;
-	time_t mtime;
-	int pack_fd;
-	unsigned pack_local:1,
-		 pack_keep:1,
-		 freshened:1,
-		 do_not_close:1,
-		 pack_promisor:1;
-	unsigned char sha1[20];
-	struct revindex_entry *revindex;
-	/* something like ".git/objects/pack/xxxxx.pack" */
-	char pack_name[FLEX_ARRAY]; /* more */
-} *packed_git;
-
-/*
- * A most-recently-used ordered version of the packed_git list.
- */
-extern struct list_head packed_git_mru;
-
 struct pack_entry {
 	off_t offset;
 	unsigned char sha1[20];
fast-import.c +5 -2
index b70ac025e0..0dba555478 100644
@@ -154,6 +154,7 @@ Format of STDIN stream:
 
 #include "builtin.h"
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "lockfile.h"
 #include "object.h"
@@ -1110,7 +1111,8 @@ static int store_object(
 	if (e->idx.offset) {
 		duplicate_count_by_type[type]++;
 		return 1;
-	} else if (find_sha1_pack(oid.hash, packed_git)) {
+	} else if (find_sha1_pack(oid.hash,
+				  get_packed_git(the_repository))) {
 		e->type = type;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
@@ -1305,7 +1307,8 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
 		duplicate_count_by_type[OBJ_BLOB]++;
 		truncate_pack(&checkpoint);
 
-	} else if (find_sha1_pack(oid.hash, packed_git)) {
+	} else if (find_sha1_pack(oid.hash,
+				  get_packed_git(the_repository))) {
 		e->type = OBJ_BLOB;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
http-backend.c +3 -2
index f3dc218b2a..22921d169a 100644
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "config.h"
+#include "repository.h"
 #include "refs.h"
 #include "pkt-line.h"
 #include "object.h"
@@ -518,13 +519,13 @@ static void get_info_packs(struct strbuf *hdr, char *arg)
 
 	select_getanyfile(hdr);
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = get_packed_git(the_repository); p; p = p->next) {
 		if (p->pack_local)
 			cnt++;
 	}
 
 	strbuf_grow(&buf, cnt * 53 + 2);
-	for (p = packed_git; p; p = p->next) {
+	for (p = get_packed_git(the_repository); p; p = p->next) {
 		if (p->pack_local)
 			strbuf_addf(&buf, "P %s\n", p->pack_name + objdirlen + 6);
 	}
object-store.h +28 -0
index 0ca9233a85..1f3e66a3b8 100644
@@ -52,6 +52,30 @@ void add_to_alternates_memory(const char *dir);
  */
 struct strbuf *alt_scratch_buf(struct alternate_object_database *alt);
 
+struct packed_git {
+	struct packed_git *next;
+	struct list_head mru;
+	struct pack_window *windows;
+	off_t pack_size;
+	const void *index_data;
+	size_t index_size;
+	uint32_t num_objects;
+	uint32_t num_bad_objects;
+	unsigned char *bad_object_sha1;
+	int index_version;
+	time_t mtime;
+	int pack_fd;
+	unsigned pack_local:1,
+		 pack_keep:1,
+		 freshened:1,
+		 do_not_close:1,
+		 pack_promisor:1;
+	unsigned char sha1[20];
+	struct revindex_entry *revindex;
+	/* something like ".git/objects/pack/xxxxx.pack" */
+	char pack_name[FLEX_ARRAY]; /* more */
+};
+
 struct raw_object_store {
 	/*
 	 * Path to the repository's object store.
@@ -62,6 +86,10 @@ struct raw_object_store {
 	/* Path to extra alternate object database if not NULL */
 	char *alternate_db;
 
+	struct packed_git *packed_git; /* private */
+	/* A most-recently-used ordered version of the packed_git list. */
+	struct list_head packed_git_mru; /* private */
+
 	struct alternate_object_database *alt_odb_list;
 	struct alternate_object_database **alt_odb_tail;
 };
object.c +7 -0
index a71ab34e69..83be6b1ecb 100644
@@ -449,6 +449,7 @@ void clear_commit_marks_all(unsigned int flags)
 void raw_object_store_init(struct raw_object_store *o)
 {
 	memset(o, 0, sizeof(*o));
+	INIT_LIST_HEAD(&o->packed_git_mru);
 }
 
 static void free_alt_odb(struct alternate_object_database *alt)
@@ -472,4 +473,10 @@ void raw_object_store_clear(struct raw_object_store *o)
 
 	free_alt_odbs(o);
 	o->alt_odb_tail = NULL;
+
+	INIT_LIST_HEAD(&o->packed_git_mru);
+	/*
+	 * TODO: call close_all_packs once migrated to
+	 * take an object store argument
+	 */
 }
pack-bitmap.c +2 -1
index 9270983e5f..abed43cdb5 100644
@@ -10,6 +10,7 @@
 #include "pack-revindex.h"
 #include "pack-objects.h"
 #include "packfile.h"
+#include "repository.h"
 
 /*
  * An entry on the bitmap index, representing the bitmap for a given
@@ -335,7 +336,7 @@ static int open_pack_bitmap(void)
 	assert(!bitmap_git.map && !bitmap_git.loaded);
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = get_packed_git(the_repository); p; p = p->next) {
 		if (open_pack_bitmap_1(p) == 0)
 			ret = 0;
 	}
packfile.c +30 -19
index 216ea836ee..d3c4a12ae0 100644
@@ -45,8 +45,6 @@ static unsigned int pack_open_fds;
 static unsigned int pack_max_fds;
 static size_t peak_pack_mapped;
 static size_t pack_mapped;
-struct packed_git *packed_git;
-LIST_HEAD(packed_git_mru);
 
 #define SZ_FMT PRIuMAX
 static inline uintmax_t sz_fmt(size_t s) { return s; }
@@ -246,7 +244,7 @@ static int unuse_one_window(struct packed_git *current)
 
 	if (current)
 		scan_windows(current, &lru_p, &lru_w, &lru_l);
-	for (p = packed_git; p; p = p->next)
+	for (p = the_repository->objects.packed_git; p; p = p->next)
 		scan_windows(p, &lru_p, &lru_w, &lru_l);
 	if (lru_p) {
 		munmap(lru_w->base, lru_w->len);
@@ -316,7 +314,7 @@ void close_all_packs(void)
 {
 	struct packed_git *p;
 
-	for (p = packed_git; p; p = p->next)
+	for (p = the_repository->objects.packed_git; p; p = p->next)
 		if (p->do_not_close)
 			die("BUG: want to close pack marked 'do-not-close'");
 		else
@@ -384,7 +382,7 @@ static int close_one_pack(void)
 	struct pack_window *mru_w = NULL;
 	int accept_windows_inuse = 1;
 
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if (p->pack_fd == -1)
 			continue;
 		find_lru_pack(p, &lru_p, &mru_w, &accept_windows_inuse);
@@ -686,8 +684,8 @@ void install_packed_git(struct packed_git *pack)
 	if (pack->pack_fd != -1)
 		pack_open_fds++;
 
-	pack->next = packed_git;
-	packed_git = pack;
+	pack->next = the_repository->objects.packed_git;
+	the_repository->objects.packed_git = pack;
 }
 
 void (*report_garbage)(unsigned seen_bits, const char *path);
@@ -769,7 +767,8 @@ static void prepare_packed_git_one(char *objdir, int local)
 		base_len = path.len;
 		if (strip_suffix_mem(path.buf, &base_len, ".idx")) {
 			/* Don't reopen a pack we already have. */
-			for (p = packed_git; p; p = p->next) {
+			for (p = the_repository->objects.packed_git; p;
+			     p = p->next) {
 				size_t len;
 				if (strip_suffix(p->pack_name, ".pack", &len) &&
 				    len == base_len &&
@@ -820,7 +819,7 @@ unsigned long approximate_object_count(void)
 
 		prepare_packed_git();
 		count = 0;
-		for (p = packed_git; p; p = p->next) {
+		for (p = the_repository->objects.packed_git; p; p = p->next) {
 			if (open_pack_index(p))
 				continue;
 			count += p->num_objects;
@@ -869,18 +868,19 @@ static int sort_pack(const void *a_, const void *b_)
 
 static void rearrange_packed_git(void)
 {
-	packed_git = llist_mergesort(packed_git, get_next_packed_git,
-				     set_next_packed_git, sort_pack);
+	the_repository->objects.packed_git = llist_mergesort(
+		the_repository->objects.packed_git, get_next_packed_git,
+		set_next_packed_git, sort_pack);
 }
 
 static void prepare_packed_git_mru(void)
 {
 	struct packed_git *p;
 
-	INIT_LIST_HEAD(&packed_git_mru);
+	INIT_LIST_HEAD(&the_repository->objects.packed_git_mru);
 
-	for (p = packed_git; p; p = p->next)
-		list_add_tail(&p->mru, &packed_git_mru);
+	for (p = the_repository->objects.packed_git; p; p = p->next)
+		list_add_tail(&p->mru, &the_repository->objects.packed_git_mru);
 }
 
 static int prepare_packed_git_run_once = 0;
@@ -906,6 +906,16 @@ void reprepare_packed_git(void)
 	prepare_packed_git();
 }
 
+struct packed_git *get_packed_git(struct repository *r)
+{
+	return r->objects.packed_git;
+}
+
+struct list_head *get_packed_git_mru(struct repository *r)
+{
+	return &r->objects.packed_git_mru;
+}
+
 unsigned long unpack_object_header_buffer(const unsigned char *buf,
 		unsigned long len, enum object_type *type, unsigned long *sizep)
 {
@@ -1014,7 +1024,7 @@ const struct packed_git *has_packed_and_bad(const unsigned char *sha1)
 	struct packed_git *p;
 	unsigned i;
 
-	for (p = packed_git; p; p = p->next)
+	for (p = the_repository->objects.packed_git; p; p = p->next)
 		for (i = 0; i < p->num_bad_objects; i++)
 			if (!hashcmp(sha1, p->bad_object_sha1 + 20 * i))
 				return p;
@@ -1845,13 +1855,14 @@ int find_pack_entry(const unsigned char *sha1, struct pack_entry *e)
 	struct list_head *pos;
 
 	prepare_packed_git();
-	if (!packed_git)
+	if (!the_repository->objects.packed_git)
 		return 0;
 
-	list_for_each(pos, &packed_git_mru) {
+	list_for_each(pos, &the_repository->objects.packed_git_mru) {
 		struct packed_git *p = list_entry(pos, struct packed_git, mru);
 		if (fill_pack_entry(sha1, e, p)) {
-			list_move(&p->mru, &packed_git_mru);
+			list_move(&p->mru,
+				  &the_repository->objects.packed_git_mru);
 			return 1;
 		}
 	}
@@ -1898,7 +1909,7 @@ int for_each_packed_object(each_packed_object_fn cb, void *data, unsigned flags)
 	int pack_errors = 0;
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
 			continue;
 		if ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&
packfile.h +3 -0
index a7fca598d6..76496226bb 100644
@@ -38,6 +38,9 @@ extern void prepare_packed_git(void);
 extern void reprepare_packed_git(void);
 extern void install_packed_git(struct packed_git *pack);
 
+struct packed_git *get_packed_git(struct repository *r);
+struct list_head *get_packed_git_mru(struct repository *r);
+
 /*
  * Give a rough count of objects in the repository. This sacrifices accuracy
  * for speed.
server-info.c +3 -2
index 26a6c20b7d..6fe64ede17 100644
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "repository.h"
 #include "refs.h"
 #include "object.h"
 #include "commit.h"
@@ -200,7 +201,7 @@ static void init_pack_info(const char *infofile, int force)
 	objdirlen = strlen(objdir);
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = get_packed_git(the_repository); p; p = p->next) {
 		/* we ignore things on alternate path since they are
 		 * not available to the pullers in general.
 		 */
@@ -210,7 +211,7 @@ static void init_pack_info(const char *infofile, int force)
 	}
 	num_pack = i;
 	info = xcalloc(num_pack, sizeof(struct pack_info *));
-	for (i = 0, p = packed_git; p; p = p->next) {
+	for (i = 0, p = get_packed_git(the_repository); p; p = p->next) {
 		if (!p->pack_local)
 			continue;
 		info[i] = xcalloc(1, sizeof(struct pack_info));
sha1_name.c +3 -2
index 957ce25680..bd4d7352ce 100644
@@ -196,7 +196,8 @@ static void find_short_packed_object(struct disambiguate_state *ds)
 	struct packed_git *p;
 
 	prepare_packed_git();
-	for (p = packed_git; p && !ds->ambiguous; p = p->next)
+	for (p = get_packed_git(the_repository); p && !ds->ambiguous;
+	     p = p->next)
 		unique_in_pack(p, ds);
 }
 
@@ -566,7 +567,7 @@ static void find_abbrev_len_packed(struct min_abbrev_data *mad)
 	struct packed_git *p;
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next)
+	for (p = get_packed_git(the_repository); p; p = p->next)
 		find_abbrev_len_for_pack(p, mad);
 }
 
Signature · 1 line
2.16.1.435.g8f24da2e1a
Signature · 1 line
Brandon Williams
author
#27 lore raw

Re: [PATCH 10/44] object-store: move packed_git and packed_git_mru to object store

Quoted 7 lines · Brandon Williams

You're marking packed_git
as "private"...well C has no notion of private vs public fields in a struct so it might be difficult to keep that convention, it also took me a second to realize that it was only in the scope of packfile.c where it was ok to reference it directly. Maybe it'll be ok? If we really wanted something to be private we'd need it to be an opaque type instead, which may be out of the scope of this code refactor.

It's true C syntax does not support private/public scoping, but it does not mean we must avoid that. Python has "private convention" with the underscore prefix, no special support from the language either. Yes having compiler support to enforce scoping is nice, but I think we can still live without it (Go devs live fine without "const" arguments, e.g.)

So I'm going to make it clearer that these fields are not supposed to be accessed outside packfile.c

Signature · 1 line
Duy
author
#28 lore raw

Re: [PATCH 10/44] object-store: move packed_git and packed_git_mru to object store

Quoted 16 lines · Duy Nguyen
Quoted 7 lines · Brandon Williams

You're marking packed_git
as "private"...well C has no notion of private vs public fields in a struct so it might be difficult to keep that convention, it also took me a second to realize that it was only in the scope of packfile.c where it was ok to reference it directly. Maybe it'll be ok? If we really wanted something to be private we'd need it to be an opaque type instead, which may be out of the scope of this code refactor.

It's true C syntax does not support private/public scoping, but it does not mean we must avoid that. Python has "private convention" with the underscore prefix, no special support from the language either. Yes having compiler support to enforce scoping is nice, but I think we can still live without it (Go devs live fine without "const" arguments, e.g.)

Quoted 2 lines

So I'm going to make it clearer that these fields are not supposed to be accessed outside packfile.c

I'm not counting out the making these fields completely opaque of course. And with your suggestion of not embedding raw_object_store to repository, that's actually possible to do. But I'm still not doing it now :) The series is getting long and extending its scope will drag it even longer (in terms of both time and number of patches)

Signature · 1 line
Duy
#29 lore raw

Re: [PATCH 10/44] object-store: move packed_git and packed_git_mru to object store

Quoted 17 lines · Duy Nguyen
Quoted 16 lines · Duy Nguyen
Quoted 7 lines · Brandon Williams

You're marking packed_git
as "private"...well C has no notion of private vs public fields in a struct so it might be difficult to keep that convention, it also took me a second to realize that it was only in the scope of packfile.c where it was ok to reference it directly. Maybe it'll be ok? If we really wanted something to be private we'd need it to be an opaque type instead, which may be out of the scope of this code refactor.

It's true C syntax does not support private/public scoping, but it does not mean we must avoid that. Python has "private convention" with the underscore prefix, no special support from the language either. Yes having compiler support to enforce scoping is nice, but I think we can still live without it (Go devs live fine without "const" arguments, e.g.)

Quoted 8 lines
Quoted 2 lines

So I'm going to make it clearer that these fields are not supposed to be accessed outside packfile.c

I'm not counting out the making these fields completely opaque of course. And with your suggestion of not embedding raw_object_store to repository, that's actually possible to do. But I'm still not doing it now :) The series is getting long and extending its scope will drag it even longer (in terms of both time and number of patches)

Oh of course. I'm a big proponent of taking small steps, so definitely avoid scope creep and hopefully this series can land quicker so we have a better base to work with to make some of those other improvements if we still want to down the road.

Signature · 1 line
Brandon Williams
#30 lore raw

PATCH 11/44 object-store: close all packs upon clearing the object store

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

---
 builtin/am.c           | 2 +-
 builtin/clone.c        | 2 +-
 builtin/fetch.c        | 2 +-
 builtin/merge.c        | 2 +-
 builtin/receive-pack.c | 2 +-
 object.c               | 7 +++----
 packfile.c             | 4 ++--
 packfile.h             | 2 +-
 8 files changed, 11 insertions(+), 12 deletions(-)
builtin/am.c +1 -1
index 5bdd2d7578..4762a702e3 100644
@@ -1859,7 +1859,7 @@ static void am_run(struct am_state *state, int resume)
 	 */
 	if (!state->rebasing) {
 		am_destroy(state);
-		close_all_packs();
+		close_all_packs(&the_repository->objects);
 		run_command_v_opt(argv_gc_auto, RUN_GIT_CMD);
 	}
 }
builtin/clone.c +1 -1
index 101c27a593..13cfaa6488 100644
@@ -1217,7 +1217,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 	transport_disconnect(transport);
 
 	if (option_dissociate) {
-		close_all_packs();
+		close_all_packs(&the_repository->objects);
 		dissociate_from_references();
 	}
 
builtin/fetch.c +1 -1
index 8ee998ea2e..4d72efca78 100644
@@ -1478,7 +1478,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
 
 	string_list_clear(&list, 0);
 
-	close_all_packs();
+	close_all_packs(&the_repository->objects);
 
 	argv_array_pushl(&argv_gc_auto, "gc", "--auto", NULL);
 	if (verbosity < 0)
builtin/merge.c +1 -1
index 30264cfd7c..907ae44ab5 100644
@@ -411,7 +411,7 @@ static void finish(struct commit *head_commit,
 			 * We ignore errors in 'gc --auto', since the
 			 * user should see them.
 			 */
-			close_all_packs();
+			close_all_packs(&the_repository->objects);
 			run_command_v_opt(argv_gc_auto, RUN_GIT_CMD);
 		}
 	}
builtin/receive-pack.c +1 -1
index b7ce7c7f52..ac478b7b99 100644
@@ -2026,7 +2026,7 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)
 			proc.git_cmd = 1;
 			proc.argv = argv_gc_auto;
 
-			close_all_packs();
+			close_all_packs(&the_repository->objects);
 			if (!start_command(&proc)) {
 				if (use_sideband)
 					copy_to_sideband(proc.err, -1, NULL);
object.c +3 -4
index 83be6b1ecb..60ca76b285 100644
@@ -4,6 +4,7 @@
 #include "tree.h"
 #include "commit.h"
 #include "tag.h"
+#include "packfile.h"
 
 static struct object **obj_hash;
 static int nr_objs, obj_hash_size;
@@ -475,8 +476,6 @@ void raw_object_store_clear(struct raw_object_store *o)
 	o->alt_odb_tail = NULL;
 
 	INIT_LIST_HEAD(&o->packed_git_mru);
-	/*
-	 * TODO: call close_all_packs once migrated to
-	 * take an object store argument
-	 */
+	close_all_packs(o);
+	o->packed_git = NULL;
 }
packfile.c +2 -2
index d3c4a12ae0..1e38334ba2 100644
@@ -310,11 +310,11 @@ static void close_pack(struct packed_git *p)
 	close_pack_index(p);
 }
 
-void close_all_packs(void)
+void close_all_packs(struct raw_object_store *o)
 {
 	struct packed_git *p;
 
-	for (p = the_repository->objects.packed_git; p; p = p->next)
+	for (p = o->packed_git; p; p = p->next)
 		if (p->do_not_close)
 			die("BUG: want to close pack marked 'do-not-close'");
 		else
packfile.h +1 -1
index 76496226bb..5b1ce00f84 100644
@@ -66,7 +66,7 @@ extern void close_pack_index(struct packed_git *);
 
 extern unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t, unsigned long *);
 extern void close_pack_windows(struct packed_git *);
-extern void close_all_packs(void);
+extern void close_all_packs(struct raw_object_store *o);
 extern void unuse_pack(struct pack_window **);
 extern void clear_delta_base_cache(void);
 extern struct packed_git *add_packed_git(const char *path, size_t path_len, int local);
Signature · 1 line
2.16.1.435.g8f24da2e1a
#31 lore raw

Re: [PATCH 11/44] object-store: close all packs upon clearing the object store

Quoted 5 lines · Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Looks good.

Quoted 133 lines
---
 builtin/am.c           | 2 +-
 builtin/clone.c        | 2 +-
 builtin/fetch.c        | 2 +-
 builtin/merge.c        | 2 +-
 builtin/receive-pack.c | 2 +-
 object.c               | 7 +++----
 packfile.c             | 4 ++--
 packfile.h             | 2 +-
 8 files changed, 11 insertions(+), 12 deletions(-)
builtin/am.c +1 -1
index 5bdd2d7578..4762a702e3 100644
@@ -1859,7 +1859,7 @@ static void am_run(struct am_state *state, int resume)
 	 */
 	if (!state->rebasing) {
 		am_destroy(state);
-		close_all_packs();
+		close_all_packs(&the_repository->objects);
 		run_command_v_opt(argv_gc_auto, RUN_GIT_CMD);
 	}
 }
builtin/clone.c +1 -1
index 101c27a593..13cfaa6488 100644
@@ -1217,7 +1217,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 	transport_disconnect(transport);
 
 	if (option_dissociate) {
-		close_all_packs();
+		close_all_packs(&the_repository->objects);
 		dissociate_from_references();
 	}
 
builtin/fetch.c +1 -1
index 8ee998ea2e..4d72efca78 100644
@@ -1478,7 +1478,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
 
 	string_list_clear(&list, 0);
 
-	close_all_packs();
+	close_all_packs(&the_repository->objects);
 
 	argv_array_pushl(&argv_gc_auto, "gc", "--auto", NULL);
 	if (verbosity < 0)
builtin/merge.c +1 -1
index 30264cfd7c..907ae44ab5 100644
@@ -411,7 +411,7 @@ static void finish(struct commit *head_commit,
 			 * We ignore errors in 'gc --auto', since the
 			 * user should see them.
 			 */
-			close_all_packs();
+			close_all_packs(&the_repository->objects);
 			run_command_v_opt(argv_gc_auto, RUN_GIT_CMD);
 		}
 	}
builtin/receive-pack.c +1 -1
index b7ce7c7f52..ac478b7b99 100644
@@ -2026,7 +2026,7 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)
 			proc.git_cmd = 1;
 			proc.argv = argv_gc_auto;
 
-			close_all_packs();
+			close_all_packs(&the_repository->objects);
 			if (!start_command(&proc)) {
 				if (use_sideband)
 					copy_to_sideband(proc.err, -1, NULL);
object.c +3 -4
index 83be6b1ecb..60ca76b285 100644
@@ -4,6 +4,7 @@
 #include "tree.h"
 #include "commit.h"
 #include "tag.h"
+#include "packfile.h"
 
 static struct object **obj_hash;
 static int nr_objs, obj_hash_size;
@@ -475,8 +476,6 @@ void raw_object_store_clear(struct raw_object_store *o)
 	o->alt_odb_tail = NULL;
 
 	INIT_LIST_HEAD(&o->packed_git_mru);
-	/*
-	 * TODO: call close_all_packs once migrated to
-	 * take an object store argument
-	 */
+	close_all_packs(o);
+	o->packed_git = NULL;
 }
packfile.c +2 -2
index d3c4a12ae0..1e38334ba2 100644
@@ -310,11 +310,11 @@ static void close_pack(struct packed_git *p)
 	close_pack_index(p);
 }
 
-void close_all_packs(void)
+void close_all_packs(struct raw_object_store *o)
 {
 	struct packed_git *p;
 
-	for (p = the_repository->objects.packed_git; p; p = p->next)
+	for (p = o->packed_git; p; p = p->next)
 		if (p->do_not_close)
 			die("BUG: want to close pack marked 'do-not-close'");
 		else
packfile.h +1 -1
index 76496226bb..5b1ce00f84 100644
@@ -66,7 +66,7 @@ extern void close_pack_index(struct packed_git *);
 
 extern unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t, unsigned long *);
 extern void close_pack_windows(struct packed_git *);
-extern void close_all_packs(void);
+extern void close_all_packs(struct raw_object_store *o);
 extern void unuse_pack(struct pack_window **);
 extern void clear_delta_base_cache(void);
 extern struct packed_git *add_packed_git(const char *path, size_t path_len, int local);
Signature · 1 line
2.16.1.435.g8f24da2e1a
Signature · 1 line
Brandon Williams
#32 lore raw

PATCH 12/44 pack: move prepare_packed_git_run_once to object store

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Each repository's object store can be initialized independently, so they must not share a run_once variable.

---
 object-store.h | 6 ++++++
 packfile.c     | 7 +++----
 2 files changed, 9 insertions(+), 4 deletions(-)
object-store.h +6 -0
index 1f3e66a3b8..b954396615 100644
@@ -92,6 +92,12 @@ struct raw_object_store {
 
 	struct alternate_object_database *alt_odb_list;
 	struct alternate_object_database **alt_odb_tail;
+
+	/*
+	 * Whether packed_git has already been populated with this repository's
+	 * packs.
+	 */
+	unsigned packed_git_initialized : 1;
 };
 
 void raw_object_store_init(struct raw_object_store *o);
packfile.c +3 -4
index 1e38334ba2..caeab0f68c 100644
@@ -883,12 +883,11 @@ static void prepare_packed_git_mru(void)
 		list_add_tail(&p->mru, &the_repository->objects.packed_git_mru);
 }
 
-static int prepare_packed_git_run_once = 0;
 void prepare_packed_git(void)
 {
 	struct alternate_object_database *alt;
 
-	if (prepare_packed_git_run_once)
+	if (the_repository->objects.packed_git_initialized)
 		return;
 	prepare_packed_git_one(get_object_directory(), 1);
 	prepare_alt_odb();
@@ -896,13 +895,13 @@ void prepare_packed_git(void)
 		prepare_packed_git_one(alt->path, 0);
 	rearrange_packed_git();
 	prepare_packed_git_mru();
-	prepare_packed_git_run_once = 1;
+	the_repository->objects.packed_git_initialized = 1;
 }
 
 void reprepare_packed_git(void)
 {
 	approximate_object_count_valid = 0;
-	prepare_packed_git_run_once = 0;
+	the_repository->objects.packed_git_initialized = 0;
 	prepare_packed_git();
 }
 
Signature · 1 line
2.16.1.435.g8f24da2e1a
#33 lore raw

Re: [PATCH 12/44] pack: move prepare_packed_git_run_once to object store

Quoted 4 lines · Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Each repository's object store can be initialized independently, so they must not share a run_once variable.

Looks good.

Quoted 63 lines
---
 object-store.h | 6 ++++++
 packfile.c     | 7 +++----
 2 files changed, 9 insertions(+), 4 deletions(-)
object-store.h +6 -0
index 1f3e66a3b8..b954396615 100644
@@ -92,6 +92,12 @@ struct raw_object_store {
 
 	struct alternate_object_database *alt_odb_list;
 	struct alternate_object_database **alt_odb_tail;
+
+	/*
+	 * Whether packed_git has already been populated with this repository's
+	 * packs.
+	 */
+	unsigned packed_git_initialized : 1;
 };
 
 void raw_object_store_init(struct raw_object_store *o);
packfile.c +3 -4
index 1e38334ba2..caeab0f68c 100644
@@ -883,12 +883,11 @@ static void prepare_packed_git_mru(void)
 		list_add_tail(&p->mru, &the_repository->objects.packed_git_mru);
 }
 
-static int prepare_packed_git_run_once = 0;
 void prepare_packed_git(void)
 {
 	struct alternate_object_database *alt;
 
-	if (prepare_packed_git_run_once)
+	if (the_repository->objects.packed_git_initialized)
 		return;
 	prepare_packed_git_one(get_object_directory(), 1);
 	prepare_alt_odb();
@@ -896,13 +895,13 @@ void prepare_packed_git(void)
 		prepare_packed_git_one(alt->path, 0);
 	rearrange_packed_git();
 	prepare_packed_git_mru();
-	prepare_packed_git_run_once = 1;
+	the_repository->objects.packed_git_initialized = 1;
 }
 
 void reprepare_packed_git(void)
 {
 	approximate_object_count_valid = 0;
-	prepare_packed_git_run_once = 0;
+	the_repository->objects.packed_git_initialized = 0;
 	prepare_packed_git();
 }
 
Signature · 1 line
2.16.1.435.g8f24da2e1a
Signature · 1 line
Brandon Williams
#34 lore raw

PATCH 13/44 pack: move approximate object count to object store

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

The approximate_object_count() function maintains a rough count of objects in a repository to estimate how long object name abbreviates should be. Object names are scoped to a repository and the appropriate length may differ by repository, so the object count should not be global.

---
 object-store.h |  8 ++++++++
 packfile.c     | 11 +++++------
 2 files changed, 13 insertions(+), 6 deletions(-)
object-store.h +8 -0
index b954396615..56ed4bd9e9 100644
@@ -93,6 +93,14 @@ struct raw_object_store {
 	struct alternate_object_database *alt_odb_list;
 	struct alternate_object_database **alt_odb_tail;
 
+	/*
+	 * A fast, rough count of the number of objects in the repository.
+	 * These two fields are not meant for direct access. Use
+	 * approximate_object_count() instead.
+	 */
+	unsigned long approximate_object_count;
+	unsigned approximate_object_count_valid : 1;
+
 	/*
 	 * Whether packed_git has already been populated with this repository's
 	 * packs.
packfile.c +5 -6
index caeab0f68c..ce4cd7d5c9 100644
@@ -802,8 +802,6 @@ static void prepare_packed_git_one(char *objdir, int local)
 	strbuf_release(&path);
 }
 
-static int approximate_object_count_valid;
-
 /*
  * Give a fast, rough count of the number of objects in the repository. This
  * ignores loose objects completely. If you have a lot of them, then either
@@ -813,8 +811,8 @@ static int approximate_object_count_valid;
  */
 unsigned long approximate_object_count(void)
 {
-	static unsigned long count;
-	if (!approximate_object_count_valid) {
+	if (!the_repository->objects.approximate_object_count_valid) {
+		unsigned long count;
 		struct packed_git *p;
 
 		prepare_packed_git();
@@ -824,8 +822,9 @@ unsigned long approximate_object_count(void)
 				continue;
 			count += p->num_objects;
 		}
+		the_repository->objects.approximate_object_count = count;
 	}
-	return count;
+	return the_repository->objects.approximate_object_count;
 }
 
 static void *get_next_packed_git(const void *p)
@@ -900,7 +899,7 @@ void prepare_packed_git(void)
 
 void reprepare_packed_git(void)
 {
-	approximate_object_count_valid = 0;
+	the_repository->objects.approximate_object_count_valid = 0;
 	the_repository->objects.packed_git_initialized = 0;
 	prepare_packed_git();
 }
Signature · 1 line
2.16.1.435.g8f24da2e1a
#35 lore raw

Re: [PATCH 13/44] pack: move approximate object count to object store

Quoted 31 lines · Nguyễn Thái Ngọc Duy

The approximate_object_count() function maintains a rough count of objects in a repository to estimate how long object name abbreviates should be. Object names are scoped to a repository and the appropriate length may differ by repository, so the object count should not be global.

---

packfile.c +4 -3
@@ -813,8 +811,8 @@ static int approximate_object_count_valid;
 unsigned long approximate_object_count(void)
 {
-       static unsigned long count;
-       if (!approximate_object_count_valid) {
+       if (!the_repository->objects.approximate_object_count_valid) {
+               unsigned long count;
                struct packed_git *p;
 
                prepare_packed_git();
@@ -824,8 +822,9 @@ unsigned long approximate_object_count(void)
                                continue;
                        count += p->num_objects;
                }
+               the_repository->objects.approximate_object_count = count;
        }
-       return count;
+       return the_repository->objects.approximate_object_count;
 }

packfile.c void prepare_packed_git(void) · line 901

Quoted 4 lines
packfile.c +1 -1
@@ -900,7 +899,7 @@ void prepare_packed_git(void)
 void reprepare_packed_git(void)
 {
-       approximate_object_count_valid = 0;
+       the_repository->objects.approximate_object_count_valid = 0;

Not an issue specific to this patch, but where, how, when does 'approximate_object_count_valid' ever get set to anything other than 0? Even in the existing code (without this patch), there doesn't seem to be anyplace which sets this to a non-zero value. Am I missing something obvious (or subtle)?

author
#36 lore raw

Re: [PATCH 13/44] pack: move approximate object count to object store

Quoted 34 lines · Eric Sunshine
Quoted 33 lines · Nguyễn Thái Ngọc Duy

The approximate_object_count() function maintains a rough count of objects in a repository to estimate how long object name abbreviates should be. Object names are scoped to a repository and the appropriate length may differ by repository, so the object count should not be global.

---

packfile.c +4 -3
@@ -813,8 +811,8 @@ static int approximate_object_count_valid;
 unsigned long approximate_object_count(void)
 {
-       static unsigned long count;
-       if (!approximate_object_count_valid) {
+       if (!the_repository->objects.approximate_object_count_valid) {
+               unsigned long count;
                struct packed_git *p;
 
                prepare_packed_git();
@@ -824,8 +822,9 @@ unsigned long approximate_object_count(void)
                                continue;
                        count += p->num_objects;
                }
+               the_repository->objects.approximate_object_count = count;
        }
-       return count;
+       return the_repository->objects.approximate_object_count;
 }
@@ -900,7 +899,7 @@ void prepare_packed_git(void)
 void reprepare_packed_git(void)
 {
Quoted 8 lines

packfile.c void prepare_packed_git(void) · line 902

Quoted 2 lines
packfile.c +1 -1
-       approximate_object_count_valid = 0;
+       the_repository->objects.approximate_object_count_valid = 0;

Not an issue specific to this patch, but where, how, when does 'approximate_object_count_valid' ever get set to anything other than 0? Even in the existing code (without this patch), there doesn't seem to be anyplace which sets this to a non-zero value. Am I missing something obvious (or subtle)?

Probably related to this
https://public-inbox.org/git/[email protected]/#t

Signature · 1 line
Duy
#37 lore raw

Re: [PATCH 13/44] pack: move approximate object count to object store

Quoted 44 lines · Duy Nguyen
Quoted 34 lines · Eric Sunshine
Quoted 33 lines · Nguyễn Thái Ngọc Duy

The approximate_object_count() function maintains a rough count of objects in a repository to estimate how long object name abbreviates should be. Object names are scoped to a repository and the appropriate length may differ by repository, so the object count should not be global.

---

packfile.c +4 -3
@@ -813,8 +811,8 @@ static int approximate_object_count_valid;
 unsigned long approximate_object_count(void)
 {
-       static unsigned long count;
-       if (!approximate_object_count_valid) {
+       if (!the_repository->objects.approximate_object_count_valid) {
+               unsigned long count;
                struct packed_git *p;
 
                prepare_packed_git();
@@ -824,8 +822,9 @@ unsigned long approximate_object_count(void)
                                continue;
                        count += p->num_objects;
                }
+               the_repository->objects.approximate_object_count = count;
        }
-       return count;
+       return the_repository->objects.approximate_object_count;
 }
@@ -900,7 +899,7 @@ void prepare_packed_git(void)
 void reprepare_packed_git(void)
 {
Quoted 8 lines

packfile.c void prepare_packed_git(void) · line 902

Quoted 2 lines
packfile.c +1 -1
-       approximate_object_count_valid = 0;
+       the_repository->objects.approximate_object_count_valid = 0;

Not an issue specific to this patch, but where, how, when does 'approximate_object_count_valid' ever get set to anything other than 0? Even in the existing code (without this patch), there doesn't seem to be anyplace which sets this to a non-zero value. Am I missing something obvious (or subtle)?

Quoted 2 lines

Probably related to this
https://public-inbox.org/git/[email protected]/#t

Yeah as far as doing a conversion this should be fine, we can come by and clean it up at a later point.

Signature · 1 line
Brandon Williams
#38 lore raw

PATCH 14/44 sha1_file: add raw_object_store argument to alt_odb_usable

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Add a raw_object_store to alt_odb_usable to be more specific about which repository to act on. The choice of the repository is delegated to its only caller link_alt_odb_entry.

---
 sha1_file.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
sha1_file.c +5 -3
index 4c9635dada..77c099dadd 100644
@@ -346,7 +346,9 @@ static const char *alt_sha1_path(struct alternate_object_database *alt,
 /*
  * Return non-zero iff the path is usable as an alternate object database.
  */
-static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)
+static int alt_odb_usable(struct raw_object_store *o,
+			  struct strbuf *path,
+			  const char *normalized_objdir)
 {
 	struct alternate_object_database *alt;
 
@@ -362,7 +364,7 @@ static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)
 	 * Prevent the common mistake of listing the same
 	 * thing twice, or object directory itself.
 	 */
-	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
+	for (alt = o->alt_odb_list; alt; alt = alt->next) {
 		if (!fspathcmp(path->buf, alt->path))
 			return 0;
 	}
@@ -414,7 +416,7 @@ static int link_alt_odb_entry(const char *entry, const char *relative_base,
 	while (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')
 		strbuf_setlen(&pathbuf, pathbuf.len - 1);
 
-	if (!alt_odb_usable(&pathbuf, normalized_objdir)) {
+	if (!alt_odb_usable(&the_repository->objects, &pathbuf, normalized_objdir)) {
 		strbuf_release(&pathbuf);
 		return -1;
 	}
Signature · 1 line
2.16.1.435.g8f24da2e1a
#39 lore raw

PATCH 15/44 sha1_file: add repository argument to link_alt_odb_entry

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Add a repository argument to allow the link_alt_odb_entry caller to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

Since the implementation does not yet work with other repositories, use a wrapper macro to enforce that the caller passes in the_repository as the first argument. It would be more appealing to use BUILD_ASSERT_OR_ZERO to enforce this, but that doesn't work because it requires a compile-time constant and common compilers like gcc 4.8.4 do not consider "r == the_repository" a compile-time constant.

This and the following three patches add repository arguments to link_alt_odb_entry, read_info_alternates, link_alt_odb_entries and prepare_alt_odb. Three out of the four functions are found in a recursive call chain, calling each other, and one of them accesses the repositories objectdir (which was migrated; it was an obvious choice) and ignore_env (which we need to keep in the repository struct for clarify); hence we will pass through the repository unlike just the object store object + the ignore_env flag.

---
 sha1_file.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
sha1_file.c +5 -3
index 77c099dadd..2ee5fe6ba3 100644
@@ -390,8 +390,9 @@ static int alt_odb_usable(struct raw_object_store *o,
  * terminating NUL.
  */
 static void read_info_alternates(const char * relative_base, int depth);
-static int link_alt_odb_entry(const char *entry, const char *relative_base,
-	int depth, const char *normalized_objdir)
+#define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)
+static int link_alt_odb_entry_the_repository(const char *entry,
+	const char *relative_base, int depth, const char *normalized_objdir)
 {
 	struct alternate_object_database *ent;
 	struct strbuf pathbuf = STRBUF_INIT;
@@ -488,7 +489,8 @@ static void link_alt_odb_entries(const char *alt, int sep,
 		alt = parse_alt_odb_entry(alt, sep, &entry);
 		if (!entry.len)
 			continue;
-		link_alt_odb_entry(entry.buf, relative_base, depth, objdirbuf.buf);
+		link_alt_odb_entry(the_repository, entry.buf,
+				   relative_base, depth, objdirbuf.buf);
 	}
 	strbuf_release(&entry);
 	strbuf_release(&objdirbuf);
Signature · 1 line
2.16.1.435.g8f24da2e1a
#40 lore raw

PATCH 16/44 sha1_file: add repository argument to read_info_alternates

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

See previous patch for explanation.

---
 sha1_file.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
sha1_file.c +7 -4
index 2ee5fe6ba3..175ea58686 100644
@@ -389,7 +389,9 @@ static int alt_odb_usable(struct raw_object_store *o,
  * SHA1, an extra slash for the first level indirection, and the
  * terminating NUL.
  */
-static void read_info_alternates(const char * relative_base, int depth);
+#define read_info_alternates(r, rb, d) read_info_alternates_##r(rb, d)
+static void read_info_alternates_the_repository(const char *relative_base,
+						int depth);
 #define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)
 static int link_alt_odb_entry_the_repository(const char *entry,
 	const char *relative_base, int depth, const char *normalized_objdir)
@@ -430,7 +432,7 @@ static int link_alt_odb_entry_the_repository(const char *entry,
 	ent->next = NULL;
 
 	/* recursively add alternates */
-	read_info_alternates(pathbuf.buf, depth + 1);
+	read_info_alternates(the_repository, pathbuf.buf, depth + 1);
 
 	strbuf_release(&pathbuf);
 	return 0;
@@ -496,7 +498,8 @@ static void link_alt_odb_entries(const char *alt, int sep,
 	strbuf_release(&objdirbuf);
 }
 
-static void read_info_alternates(const char * relative_base, int depth)
+static void read_info_alternates_the_repository(const char *relative_base,
+						int depth)
 {
 	char *path;
 	struct strbuf buf = STRBUF_INIT;
@@ -677,7 +680,7 @@ void prepare_alt_odb(void)
 	link_alt_odb_entries(the_repository->objects.alternate_db,
 			     PATH_SEP, NULL, 0);
 
-	read_info_alternates(get_object_directory(), 0);
+	read_info_alternates(the_repository, get_object_directory(), 0);
 }
 
 /* Returns 1 if we have successfully freshened the file, 0 otherwise. */
Signature · 1 line
2.16.1.435.g8f24da2e1a
#41 lore raw

PATCH 17/44 sha1_file: add repository argument to link_alt_odb_entries

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

See previous patch for explanation.

---
 sha1_file.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)
sha1_file.c +12 -6
index 175ea58686..4f9c07b547 100644
@@ -467,8 +467,12 @@ static const char *parse_alt_odb_entry(const char *string,
 	return end;
 }
 
-static void link_alt_odb_entries(const char *alt, int sep,
-				 const char *relative_base, int depth)
+#define link_alt_odb_entries(r, a, s, rb, d) \
+	link_alt_odb_entries_##r(a, s, rb, d)
+static void link_alt_odb_entries_the_repository(const char *alt,
+						int sep,
+						const char *relative_base,
+						int depth)
 {
 	struct strbuf objdirbuf = STRBUF_INIT;
 	struct strbuf entry = STRBUF_INIT;
@@ -511,7 +515,7 @@ static void read_info_alternates_the_repository(const char *relative_base,
 		return;
 	}
 
-	link_alt_odb_entries(buf.buf, '\n', relative_base, depth);
+	link_alt_odb_entries(the_repository, buf.buf, '\n', relative_base, depth);
 	strbuf_release(&buf);
 	free(path);
 }
@@ -565,7 +569,8 @@ void add_to_alternates_file(const char *reference)
 		if (commit_lock_file(&lock))
 			die_errno("unable to move new alternates file into place");
 		if (the_repository->objects.alt_odb_tail)
-			link_alt_odb_entries(reference, '\n', NULL, 0);
+			link_alt_odb_entries(the_repository, reference,
+					     '\n', NULL, 0);
 	}
 	free(alts);
 }
@@ -578,7 +583,8 @@ void add_to_alternates_memory(const char *reference)
 	 */
 	prepare_alt_odb();
 
-	link_alt_odb_entries(reference, '\n', NULL, 0);
+	link_alt_odb_entries(the_repository, reference,
+			     '\n', NULL, 0);
 }
 
 /*
@@ -677,7 +683,7 @@ void prepare_alt_odb(void)
 
 	the_repository->objects.alt_odb_tail =
 			&the_repository->objects.alt_odb_list;
-	link_alt_odb_entries(the_repository->objects.alternate_db,
+	link_alt_odb_entries(the_repository, the_repository->objects.alternate_db,
 			     PATH_SEP, NULL, 0);
 
 	read_info_alternates(the_repository, get_object_directory(), 0);
Signature · 1 line
2.16.1.435.g8f24da2e1a
#42 lore raw

PATCH 18/44 sha1_file: add repository argument to prepare_alt_odb

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

See previous patch for explanation.

---
 builtin/fsck.c |  2 +-
 object-store.h |  3 ++-
 packfile.c     |  2 +-
 sha1_file.c    | 13 +++++++------
 sha1_name.c    |  2 +-
 5 files changed, 12 insertions(+), 10 deletions(-)
builtin/fsck.c +1 -1
index 7aca9699f6..b0abba6e04 100644
@@ -716,7 +716,7 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 	} else {
 		fsck_object_dir(get_object_directory());
 
-		prepare_alt_odb();
+		prepare_alt_odb(the_repository);
 		for (alt = the_repository->objects.alt_odb_list;
 		     alt; alt = alt->next)
 			fsck_object_dir(alt->path);
object-store.h +2 -1
index 56ed4bd9e9..3d0f8a87cb 100644
@@ -20,7 +20,8 @@ struct alternate_object_database {
 
 	char path[FLEX_ARRAY];
 };
-void prepare_alt_odb(void);
+#define prepare_alt_odb(r) prepare_alt_odb_##r()
+void prepare_alt_odb_the_repository(void);
 char *compute_alternate_path(const char *path, struct strbuf *err);
 typedef int alt_odb_fn(struct alternate_object_database *, void *);
 int foreach_alt_odb(alt_odb_fn, void*);
packfile.c +1 -1
index ce4cd7d5c9..a0842521e5 100644
@@ -889,7 +889,7 @@ void prepare_packed_git(void)
 	if (the_repository->objects.packed_git_initialized)
 		return;
 	prepare_packed_git_one(get_object_directory(), 1);
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next)
 		prepare_packed_git_one(alt->path, 0);
 	rearrange_packed_git();
sha1_file.c +7 -6
index 4f9c07b547..027e0f3741 100644
@@ -23,6 +23,7 @@
 #include "sha1-lookup.h"
 #include "bulk-checkin.h"
 #include "repository.h"
+#include "object-store.h"
 #include "streaming.h"
 #include "dir.h"
 #include "list.h"
@@ -581,7 +582,7 @@ void add_to_alternates_memory(const char *reference)
 	 * Make sure alternates are initialized, or else our entry may be
 	 * overwritten when they are.
 	 */
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 
 	link_alt_odb_entries(the_repository, reference,
 			     '\n', NULL, 0);
@@ -667,7 +668,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 	struct alternate_object_database *ent;
 	int r = 0;
 
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	for (ent = the_repository->objects.alt_odb_list; ent; ent = ent->next) {
 		r = fn(ent, cb);
 		if (r)
@@ -676,7 +677,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 	return r;
 }
 
-void prepare_alt_odb(void)
+void prepare_alt_odb_the_repository(void)
 {
 	if (the_repository->objects.alt_odb_tail)
 		return;
@@ -726,7 +727,7 @@ static int check_and_freshen_local(const unsigned char *sha1, int freshen)
 static int check_and_freshen_nonlocal(const unsigned char *sha1, int freshen)
 {
 	struct alternate_object_database *alt;
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		const char *path = alt_sha1_path(alt, sha1);
 		if (check_and_freshen_file(path, freshen))
@@ -885,7 +886,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,
 	if (!lstat(*path, st))
 		return 0;
 
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	errno = ENOENT;
 	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
@@ -916,7 +917,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)
 		return fd;
 	most_interesting_errno = errno;
 
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		fd = git_open(*path);
sha1_name.c +1 -1
index bd4d7352ce..2065be90d2 100644
@@ -353,7 +353,7 @@ static int init_object_disambiguation(const char *name, int len,
 
 	ds->len = len;
 	ds->hex_pfx[len] = '\0';
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	return 0;
 }
 
Signature · 1 line
2.16.1.435.g8f24da2e1a
#43 lore raw

PATCH 19/44 sha1_file: allow link_alt_odb_entries to handle arbitrary repositories

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Actually this also allows read_info_alternates and link_alt_odb_entry to handle arbitrary repositories, but link_alt_odb_entries is the most interesting function in this set of functions, hence the commit subject.

These functions span a strongly connected component in the function graph, i.e. the recursive call chain might look like

-> link_alt_odb_entries

    -> link_alt_odb_entry
      -> read_info_alternates
        -> link_alt_odb_entries

That is why we need to convert all these functions at the same time.

---
 object-store.h |  4 ++++
 sha1_file.c    | 36 ++++++++++++++++--------------------
 2 files changed, 20 insertions(+), 20 deletions(-)
object-store.h +4 -0
index 3d0f8a87cb..20ba136c1f 100644
@@ -18,6 +18,10 @@ struct alternate_object_database {
 	char loose_objects_subdir_seen[256];
 	struct oid_array loose_objects_cache;
 
+	/*
+	 * Path to the alternative object store. If this is a relative path,
+	 * it is relative to the current working directory.
+	 */
 	char path[FLEX_ARRAY];
 };
 #define prepare_alt_odb(r) prepare_alt_odb_##r()
sha1_file.c +16 -20
index 027e0f3741..f34eb69e39 100644
@@ -390,11 +390,10 @@ static int alt_odb_usable(struct raw_object_store *o,
  * SHA1, an extra slash for the first level indirection, and the
  * terminating NUL.
  */
-#define read_info_alternates(r, rb, d) read_info_alternates_##r(rb, d)
-static void read_info_alternates_the_repository(const char *relative_base,
-						int depth);
-#define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)
-static int link_alt_odb_entry_the_repository(const char *entry,
+static void read_info_alternates(struct repository *r,
+				 const char *relative_base,
+				 int depth);
+static int link_alt_odb_entry(struct repository *r, const char *entry,
 	const char *relative_base, int depth, const char *normalized_objdir)
 {
 	struct alternate_object_database *ent;
@@ -420,7 +419,7 @@ static int link_alt_odb_entry_the_repository(const char *entry,
 	while (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')
 		strbuf_setlen(&pathbuf, pathbuf.len - 1);
 
-	if (!alt_odb_usable(&the_repository->objects, &pathbuf, normalized_objdir)) {
+	if (!alt_odb_usable(&r->objects, &pathbuf, normalized_objdir)) {
 		strbuf_release(&pathbuf);
 		return -1;
 	}
@@ -428,12 +427,12 @@ static int link_alt_odb_entry_the_repository(const char *entry,
 	ent = alloc_alt_odb(pathbuf.buf);
 
 	/* add the alternate entry */
-	*the_repository->objects.alt_odb_tail = ent;
-	the_repository->objects.alt_odb_tail = &(ent->next);
+	*r->objects.alt_odb_tail = ent;
+	r->objects.alt_odb_tail = &(ent->next);
 	ent->next = NULL;
 
 	/* recursively add alternates */
-	read_info_alternates(the_repository, pathbuf.buf, depth + 1);
+	read_info_alternates(r, pathbuf.buf, depth + 1);
 
 	strbuf_release(&pathbuf);
 	return 0;
@@ -468,12 +467,8 @@ static const char *parse_alt_odb_entry(const char *string,
 	return end;
 }
 
-#define link_alt_odb_entries(r, a, s, rb, d) \
-	link_alt_odb_entries_##r(a, s, rb, d)
-static void link_alt_odb_entries_the_repository(const char *alt,
-						int sep,
-						const char *relative_base,
-						int depth)
+static void link_alt_odb_entries(struct repository *r, const char *alt,
+				 int sep, const char *relative_base, int depth)
 {
 	struct strbuf objdirbuf = STRBUF_INIT;
 	struct strbuf entry = STRBUF_INIT;
@@ -487,7 +482,7 @@ static void link_alt_odb_entries_the_repository(const char *alt,
 		return;
 	}
 
-	strbuf_add_absolute_path(&objdirbuf, get_object_directory());
+	strbuf_add_absolute_path(&objdirbuf, r->objects.objectdir);
 	if (strbuf_normalize_path(&objdirbuf) < 0)
 		die("unable to normalize object directory: %s",
 		    objdirbuf.buf);
@@ -496,15 +491,16 @@ static void link_alt_odb_entries_the_repository(const char *alt,
 		alt = parse_alt_odb_entry(alt, sep, &entry);
 		if (!entry.len)
 			continue;
-		link_alt_odb_entry(the_repository, entry.buf,
+		link_alt_odb_entry(r, entry.buf,
 				   relative_base, depth, objdirbuf.buf);
 	}
 	strbuf_release(&entry);
 	strbuf_release(&objdirbuf);
 }
 
-static void read_info_alternates_the_repository(const char *relative_base,
-						int depth)
+static void read_info_alternates(struct repository *r,
+				 const char *relative_base,
+				 int depth)
 {
 	char *path;
 	struct strbuf buf = STRBUF_INIT;
@@ -516,7 +512,7 @@ static void read_info_alternates_the_repository(const char *relative_base,
 		return;
 	}
 
-	link_alt_odb_entries(the_repository, buf.buf, '\n', relative_base, depth);
+	link_alt_odb_entries(r, buf.buf, '\n', relative_base, depth);
 	strbuf_release(&buf);
 	free(path);
 }
Signature · 1 line
2.16.1.435.g8f24da2e1a
#44 lore raw

Re: [PATCH 19/44] sha1_file: allow link_alt_odb_entries to handle arbitrary repositories

Quoted 9 lines · Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Actually this also allows read_info_alternates and link_alt_odb_entry to handle arbitrary repositories, but link_alt_odb_entries is the most interesting function in this set of functions, hence the commit subject.

These functions span a strongly connected component in the function graph, i.e. the recursive call chain might look like

Quoted 6 lines

-> link_alt_odb_entries

    -> link_alt_odb_entry
      -> read_info_alternates
        -> link_alt_odb_entries

That is why we need to convert all these functions at the same time.

This conversion looks good.

Quoted 126 lines
---
 object-store.h |  4 ++++
 sha1_file.c    | 36 ++++++++++++++++--------------------
 2 files changed, 20 insertions(+), 20 deletions(-)
object-store.h +4 -0
index 3d0f8a87cb..20ba136c1f 100644
@@ -18,6 +18,10 @@ struct alternate_object_database {
 	char loose_objects_subdir_seen[256];
 	struct oid_array loose_objects_cache;
 
+	/*
+	 * Path to the alternative object store. If this is a relative path,
+	 * it is relative to the current working directory.
+	 */
 	char path[FLEX_ARRAY];
 };
 #define prepare_alt_odb(r) prepare_alt_odb_##r()
sha1_file.c +16 -20
index 027e0f3741..f34eb69e39 100644
@@ -390,11 +390,10 @@ static int alt_odb_usable(struct raw_object_store *o,
  * SHA1, an extra slash for the first level indirection, and the
  * terminating NUL.
  */
-#define read_info_alternates(r, rb, d) read_info_alternates_##r(rb, d)
-static void read_info_alternates_the_repository(const char *relative_base,
-						int depth);
-#define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)
-static int link_alt_odb_entry_the_repository(const char *entry,
+static void read_info_alternates(struct repository *r,
+				 const char *relative_base,
+				 int depth);
+static int link_alt_odb_entry(struct repository *r, const char *entry,
 	const char *relative_base, int depth, const char *normalized_objdir)
 {
 	struct alternate_object_database *ent;
@@ -420,7 +419,7 @@ static int link_alt_odb_entry_the_repository(const char *entry,
 	while (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')
 		strbuf_setlen(&pathbuf, pathbuf.len - 1);
 
-	if (!alt_odb_usable(&the_repository->objects, &pathbuf, normalized_objdir)) {
+	if (!alt_odb_usable(&r->objects, &pathbuf, normalized_objdir)) {
 		strbuf_release(&pathbuf);
 		return -1;
 	}
@@ -428,12 +427,12 @@ static int link_alt_odb_entry_the_repository(const char *entry,
 	ent = alloc_alt_odb(pathbuf.buf);
 
 	/* add the alternate entry */
-	*the_repository->objects.alt_odb_tail = ent;
-	the_repository->objects.alt_odb_tail = &(ent->next);
+	*r->objects.alt_odb_tail = ent;
+	r->objects.alt_odb_tail = &(ent->next);
 	ent->next = NULL;
 
 	/* recursively add alternates */
-	read_info_alternates(the_repository, pathbuf.buf, depth + 1);
+	read_info_alternates(r, pathbuf.buf, depth + 1);
 
 	strbuf_release(&pathbuf);
 	return 0;
@@ -468,12 +467,8 @@ static const char *parse_alt_odb_entry(const char *string,
 	return end;
 }
 
-#define link_alt_odb_entries(r, a, s, rb, d) \
-	link_alt_odb_entries_##r(a, s, rb, d)
-static void link_alt_odb_entries_the_repository(const char *alt,
-						int sep,
-						const char *relative_base,
-						int depth)
+static void link_alt_odb_entries(struct repository *r, const char *alt,
+				 int sep, const char *relative_base, int depth)
 {
 	struct strbuf objdirbuf = STRBUF_INIT;
 	struct strbuf entry = STRBUF_INIT;
@@ -487,7 +482,7 @@ static void link_alt_odb_entries_the_repository(const char *alt,
 		return;
 	}
 
-	strbuf_add_absolute_path(&objdirbuf, get_object_directory());
+	strbuf_add_absolute_path(&objdirbuf, r->objects.objectdir);
 	if (strbuf_normalize_path(&objdirbuf) < 0)
 		die("unable to normalize object directory: %s",
 		    objdirbuf.buf);
@@ -496,15 +491,16 @@ static void link_alt_odb_entries_the_repository(const char *alt,
 		alt = parse_alt_odb_entry(alt, sep, &entry);
 		if (!entry.len)
 			continue;
-		link_alt_odb_entry(the_repository, entry.buf,
+		link_alt_odb_entry(r, entry.buf,
 				   relative_base, depth, objdirbuf.buf);
 	}
 	strbuf_release(&entry);
 	strbuf_release(&objdirbuf);
 }
 
-static void read_info_alternates_the_repository(const char *relative_base,
-						int depth)
+static void read_info_alternates(struct repository *r,
+				 const char *relative_base,
+				 int depth)
 {
 	char *path;
 	struct strbuf buf = STRBUF_INIT;
@@ -516,7 +512,7 @@ static void read_info_alternates_the_repository(const char *relative_base,
 		return;
 	}
 
-	link_alt_odb_entries(the_repository, buf.buf, '\n', relative_base, depth);
+	link_alt_odb_entries(r, buf.buf, '\n', relative_base, depth);
 	strbuf_release(&buf);
 	free(path);
 }
Signature · 1 line
2.16.1.435.g8f24da2e1a
Signature · 1 line
Brandon Williams
#45 lore raw

PATCH 20/44 sha1_file: allow prepare_alt_odb to handle arbitrary repositories

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

---
 object-store.h |  3 +--
 sha1_file.c    | 12 +++++-------
 2 files changed, 6 insertions(+), 9 deletions(-)
object-store.h +1 -2
index 20ba136c1f..141455b5b2 100644
@@ -24,8 +24,7 @@ struct alternate_object_database {
 	 */
 	char path[FLEX_ARRAY];
 };
-#define prepare_alt_odb(r) prepare_alt_odb_##r()
-void prepare_alt_odb_the_repository(void);
+void prepare_alt_odb(struct repository *r);
 char *compute_alternate_path(const char *path, struct strbuf *err);
 typedef int alt_odb_fn(struct alternate_object_database *, void *);
 int foreach_alt_odb(alt_odb_fn, void*);
sha1_file.c +5 -7
index f34eb69e39..a3e4ef4253 100644
@@ -673,17 +673,15 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 	return r;
 }
 
-void prepare_alt_odb_the_repository(void)
+void prepare_alt_odb(struct repository *r)
 {
-	if (the_repository->objects.alt_odb_tail)
+	if (r->objects.alt_odb_tail)
 		return;
 
-	the_repository->objects.alt_odb_tail =
-			&the_repository->objects.alt_odb_list;
-	link_alt_odb_entries(the_repository, the_repository->objects.alternate_db,
-			     PATH_SEP, NULL, 0);
+	r->objects.alt_odb_tail = &r->objects.alt_odb_list;
+	link_alt_odb_entries(r, r->objects.alternate_db, PATH_SEP, NULL, 0);
 
-	read_info_alternates(the_repository, get_object_directory(), 0);
+	read_info_alternates(r, r->objects.objectdir, 0);
 }
 
 /* Returns 1 if we have successfully freshened the file, 0 otherwise. */
Signature · 1 line
2.16.1.435.g8f24da2e1a
#46 lore raw

PATCH 21/44 sha1_file: add repository argument to sha1_file_name

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Add a repository argument to allow sha1_file_name callers to be more specific about which repository to handle. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

While at it, move the declaration to object-store.h, where it should be easier to find.

---
 cache.h        |  6 ------
 http-walker.c  |  3 ++-
 http.c         |  5 ++---
 object-store.h |  7 +++++++
 sha1_file.c    | 10 +++++-----
 5 files changed, 16 insertions(+), 15 deletions(-)
cache.h +0 -6
index d3429a0d48..cbec0ecd23 100644
@@ -961,12 +961,6 @@ extern void check_repository_format(void);
 #define DATA_CHANGED    0x0020
 #define TYPE_CHANGED    0x0040
 
-/*
- * Put in `buf` the name of the file in the local object database that
- * would be used to store a loose object with the specified sha1.
- */
-extern void sha1_file_name(struct strbuf *buf, const unsigned char *sha1);
-
 /*
  * Return an abbreviated sha1 unique within this repository's object database.
  * The result will be at least `len` characters long, and will be NUL
http-walker.c +2 -1
index 07c2b1af82..2c33969123 100644
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "repository.h"
 #include "commit.h"
 #include "walker.h"
 #include "http.h"
@@ -545,7 +546,7 @@ static int fetch_object(struct walker *walker, unsigned char *sha1)
 		ret = error("File %s has bad hash", hex);
 	} else if (req->rename < 0) {
 		struct strbuf buf = STRBUF_INIT;
-		sha1_file_name(&buf, req->sha1);
+		sha1_file_name(the_repository, &buf, req->sha1);
 		ret = error("unable to write sha1 filename %s", buf.buf);
 		strbuf_release(&buf);
 	}
http.c +2 -3
index 31755023a4..df9dbea59c 100644
@@ -2246,7 +2246,7 @@ struct http_object_request *new_http_object_request(const char *base_url,
 	hashcpy(freq->sha1, sha1);
 	freq->localfile = -1;
 
-	sha1_file_name(&filename, sha1);
+	sha1_file_name(the_repository, &filename, sha1);
 	snprintf(freq->tmpfile, sizeof(freq->tmpfile),
 		 "%s.temp", filename.buf);
 
@@ -2395,8 +2395,7 @@ int finish_http_object_request(struct http_object_request *freq)
 		unlink_or_warn(freq->tmpfile);
 		return -1;
 	}
-
-	sha1_file_name(&filename, freq->sha1);
+	sha1_file_name(the_repository, &filename, freq->sha1);
 	freq->rename = finalize_object_file(freq->tmpfile, filename.buf);
 	strbuf_release(&filename);
 
object-store.h +7 -0
index 141455b5b2..f09acfbf5b 100644
@@ -115,4 +115,11 @@ struct raw_object_store {
 void raw_object_store_init(struct raw_object_store *o);
 void raw_object_store_clear(struct raw_object_store *o);
 
+/*
+ * Put in `buf` the name of the file in the local object database that
+ * would be used to store a loose object with the specified sha1.
+ */
+#define sha1_file_name(r, b, s) sha1_file_name_##r(b, s)
+void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1);
+
 #endif /* OBJECT_STORE_H */
sha1_file.c +5 -5
index a3e4ef4253..e06d62a451 100644
@@ -323,7 +323,7 @@ static void fill_sha1_path(struct strbuf *buf, const unsigned char *sha1)
 	}
 }
 
-void sha1_file_name(struct strbuf *buf, const unsigned char *sha1)
+void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1)
 {
 	strbuf_addstr(buf, get_object_directory());
 	strbuf_addch(buf, '/');
@@ -713,7 +713,7 @@ static int check_and_freshen_local(const unsigned char *sha1, int freshen)
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(&buf, sha1);
+	sha1_file_name(the_repository, &buf, sha1);
 
 	return check_and_freshen_file(buf.buf, freshen);
 }
@@ -874,7 +874,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(&buf, sha1);
+	sha1_file_name(the_repository, &buf, sha1);
 	*path = buf.buf;
 
 	if (!lstat(*path, st))
@@ -903,7 +903,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(&buf, sha1);
+	sha1_file_name(the_repository, &buf, sha1);
 	*path = buf.buf;
 
 	fd = git_open(*path);
@@ -1588,7 +1588,7 @@ static int write_loose_object(const unsigned char *sha1, char *hdr, int hdrlen,
 	static struct strbuf filename = STRBUF_INIT;
 
 	strbuf_reset(&filename);
-	sha1_file_name(&filename, sha1);
+	sha1_file_name(the_repository, &filename, sha1);
 
 	fd = create_tmpfile(&tmp_file, filename.buf);
 	if (fd < 0) {
Signature · 1 line
2.16.1.435.g8f24da2e1a
#47 lore raw

PATCH 22/44 sha1_file: add repository argument to stat_sha1_file

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Add a repository argument to allow the stat_sha1_file caller to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

---
 sha1_file.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
sha1_file.c +5 -4
index e06d62a451..7b640cca28 100644
@@ -867,8 +867,9 @@ int git_open_cloexec(const char *name, int flags)
  * Note that it may point to static storage and is only valid until another
  * call to sha1_file_name(), etc.
  */
-static int stat_sha1_file(const unsigned char *sha1, struct stat *st,
-			  const char **path)
+#define stat_sha1_file(r, s, st, p) stat_sha1_file_##r(s, st, p)
+static int stat_sha1_file_the_repository(const unsigned char *sha1,
+					 struct stat *st, const char **path)
 {
 	struct alternate_object_database *alt;
 	static struct strbuf buf = STRBUF_INIT;
@@ -1174,7 +1175,7 @@ static int sha1_loose_object_info(const unsigned char *sha1,
 	if (!oi->typep && !oi->typename && !oi->sizep && !oi->contentp) {
 		const char *path;
 		struct stat st;
-		if (stat_sha1_file(sha1, &st, &path) < 0)
+		if (stat_sha1_file(the_repository, sha1, &st, &path) < 0)
 			return -1;
 		if (oi->disk_sizep)
 			*oi->disk_sizep = st.st_size;
@@ -1388,7 +1389,7 @@ void *read_sha1_file_extended(const unsigned char *sha1,
 		die("replacement %s not found for %s",
 		    sha1_to_hex(repl), sha1_to_hex(sha1));
 
-	if (!stat_sha1_file(repl, &st, &path))
+	if (!stat_sha1_file(the_repository, repl, &st, &path))
 		die("loose object %s (stored in %s) is corrupt",
 		    sha1_to_hex(repl), path);
 
Signature · 1 line
2.16.1.435.g8f24da2e1a
#48 lore raw

PATCH 23/44 sha1_file: add repository argument to open_sha1_file

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Add a repository argument to allow the open_sha1_file caller to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

---
 sha1_file.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
sha1_file.c +4 -2
index 7b640cca28..d4298805e3 100644
@@ -896,7 +896,9 @@ static int stat_sha1_file_the_repository(const unsigned char *sha1,
  * Like stat_sha1_file(), but actually open the object and return the
  * descriptor. See the caveats on the "path" parameter above.
  */
-static int open_sha1_file(const unsigned char *sha1, const char **path)
+#define open_sha1_file(r, s, p) open_sha1_file_##r(s, p)
+static int open_sha1_file_the_repository(const unsigned char *sha1,
+					 const char **path)
 {
 	int fd;
 	struct alternate_object_database *alt;
@@ -939,7 +941,7 @@ static void *map_sha1_file_1(const char *path,
 	if (path)
 		fd = git_open(path);
 	else
-		fd = open_sha1_file(sha1, &path);
+		fd = open_sha1_file(the_repository, sha1, &path);
 	map = NULL;
 	if (fd >= 0) {
 		struct stat st;
Signature · 1 line
2.16.1.435.g8f24da2e1a
#49 lore raw

PATCH 24/44 sha1_file: add repository argument to map_sha1_file_1

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Add a repository argument to allow the map_sha1_file_1 caller to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

---
 sha1_file.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
sha1_file.c +6 -5
index d4298805e3..78d5ffe876 100644
@@ -931,9 +931,10 @@ static int open_sha1_file_the_repository(const unsigned char *sha1,
  * Map the loose object at "path" if it is not NULL, or the path found by
  * searching for a loose object named "sha1".
  */
-static void *map_sha1_file_1(const char *path,
-			     const unsigned char *sha1,
-			     unsigned long *size)
+#define map_sha1_file_1(r, p, s, si) map_sha1_file_1_##r(p, s, si)
+static void *map_sha1_file_1_the_repository(const char *path,
+					    const unsigned char *sha1,
+					    unsigned long *size)
 {
 	void *map;
 	int fd;
@@ -962,7 +963,7 @@ static void *map_sha1_file_1(const char *path,
 
 void *map_sha1_file(const unsigned char *sha1, unsigned long *size)
 {
-	return map_sha1_file_1(NULL, sha1, size);
+	return map_sha1_file_1(the_repository, NULL, sha1, size);
 }
 
 static int unpack_sha1_short_header(git_zstream *stream,
@@ -2192,7 +2193,7 @@ int read_loose_object(const char *path,
 
 	*contents = NULL;
 
-	map = map_sha1_file_1(path, NULL, &mapsize);
+	map = map_sha1_file_1(the_repository, path, NULL, &mapsize);
 	if (!map) {
 		error_errno("unable to mmap %s", path);
 		goto out;
Signature · 1 line
2.16.1.435.g8f24da2e1a
#50 lore raw

PATCH 25/44 sha1_file: add repository argument to map_sha1_file

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Add a repository argument to allow map_sha1_file callers to be more specific about which repository to handle. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

While at it, move the declaration to object-store.h, where it should be easier to find.

---
 cache.h        | 1 -
 object-store.h | 3 +++
 sha1_file.c    | 4 ++--
 streaming.c    | 5 ++++-
 4 files changed, 9 insertions(+), 4 deletions(-)
cache.h +0 -1
index cbec0ecd23..720664e394 100644
@@ -1242,7 +1242,6 @@ extern int pretend_sha1_file(void *, unsigned long, enum object_type, unsigned c
 extern int force_object_loose(const unsigned char *sha1, time_t mtime);
 extern int git_open_cloexec(const char *name, int flags);
 #define git_open(name) git_open_cloexec(name, O_RDONLY)
-extern void *map_sha1_file(const unsigned char *sha1, unsigned long *size);
 extern int unpack_sha1_header(git_zstream *stream, unsigned char *map, unsigned long mapsize, void *buffer, unsigned long bufsiz);
 extern int parse_sha1_header(const char *hdr, unsigned long *sizep);
 
object-store.h +3 -0
index f09acfbf5b..ad4c7501d7 100644
@@ -122,4 +122,7 @@ void raw_object_store_clear(struct raw_object_store *o);
 #define sha1_file_name(r, b, s) sha1_file_name_##r(b, s)
 void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1);
 
+#define map_sha1_file(r, s, sz) map_sha1_file_##r(s, sz)
+void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size);
+
 #endif /* OBJECT_STORE_H */
sha1_file.c +2 -2
index 78d5ffe876..536c45589b 100644
@@ -961,7 +961,7 @@ static void *map_sha1_file_1_the_repository(const char *path,
 	return map;
 }
 
-void *map_sha1_file(const unsigned char *sha1, unsigned long *size)
+void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size)
 {
 	return map_sha1_file_1(the_repository, NULL, sha1, size);
 }
@@ -1185,7 +1185,7 @@ static int sha1_loose_object_info(const unsigned char *sha1,
 		return 0;
 	}
 
-	map = map_sha1_file(sha1, &mapsize);
+	map = map_sha1_file(the_repository, sha1, &mapsize);
 	if (!map)
 		return -1;
 
streaming.c +4 -1
index 5892b50bd8..22d27df55e 100644
@@ -3,6 +3,8 @@
  */
 #include "cache.h"
 #include "streaming.h"
+#include "repository.h"
+#include "object-store.h"
 #include "packfile.h"
 
 enum input_source {
@@ -335,7 +337,8 @@ static struct stream_vtbl loose_vtbl = {
 
 static open_method_decl(loose)
 {
-	st->u.loose.mapped = map_sha1_file(sha1, &st->u.loose.mapsize);
+	st->u.loose.mapped = map_sha1_file(the_repository,
+					   sha1, &st->u.loose.mapsize);
 	if (!st->u.loose.mapped)
 		return -1;
 	if ((unpack_sha1_header(&st->z,
Signature · 1 line
2.16.1.435.g8f24da2e1a
#51 lore raw

PATCH 26/44 sha1_file: add repository argument to sha1_loose_object_info

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Add a repository argument to allow the sha1_loose_object_info caller to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

---
 sha1_file.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
sha1_file.c +5 -4
index 536c45589b..ecbf846f12 100644
@@ -1152,9 +1152,10 @@ int parse_sha1_header(const char *hdr, unsigned long *sizep)
 	return parse_sha1_header_extended(hdr, &oi, 0);
 }
 
-static int sha1_loose_object_info(const unsigned char *sha1,
-				  struct object_info *oi,
-				  int flags)
+#define sha1_loose_object_info(r, s, o, f) sha1_loose_object_info_##r(s, o, f)
+static int sha1_loose_object_info_the_repository(const unsigned char *sha1,
+						 struct object_info *oi,
+						 int flags)
 {
 	int status = 0;
 	unsigned long mapsize;
@@ -1273,7 +1274,7 @@ int sha1_object_info_extended(const unsigned char *sha1, struct object_info *oi,
 			break;
 
 		/* Most likely it's a loose object. */
-		if (!sha1_loose_object_info(real, oi, flags))
+		if (!sha1_loose_object_info(the_repository, real, oi, flags))
 			return 0;
 
 		/* Not a loose object; someone else may have just packed it. */
Signature · 1 line
2.16.1.435.g8f24da2e1a
#52 lore raw

PATCH 27/44 sha1_file: allow sha1_file_name to handle arbitrary repositories

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

---
 object-store.h | 3 +--
 sha1_file.c    | 4 ++--
 2 files changed, 3 insertions(+), 4 deletions(-)
object-store.h +1 -2
index ad4c7501d7..98622a2b98 100644
@@ -119,8 +119,7 @@ void raw_object_store_clear(struct raw_object_store *o);
  * Put in `buf` the name of the file in the local object database that
  * would be used to store a loose object with the specified sha1.
  */
-#define sha1_file_name(r, b, s) sha1_file_name_##r(b, s)
-void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1);
+void sha1_file_name(struct repository *r, struct strbuf *buf, const unsigned char *sha1);
 
 #define map_sha1_file(r, s, sz) map_sha1_file_##r(s, sz)
 void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size);
sha1_file.c +2 -2
index ecbf846f12..e04e0587c2 100644
@@ -323,9 +323,9 @@ static void fill_sha1_path(struct strbuf *buf, const unsigned char *sha1)
 	}
 }
 
-void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1)
+void sha1_file_name(struct repository *r, struct strbuf *buf, const unsigned char *sha1)
 {
-	strbuf_addstr(buf, get_object_directory());
+	strbuf_addstr(buf, r->objects.objectdir);
 	strbuf_addch(buf, '/');
 	fill_sha1_path(buf, sha1);
 }
Signature · 1 line
2.16.1.435.g8f24da2e1a
#53 lore raw

PATCH 28/44 sha1_file: allow stat_sha1_file to handle arbitrary repositories

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

---
 sha1_file.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)
sha1_file.c +5 -6
index e04e0587c2..f2533dd9e4 100644
@@ -867,23 +867,22 @@ int git_open_cloexec(const char *name, int flags)
  * Note that it may point to static storage and is only valid until another
  * call to sha1_file_name(), etc.
  */
-#define stat_sha1_file(r, s, st, p) stat_sha1_file_##r(s, st, p)
-static int stat_sha1_file_the_repository(const unsigned char *sha1,
-					 struct stat *st, const char **path)
+static int stat_sha1_file(struct repository *r, const unsigned char *sha1,
+			  struct stat *st, const char **path)
 {
 	struct alternate_object_database *alt;
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(the_repository, &buf, sha1);
+	sha1_file_name(r, &buf, sha1);
 	*path = buf.buf;
 
 	if (!lstat(*path, st))
 		return 0;
 
-	prepare_alt_odb(the_repository);
+	prepare_alt_odb(r);
 	errno = ENOENT;
-	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
+	for (alt = r->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		if (!lstat(*path, st))
 			return 0;
Signature · 1 line
2.16.1.435.g8f24da2e1a
#54 lore raw

PATCH 29/44 sha1_file: allow open_sha1_file to handle arbitrary repositories

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

---
 sha1_file.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)
sha1_file.c +5 -6
index f2533dd9e4..b3f7656c35 100644
@@ -895,9 +895,8 @@ static int stat_sha1_file(struct repository *r, const unsigned char *sha1,
  * Like stat_sha1_file(), but actually open the object and return the
  * descriptor. See the caveats on the "path" parameter above.
  */
-#define open_sha1_file(r, s, p) open_sha1_file_##r(s, p)
-static int open_sha1_file_the_repository(const unsigned char *sha1,
-					 const char **path)
+static int open_sha1_file(struct repository *r,
+			  const unsigned char *sha1, const char **path)
 {
 	int fd;
 	struct alternate_object_database *alt;
@@ -905,7 +904,7 @@ static int open_sha1_file_the_repository(const unsigned char *sha1,
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(the_repository, &buf, sha1);
+	sha1_file_name(r, &buf, sha1);
 	*path = buf.buf;
 
 	fd = git_open(*path);
@@ -913,8 +912,8 @@ static int open_sha1_file_the_repository(const unsigned char *sha1,
 		return fd;
 	most_interesting_errno = errno;
 
-	prepare_alt_odb(the_repository);
-	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
+	prepare_alt_odb(r);
+	for (alt = r->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		fd = git_open(*path);
 		if (fd >= 0)
Signature · 1 line
2.16.1.435.g8f24da2e1a
#55 lore raw

PATCH 30/44 sha1_file: allow map_sha1_file_1 to handle arbitrary repositories

In reply to Nguyễn Thái Ngọc Duy

From: Jonathan Nieder <[email protected]>

---
 sha1_file.c | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)
sha1_file.c +3 -5
index b3f7656c35..d4678f8162 100644
@@ -929,10 +929,8 @@ static int open_sha1_file(struct repository *r,
  * Map the loose object at "path" if it is not NULL, or the path found by
  * searching for a loose object named "sha1".
  */
-#define map_sha1_file_1(r, p, s, si) map_sha1_file_1_##r(p, s, si)
-static void *map_sha1_file_1_the_repository(const char *path,
-					    const unsigned char *sha1,
-					    unsigned long *size)
+static void *map_sha1_file_1(struct repository *r, const char *path,
+			     const unsigned char *sha1, unsigned long *size)
 {
 	void *map;
 	int fd;
@@ -940,7 +938,7 @@ static void *map_sha1_file_1_the_repository(const char *path,
 	if (path)
 		fd = git_open(path);
 	else
-		fd = open_sha1_file(the_repository, sha1, &path);
+		fd = open_sha1_file(r, sha1, &path);
 	map = NULL;
 	if (fd >= 0) {
 		struct stat st;
Signature · 1 line
2.16.1.435.g8f24da2e1a
#56 lore raw

PATCH 31/44 sha1_file: allow map_sha1_file to handle arbitrary repositories

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

---
 object-store.h | 3 +--
 sha1_file.c    | 5 +++--
 2 files changed, 4 insertions(+), 4 deletions(-)
object-store.h +1 -2
index 98622a2b98..521f5a1755 100644
@@ -121,7 +121,6 @@ void raw_object_store_clear(struct raw_object_store *o);
  */
 void sha1_file_name(struct repository *r, struct strbuf *buf, const unsigned char *sha1);
 
-#define map_sha1_file(r, s, sz) map_sha1_file_##r(s, sz)
-void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size);
+void *map_sha1_file(struct repository *r, const unsigned char *sha1, unsigned long *size);
 
 #endif /* OBJECT_STORE_H */
sha1_file.c +3 -2
index d4678f8162..c5a8b00aed 100644
@@ -957,9 +957,10 @@ static void *map_sha1_file_1(struct repository *r, const char *path,
 	return map;
 }
 
-void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size)
+void *map_sha1_file(struct repository *r,
+		    const unsigned char *sha1, unsigned long *size)
 {
-	return map_sha1_file_1(the_repository, NULL, sha1, size);
+	return map_sha1_file_1(r, NULL, sha1, size);
 }
 
 static int unpack_sha1_short_header(git_zstream *stream,
Signature · 1 line
2.16.1.435.g8f24da2e1a
#57 lore raw

PATCH 32/44 sha1_file: allow sha1_loose_object_info to handle arbitrary repositories

In reply to Nguyễn Thái Ngọc Duy

From: Jonathan Nieder <[email protected]>

---
 sha1_file.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)
sha1_file.c +5 -6
index c5a8b00aed..7066d4c9ce 100644
@@ -1149,10 +1149,9 @@ int parse_sha1_header(const char *hdr, unsigned long *sizep)
 	return parse_sha1_header_extended(hdr, &oi, 0);
 }
 
-#define sha1_loose_object_info(r, s, o, f) sha1_loose_object_info_##r(s, o, f)
-static int sha1_loose_object_info_the_repository(const unsigned char *sha1,
-						 struct object_info *oi,
-						 int flags)
+static int sha1_loose_object_info(struct repository *r,
+				  const unsigned char *sha1,
+				  struct object_info *oi, int flags)
 {
 	int status = 0;
 	unsigned long mapsize;
@@ -1176,14 +1175,14 @@ static int sha1_loose_object_info_the_repository(const unsigned char *sha1,
 	if (!oi->typep && !oi->typename && !oi->sizep && !oi->contentp) {
 		const char *path;
 		struct stat st;
-		if (stat_sha1_file(the_repository, sha1, &st, &path) < 0)
+		if (stat_sha1_file(r, sha1, &st, &path) < 0)
 			return -1;
 		if (oi->disk_sizep)
 			*oi->disk_sizep = st.st_size;
 		return 0;
 	}
 
-	map = map_sha1_file(the_repository, sha1, &mapsize);
+	map = map_sha1_file(r, sha1, &mapsize);
 	if (!map)
 		return -1;
 
Signature · 1 line
2.16.1.435.g8f24da2e1a
#58 lore raw

Re: [PATCH 32/44] sha1_file: allow sha1_loose_object_info to handle arbitrary repositories

Quoted 6 lines · Nguyễn Thái Ngọc Duy

From: Jonathan Nieder <[email protected]>

Patches up to here look sane.

Quoted 42 lines
---
 sha1_file.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)
sha1_file.c +5 -6
index c5a8b00aed..7066d4c9ce 100644
@@ -1149,10 +1149,9 @@ int parse_sha1_header(const char *hdr, unsigned long *sizep)
 	return parse_sha1_header_extended(hdr, &oi, 0);
 }
 
-#define sha1_loose_object_info(r, s, o, f) sha1_loose_object_info_##r(s, o, f)
-static int sha1_loose_object_info_the_repository(const unsigned char *sha1,
-						 struct object_info *oi,
-						 int flags)
+static int sha1_loose_object_info(struct repository *r,
+				  const unsigned char *sha1,
+				  struct object_info *oi, int flags)
 {
 	int status = 0;
 	unsigned long mapsize;
@@ -1176,14 +1175,14 @@ static int sha1_loose_object_info_the_repository(const unsigned char *sha1,
 	if (!oi->typep && !oi->typename && !oi->sizep && !oi->contentp) {
 		const char *path;
 		struct stat st;
-		if (stat_sha1_file(the_repository, sha1, &st, &path) < 0)
+		if (stat_sha1_file(r, sha1, &st, &path) < 0)
 			return -1;
 		if (oi->disk_sizep)
 			*oi->disk_sizep = st.st_size;
 		return 0;
 	}
 
-	map = map_sha1_file(the_repository, sha1, &mapsize);
+	map = map_sha1_file(r, sha1, &mapsize);
 	if (!map)
 		return -1;
 
Signature · 1 line
2.16.1.435.g8f24da2e1a
Signature · 1 line
Brandon Williams
#59 lore raw

PATCH 33/44 packfile: allow prepare_packed_git_mru to handle arbitrary repositories

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

This conversion was done without the #define trick used in the earlier series refactoring to have better repository access, because this function is easy to review, as all lines are converted and it has only one caller

---
 packfile.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
packfile.c +5 -5
index a0842521e5..0eaf5ee100 100644
@@ -872,14 +872,14 @@ static void rearrange_packed_git(void)
 		set_next_packed_git, sort_pack);
 }
 
-static void prepare_packed_git_mru(void)
+static void prepare_packed_git_mru(struct repository *r)
 {
 	struct packed_git *p;
 
-	INIT_LIST_HEAD(&the_repository->objects.packed_git_mru);
+	INIT_LIST_HEAD(&r->objects.packed_git_mru);
 
-	for (p = the_repository->objects.packed_git; p; p = p->next)
-		list_add_tail(&p->mru, &the_repository->objects.packed_git_mru);
+	for (p = r->objects.packed_git; p; p = p->next)
+		list_add_tail(&p->mru, &r->objects.packed_git_mru);
 }
 
 void prepare_packed_git(void)
@@ -893,7 +893,7 @@ void prepare_packed_git(void)
 	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next)
 		prepare_packed_git_one(alt->path, 0);
 	rearrange_packed_git();
-	prepare_packed_git_mru();
+	prepare_packed_git_mru(the_repository);
 	the_repository->objects.packed_git_initialized = 1;
 }
 
Signature · 1 line
2.16.1.435.g8f24da2e1a
#60 lore raw

PATCH 34/44 packfile: allow rearrange_packed_git to handle arbitrary repositories

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

---
 packfile.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
packfile.c +4 -4
index 0eaf5ee100..5356712717 100644
@@ -865,10 +865,10 @@ static int sort_pack(const void *a_, const void *b_)
 	return -1;
 }
 
-static void rearrange_packed_git(void)
+static void rearrange_packed_git(struct repository *r)
 {
-	the_repository->objects.packed_git = llist_mergesort(
-		the_repository->objects.packed_git, get_next_packed_git,
+	r->objects.packed_git = llist_mergesort(
+		r->objects.packed_git, get_next_packed_git,
 		set_next_packed_git, sort_pack);
 }
 
@@ -892,7 +892,7 @@ void prepare_packed_git(void)
 	prepare_alt_odb(the_repository);
 	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next)
 		prepare_packed_git_one(alt->path, 0);
-	rearrange_packed_git();
+	rearrange_packed_git(the_repository);
 	prepare_packed_git_mru(the_repository);
 	the_repository->objects.packed_git_initialized = 1;
 }
Signature · 1 line
2.16.1.435.g8f24da2e1a
#61 lore raw

PATCH 35/44 packfile: allow install_packed_git to handle arbitrary repositories

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

This conversion was done without the #define trick used in the earlier series refactoring to have better repository access, because this function is easy to review, as it only has one caller and all lines but the first two are converted.

We must not convert 'pack_open_fds' to be a repository specific variable, as it is used to monitor resource usage of the machine that Git executes on.

---
 fast-import.c | 2 +-
 http.c        | 2 +-
 packfile.c    | 8 ++++----
 packfile.h    | 2 +-
 4 files changed, 7 insertions(+), 7 deletions(-)
fast-import.c +1 -1
index 0dba555478..ec78e8ff47 100644
@@ -1037,7 +1037,7 @@ static void end_packfile(void)
 		if (!new_p)
 			die("core git rejected index %s", idx_name);
 		all_packs[pack_id] = new_p;
-		install_packed_git(new_p);
+		install_packed_git(the_repository, new_p);
 		free(idx_name);
 
 		/* Print the boundary */
http.c +1 -1
index df9dbea59c..649656b87c 100644
@@ -2133,7 +2133,7 @@ int finish_http_pack_request(struct http_pack_request *preq)
 		return -1;
 	}
 
-	install_packed_git(p);
+	install_packed_git(the_repository, p);
 	free(tmp_idx);
 	return 0;
 }
packfile.c +4 -4
index 5356712717..ba185daec2 100644
@@ -679,13 +679,13 @@ struct packed_git *add_packed_git(const char *path, size_t path_len, int local)
 	return p;
 }
 
-void install_packed_git(struct packed_git *pack)
+void install_packed_git(struct repository *r, struct packed_git *pack)
 {
 	if (pack->pack_fd != -1)
 		pack_open_fds++;
 
-	pack->next = the_repository->objects.packed_git;
-	the_repository->objects.packed_git = pack;
+	pack->next = r->objects.packed_git;
+	r->objects.packed_git = pack;
 }
 
 void (*report_garbage)(unsigned seen_bits, const char *path);
@@ -781,7 +781,7 @@ static void prepare_packed_git_one(char *objdir, int local)
 			     * corresponding .pack file that we can map.
 			     */
 			    (p = add_packed_git(path.buf, path.len, local)) != NULL)
-				install_packed_git(p);
+				install_packed_git(the_repository, p);
 		}
 
 		if (!report_garbage)
packfile.h +1 -1
index 5b1ce00f84..77442172f0 100644
@@ -36,7 +36,7 @@ extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
 extern void prepare_packed_git(void);
 extern void reprepare_packed_git(void);
-extern void install_packed_git(struct packed_git *pack);
+extern void install_packed_git(struct repository *r, struct packed_git *pack);
 
 struct packed_git *get_packed_git(struct repository *r);
 struct list_head *get_packed_git_mru(struct repository *r);
Signature · 1 line
2.16.1.435.g8f24da2e1a
#62 lore raw

PATCH 36/44 packfile: add repository argument to prepare_packed_git_one

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

---
 packfile.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
packfile.c +4 -3
index ba185daec2..d7658e6c45 100644
@@ -734,7 +734,8 @@ static void report_pack_garbage(struct string_list *list)
 	report_helper(list, seen_bits, first, list->nr);
 }
 
-static void prepare_packed_git_one(char *objdir, int local)
+#define prepare_packed_git_one(r, o, l) prepare_packed_git_one_##r(o, l)
+static void prepare_packed_git_one_the_repository(char *objdir, int local)
 {
 	struct strbuf path = STRBUF_INIT;
 	size_t dirnamelen;
@@ -888,10 +889,10 @@ void prepare_packed_git(void)
 
 	if (the_repository->objects.packed_git_initialized)
 		return;
-	prepare_packed_git_one(get_object_directory(), 1);
+	prepare_packed_git_one(the_repository, get_object_directory(), 1);
 	prepare_alt_odb(the_repository);
 	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next)
-		prepare_packed_git_one(alt->path, 0);
+		prepare_packed_git_one(the_repository, alt->path, 0);
 	rearrange_packed_git(the_repository);
 	prepare_packed_git_mru(the_repository);
 	the_repository->objects.packed_git_initialized = 1;
Signature · 1 line
2.16.1.435.g8f24da2e1a
#63 lore raw

PATCH 37/44 packfile: add repository argument to prepare_packed_git

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Add a repository argument to allow prepare_packed_git callers to be more specific about which repository to handle. See c28d027a52c (sha1_file: add repository argument to link_alt_odb_entry, 2018-02-20) for an explanation of the #define trick.

---
 builtin/count-objects.c  |  2 +-
 builtin/fsck.c           |  2 +-
 builtin/gc.c             |  2 +-
 builtin/pack-objects.c   |  2 +-
 builtin/pack-redundant.c |  2 +-
 fast-import.c            |  2 +-
 http-backend.c           |  2 +-
 pack-bitmap.c            |  2 +-
 packfile.c               | 10 +++++-----
 packfile.h               |  3 ++-
 server-info.c            |  2 +-
 sha1_name.c              |  4 ++--
 12 files changed, 18 insertions(+), 17 deletions(-)
builtin/count-objects.c +1 -1
index 5c7c3c6ae3..2793c98ed3 100644
@@ -122,7 +122,7 @@ int cmd_count_objects(int argc, const char **argv, const char *prefix)
 		struct strbuf pack_buf = STRBUF_INIT;
 		struct strbuf garbage_buf = STRBUF_INIT;
 		if (!get_packed_git(the_repository))
-			prepare_packed_git();
+			prepare_packed_git(the_repository);
 		for (p = get_packed_git(the_repository); p; p = p->next) {
 			if (!p->pack_local)
 				continue;
builtin/fsck.c +1 -1
index b0abba6e04..7a3e323e9e 100644
@@ -726,7 +726,7 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 			uint32_t total = 0, count = 0;
 			struct progress *progress = NULL;
 
-			prepare_packed_git();
+			prepare_packed_git(the_repository);
 
 			if (show_progress) {
 				for (p = get_packed_git(the_repository); p;
builtin/gc.c +1 -1
index dd30067ac1..526f06173e 100644
@@ -173,7 +173,7 @@ static int too_many_packs(void)
 	if (gc_auto_pack_limit <= 0)
 		return 0;
 
-	prepare_packed_git();
+	prepare_packed_git(the_repository);
 	for (cnt = 0, p = get_packed_git(the_repository); p; p = p->next) {
 		if (!p->pack_local)
 			continue;
builtin/pack-objects.c +1 -1
index 0e5fde1d6b..6020a7e230 100644
@@ -3151,7 +3151,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 	if (progress && all_progress_implied)
 		progress = 2;
 
-	prepare_packed_git();
+	prepare_packed_git(the_repository);
 	if (ignore_packed_keep) {
 		struct packed_git *p;
 		for (p = get_packed_git(the_repository); p; p = p->next)
builtin/pack-redundant.c +1 -1
index d6d8a44959..bf42e164eb 100644
@@ -630,7 +630,7 @@ int cmd_pack_redundant(int argc, const char **argv, const char *prefix)
 			break;
 	}
 
-	prepare_packed_git();
+	prepare_packed_git(the_repository);
 
 	if (load_all_packs)
 		load_all();
fast-import.c +1 -1
index ec78e8ff47..985eb2eccc 100644
@@ -3472,7 +3472,7 @@ int cmd_main(int argc, const char **argv)
 		rc_free[i].next = &rc_free[i + 1];
 	rc_free[cmd_save - 1].next = NULL;
 
-	prepare_packed_git();
+	prepare_packed_git(the_repository);
 	start_packfile();
 	set_die_routine(die_nicely);
 	set_checkpoint_signal();
http-backend.c +1 -1
index 22921d169a..659ddfb5f1 100644
@@ -518,7 +518,7 @@ static void get_info_packs(struct strbuf *hdr, char *arg)
 	size_t cnt = 0;
 
 	select_getanyfile(hdr);
-	prepare_packed_git();
+	prepare_packed_git(the_repository);
 	for (p = get_packed_git(the_repository); p; p = p->next) {
 		if (p->pack_local)
 			cnt++;
pack-bitmap.c +1 -1
index abed43cdb5..01c9cd1642 100644
@@ -335,7 +335,7 @@ static int open_pack_bitmap(void)
 
 	assert(!bitmap_git.map && !bitmap_git.loaded);
 
-	prepare_packed_git();
+	prepare_packed_git(the_repository);
 	for (p = get_packed_git(the_repository); p; p = p->next) {
 		if (open_pack_bitmap_1(p) == 0)
 			ret = 0;
packfile.c +5 -5
index d7658e6c45..b5ad6838f8 100644
@@ -816,7 +816,7 @@ unsigned long approximate_object_count(void)
 		unsigned long count;
 		struct packed_git *p;
 
-		prepare_packed_git();
+		prepare_packed_git(the_repository);
 		count = 0;
 		for (p = the_repository->objects.packed_git; p; p = p->next) {
 			if (open_pack_index(p))
@@ -883,7 +883,7 @@ static void prepare_packed_git_mru(struct repository *r)
 		list_add_tail(&p->mru, &r->objects.packed_git_mru);
 }
 
-void prepare_packed_git(void)
+void prepare_packed_git_the_repository(void)
 {
 	struct alternate_object_database *alt;
 
@@ -902,7 +902,7 @@ void reprepare_packed_git(void)
 {
 	the_repository->objects.approximate_object_count_valid = 0;
 	the_repository->objects.packed_git_initialized = 0;
-	prepare_packed_git();
+	prepare_packed_git(the_repository);
 }
 
 struct packed_git *get_packed_git(struct repository *r)
@@ -1853,7 +1853,7 @@ int find_pack_entry(const unsigned char *sha1, struct pack_entry *e)
 {
 	struct list_head *pos;
 
-	prepare_packed_git();
+	prepare_packed_git(the_repository);
 	if (!the_repository->objects.packed_git)
 		return 0;
 
@@ -1907,7 +1907,7 @@ int for_each_packed_object(each_packed_object_fn cb, void *data, unsigned flags)
 	int r = 0;
 	int pack_errors = 0;
 
-	prepare_packed_git();
+	prepare_packed_git(the_repository);
 	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
 			continue;
packfile.h +2 -1
index 77442172f0..3f59456e7e 100644
@@ -34,7 +34,8 @@ extern struct packed_git *parse_pack_index(unsigned char *sha1, const char *idx_
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-extern void prepare_packed_git(void);
+#define prepare_packed_git(r) prepare_packed_git_##r()
+extern void prepare_packed_git_the_repository(void);
 extern void reprepare_packed_git(void);
 extern void install_packed_git(struct repository *r, struct packed_git *pack);
 
server-info.c +1 -1
index 6fe64ede17..9604772522 100644
@@ -200,7 +200,7 @@ static void init_pack_info(const char *infofile, int force)
 	objdir = get_object_directory();
 	objdirlen = strlen(objdir);
 
-	prepare_packed_git();
+	prepare_packed_git(the_repository);
 	for (p = get_packed_git(the_repository); p; p = p->next) {
 		/* we ignore things on alternate path since they are
 		 * not available to the pullers in general.
sha1_name.c +2 -2
index 2065be90d2..cb42e398b2 100644
@@ -195,7 +195,7 @@ static void find_short_packed_object(struct disambiguate_state *ds)
 {
 	struct packed_git *p;
 
-	prepare_packed_git();
+	prepare_packed_git(the_repository);
 	for (p = get_packed_git(the_repository); p && !ds->ambiguous;
 	     p = p->next)
 		unique_in_pack(p, ds);
@@ -566,7 +566,7 @@ static void find_abbrev_len_packed(struct min_abbrev_data *mad)
 {
 	struct packed_git *p;
 
-	prepare_packed_git();
+	prepare_packed_git(the_repository);
 	for (p = get_packed_git(the_repository); p; p = p->next)
 		find_abbrev_len_for_pack(p, mad);
 }
Signature · 1 line
2.16.1.435.g8f24da2e1a
#64 lore raw

Re: [PATCH 37/44] packfile: add repository argument to prepare_packed_git

Quoted 4 lines · Nguyễn Thái Ngọc Duy

Add a repository argument to allow prepare_packed_git callers to be more specific about which repository to handle. See c28d027a52c (sha1_file: add repository argument to link_alt_odb_entry, 2018-02-20) for an explanation of the #define trick.

Object c28d027a52c doesn't exist. Most likely it was a reference to a change existing only in 'pu' which got replaced by a later revision of the series.

Quoted 2 lines
#65 lore raw

PATCH 38/44 packfile: add repository argument to reprepare_packed_git

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

See previous patch for explanation.

---
 builtin/gc.c           | 2 +-
 builtin/receive-pack.c | 3 ++-
 bulk-checkin.c         | 3 ++-
 fetch-pack.c           | 3 ++-
 packfile.c             | 2 +-
 packfile.h             | 3 ++-
 sha1_file.c            | 2 +-
 7 files changed, 11 insertions(+), 7 deletions(-)
builtin/gc.c +1 -1
index 526f06173e..560d58daec 100644
@@ -477,7 +477,7 @@ int cmd_gc(int argc, const char **argv, const char *prefix)
 		return error(FAILED_RUN, rerere.argv[0]);
 
 	report_garbage = report_pack_garbage;
-	reprepare_packed_git();
+	reprepare_packed_git(the_repository);
 	if (pack_garbage.nr > 0)
 		clean_pack_garbage();
 
builtin/receive-pack.c +2 -1
index ac478b7b99..dfc69be1a2 100644
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "repository.h"
 #include "config.h"
 #include "lockfile.h"
 #include "pack.h"
@@ -1777,7 +1778,7 @@ static const char *unpack(int err_fd, struct shallow_info *si)
 		status = finish_command(&child);
 		if (status)
 			return "index-pack abnormal exit";
-		reprepare_packed_git();
+		reprepare_packed_git(the_repository);
 	}
 	return NULL;
 }
bulk-checkin.c +2 -1
index 3310fd210a..eadc2d5172 100644
@@ -3,6 +3,7 @@
  */
 #include "cache.h"
 #include "bulk-checkin.h"
+#include "repository.h"
 #include "csum-file.h"
 #include "pack.h"
 #include "strbuf.h"
@@ -57,7 +58,7 @@ static void finish_bulk_checkin(struct bulk_checkin_state *state)
 
 	strbuf_release(&packname);
 	/* Make objects we just wrote available to ourselves */
-	reprepare_packed_git();
+	reprepare_packed_git(the_repository);
 }
 
 static int already_written(struct bulk_checkin_state *state, unsigned char sha1[])
fetch-pack.c +2 -1
index 8253d746e0..eac5928a27 100644
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "lockfile.h"
 #include "refs.h"
@@ -1192,7 +1193,7 @@ struct ref *fetch_pack(struct fetch_pack_args *args,
 	prepare_shallow_info(&si, shallow);
 	ref_cpy = do_fetch_pack(args, fd, ref, sought, nr_sought,
 				&si, pack_lockfile);
-	reprepare_packed_git();
+	reprepare_packed_git(the_repository);
 	update_shallow(args, sought, nr_sought, &si);
 	clear_shallow_info(&si);
 	return ref_cpy;
packfile.c +1 -1
index b5ad6838f8..53db240d0f 100644
@@ -898,7 +898,7 @@ void prepare_packed_git_the_repository(void)
 	the_repository->objects.packed_git_initialized = 1;
 }
 
-void reprepare_packed_git(void)
+void reprepare_packed_git_the_repository(void)
 {
 	the_repository->objects.approximate_object_count_valid = 0;
 	the_repository->objects.packed_git_initialized = 0;
packfile.h +2 -1
index 3f59456e7e..ab5046938c 100644
@@ -36,7 +36,8 @@ extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
 #define prepare_packed_git(r) prepare_packed_git_##r()
 extern void prepare_packed_git_the_repository(void);
-extern void reprepare_packed_git(void);
+#define reprepare_packed_git(r) reprepare_packed_git_##r()
+extern void reprepare_packed_git_the_repository(void);
 extern void install_packed_git(struct repository *r, struct packed_git *pack);
 
 struct packed_git *get_packed_git(struct repository *r);
sha1_file.c +1 -1
index 7066d4c9ce..be84f84373 100644
@@ -1274,7 +1274,7 @@ int sha1_object_info_extended(const unsigned char *sha1, struct object_info *oi,
 			return 0;
 
 		/* Not a loose object; someone else may have just packed it. */
-		reprepare_packed_git();
+		reprepare_packed_git(the_repository);
 		if (find_pack_entry(real, &e))
 			break;
 
Signature · 1 line
2.16.1.435.g8f24da2e1a
#66 lore raw

PATCH 39/44 packfile: allow prepare_packed_git_one to handle arbitrary repositories

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

---
 packfile.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
packfile.c +3 -4
index 53db240d0f..52febba932 100644
@@ -734,8 +734,7 @@ static void report_pack_garbage(struct string_list *list)
 	report_helper(list, seen_bits, first, list->nr);
 }
 
-#define prepare_packed_git_one(r, o, l) prepare_packed_git_one_##r(o, l)
-static void prepare_packed_git_one_the_repository(char *objdir, int local)
+static void prepare_packed_git_one(struct repository *r, char *objdir, int local)
 {
 	struct strbuf path = STRBUF_INIT;
 	size_t dirnamelen;
@@ -768,7 +767,7 @@ static void prepare_packed_git_one_the_repository(char *objdir, int local)
 		base_len = path.len;
 		if (strip_suffix_mem(path.buf, &base_len, ".idx")) {
 			/* Don't reopen a pack we already have. */
-			for (p = the_repository->objects.packed_git; p;
+			for (p = r->objects.packed_git; p;
 			     p = p->next) {
 				size_t len;
 				if (strip_suffix(p->pack_name, ".pack", &len) &&
@@ -782,7 +781,7 @@ static void prepare_packed_git_one_the_repository(char *objdir, int local)
 			     * corresponding .pack file that we can map.
 			     */
 			    (p = add_packed_git(path.buf, path.len, local)) != NULL)
-				install_packed_git(the_repository, p);
+				install_packed_git(r, p);
 		}
 
 		if (!report_garbage)
Signature · 1 line
2.16.1.435.g8f24da2e1a
#67 lore raw

Re: [PATCH 39/44] packfile: allow prepare_packed_git_one to handle arbitrary repositories

Quoted 5 lines · On Sat, 3 Mar 2018 18:36:32 +0700 Nguyễn Thái Ngọc Duy <[email protected]>…

From: Stefan Beller <[email protected]>

Thanks - I've checked that none of the functions invoked in prepare_packed_git_one() access the_repository. (add_packed_git() does not, despite its name.)

The patches up to this one are fine.

[snip]

packfile.c static void prepare_packed_git_one_the_repository(char *objdir, int local) · line 770

Quoted 2 lines
packfile.c +1 -1
-			for (p = the_repository->objects.packed_git; p;
+			for (p = r->objects.packed_git; p;

Optional: this could be get_packed_git(r).

#68 lore raw

PATCH 40/44 packfile: allow prepare_packed_git to handle arbitrary repositories

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

---
 packfile.c | 18 +++++++++---------
 packfile.h |  3 +--
 2 files changed, 10 insertions(+), 11 deletions(-)
packfile.c +9 -9
index 52febba932..2276e2ad26 100644
@@ -882,19 +882,19 @@ static void prepare_packed_git_mru(struct repository *r)
 		list_add_tail(&p->mru, &r->objects.packed_git_mru);
 }
 
-void prepare_packed_git_the_repository(void)
+void prepare_packed_git(struct repository *r)
 {
 	struct alternate_object_database *alt;
 
-	if (the_repository->objects.packed_git_initialized)
+	if (r->objects.packed_git_initialized)
 		return;
-	prepare_packed_git_one(the_repository, get_object_directory(), 1);
-	prepare_alt_odb(the_repository);
-	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next)
-		prepare_packed_git_one(the_repository, alt->path, 0);
-	rearrange_packed_git(the_repository);
-	prepare_packed_git_mru(the_repository);
-	the_repository->objects.packed_git_initialized = 1;
+	prepare_packed_git_one(r, get_object_directory(), 1);
+	prepare_alt_odb(r);
+	for (alt = r->objects.alt_odb_list; alt; alt = alt->next)
+		prepare_packed_git_one(r, alt->path, 0);
+	rearrange_packed_git(r);
+	prepare_packed_git_mru(r);
+	r->objects.packed_git_initialized = 1;
 }
 
 void reprepare_packed_git_the_repository(void)
packfile.h +1 -2
index ab5046938c..3fd9092472 100644
@@ -34,8 +34,7 @@ extern struct packed_git *parse_pack_index(unsigned char *sha1, const char *idx_
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-#define prepare_packed_git(r) prepare_packed_git_##r()
-extern void prepare_packed_git_the_repository(void);
+extern void prepare_packed_git(struct repository *r);
 #define reprepare_packed_git(r) reprepare_packed_git_##r()
 extern void reprepare_packed_git_the_repository(void);
 extern void install_packed_git(struct repository *r, struct packed_git *pack);
Signature · 1 line
2.16.1.435.g8f24da2e1a
#69 lore raw

Re: [PATCH 40/44] packfile: allow prepare_packed_git to handle arbitrary repositories

Quoted 6 lines · Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

---

This is an invalid conversion.

Quoted 18 lines
 packfile.c | 18 +++++++++---------
 packfile.h |  3 +--
 2 files changed, 10 insertions(+), 11 deletions(-)
packfile.c +1 -2
index 52febba932..2276e2ad26 100644
@@ -882,19 +882,19 @@ static void prepare_packed_git_mru(struct repository *r)
 		list_add_tail(&p->mru, &r->objects.packed_git_mru);
 }
 
-void prepare_packed_git_the_repository(void)
+void prepare_packed_git(struct repository *r)
 {
 	struct alternate_object_database *alt;
 
-	if (the_repository->objects.packed_git_initialized)

packfile.c static void prepare_packed_git_mru(struct repository *r) · line 891

Quoted 10 lines
packfile.c +2 -7
+	if (r->objects.packed_git_initialized)
 		return;
-	prepare_packed_git_one(the_repository, get_object_directory(), 1);
-	prepare_alt_odb(the_repository);
-	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next)
-		prepare_packed_git_one(the_repository, alt->path, 0);
-	rearrange_packed_git(the_repository);
-	prepare_packed_git_mru(the_repository);
-	the_repository->objects.packed_git_initialized = 1;
+	prepare_packed_git_one(r, get_object_directory(), 1);

Calling get_object_directory() returns the_repository's object dir, this needs to be replaced with r->objects.objectdir.

Quoted 26 lines
packfile.c +6 -0
+	prepare_alt_odb(r);
+	for (alt = r->objects.alt_odb_list; alt; alt = alt->next)
+		prepare_packed_git_one(r, alt->path, 0);
+	rearrange_packed_git(r);
+	prepare_packed_git_mru(r);
+	r->objects.packed_git_initialized = 1;
 }
 
 void reprepare_packed_git_the_repository(void)
packfile.h +1 -2
index ab5046938c..3fd9092472 100644
@@ -34,8 +34,7 @@ extern struct packed_git *parse_pack_index(unsigned char *sha1, const char *idx_
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-#define prepare_packed_git(r) prepare_packed_git_##r()
-extern void prepare_packed_git_the_repository(void);
+extern void prepare_packed_git(struct repository *r);
 #define reprepare_packed_git(r) reprepare_packed_git_##r()
 extern void reprepare_packed_git_the_repository(void);
 extern void install_packed_git(struct repository *r, struct packed_git *pack);
Signature · 1 line
2.16.1.435.g8f24da2e1a
Signature · 1 line
Brandon Williams
#70 lore raw

Re: [PATCH 40/44] packfile: allow prepare_packed_git to handle arbitrary repositories

Quoted 27 lines · Brandon Williams
Quoted 6 lines · Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

---

This is an invalid conversion.

Quoted 17 lines
 packfile.c | 18 +++++++++---------
 packfile.h |  3 +--
 2 files changed, 10 insertions(+), 11 deletions(-)
packfile.c +1 -1
index 52febba932..2276e2ad26 100644
@@ -882,19 +882,19 @@ static void prepare_packed_git_mru(struct repository *r)
 		list_add_tail(&p->mru, &r->objects.packed_git_mru);
 }
 
-void prepare_packed_git_the_repository(void)
+void prepare_packed_git(struct repository *r)
 {
 	struct alternate_object_database *alt;
 
Quoted 14 lines

packfile.c static void prepare_packed_git_mru(struct repository *r) · line 891

Quoted 11 lines
packfile.c +2 -8
-	if (the_repository->objects.packed_git_initialized)
+	if (r->objects.packed_git_initialized)
 		return;
-	prepare_packed_git_one(the_repository, get_object_directory(), 1);
-	prepare_alt_odb(the_repository);
-	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next)
-		prepare_packed_git_one(the_repository, alt->path, 0);
-	rearrange_packed_git(the_repository);
-	prepare_packed_git_mru(the_repository);
-	the_repository->objects.packed_git_initialized = 1;
+	prepare_packed_git_one(r, get_object_directory(), 1);

Calling get_object_directory() returns the_repository's object dir, this needs to be replaced with r->objects.objectdir.

Nicely spotted. I think this was inherited from the orginal, e.g. the one from the end of last month

https://public-inbox.org/git/[email protected]/

also calls get_object_directory().

Thanks.

#71 lore raw

PATCH 41/44 packfile: allow reprepare_packed_git to handle arbitrary repositories

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

---
 packfile.c | 8 ++++----
 packfile.h | 3 +--
 2 files changed, 5 insertions(+), 6 deletions(-)
packfile.c +4 -4
index 2276e2ad26..fbf5d56754 100644
@@ -897,11 +897,11 @@ void prepare_packed_git(struct repository *r)
 	r->objects.packed_git_initialized = 1;
 }
 
-void reprepare_packed_git_the_repository(void)
+void reprepare_packed_git(struct repository *r)
 {
-	the_repository->objects.approximate_object_count_valid = 0;
-	the_repository->objects.packed_git_initialized = 0;
-	prepare_packed_git(the_repository);
+	r->objects.approximate_object_count_valid = 0;
+	r->objects.packed_git_initialized = 0;
+	prepare_packed_git(r);
 }
 
 struct packed_git *get_packed_git(struct repository *r)
packfile.h +1 -2
index 3fd9092472..ee6da3a9ae 100644
@@ -35,8 +35,7 @@ extern struct packed_git *parse_pack_index(unsigned char *sha1, const char *idx_
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
 extern void prepare_packed_git(struct repository *r);
-#define reprepare_packed_git(r) reprepare_packed_git_##r()
-extern void reprepare_packed_git_the_repository(void);
+extern void reprepare_packed_git(struct repository *r);
 extern void install_packed_git(struct repository *r, struct packed_git *pack);
 
 struct packed_git *get_packed_git(struct repository *r);
Signature · 1 line
2.16.1.435.g8f24da2e1a
#72 lore raw

PATCH 42/44 packfile: add repository argument to find_pack_entry

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

While at it move the documentation to the header and mention which pack files are searched.

---
 packfile.c  | 8 ++------
 packfile.h  | 7 ++++++-
 sha1_file.c | 6 +++---
 3 files changed, 11 insertions(+), 10 deletions(-)
packfile.c +2 -6
index fbf5d56754..7388debbb2 100644
@@ -1844,11 +1844,7 @@ static int fill_pack_entry(const unsigned char *sha1,
 	return 1;
 }
 
-/*
- * Iff a pack file contains the object named by sha1, return true and
- * store its location to e.
- */
-int find_pack_entry(const unsigned char *sha1, struct pack_entry *e)
+int find_pack_entry_the_repository(const unsigned char *sha1, struct pack_entry *e)
 {
 	struct list_head *pos;
 
@@ -1870,7 +1866,7 @@ int find_pack_entry(const unsigned char *sha1, struct pack_entry *e)
 int has_sha1_pack(const unsigned char *sha1)
 {
 	struct pack_entry e;
-	return find_pack_entry(sha1, &e);
+	return find_pack_entry(the_repository, sha1, &e);
 }
 
 int has_pack_index(const unsigned char *sha1)
packfile.h +6 -1
index ee6da3a9ae..e68f790ea7 100644
@@ -123,7 +123,12 @@ extern int packed_object_info(struct packed_git *pack, off_t offset, struct obje
 extern void mark_bad_packed_object(struct packed_git *p, const unsigned char *sha1);
 extern const struct packed_git *has_packed_and_bad(const unsigned char *sha1);
 
-extern int find_pack_entry(const unsigned char *sha1, struct pack_entry *e);
+/*
+ * Iff a pack file in the given repository contains the object named by sha1,
+ * return true and store its location to e.
+ */
+#define find_pack_entry(r, s, e) find_pack_entry_##r(s, e)
+extern int find_pack_entry_the_repository(const unsigned char *sha1, struct pack_entry *e);
 
 extern int has_sha1_pack(const unsigned char *sha1);
 
sha1_file.c +3 -3
index be84f84373..ba47963e63 100644
@@ -1266,7 +1266,7 @@ int sha1_object_info_extended(const unsigned char *sha1, struct object_info *oi,
 	}
 
 	while (1) {
-		if (find_pack_entry(real, &e))
+		if (find_pack_entry(the_repository, real, &e))
 			break;
 
 		/* Most likely it's a loose object. */
@@ -1275,7 +1275,7 @@ int sha1_object_info_extended(const unsigned char *sha1, struct object_info *oi,
 
 		/* Not a loose object; someone else may have just packed it. */
 		reprepare_packed_git(the_repository);
-		if (find_pack_entry(real, &e))
+		if (find_pack_entry(the_repository, real, &e))
 			break;
 
 		/* Check if it is a missing object */
@@ -1655,7 +1655,7 @@ static int freshen_loose_object(const unsigned char *sha1)
 static int freshen_packed_object(const unsigned char *sha1)
 {
 	struct pack_entry e;
-	if (!find_pack_entry(sha1, &e))
+	if (!find_pack_entry(the_repository, sha1, &e))
 		return 0;
 	if (e.p->freshened)
 		return 1;
Signature · 1 line
2.16.1.435.g8f24da2e1a
#73 lore raw

PATCH 43/44 packfile: allow find_pack_entry to handle arbitrary repositories

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

---
 packfile.c | 11 +++++------
 packfile.h |  3 +--
 2 files changed, 6 insertions(+), 8 deletions(-)
packfile.c +5 -6
index 7388debbb2..bafe81544d 100644
@@ -1844,19 +1844,18 @@ static int fill_pack_entry(const unsigned char *sha1,
 	return 1;
 }
 
-int find_pack_entry_the_repository(const unsigned char *sha1, struct pack_entry *e)
+int find_pack_entry(struct repository *r, const unsigned char *sha1, struct pack_entry *e)
 {
 	struct list_head *pos;
 
-	prepare_packed_git(the_repository);
-	if (!the_repository->objects.packed_git)
+	prepare_packed_git(r);
+	if (!r->objects.packed_git)
 		return 0;
 
-	list_for_each(pos, &the_repository->objects.packed_git_mru) {
+	list_for_each(pos, &r->objects.packed_git_mru) {
 		struct packed_git *p = list_entry(pos, struct packed_git, mru);
 		if (fill_pack_entry(sha1, e, p)) {
-			list_move(&p->mru,
-				  &the_repository->objects.packed_git_mru);
+			list_move(&p->mru, &r->objects.packed_git_mru);
 			return 1;
 		}
 	}
packfile.h +1 -2
index e68f790ea7..fe1a6380e6 100644
@@ -127,8 +127,7 @@ extern const struct packed_git *has_packed_and_bad(const unsigned char *sha1);
  * Iff a pack file in the given repository contains the object named by sha1,
  * return true and store its location to e.
  */
-#define find_pack_entry(r, s, e) find_pack_entry_##r(s, e)
-extern int find_pack_entry_the_repository(const unsigned char *sha1, struct pack_entry *e);
+extern int find_pack_entry(struct repository *r, const unsigned char *sha1, struct pack_entry *e);
 
 extern int has_sha1_pack(const unsigned char *sha1);
 
Signature · 1 line
2.16.1.435.g8f24da2e1a
#74 lore raw

PATCH 44/44 packfile: keep prepare_packed_git() private

In reply to Nguyễn Thái Ngọc Duy

The reason callers have to call this is to make sure either packed_git or packed_git_mru pointers are initialized since we don't do that by default. Sometimes it's hard to see this connection between where the function is called and where packed_git pointer is used (sometimes in separate functions).

Keep this dependency internal because now all access to packed_git and packed_git_mru must go through get_xxx() wrappers.

---
 builtin/count-objects.c  | 3 +--
 builtin/fsck.c           | 2 --
 builtin/gc.c             | 1 -
 builtin/pack-objects.c   | 1 -
 builtin/pack-redundant.c | 2 --
 fast-import.c            | 1 -
 http-backend.c           | 1 -
 pack-bitmap.c            | 1 -
 packfile.c               | 5 ++++-
 packfile.h               | 1 -
 server-info.c            | 1 -
 sha1_name.c              | 2 --
 12 files changed, 5 insertions(+), 16 deletions(-)
builtin/count-objects.c +1 -2
index 2793c98ed3..ee6ae35244 100644
@@ -121,8 +121,7 @@ int cmd_count_objects(int argc, const char **argv, const char *prefix)
 		struct strbuf loose_buf = STRBUF_INIT;
 		struct strbuf pack_buf = STRBUF_INIT;
 		struct strbuf garbage_buf = STRBUF_INIT;
-		if (!get_packed_git(the_repository))
-			prepare_packed_git(the_repository);
+
 		for (p = get_packed_git(the_repository); p; p = p->next) {
 			if (!p->pack_local)
 				continue;
builtin/fsck.c +0 -2
index 7a3e323e9e..9911c52bc8 100644
@@ -726,8 +726,6 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 			uint32_t total = 0, count = 0;
 			struct progress *progress = NULL;
 
-			prepare_packed_git(the_repository);
-
 			if (show_progress) {
 				for (p = get_packed_git(the_repository); p;
 				     p = p->next) {
builtin/gc.c +0 -1
index 560d58daec..be63bec09c 100644
@@ -173,7 +173,6 @@ static int too_many_packs(void)
 	if (gc_auto_pack_limit <= 0)
 		return 0;
 
-	prepare_packed_git(the_repository);
 	for (cnt = 0, p = get_packed_git(the_repository); p; p = p->next) {
 		if (!p->pack_local)
 			continue;
builtin/pack-objects.c +0 -1
index 6020a7e230..435f091a69 100644
@@ -3151,7 +3151,6 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 	if (progress && all_progress_implied)
 		progress = 2;
 
-	prepare_packed_git(the_repository);
 	if (ignore_packed_keep) {
 		struct packed_git *p;
 		for (p = get_packed_git(the_repository); p; p = p->next)
builtin/pack-redundant.c +0 -2
index bf42e164eb..02b5f0becc 100644
@@ -630,8 +630,6 @@ int cmd_pack_redundant(int argc, const char **argv, const char *prefix)
 			break;
 	}
 
-	prepare_packed_git(the_repository);
-
 	if (load_all_packs)
 		load_all();
 	else
fast-import.c +0 -1
index 985eb2eccc..2298bfcdfd 100644
@@ -3472,7 +3472,6 @@ int cmd_main(int argc, const char **argv)
 		rc_free[i].next = &rc_free[i + 1];
 	rc_free[cmd_save - 1].next = NULL;
 
-	prepare_packed_git(the_repository);
 	start_packfile();
 	set_die_routine(die_nicely);
 	set_checkpoint_signal();
http-backend.c +0 -1
index 659ddfb5f1..22d2e1668e 100644
@@ -518,7 +518,6 @@ static void get_info_packs(struct strbuf *hdr, char *arg)
 	size_t cnt = 0;
 
 	select_getanyfile(hdr);
-	prepare_packed_git(the_repository);
 	for (p = get_packed_git(the_repository); p; p = p->next) {
 		if (p->pack_local)
 			cnt++;
pack-bitmap.c +0 -1
index 01c9cd1642..2a007b5539 100644
@@ -335,7 +335,6 @@ static int open_pack_bitmap(void)
 
 	assert(!bitmap_git.map && !bitmap_git.loaded);
 
-	prepare_packed_git(the_repository);
 	for (p = get_packed_git(the_repository); p; p = p->next) {
 		if (open_pack_bitmap_1(p) == 0)
 			ret = 0;
packfile.c +4 -1
index bafe81544d..773cd99a13 100644
@@ -802,6 +802,7 @@ static void prepare_packed_git_one(struct repository *r, char *objdir, int local
 	strbuf_release(&path);
 }
 
+static void prepare_packed_git(struct repository *r);
 /*
  * Give a fast, rough count of the number of objects in the repository. This
  * ignores loose objects completely. If you have a lot of them, then either
@@ -882,7 +883,7 @@ static void prepare_packed_git_mru(struct repository *r)
 		list_add_tail(&p->mru, &r->objects.packed_git_mru);
 }
 
-void prepare_packed_git(struct repository *r)
+static void prepare_packed_git(struct repository *r)
 {
 	struct alternate_object_database *alt;
 
@@ -906,11 +907,13 @@ void reprepare_packed_git(struct repository *r)
 
 struct packed_git *get_packed_git(struct repository *r)
 {
+	prepare_packed_git(r);
 	return r->objects.packed_git;
 }
 
 struct list_head *get_packed_git_mru(struct repository *r)
 {
+	prepare_packed_git(r);
 	return &r->objects.packed_git_mru;
 }
 
packfile.h +0 -1
index fe1a6380e6..efda10329c 100644
@@ -34,7 +34,6 @@ extern struct packed_git *parse_pack_index(unsigned char *sha1, const char *idx_
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-extern void prepare_packed_git(struct repository *r);
 extern void reprepare_packed_git(struct repository *r);
 extern void install_packed_git(struct repository *r, struct packed_git *pack);
 
server-info.c +0 -1
index 9604772522..af737acd24 100644
@@ -200,7 +200,6 @@ static void init_pack_info(const char *infofile, int force)
 	objdir = get_object_directory();
 	objdirlen = strlen(objdir);
 
-	prepare_packed_git(the_repository);
 	for (p = get_packed_git(the_repository); p; p = p->next) {
 		/* we ignore things on alternate path since they are
 		 * not available to the pullers in general.
sha1_name.c +0 -2
index cb42e398b2..f25a0970ca 100644
@@ -195,7 +195,6 @@ static void find_short_packed_object(struct disambiguate_state *ds)
 {
 	struct packed_git *p;
 
-	prepare_packed_git(the_repository);
 	for (p = get_packed_git(the_repository); p && !ds->ambiguous;
 	     p = p->next)
 		unique_in_pack(p, ds);
@@ -566,7 +565,6 @@ static void find_abbrev_len_packed(struct min_abbrev_data *mad)
 {
 	struct packed_git *p;
 
-	prepare_packed_git(the_repository);
 	for (p = get_packed_git(the_repository); p; p = p->next)
 		find_abbrev_len_for_pack(p, mad);
 }
Signature · 1 line
2.16.1.435.g8f24da2e1a
#75 lore raw

Re: [PATCH 44/44] packfile: keep prepare_packed_git() private

Quoted 10 lines · On Sat, 3 Mar 2018 18:36:37 +0700 Nguyễn Thái Ngọc Duy <[email protected]>…

The reason callers have to call this is to make sure either packed_git or packed_git_mru pointers are initialized since we don't do that by default. Sometimes it's hard to see this connection between where the function is called and where packed_git pointer is used (sometimes in separate functions).

Keep this dependency internal because now all access to packed_git and packed_git_mru must go through get_xxx() wrappers.

The patches up to and including this one look good.

I also see that the question I asked in patch 10 about lazily initializing some fields is answered here.

If we're planning to avoid making the user call prepare_packed_git() (which I agree with), I think we also need to ensure that we always use the get_xxx() wrapper whenever we access objects.packed_git. Currently, there are still some functions in packfile.c that do not do so (notably for_each_packed_object()). Could these be changed too? (This should not be too difficult for you to do on your own, but I can send a fixup patch if you want.)

#76 lore raw

Re: [PATCH 44/44] packfile: keep prepare_packed_git() private

In reply to Nguyễn Thái Ngọc Duy
Quoted 8 lines · Nguyễn Thái Ngọc Duy

The reason callers have to call this is to make sure either packed_git or packed_git_mru pointers are initialized since we don't do that by default. Sometimes it's hard to see this connection between where the function is called and where packed_git pointer is used (sometimes in separate functions).

Keep this dependency internal because now all access to packed_git and packed_git_mru must go through get_xxx() wrappers.

Ahh now I understand the rational for trying to make it "private".

Quoted 199 lines
---
 builtin/count-objects.c  | 3 +--
 builtin/fsck.c           | 2 --
 builtin/gc.c             | 1 -
 builtin/pack-objects.c   | 1 -
 builtin/pack-redundant.c | 2 --
 fast-import.c            | 1 -
 http-backend.c           | 1 -
 pack-bitmap.c            | 1 -
 packfile.c               | 5 ++++-
 packfile.h               | 1 -
 server-info.c            | 1 -
 sha1_name.c              | 2 --
 12 files changed, 5 insertions(+), 16 deletions(-)
builtin/count-objects.c +1 -2
index 2793c98ed3..ee6ae35244 100644
@@ -121,8 +121,7 @@ int cmd_count_objects(int argc, const char **argv, const char *prefix)
 		struct strbuf loose_buf = STRBUF_INIT;
 		struct strbuf pack_buf = STRBUF_INIT;
 		struct strbuf garbage_buf = STRBUF_INIT;
-		if (!get_packed_git(the_repository))
-			prepare_packed_git(the_repository);
+
 		for (p = get_packed_git(the_repository); p; p = p->next) {
 			if (!p->pack_local)
 				continue;
builtin/fsck.c +0 -2
index 7a3e323e9e..9911c52bc8 100644
@@ -726,8 +726,6 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 			uint32_t total = 0, count = 0;
 			struct progress *progress = NULL;
 
-			prepare_packed_git(the_repository);
-
 			if (show_progress) {
 				for (p = get_packed_git(the_repository); p;
 				     p = p->next) {
builtin/gc.c +0 -1
index 560d58daec..be63bec09c 100644
@@ -173,7 +173,6 @@ static int too_many_packs(void)
 	if (gc_auto_pack_limit <= 0)
 		return 0;
 
-	prepare_packed_git(the_repository);
 	for (cnt = 0, p = get_packed_git(the_repository); p; p = p->next) {
 		if (!p->pack_local)
 			continue;
builtin/pack-objects.c +0 -1
index 6020a7e230..435f091a69 100644
@@ -3151,7 +3151,6 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 	if (progress && all_progress_implied)
 		progress = 2;
 
-	prepare_packed_git(the_repository);
 	if (ignore_packed_keep) {
 		struct packed_git *p;
 		for (p = get_packed_git(the_repository); p; p = p->next)
builtin/pack-redundant.c +0 -2
index bf42e164eb..02b5f0becc 100644
@@ -630,8 +630,6 @@ int cmd_pack_redundant(int argc, const char **argv, const char *prefix)
 			break;
 	}
 
-	prepare_packed_git(the_repository);
-
 	if (load_all_packs)
 		load_all();
 	else
fast-import.c +0 -1
index 985eb2eccc..2298bfcdfd 100644
@@ -3472,7 +3472,6 @@ int cmd_main(int argc, const char **argv)
 		rc_free[i].next = &rc_free[i + 1];
 	rc_free[cmd_save - 1].next = NULL;
 
-	prepare_packed_git(the_repository);
 	start_packfile();
 	set_die_routine(die_nicely);
 	set_checkpoint_signal();
http-backend.c +0 -1
index 659ddfb5f1..22d2e1668e 100644
@@ -518,7 +518,6 @@ static void get_info_packs(struct strbuf *hdr, char *arg)
 	size_t cnt = 0;
 
 	select_getanyfile(hdr);
-	prepare_packed_git(the_repository);
 	for (p = get_packed_git(the_repository); p; p = p->next) {
 		if (p->pack_local)
 			cnt++;
pack-bitmap.c +0 -1
index 01c9cd1642..2a007b5539 100644
@@ -335,7 +335,6 @@ static int open_pack_bitmap(void)
 
 	assert(!bitmap_git.map && !bitmap_git.loaded);
 
-	prepare_packed_git(the_repository);
 	for (p = get_packed_git(the_repository); p; p = p->next) {
 		if (open_pack_bitmap_1(p) == 0)
 			ret = 0;
packfile.c +4 -1
index bafe81544d..773cd99a13 100644
@@ -802,6 +802,7 @@ static void prepare_packed_git_one(struct repository *r, char *objdir, int local
 	strbuf_release(&path);
 }
 
+static void prepare_packed_git(struct repository *r);
 /*
  * Give a fast, rough count of the number of objects in the repository. This
  * ignores loose objects completely. If you have a lot of them, then either
@@ -882,7 +883,7 @@ static void prepare_packed_git_mru(struct repository *r)
 		list_add_tail(&p->mru, &r->objects.packed_git_mru);
 }
 
-void prepare_packed_git(struct repository *r)
+static void prepare_packed_git(struct repository *r)
 {
 	struct alternate_object_database *alt;
 
@@ -906,11 +907,13 @@ void reprepare_packed_git(struct repository *r)
 
 struct packed_git *get_packed_git(struct repository *r)
 {
+	prepare_packed_git(r);
 	return r->objects.packed_git;
 }
 
 struct list_head *get_packed_git_mru(struct repository *r)
 {
+	prepare_packed_git(r);
 	return &r->objects.packed_git_mru;
 }
 
packfile.h +0 -1
index fe1a6380e6..efda10329c 100644
@@ -34,7 +34,6 @@ extern struct packed_git *parse_pack_index(unsigned char *sha1, const char *idx_
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-extern void prepare_packed_git(struct repository *r);
 extern void reprepare_packed_git(struct repository *r);
 extern void install_packed_git(struct repository *r, struct packed_git *pack);
 
server-info.c +0 -1
index 9604772522..af737acd24 100644
@@ -200,7 +200,6 @@ static void init_pack_info(const char *infofile, int force)
 	objdir = get_object_directory();
 	objdirlen = strlen(objdir);
 
-	prepare_packed_git(the_repository);
 	for (p = get_packed_git(the_repository); p; p = p->next) {
 		/* we ignore things on alternate path since they are
 		 * not available to the pullers in general.
sha1_name.c +0 -2
index cb42e398b2..f25a0970ca 100644
@@ -195,7 +195,6 @@ static void find_short_packed_object(struct disambiguate_state *ds)
 {
 	struct packed_git *p;
 
-	prepare_packed_git(the_repository);
 	for (p = get_packed_git(the_repository); p && !ds->ambiguous;
 	     p = p->next)
 		unique_in_pack(p, ds);
@@ -566,7 +565,6 @@ static void find_abbrev_len_packed(struct min_abbrev_data *mad)
 {
 	struct packed_git *p;
 
-	prepare_packed_git(the_repository);
 	for (p = get_packed_git(the_repository); p; p = p->next)
 		find_abbrev_len_for_pack(p, mad);
 }
Signature · 1 line
2.16.1.435.g8f24da2e1a
Signature · 1 line
Brandon Williams
#77 lore raw

Re: [PATCH 00/44] reroll nd/remove-ignore-env.. sb/object-store and sb/packfiles..

In reply to Nguyễn Thái Ngọc Duy
Quoted 5 lines · Nguyễn Thái Ngọc Duy

01/44 - 05/44: nd/remove-ignore-env-field

This series is moved up top. After this the patch that touch alternate-db in sha1_file.c looks natural because no env is involved anymore

Yes, I do like having this upfront and being able to merge it before having to wait for the rest of the huge pile.

Quoted 4 lines

I also take this opportunity to introduce a new patch 01/44 to avoid struct initialization that makes it hard to read and update. Later patches are also simplified thanks to this.

Quoted 8 lines

06/44 - 32/44: sb/object-store

06/44 is updated to introduce raw_object_store_init() instead of RAW_OBJECT_STORE_INIT macro. This function is now used to initialize both main repo and submodule ones.

10/44 (moving "packed_git") also introduces two new access wrapper get_packed_git() and get_packed_git_mru()

I haven't studied individual patches in this round for these, but the interdiff you show below looks quite sensible.

Thanks, will take a bit deeper look at the rest.

#78 lore raw

Re: [PATCH 00/44] reroll nd/remove-ignore-env.. sb/object-store and sb/packfiles..

In reply to Nguyễn Thái Ngọc Duy
Quoted 56 lines · Nguyễn Thái Ngọc Duy
Quoted 16 lines · Duy Nguyen
Quoted 15 lines · Junio C Hamano
Quoted 12 lines · Stefan Beller
Quoted 8 lines · Junio C Hamano
Quoted 4 lines · Duy Nguyen

Looking at the full-series diff though, it makes me wonder if we should keep prepare_packed_git() private (i.e. how we initialize the object store, packfile included, is a private matter). How about something like this on top?

Yup, that looks cleaner.

I agree that it looks cleaner. So we plan on just putting it on top of that series?

Quoted 7 lines
Quoted 4 lines

We tend to avoid "oops, that was wrong and here is a band aid on top" for things that are still mushy, so it would be preferrable to get it fixed inline, especially if there are more changes to the other parts of the series coming.

I agree with this. Stefan, if you're getting bored with rerolling refactor patches, I can update this series and send out v2.

Since Stefan is traveling, I take this opportunity to reroll it. Unfortunately, I think the fix should go in 46cd557bd9 (object-store:
move packed_git and packed_git_mru to object store - 2018-02-23) where we start removing the global "packed_git". But that's in sb/object-store, so.. I'm rerolling all three

01/44 - 05/44: nd/remove-ignore-env-field

This series is moved up top. After this the patch that touch alternate-db in sha1_file.c looks natural because no env is involved anymore

I also take this opportunity to introduce a new patch 01/44 to avoid struct initialization that makes it hard to read and update. Later patches are also simplified thanks to this.

06/44 - 32/44: sb/object-store

06/44 is updated to introduce raw_object_store_init() instead of RAW_OBJECT_STORE_INIT macro. This function is now used to initialize both main repo and submodule ones.

10/44 (moving "packed_git") also introduces two new access wrapper get_packed_git() and get_packed_git_mru()

33/44 - 44/44: sb/packfiles-in-repository

The only thing new here is 44/44 which makes prepare_packed_git() internal. get_packed_git() and get_packed_git_mru() introduced earlier will call prepare_packed_git() automatically.

Quoted 6 lines

The whole thing is also available at

https://github.com/pclouds/git/tree/ignore-env-object-store-packfiles

And interdiff of all three, compared to what is currently in 'pu'. Looks pretty good in my opinon:

The only major issue I could find is with patch 40 and must be fixed before this can be merged. The rest of the series looks good.

Signature · 1 line
Brandon Williams
#79 lore raw

cover nd/remove-ignore-env-field updates

In reply to Nguyễn Thái Ngọc Duy

This is a very small, non-functional update. On top of 'next'

Nguyễn Thái Ngọc Duy (1):
repository.h: add comment and clarify repo_set_gitdir

 repository.h | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#80 lore raw

PATCH 1/1 repository.h: add comment and clarify repo_set_gitdir

The argument name "optional" may mislead the reader to think this option could be NULL. But it can't be. While at there, document a bit more about struct set_gitdir_args.

---
 repository.h | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
repository.h +5 -1
index e7127baffb..f21fd93f72 100644
@@ -82,6 +82,10 @@ struct repository {
 
 extern struct repository *the_repository;
 
+/*
+ * Define a custom repository layout. Any field can be NULL, which
+ * will default back to the path according to the default layout.
+ */
 struct set_gitdir_args {
 	const char *commondir;
 	const char *object_dir;
@@ -92,7 +96,7 @@ struct set_gitdir_args {
 
 extern void repo_set_gitdir(struct repository *repo,
 			    const char *root,
-			    const struct set_gitdir_args *optional);
+			    const struct set_gitdir_args *extra_args);
 extern void repo_set_worktree(struct repository *repo, const char *path);
 extern void repo_set_hash_algo(struct repository *repo, int algo);
 extern void initialize_the_repository(void);
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#82 lore raw

Re: [PATCH 1/1] repository.h: add comment and clarify repo_set_gitdir

Quoted 1 line · Stefan Beller

On Fri, Mar 23, 2018 at 8:55 AM Nguyễn Thái Ngọc Duy <[email protected]>

Quoted 8 lines · wrote:
Quoted 6 lines

The argument name "optional" may mislead the reader to think this option could be NULL. But it can't be. While at there, document a bit more about struct set_gitdir_args.

Uh, not so fast. Is it worthwhile to align the parameter name in the c file as well?

#83 lore raw

cover sb/object-store updates

In reply to Nguyễn Thái Ngọc Duy

I think I have addressed all comments I've received so far. What I decided not to do, I have responded individually. One comment I did not respond nor do is the approximate thing, which could be done later.

Interdiff is big due to the "objects." to "objects->" conversion (blame Brandon for his suggestion, don't blame me :D)

builtin/am.c +1 -1
index 4762a702e3..47beddbe24 100644
@@ -1859,7 +1859,7 @@ static void am_run(struct am_state *state, int resume)
 	 */
 	if (!state->rebasing) {
 		am_destroy(state);
-		close_all_packs(&the_repository->objects);
+		close_all_packs(the_repository->objects);
 		run_command_v_opt(argv_gc_auto, RUN_GIT_CMD);
 	}
 }
builtin/clone.c +2 -1
index 13cfaa6488..7df5932b85 100644
@@ -27,6 +27,7 @@
 #include "connected.h"
 #include "packfile.h"
 #include "list-objects-filter-options.h"
+#include "object-store.h"
 
 /*
  * Overall FIXMEs:
@@ -1217,7 +1218,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 	transport_disconnect(transport);
 
 	if (option_dissociate) {
-		close_all_packs(&the_repository->objects);
+		close_all_packs(the_repository->objects);
 		dissociate_from_references();
 	}
 
builtin/count-objects.c +1 -0
index 5c7c3c6ae3..b28ff00be2 100644
@@ -12,6 +12,7 @@
 #include "parse-options.h"
 #include "quote.h"
 #include "packfile.h"
+#include "object-store.h"
 
 static unsigned long garbage;
 static off_t size_garbage;
builtin/fetch.c +1 -1
index 4d72efca78..a39e9d7b15 100644
@@ -1478,7 +1478,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
 
 	string_list_clear(&list, 0);
 
-	close_all_packs(&the_repository->objects);
+	close_all_packs(the_repository->objects);
 
 	argv_array_pushl(&argv_gc_auto, "gc", "--auto", NULL);
 	if (verbosity < 0)
builtin/fsck.c +5 -2
index b0abba6e04..3ef25fab97 100644
@@ -17,6 +17,7 @@
 #include "streaming.h"
 #include "decorate.h"
 #include "packfile.h"
+#include "object-store.h"
 
 #define REACHABLE 0x0001
 #define SEEN      0x0002
@@ -714,11 +715,13 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 		for_each_loose_object(mark_loose_for_connectivity, NULL, 0);
 		for_each_packed_object(mark_packed_for_connectivity, NULL, 0);
 	} else {
+		struct alternate_object_database *alt_odb_list;
+
 		fsck_object_dir(get_object_directory());
 
 		prepare_alt_odb(the_repository);
-		for (alt = the_repository->objects.alt_odb_list;
-		     alt; alt = alt->next)
+		alt_odb_list = the_repository->objects->alt_odb_list;
+		for (alt = alt_odb_list; alt; alt = alt->next)
 			fsck_object_dir(alt->path);
 
 		if (check_full) {
builtin/gc.c +1 -0
index dd30067ac1..b00238cd5d 100644
@@ -21,6 +21,7 @@
 #include "argv-array.h"
 #include "commit.h"
 #include "packfile.h"
+#include "object-store.h"
 
 #define FAILED_RUN "failed to run %s"
 
builtin/grep.c +2 -1
index 0f0c195705..1e9cdbdf78 100644
@@ -22,6 +22,7 @@
 #include "pathspec.h"
 #include "submodule.h"
 #include "submodule-config.h"
+#include "object-store.h"
 
 static char const * const grep_usage[] = {
 	N_("git grep [<options>] [-e] <pattern> [<rev>...] [[--] <path>...]"),
@@ -432,7 +433,7 @@ static int grep_submodule(struct grep_opt *opt, struct repository *superproject,
 	 * object.
 	 */
 	grep_read_lock();
-	add_to_alternates_memory(submodule.objects.objectdir);
+	add_to_alternates_memory(submodule.objects->objectdir);
 	grep_read_unlock();
 
 	if (oid) {
builtin/index-pack.c +1 -0
index 5ebd370c56..1d6bc87b76 100644
@@ -13,6 +13,7 @@
 #include "streaming.h"
 #include "thread-utils.h"
 #include "packfile.h"
+#include "object-store.h"
 
 static const char index_pack_usage[] =
 "git index-pack [-v] [-o <index-file>] [--keep | --keep=<msg>] [--verify] [--strict] (<pack-file> | --stdin [--fix-thin] [<pack-file>])";
builtin/merge.c +1 -1
index 907ae44ab5..96d56cbdd2 100644
@@ -411,7 +411,7 @@ static void finish(struct commit *head_commit,
 			 * We ignore errors in 'gc --auto', since the
 			 * user should see them.
 			 */
-			close_all_packs(&the_repository->objects);
+			close_all_packs(the_repository->objects);
 			run_command_v_opt(argv_gc_auto, RUN_GIT_CMD);
 		}
 	}
builtin/pack-objects.c +1 -0
index 0e5fde1d6b..223f2d9fc0 100644
@@ -29,6 +29,7 @@
 #include "argv-array.h"
 #include "list.h"
 #include "packfile.h"
+#include "object-store.h"
 
 static const char *pack_usage[] = {
 	N_("git pack-objects --stdout [<options>...] [< <ref-list> | < <object-list>]"),
builtin/pack-redundant.c +1 -0
index d6d8a44959..b5b007e706 100644
@@ -9,6 +9,7 @@
 #include "builtin.h"
 #include "repository.h"
 #include "packfile.h"
+#include "object-store.h"
 
 #define BLKSIZE 512
 
builtin/receive-pack.c +1 -1
index ac478b7b99..1a298a6711 100644
@@ -2026,7 +2026,7 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)
 			proc.git_cmd = 1;
 			proc.argv = argv_gc_auto;
 
-			close_all_packs(&the_repository->objects);
+			close_all_packs(the_repository->objects);
 			if (!start_command(&proc)) {
 				if (use_sideband)
 					copy_to_sideband(proc.err, -1, NULL);
builtin/submodule--helper.c +1 -0
index b1daca995f..6d8e002be7 100644
@@ -16,6 +16,7 @@
 #include "revision.h"
 #include "diffcore.h"
 #include "diff.h"
+#include "object-store.h"
 
 #define OPT_QUIET (1 << 0)
 #define OPT_CACHED (1 << 1)
environment.c +2 -2
index c05705e384..93c9fbb0ba 100644
@@ -271,9 +271,9 @@ const char *get_git_work_tree(void)
 
 char *get_object_directory(void)
 {
-	if (!the_repository->objects.objectdir)
+	if (!the_repository->objects->objectdir)
 		BUG("git environment hasn't been setup");
-	return the_repository->objects.objectdir;
+	return the_repository->objects->objectdir;
 }
 
 int odb_mkstemp(struct strbuf *template, const char *pattern)
fast-import.c +1 -0
index 0dba555478..b3492fce5c 100644
@@ -169,6 +169,7 @@ Format of STDIN stream:
 #include "dir.h"
 #include "run-command.h"
 #include "packfile.h"
+#include "object-store.h"
 
 #define PACK_ID_BITS 16
 #define MAX_PACK_ID ((1<<PACK_ID_BITS)-1)
http-backend.c +1 -0
index 22921d169a..64dde78c1b 100644
@@ -11,6 +11,7 @@
 #include "url.h"
 #include "argv-array.h"
 #include "packfile.h"
+#include "object-store.h"
 
 static const char content_type[] = "Content-Type";
 static const char content_length[] = "Content-Length";
http-push.c +1 -0
index 0913f8ab86..97fe22a705 100644
@@ -12,6 +12,7 @@
 #include "sigchain.h"
 #include "argv-array.h"
 #include "packfile.h"
+#include "object-store.h"
 
 #ifdef EXPAT_NEEDS_XMLPARSE_H
 #include <xmlparse.h>
http-walker.c +1 -0
index 2c33969123..75d55d42a9 100644
@@ -6,6 +6,7 @@
 #include "list.h"
 #include "transport.h"
 #include "packfile.h"
+#include "object-store.h"
 
 struct alt_base {
 	char *base;
http.c +1 -0
index df9dbea59c..4d613d5f6b 100644
@@ -14,6 +14,7 @@
 #include "packfile.h"
 #include "protocol.h"
 #include "string-list.h"
+#include "object-store.h"
 
 static struct trace_key trace_curl = TRACE_KEY_INIT(CURL);
 static int trace_curl_data = 1;
object-store.h +11 -5
index 521f5a1755..fef33f345f 100644
@@ -90,13 +90,19 @@ struct raw_object_store {
 	/* Path to extra alternate object database if not NULL */
 	char *alternate_db;
 
-	struct packed_git *packed_git; /* private */
-	/* A most-recently-used ordered version of the packed_git list. */
-	struct list_head packed_git_mru; /* private */
-
 	struct alternate_object_database *alt_odb_list;
 	struct alternate_object_database **alt_odb_tail;
 
+	/*
+	 * private data
+	 *
+	 * should only be accessed directly by packfile.c
+	 */
+
+	struct packed_git *packed_git;
+	/* A most-recently-used ordered version of the packed_git list. */
+	struct list_head packed_git_mru;
+
 	/*
 	 * A fast, rough count of the number of objects in the repository.
 	 * These two fields are not meant for direct access. Use
@@ -112,7 +118,7 @@ struct raw_object_store {
 	unsigned packed_git_initialized : 1;
 };
 
-void raw_object_store_init(struct raw_object_store *o);
+struct raw_object_store *raw_object_store_new(void);
 void raw_object_store_clear(struct raw_object_store *o);
 
 /*
object.c +10 -2
index 60ca76b285..4c2cf7ff5d 100644
@@ -4,6 +4,7 @@
 #include "tree.h"
 #include "commit.h"
 #include "tag.h"
+#include "object-store.h"
 #include "packfile.h"
 
 static struct object **obj_hash;
@@ -447,23 +448,30 @@ void clear_commit_marks_all(unsigned int flags)
 	}
 }
 
-void raw_object_store_init(struct raw_object_store *o)
+struct raw_object_store *raw_object_store_new(void)
 {
+	struct raw_object_store *o = xmalloc(sizeof(*o));
+
 	memset(o, 0, sizeof(*o));
 	INIT_LIST_HEAD(&o->packed_git_mru);
+	return o;
 }
 
 static void free_alt_odb(struct alternate_object_database *alt)
 {
 	strbuf_release(&alt->scratch);
 	oid_array_clear(&alt->loose_objects_cache);
+	free(alt);
 }
 
 static void free_alt_odbs(struct raw_object_store *o)
 {
 	while (o->alt_odb_list) {
+		struct alternate_object_database *next;
+
+		next = o->alt_odb_list->next;
 		free_alt_odb(o->alt_odb_list);
-		o->alt_odb_list = o->alt_odb_list->next;
+		o->alt_odb_list = next;
 	}
 }
 
pack-bitmap.c +1 -0
index abed43cdb5..22cd425788 100644
@@ -11,6 +11,7 @@
 #include "pack-objects.h"
 #include "packfile.h"
 #include "repository.h"
+#include "object-store.h"
 
 /*
  * An entry on the bitmap index, representing the bitmap for a given
pack-check.c +1 -0
index 073c1fbd46..2378f25999 100644
@@ -3,6 +3,7 @@
 #include "pack-revindex.h"
 #include "progress.h"
 #include "packfile.h"
+#include "object-store.h"
 
 struct idx_entry {
 	off_t                offset;
pack-revindex.c +1 -0
index ff5f62c033..bb521cf7fb 100644
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "pack-revindex.h"
+#include "object-store.h"
 
 /*
  * Pack index for existing packs give us easy access to the offsets into
packfile.c +28 -27

168 lines not on this page · raw message

path.c +2 -1
index 81a42d9115..3308b7b958 100644
@@ -10,6 +10,7 @@
 #include "submodule-config.h"
 #include "path.h"
 #include "packfile.h"
+#include "object-store.h"
 
 static int get_st_mode_bits(const char *path, int *mode)
 {
@@ -382,7 +383,7 @@ static void adjust_git_path(const struct repository *repo,
 		strbuf_splice(buf, 0, buf->len,
 			      repo->index_file, strlen(repo->index_file));
 	else if (dir_prefix(base, "objects"))
-		replace_dir(buf, git_dir_len + 7, repo->objects.objectdir);
+		replace_dir(buf, git_dir_len + 7, repo->objects->objectdir);
 	else if (git_hooks_path && dir_prefix(base, "hooks"))
 		replace_dir(buf, git_dir_len + 5, git_hooks_path);
 	else if (repo->different_commondir)
reachable.c +1 -0
index 88d7d679da..25cfd99d1c 100644
@@ -11,6 +11,7 @@
 #include "list-objects.h"
 #include "packfile.h"
 #include "worktree.h"
+#include "object-store.h"
 
 struct connectivity_progress {
 	struct progress *progress;
repository.c +7 -6
index 34c0a7f180..a4848c1bd0 100644
@@ -13,7 +13,7 @@ void initialize_the_repository(void)
 	the_repository = &the_repo;
 
 	the_repo.index = &the_index;
-	raw_object_store_init(&the_repo.objects);
+	the_repo.objects = raw_object_store_new();
 	repo_set_hash_algo(&the_repo, GIT_HASH_SHA1);
 }
 
@@ -60,10 +60,10 @@ void repo_set_gitdir(struct repository *repo,
 	free(old_gitdir);
 
 	repo_set_commondir(repo, o->commondir);
-	expand_base_dir(&repo->objects.objectdir, o->object_dir,
+	expand_base_dir(&repo->objects->objectdir, o->object_dir,
 			repo->commondir, "objects");
-	free(repo->objects.alternate_db);
-	repo->objects.alternate_db = xstrdup_or_null(o->alternate_db);
+	free(repo->objects->alternate_db);
+	repo->objects->alternate_db = xstrdup_or_null(o->alternate_db);
 	expand_base_dir(&repo->graft_file, o->graft_file,
 			repo->commondir, "info/grafts");
 	expand_base_dir(&repo->index_file, o->index_file,
@@ -142,7 +142,7 @@ static int repo_init(struct repository *repo,
 	struct repository_format format;
 	memset(repo, 0, sizeof(*repo));
 
-	raw_object_store_init(&repo->objects);
+	repo->objects = raw_object_store_new();
 
 	if (repo_init_gitdir(repo, gitdir))
 		goto error;
@@ -223,7 +223,8 @@ void repo_clear(struct repository *repo)
 	FREE_AND_NULL(repo->worktree);
 	FREE_AND_NULL(repo->submodule_prefix);
 
-	raw_object_store_clear(&repo->objects);
+	raw_object_store_clear(repo->objects);
+	FREE_AND_NULL(repo->objects);
 
 	if (repo->config) {
 		git_configset_clear(repo->config);
repository.h +3 -4
index 6c383c05c6..abba2c09bc 100644
@@ -1,12 +1,11 @@
 #ifndef REPOSITORY_H
 #define REPOSITORY_H
 
-#include "object-store.h"
-
 struct config_set;
+struct git_hash_algo;
 struct index_state;
+struct raw_object_store;
 struct submodule_cache;
-struct git_hash_algo;
 
 struct repository {
 	/* Environment */
@@ -25,7 +24,7 @@ struct repository {
 	/*
 	 * Holds any information related to accessing the raw object content.
 	 */
-	struct raw_object_store objects;
+	struct raw_object_store *objects;
 
 	/*
 	 * Path to the repository's graft file.
server-info.c +1 -0
index 6fe64ede17..75a8b65e47 100644
@@ -5,6 +5,7 @@
 #include "commit.h"
 #include "tag.h"
 #include "packfile.h"
+#include "object-store.h"
 
 /*
  * Create the file "path" by writing to a temporary file and renaming
sha1_file.c +15 -15
index 7066d4c9ce..0989bbd948 100644
@@ -23,7 +23,6 @@
 #include "sha1-lookup.h"
 #include "bulk-checkin.h"
 #include "repository.h"
-#include "object-store.h"
 #include "streaming.h"
 #include "dir.h"
 #include "list.h"
@@ -31,6 +30,7 @@
 #include "quote.h"
 #include "packfile.h"
 #include "fetch-object.h"
+#include "object-store.h"
 
 const unsigned char null_sha1[GIT_MAX_RAWSZ];
 const struct object_id null_oid;
@@ -325,7 +325,7 @@ static void fill_sha1_path(struct strbuf *buf, const unsigned char *sha1)
 
 void sha1_file_name(struct repository *r, struct strbuf *buf, const unsigned char *sha1)
 {
-	strbuf_addstr(buf, r->objects.objectdir);
+	strbuf_addstr(buf, r->objects->objectdir);
 	strbuf_addch(buf, '/');
 	fill_sha1_path(buf, sha1);
 }
@@ -419,7 +419,7 @@ static int link_alt_odb_entry(struct repository *r, const char *entry,
 	while (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')
 		strbuf_setlen(&pathbuf, pathbuf.len - 1);
 
-	if (!alt_odb_usable(&r->objects, &pathbuf, normalized_objdir)) {
+	if (!alt_odb_usable(r->objects, &pathbuf, normalized_objdir)) {
 		strbuf_release(&pathbuf);
 		return -1;
 	}
@@ -427,8 +427,8 @@ static int link_alt_odb_entry(struct repository *r, const char *entry,
 	ent = alloc_alt_odb(pathbuf.buf);
 
 	/* add the alternate entry */
-	*r->objects.alt_odb_tail = ent;
-	r->objects.alt_odb_tail = &(ent->next);
+	*r->objects->alt_odb_tail = ent;
+	r->objects->alt_odb_tail = &(ent->next);
 	ent->next = NULL;
 
 	/* recursively add alternates */
@@ -482,7 +482,7 @@ static void link_alt_odb_entries(struct repository *r, const char *alt,
 		return;
 	}
 
-	strbuf_add_absolute_path(&objdirbuf, r->objects.objectdir);
+	strbuf_add_absolute_path(&objdirbuf, r->objects->objectdir);
 	if (strbuf_normalize_path(&objdirbuf) < 0)
 		die("unable to normalize object directory: %s",
 		    objdirbuf.buf);
@@ -565,7 +565,7 @@ void add_to_alternates_file(const char *reference)
 		fprintf_or_die(out, "%s\n", reference);
 		if (commit_lock_file(&lock))
 			die_errno("unable to move new alternates file into place");
-		if (the_repository->objects.alt_odb_tail)
+		if (the_repository->objects->alt_odb_tail)
 			link_alt_odb_entries(the_repository, reference,
 					     '\n', NULL, 0);
 	}
@@ -665,7 +665,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 	int r = 0;
 
 	prepare_alt_odb(the_repository);
-	for (ent = the_repository->objects.alt_odb_list; ent; ent = ent->next) {
+	for (ent = the_repository->objects->alt_odb_list; ent; ent = ent->next) {
 		r = fn(ent, cb);
 		if (r)
 			break;
@@ -675,13 +675,13 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 
 void prepare_alt_odb(struct repository *r)
 {
-	if (r->objects.alt_odb_tail)
+	if (r->objects->alt_odb_tail)
 		return;
 
-	r->objects.alt_odb_tail = &r->objects.alt_odb_list;
-	link_alt_odb_entries(r, r->objects.alternate_db, PATH_SEP, NULL, 0);
+	r->objects->alt_odb_tail = &r->objects->alt_odb_list;
+	link_alt_odb_entries(r, r->objects->alternate_db, PATH_SEP, NULL, 0);
 
-	read_info_alternates(r, r->objects.objectdir, 0);
+	read_info_alternates(r, r->objects->objectdir, 0);
 }
 
 /* Returns 1 if we have successfully freshened the file, 0 otherwise. */
@@ -722,7 +722,7 @@ static int check_and_freshen_nonlocal(const unsigned char *sha1, int freshen)
 {
 	struct alternate_object_database *alt;
 	prepare_alt_odb(the_repository);
-	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
+	for (alt = the_repository->objects->alt_odb_list; alt; alt = alt->next) {
 		const char *path = alt_sha1_path(alt, sha1);
 		if (check_and_freshen_file(path, freshen))
 			return 1;
@@ -882,7 +882,7 @@ static int stat_sha1_file(struct repository *r, const unsigned char *sha1,
 
 	prepare_alt_odb(r);
 	errno = ENOENT;
-	for (alt = r->objects.alt_odb_list; alt; alt = alt->next) {
+	for (alt = r->objects->alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		if (!lstat(*path, st))
 			return 0;
@@ -913,7 +913,7 @@ static int open_sha1_file(struct repository *r,
 	most_interesting_errno = errno;
 
 	prepare_alt_odb(r);
-	for (alt = r->objects.alt_odb_list; alt; alt = alt->next) {
+	for (alt = r->objects->alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		fd = git_open(*path);
 		if (fd >= 0)
sha1_name.c +2 -1
index 2065be90d2..4325f74e0c 100644
@@ -10,6 +10,7 @@
 #include "dir.h"
 #include "sha1-array.h"
 #include "packfile.h"
+#include "object-store.h"
 #include "repository.h"
 
 static int get_oid_oneline(const char *, struct object_id *, struct commit_list *);
@@ -105,7 +106,7 @@ static void find_short_object_filename(struct disambiguate_state *ds)
 		 */
 		fakeent = alloc_alt_odb(get_object_directory());
 	}
-	fakeent->next = the_repository->objects.alt_odb_list;
+	fakeent->next = the_repository->objects->alt_odb_list;
 
 	for (alt = fakeent; alt && !ds->ambiguous; alt = alt->next) {
 		int pos;
submodule.c +1 -0
index 47ddc9b273..b03e5f5045 100644
@@ -21,6 +21,7 @@
 #include "remote.h"
 #include "worktree.h"
 #include "parse-options.h"
+#include "object-store.h"
 
 static int config_update_recurse_submodules = RECURSE_SUBMODULES_OFF;
 static struct string_list changed_submodule_names = STRING_LIST_INIT_DUP;
t/helper/test-ref-store.c +1 -0
index 7120634b04..7314b5943e 100644
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "refs.h"
 #include "worktree.h"
+#include "object-store.h"
 
 static const char *notnull(const char *arg, const char *name)
 {
tmp-objdir.c +1 -0
index b2d9280f10..fea3f55545 100644
@@ -6,6 +6,7 @@
 #include "strbuf.h"
 #include "argv-array.h"
 #include "quote.h"
+#include "object-store.h"
 
 struct tmp_objdir {
 	struct strbuf path;
transport.c +1 -0
index 00d48b5b56..3afc632472 100644
@@ -18,6 +18,7 @@
 #include "sha1-array.h"
 #include "sigchain.h"
 #include "transport-internal.h"
+#include "object-store.h"
 
 static void set_upstreams(struct transport *transport, struct ref *refs,
 	int pretend)
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#84 lore raw

PATCH 1/27 repository: introduce raw object store field

From: Stefan Beller <[email protected]>

The raw object store field will contain any objects needed for access to objects in a given repository.

This patch introduces the raw object store and populates it with the objectdir, which used to be part of the repository struct.

As the struct gains members, we'll also populate the function to clear the memory for these members.

In a later step, we'll introduce a struct object_parser, that will complement the object parsing in a repository struct: The raw object parser is the layer that will provide access to raw object content, while the higher level object parser code will parse raw objects and keeps track of parenthood and other object relationships using 'struct object'. For now only add the lower level to the repository struct.

---
 builtin/grep.c |  3 ++-
 environment.c  |  5 +++--
 object-store.h | 18 ++++++++++++++++++
 object.c       | 14 ++++++++++++++
 path.c         |  3 ++-
 repository.c   | 15 ++++++++++-----
 repository.h   | 11 ++++-------
 sha1_file.c    |  4 +++-
 8 files changed, 56 insertions(+), 17 deletions(-)
 create mode 100644 object-store.h
builtin/grep.c +2 -1
index 3ca4ac80d8..1e9cdbdf78 100644
@@ -22,6 +22,7 @@
 #include "pathspec.h"
 #include "submodule.h"
 #include "submodule-config.h"
+#include "object-store.h"
 
 static char const * const grep_usage[] = {
 	N_("git grep [<options>] [-e] <pattern> [<rev>...] [[--] <path>...]"),
@@ -432,7 +433,7 @@ static int grep_submodule(struct grep_opt *opt, struct repository *superproject,
 	 * object.
 	 */
 	grep_read_lock();
-	add_to_alternates_memory(submodule.objectdir);
+	add_to_alternates_memory(submodule.objects->objectdir);
 	grep_read_unlock();
 
 	if (oid) {
environment.c +3 -2
index a5eaa97fb1..93c9fbb0ba 100644
@@ -14,6 +14,7 @@
 #include "fmt-merge-msg.h"
 #include "commit.h"
 #include "argv-array.h"
+#include "object-store.h"
 
 int trust_executable_bit = 1;
 int trust_ctime = 1;
@@ -270,9 +271,9 @@ const char *get_git_work_tree(void)
 
 char *get_object_directory(void)
 {
-	if (!the_repository->objectdir)
+	if (!the_repository->objects->objectdir)
 		BUG("git environment hasn't been setup");
-	return the_repository->objectdir;
+	return the_repository->objects->objectdir;
 }
 
 int odb_mkstemp(struct strbuf *template, const char *pattern)
object-store.h new file +18 -0
new file mode 100644
index 0000000000..abfaae059b
@@ -0,0 +1,18 @@
+#ifndef OBJECT_STORE_H
+#define OBJECT_STORE_H
+
+struct raw_object_store {
+	/*
+	 * Path to the repository's object store.
+	 * Cannot be NULL after initialization.
+	 */
+	char *objectdir;
+
+	/* Path to extra alternate object database if not NULL */
+	char *alternate_db;
+};
+
+struct raw_object_store *raw_object_store_new(void);
+void raw_object_store_clear(struct raw_object_store *o);
+
+#endif /* OBJECT_STORE_H */
object.c +14 -0
index 9e6f9ff20b..6ddd61242c 100644
@@ -4,6 +4,7 @@
 #include "tree.h"
 #include "commit.h"
 #include "tag.h"
+#include "object-store.h"
 
 static struct object **obj_hash;
 static int nr_objs, obj_hash_size;
@@ -445,3 +446,16 @@ void clear_commit_marks_all(unsigned int flags)
 			obj->flags &= ~flags;
 	}
 }
+
+struct raw_object_store *raw_object_store_new(void)
+{
+	struct raw_object_store *o = xmalloc(sizeof(*o));
+
+	memset(o, 0, sizeof(*o));
+	return o;
+}
+void raw_object_store_clear(struct raw_object_store *o)
+{
+	FREE_AND_NULL(o->objectdir);
+	FREE_AND_NULL(o->alternate_db);
+}
path.c +2 -1
index da8b655730..3308b7b958 100644
@@ -10,6 +10,7 @@
 #include "submodule-config.h"
 #include "path.h"
 #include "packfile.h"
+#include "object-store.h"
 
 static int get_st_mode_bits(const char *path, int *mode)
 {
@@ -382,7 +383,7 @@ static void adjust_git_path(const struct repository *repo,
 		strbuf_splice(buf, 0, buf->len,
 			      repo->index_file, strlen(repo->index_file));
 	else if (dir_prefix(base, "objects"))
-		replace_dir(buf, git_dir_len + 7, repo->objectdir);
+		replace_dir(buf, git_dir_len + 7, repo->objects->objectdir);
 	else if (git_hooks_path && dir_prefix(base, "hooks"))
 		replace_dir(buf, git_dir_len + 5, git_hooks_path);
 	else if (repo->different_commondir)
repository.c +10 -5
index 62f52f47fc..a4848c1bd0 100644
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "repository.h"
+#include "object-store.h"
 #include "config.h"
 #include "submodule-config.h"
 
@@ -12,6 +13,7 @@ void initialize_the_repository(void)
 	the_repository = &the_repo;
 
 	the_repo.index = &the_index;
+	the_repo.objects = raw_object_store_new();
 	repo_set_hash_algo(&the_repo, GIT_HASH_SHA1);
 }
 
@@ -58,10 +60,10 @@ void repo_set_gitdir(struct repository *repo,
 	free(old_gitdir);
 
 	repo_set_commondir(repo, o->commondir);
-	expand_base_dir(&repo->objectdir, o->object_dir,
+	expand_base_dir(&repo->objects->objectdir, o->object_dir,
 			repo->commondir, "objects");
-	free(repo->alternate_db);
-	repo->alternate_db = xstrdup_or_null(o->alternate_db);
+	free(repo->objects->alternate_db);
+	repo->objects->alternate_db = xstrdup_or_null(o->alternate_db);
 	expand_base_dir(&repo->graft_file, o->graft_file,
 			repo->commondir, "info/grafts");
 	expand_base_dir(&repo->index_file, o->index_file,
@@ -140,6 +142,8 @@ static int repo_init(struct repository *repo,
 	struct repository_format format;
 	memset(repo, 0, sizeof(*repo));
 
+	repo->objects = raw_object_store_new();
+
 	if (repo_init_gitdir(repo, gitdir))
 		goto error;
 
@@ -214,13 +218,14 @@ void repo_clear(struct repository *repo)
 {
 	FREE_AND_NULL(repo->gitdir);
 	FREE_AND_NULL(repo->commondir);
-	FREE_AND_NULL(repo->objectdir);
-	FREE_AND_NULL(repo->alternate_db);
 	FREE_AND_NULL(repo->graft_file);
 	FREE_AND_NULL(repo->index_file);
 	FREE_AND_NULL(repo->worktree);
 	FREE_AND_NULL(repo->submodule_prefix);
 
+	raw_object_store_clear(repo->objects);
+	FREE_AND_NULL(repo->objects);
+
 	if (repo->config) {
 		git_configset_clear(repo->config);
 		FREE_AND_NULL(repo->config);
repository.h +4 -7
index e7127baffb..abba2c09bc 100644
@@ -2,9 +2,10 @@
 #define REPOSITORY_H
 
 struct config_set;
+struct git_hash_algo;
 struct index_state;
+struct raw_object_store;
 struct submodule_cache;
-struct git_hash_algo;
 
 struct repository {
 	/* Environment */
@@ -21,13 +22,9 @@ struct repository {
 	char *commondir;
 
 	/*
-	 * Path to the repository's object store.
-	 * Cannot be NULL after initialization.
+	 * Holds any information related to accessing the raw object content.
 	 */
-	char *objectdir;
-
-	/* Path to extra alternate object database if not NULL */
-	char *alternate_db;
+	struct raw_object_store *objects;
 
 	/*
 	 * Path to the repository's graft file.
sha1_file.c +3 -1
index 4af422e3cf..af5aa83db5 100644
@@ -29,6 +29,7 @@
 #include "quote.h"
 #include "packfile.h"
 #include "fetch-object.h"
+#include "object-store.h"
 
 const unsigned char null_sha1[GIT_MAX_RAWSZ];
 const struct object_id null_oid;
@@ -671,7 +672,8 @@ void prepare_alt_odb(void)
 		return;
 
 	alt_odb_tail = &alt_odb_list;
-	link_alt_odb_entries(the_repository->alternate_db, PATH_SEP, NULL, 0);
+	link_alt_odb_entries(the_repository->objects->alternate_db,
+			     PATH_SEP, NULL, 0);
 
 	read_info_alternates(get_object_directory(), 0);
 }
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#85 lore raw

PATCH 2/27 object-store: migrate alternates struct and functions from cache.h

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Migrate the struct alternate_object_database and all its related functions to the object store as these functions are easier found in that header. The migration is just a verbatim copy, no need to include the object store header at any C file, because cache.h includes repository.h which in turn includes the object-store.h

---
 builtin/clone.c             |  1 +
 builtin/count-objects.c     |  1 +
 builtin/fsck.c              |  1 +
 builtin/submodule--helper.c |  1 +
 cache.h                     | 51 -------------------------------------
 object-store.h              | 51 +++++++++++++++++++++++++++++++++++++
 packfile.c                  |  1 +
 sha1_name.c                 |  1 +
 submodule.c                 |  1 +
 t/helper/test-ref-store.c   |  1 +
 tmp-objdir.c                |  1 +
 transport.c                 |  1 +
 12 files changed, 61 insertions(+), 51 deletions(-)
builtin/clone.c +1 -0
index 101c27a593..855947f1ab 100644
@@ -27,6 +27,7 @@
 #include "connected.h"
 #include "packfile.h"
 #include "list-objects-filter-options.h"
+#include "object-store.h"
 
 /*
  * Overall FIXMEs:
builtin/count-objects.c +1 -0
index 33343818c8..ced8958e43 100644
@@ -11,6 +11,7 @@
 #include "parse-options.h"
 #include "quote.h"
 #include "packfile.h"
+#include "object-store.h"
 
 static unsigned long garbage;
 static off_t size_garbage;
builtin/fsck.c +1 -0
index 7a8a679d4f..b0eba4c3c9 100644
@@ -16,6 +16,7 @@
 #include "streaming.h"
 #include "decorate.h"
 #include "packfile.h"
+#include "object-store.h"
 
 #define REACHABLE 0x0001
 #define SEEN      0x0002
builtin/submodule--helper.c +1 -0
index b1daca995f..6d8e002be7 100644
@@ -16,6 +16,7 @@
 #include "revision.h"
 #include "diffcore.h"
 #include "diff.h"
+#include "object-store.h"
 
 #define OPT_QUIET (1 << 0)
 #define OPT_CACHED (1 << 1)
cache.h +0 -51

57 lines not on this page · raw message

object-store.h +51 -0

57 lines not on this page · raw message

packfile.c +1 -0
index 7dbe8739d1..c6651682a7 100644
@@ -13,6 +13,7 @@
 #include "tag.h"
 #include "tree-walk.h"
 #include "tree.h"
+#include "object-store.h"
 
 char *odb_pack_name(struct strbuf *buf,
 		    const unsigned char *sha1,
sha1_name.c +1 -0
index 611c7d24dd..434025bf03 100644
@@ -10,6 +10,7 @@
 #include "dir.h"
 #include "sha1-array.h"
 #include "packfile.h"
+#include "object-store.h"
 
 static int get_oid_oneline(const char *, struct object_id *, struct commit_list *);
 
submodule.c +1 -0
index 47ddc9b273..b03e5f5045 100644
@@ -21,6 +21,7 @@
 #include "remote.h"
 #include "worktree.h"
 #include "parse-options.h"
+#include "object-store.h"
 
 static int config_update_recurse_submodules = RECURSE_SUBMODULES_OFF;
 static struct string_list changed_submodule_names = STRING_LIST_INIT_DUP;
t/helper/test-ref-store.c +1 -0
index 7120634b04..7314b5943e 100644
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "refs.h"
 #include "worktree.h"
+#include "object-store.h"
 
 static const char *notnull(const char *arg, const char *name)
 {
tmp-objdir.c +1 -0
index b2d9280f10..fea3f55545 100644
@@ -6,6 +6,7 @@
 #include "strbuf.h"
 #include "argv-array.h"
 #include "quote.h"
+#include "object-store.h"
 
 struct tmp_objdir {
 	struct strbuf path;
transport.c +1 -0
index 00d48b5b56..3afc632472 100644
@@ -18,6 +18,7 @@
 #include "sha1-array.h"
 #include "sigchain.h"
 #include "transport-internal.h"
+#include "object-store.h"
 
 static void set_upstreams(struct transport *transport, struct ref *refs,
 	int pretend)
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#86 lore raw

PATCH 3/27 object-store: move alt_odb_list and alt_odb_tail to object store

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

In a process with multiple repositories open, alternates should be associated to a single repository and not shared globally. Move alt_odb_list and alt_odb_tail into the_repository and adjust callers to reflect this.

Now that the alternative object data base is per repository, we're leaking its memory upon freeing a repository. The next patch plugs this hole.

No functional change intended.

---
 builtin/fsck.c |  4 ++++
 object-store.h |  7 +++++--
 packfile.c     |  3 ++-
 sha1_file.c    | 25 ++++++++++++-------------
 sha1_name.c    |  3 ++-
 5 files changed, 25 insertions(+), 17 deletions(-)
builtin/fsck.c +4 -0
index b0eba4c3c9..c736a10a11 100644
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "commit.h"
 #include "tree.h"
@@ -714,9 +715,12 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 		for_each_loose_object(mark_loose_for_connectivity, NULL, 0);
 		for_each_packed_object(mark_packed_for_connectivity, NULL, 0);
 	} else {
+		struct alternate_object_database *alt_odb_list;
+
 		fsck_object_dir(get_object_directory());
 
 		prepare_alt_odb();
+		alt_odb_list = the_repository->objects->alt_odb_list;
 		for (alt = alt_odb_list; alt; alt = alt->next)
 			fsck_object_dir(alt->path);
 
object-store.h +5 -2
index 5002e373cd..0b4db5867d 100644
@@ -1,7 +1,7 @@
 #ifndef OBJECT_STORE_H
 #define OBJECT_STORE_H
 
-extern struct alternate_object_database {
+struct alternate_object_database {
 	struct alternate_object_database *next;
 
 	/* see alt_scratch_buf() */
@@ -19,7 +19,7 @@ extern struct alternate_object_database {
 	struct oid_array loose_objects_cache;
 
 	char path[FLEX_ARRAY];
-} *alt_odb_list;
+};
 void prepare_alt_odb(void);
 char *compute_alternate_path(const char *path, struct strbuf *err);
 typedef int alt_odb_fn(struct alternate_object_database *, void *);
@@ -61,6 +61,9 @@ struct raw_object_store {
 
 	/* Path to extra alternate object database if not NULL */
 	char *alternate_db;
+
+	struct alternate_object_database *alt_odb_list;
+	struct alternate_object_database **alt_odb_tail;
 };
 
 struct raw_object_store *raw_object_store_new(void);
packfile.c +2 -1
index c6651682a7..fdbf240f81 100644
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "list.h"
 #include "pack.h"
+#include "repository.h"
 #include "dir.h"
 #include "mergesort.h"
 #include "packfile.h"
@@ -892,7 +893,7 @@ void prepare_packed_git(void)
 		return;
 	prepare_packed_git_one(get_object_directory(), 1);
 	prepare_alt_odb();
-	for (alt = alt_odb_list; alt; alt = alt->next)
+	for (alt = the_repository->objects->alt_odb_list; alt; alt = alt->next)
 		prepare_packed_git_one(alt->path, 0);
 	rearrange_packed_git();
 	prepare_packed_git_mru();
sha1_file.c +12 -13
index af5aa83db5..84b361c125 100644
@@ -22,6 +22,7 @@
 #include "pack-revindex.h"
 #include "sha1-lookup.h"
 #include "bulk-checkin.h"
+#include "repository.h"
 #include "streaming.h"
 #include "dir.h"
 #include "list.h"
@@ -343,9 +344,6 @@ static const char *alt_sha1_path(struct alternate_object_database *alt,
 	return buf->buf;
 }
 
-struct alternate_object_database *alt_odb_list;
-static struct alternate_object_database **alt_odb_tail;
-
 /*
  * Return non-zero iff the path is usable as an alternate object database.
  */
@@ -365,7 +363,7 @@ static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)
 	 * Prevent the common mistake of listing the same
 	 * thing twice, or object directory itself.
 	 */
-	for (alt = alt_odb_list; alt; alt = alt->next) {
+	for (alt = the_repository->objects->alt_odb_list; alt; alt = alt->next) {
 		if (!fspathcmp(path->buf, alt->path))
 			return 0;
 	}
@@ -425,8 +423,8 @@ static int link_alt_odb_entry(const char *entry, const char *relative_base,
 	ent = alloc_alt_odb(pathbuf.buf);
 
 	/* add the alternate entry */
-	*alt_odb_tail = ent;
-	alt_odb_tail = &(ent->next);
+	*the_repository->objects->alt_odb_tail = ent;
+	the_repository->objects->alt_odb_tail = &(ent->next);
 	ent->next = NULL;
 
 	/* recursively add alternates */
@@ -560,7 +558,7 @@ void add_to_alternates_file(const char *reference)
 		fprintf_or_die(out, "%s\n", reference);
 		if (commit_lock_file(&lock))
 			die_errno("unable to move new alternates file into place");
-		if (alt_odb_tail)
+		if (the_repository->objects->alt_odb_tail)
 			link_alt_odb_entries(reference, '\n', NULL, 0);
 	}
 	free(alts);
@@ -658,7 +656,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 	int r = 0;
 
 	prepare_alt_odb();
-	for (ent = alt_odb_list; ent; ent = ent->next) {
+	for (ent = the_repository->objects->alt_odb_list; ent; ent = ent->next) {
 		r = fn(ent, cb);
 		if (r)
 			break;
@@ -668,10 +666,11 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 
 void prepare_alt_odb(void)
 {
-	if (alt_odb_tail)
+	if (the_repository->objects->alt_odb_tail)
 		return;
 
-	alt_odb_tail = &alt_odb_list;
+	the_repository->objects->alt_odb_tail =
+			&the_repository->objects->alt_odb_list;
 	link_alt_odb_entries(the_repository->objects->alternate_db,
 			     PATH_SEP, NULL, 0);
 
@@ -716,7 +715,7 @@ static int check_and_freshen_nonlocal(const unsigned char *sha1, int freshen)
 {
 	struct alternate_object_database *alt;
 	prepare_alt_odb();
-	for (alt = alt_odb_list; alt; alt = alt->next) {
+	for (alt = the_repository->objects->alt_odb_list; alt; alt = alt->next) {
 		const char *path = alt_sha1_path(alt, sha1);
 		if (check_and_freshen_file(path, freshen))
 			return 1;
@@ -876,7 +875,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,
 
 	prepare_alt_odb();
 	errno = ENOENT;
-	for (alt = alt_odb_list; alt; alt = alt->next) {
+	for (alt = the_repository->objects->alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		if (!lstat(*path, st))
 			return 0;
@@ -906,7 +905,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)
 	most_interesting_errno = errno;
 
 	prepare_alt_odb();
-	for (alt = alt_odb_list; alt; alt = alt->next) {
+	for (alt = the_repository->objects->alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		fd = git_open(*path);
 		if (fd >= 0)
sha1_name.c +2 -1
index 434025bf03..a22a29cda0 100644
@@ -11,6 +11,7 @@
 #include "sha1-array.h"
 #include "packfile.h"
 #include "object-store.h"
+#include "repository.h"
 
 static int get_oid_oneline(const char *, struct object_id *, struct commit_list *);
 
@@ -105,7 +106,7 @@ static void find_short_object_filename(struct disambiguate_state *ds)
 		 */
 		fakeent = alloc_alt_odb(get_object_directory());
 	}
-	fakeent->next = alt_odb_list;
+	fakeent->next = the_repository->objects->alt_odb_list;
 
 	for (alt = fakeent; alt && !ds->ambiguous; alt = alt->next) {
 		int pos;
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#87 lore raw

PATCH 4/27 object-store: free alt_odb_list

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Free the memory and reset alt_odb_{list, tail} to NULL.

---
 object.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
object.c +22 -0
index 6ddd61242c..581347b535 100644
@@ -454,8 +454,30 @@ struct raw_object_store *raw_object_store_new(void)
 	memset(o, 0, sizeof(*o));
 	return o;
 }
+
+static void free_alt_odb(struct alternate_object_database *alt)
+{
+	strbuf_release(&alt->scratch);
+	oid_array_clear(&alt->loose_objects_cache);
+	free(alt);
+}
+
+static void free_alt_odbs(struct raw_object_store *o)
+{
+	while (o->alt_odb_list) {
+		struct alternate_object_database *next;
+
+		next = o->alt_odb_list->next;
+		free_alt_odb(o->alt_odb_list);
+		o->alt_odb_list = next;
+	}
+}
+
 void raw_object_store_clear(struct raw_object_store *o)
 {
 	FREE_AND_NULL(o->objectdir);
 	FREE_AND_NULL(o->alternate_db);
+
+	free_alt_odbs(o);
+	o->alt_odb_tail = NULL;
 }
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#88 lore raw

Re: [PATCH 04/27] object-store: free alt_odb_list

Quoted 6 lines · Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Free the memory and reset alt_odb_{list, tail} to NULL.

Thanks for fixing the memory leak. Looks good now.

Quoted 42 lines
---
 object.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
object.c +22 -0
index 6ddd61242c..581347b535 100644
@@ -454,8 +454,30 @@ struct raw_object_store *raw_object_store_new(void)
 	memset(o, 0, sizeof(*o));
 	return o;
 }
+
+static void free_alt_odb(struct alternate_object_database *alt)
+{
+	strbuf_release(&alt->scratch);
+	oid_array_clear(&alt->loose_objects_cache);
+	free(alt);
+}
+
+static void free_alt_odbs(struct raw_object_store *o)
+{
+	while (o->alt_odb_list) {
+		struct alternate_object_database *next;
+
+		next = o->alt_odb_list->next;
+		free_alt_odb(o->alt_odb_list);
+		o->alt_odb_list = next;
+	}
+}
+
 void raw_object_store_clear(struct raw_object_store *o)
 {
 	FREE_AND_NULL(o->objectdir);
 	FREE_AND_NULL(o->alternate_db);
+
+	free_alt_odbs(o);
+	o->alt_odb_tail = NULL;
 }
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
Signature · 1 line
Brandon Williams
#89 lore raw

PATCH 5/27 object-store: move packed_git and packed_git_mru to object store

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

In a process with multiple repositories open, packfile accessors should be associated to a single repository and not shared globally. Move packed_git and packed_git_mru into the_repository and adjust callers to reflect this.

[nd: while at there, wrap access to these two fields in get_packed_git() and get_packed_git_mru(). This allows us to lazily initialize these fields without caller doing that explicitly]

---
 builtin/count-objects.c  |  5 ++--
 builtin/fsck.c           |  6 +++--
 builtin/gc.c             |  4 +++-
 builtin/index-pack.c     |  1 +
 builtin/pack-objects.c   | 21 ++++++++++-------
 builtin/pack-redundant.c |  6 +++--
 cache.h                  | 29 -----------------------
 fast-import.c            |  8 +++++--
 http-backend.c           |  6 +++--
 http-push.c              |  1 +
 http-walker.c            |  1 +
 http.c                   |  1 +
 object-store.h           | 34 +++++++++++++++++++++++++++
 object.c                 |  7 ++++++
 pack-bitmap.c            |  4 +++-
 pack-check.c             |  1 +
 pack-revindex.c          |  1 +
 packfile.c               | 51 ++++++++++++++++++++++++----------------
 packfile.h               |  3 +++
 reachable.c              |  1 +
 server-info.c            |  6 +++--
 sha1_name.c              |  5 ++--
 22 files changed, 128 insertions(+), 74 deletions(-)
builtin/count-objects.c +3 -2
index ced8958e43..b28ff00be2 100644
@@ -7,6 +7,7 @@
 #include "cache.h"
 #include "config.h"
 #include "dir.h"
+#include "repository.h"
 #include "builtin.h"
 #include "parse-options.h"
 #include "quote.h"
@@ -121,9 +122,9 @@ int cmd_count_objects(int argc, const char **argv, const char *prefix)
 		struct strbuf loose_buf = STRBUF_INIT;
 		struct strbuf pack_buf = STRBUF_INIT;
 		struct strbuf garbage_buf = STRBUF_INIT;
-		if (!packed_git)
+		if (!get_packed_git(the_repository))
 			prepare_packed_git();
-		for (p = packed_git; p; p = p->next) {
+		for (p = get_packed_git(the_repository); p; p = p->next) {
 			if (!p->pack_local)
 				continue;
 			if (open_pack_index(p))
builtin/fsck.c +4 -2
index c736a10a11..7707407275 100644
@@ -732,7 +732,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 			prepare_packed_git();
 
 			if (show_progress) {
-				for (p = packed_git; p; p = p->next) {
+				for (p = get_packed_git(the_repository); p;
+				     p = p->next) {
 					if (open_pack_index(p))
 						continue;
 					total += p->num_objects;
@@ -740,7 +741,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 
 				progress = start_progress(_("Checking objects"), total);
 			}
-			for (p = packed_git; p; p = p->next) {
+			for (p = get_packed_git(the_repository); p;
+			     p = p->next) {
 				/* verify gives error messages itself */
 				if (verify_pack(p, fsck_obj_buffer,
 						progress, count))
builtin/gc.c +3 -1
index 77fa720bd0..b00238cd5d 100644
@@ -11,6 +11,7 @@
  */
 
 #include "builtin.h"
+#include "repository.h"
 #include "config.h"
 #include "tempfile.h"
 #include "lockfile.h"
@@ -20,6 +21,7 @@
 #include "argv-array.h"
 #include "commit.h"
 #include "packfile.h"
+#include "object-store.h"
 
 #define FAILED_RUN "failed to run %s"
 
@@ -173,7 +175,7 @@ static int too_many_packs(void)
 		return 0;
 
 	prepare_packed_git();
-	for (cnt = 0, p = packed_git; p; p = p->next) {
+	for (cnt = 0, p = get_packed_git(the_repository); p; p = p->next) {
 		if (!p->pack_local)
 			continue;
 		if (p->pack_keep)
builtin/index-pack.c +1 -0
index 5ebd370c56..1d6bc87b76 100644
@@ -13,6 +13,7 @@
 #include "streaming.h"
 #include "thread-utils.h"
 #include "packfile.h"
+#include "object-store.h"
 
 static const char index_pack_usage[] =
 "git index-pack [-v] [-o <index-file>] [--keep | --keep=<msg>] [--verify] [--strict] (<pack-file> | --stdin [--fix-thin] [<pack-file>])";
builtin/pack-objects.c +12 -9
index 83dcbc9773..223f2d9fc0 100644
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "attr.h"
 #include "object.h"
@@ -28,6 +29,7 @@
 #include "argv-array.h"
 #include "list.h"
 #include "packfile.h"
+#include "object-store.h"
 
 static const char *pack_usage[] = {
 	N_("git pack-objects --stdout [<options>...] [< <ref-list> | < <object-list>]"),
@@ -1025,8 +1027,7 @@ static int want_object_in_pack(const struct object_id *oid,
 		if (want != -1)
 			return want;
 	}
-
-	list_for_each(pos, &packed_git_mru) {
+	list_for_each(pos, get_packed_git_mru(the_repository)) {
 		struct packed_git *p = list_entry(pos, struct packed_git, mru);
 		off_t offset;
 
@@ -1044,7 +1045,8 @@ static int want_object_in_pack(const struct object_id *oid,
 			}
 			want = want_found_object(exclude, p);
 			if (!exclude && want > 0)
-				list_move(&p->mru, &packed_git_mru);
+				list_move(&p->mru,
+					  get_packed_git_mru(the_repository));
 			if (want != -1)
 				return want;
 		}
@@ -2673,7 +2675,7 @@ static void add_objects_in_unpacked_packs(struct rev_info *revs)
 
 	memset(&in_pack, 0, sizeof(in_pack));
 
-	for (p = packed_git; p; p = p->next) {
+	for (p = get_packed_git(the_repository); p; p = p->next) {
 		struct object_id oid;
 		struct object *o;
 
@@ -2736,7 +2738,8 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 	static struct packed_git *last_found = (void *)1;
 	struct packed_git *p;
 
-	p = (last_found != (void *)1) ? last_found : packed_git;
+	p = (last_found != (void *)1) ? last_found :
+					get_packed_git(the_repository);
 
 	while (p) {
 		if ((!p->pack_local || p->pack_keep) &&
@@ -2745,7 +2748,7 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 			return 1;
 		}
 		if (p == last_found)
-			p = packed_git;
+			p = get_packed_git(the_repository);
 		else
 			p = p->next;
 		if (p == last_found)
@@ -2781,7 +2784,7 @@ static void loosen_unused_packed_objects(struct rev_info *revs)
 	uint32_t i;
 	struct object_id oid;
 
-	for (p = packed_git; p; p = p->next) {
+	for (p = get_packed_git(the_repository); p; p = p->next) {
 		if (!p->pack_local || p->pack_keep)
 			continue;
 
@@ -3152,7 +3155,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 	prepare_packed_git();
 	if (ignore_packed_keep) {
 		struct packed_git *p;
-		for (p = packed_git; p; p = p->next)
+		for (p = get_packed_git(the_repository); p; p = p->next)
 			if (p->pack_local && p->pack_keep)
 				break;
 		if (!p) /* no keep-able packs found */
@@ -3165,7 +3168,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 		 * also covers non-local objects
 		 */
 		struct packed_git *p;
-		for (p = packed_git; p; p = p->next) {
+		for (p = get_packed_git(the_repository); p; p = p->next) {
 			if (!p->pack_local) {
 				have_non_local_packs = 1;
 				break;
builtin/pack-redundant.c +4 -2
index aaa8136322..b5b007e706 100644
@@ -7,7 +7,9 @@
 */
 
 #include "builtin.h"
+#include "repository.h"
 #include "packfile.h"
+#include "object-store.h"
 
 #define BLKSIZE 512
 
@@ -571,7 +573,7 @@ static struct pack_list * add_pack(struct packed_git *p)
 
 static struct pack_list * add_pack_file(const char *filename)
 {
-	struct packed_git *p = packed_git;
+	struct packed_git *p = get_packed_git(the_repository);
 
 	if (strlen(filename) < 40)
 		die("Bad pack filename: %s", filename);
@@ -586,7 +588,7 @@ static struct pack_list * add_pack_file(const char *filename)
 
 static void load_all(void)
 {
-	struct packed_git *p = packed_git;
+	struct packed_git *p = get_packed_git(the_repository);
 
 	while (p) {
 		add_pack(p);
cache.h +0 -29
index 41530d5d16..d3429a0d48 100644
@@ -1585,35 +1585,6 @@ struct pack_window {
 	unsigned int inuse_cnt;
 };
 
-extern struct packed_git {
-	struct packed_git *next;
-	struct list_head mru;
-	struct pack_window *windows;
-	off_t pack_size;
-	const void *index_data;
-	size_t index_size;
-	uint32_t num_objects;
-	uint32_t num_bad_objects;
-	unsigned char *bad_object_sha1;
-	int index_version;
-	time_t mtime;
-	int pack_fd;
-	unsigned pack_local:1,
-		 pack_keep:1,
-		 freshened:1,
-		 do_not_close:1,
-		 pack_promisor:1;
-	unsigned char sha1[20];
-	struct revindex_entry *revindex;
-	/* something like ".git/objects/pack/xxxxx.pack" */
-	char pack_name[FLEX_ARRAY]; /* more */
-} *packed_git;
-
-/*
- * A most-recently-used ordered version of the packed_git list.
- */
-extern struct list_head packed_git_mru;
-
 struct pack_entry {
 	off_t offset;
 	unsigned char sha1[20];
fast-import.c +6 -2
index b70ac025e0..b3492fce5c 100644
@@ -154,6 +154,7 @@ Format of STDIN stream:
 
 #include "builtin.h"
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "lockfile.h"
 #include "object.h"
@@ -168,6 +169,7 @@ Format of STDIN stream:
 #include "dir.h"
 #include "run-command.h"
 #include "packfile.h"
+#include "object-store.h"
 
 #define PACK_ID_BITS 16
 #define MAX_PACK_ID ((1<<PACK_ID_BITS)-1)
@@ -1110,7 +1112,8 @@ static int store_object(
 	if (e->idx.offset) {
 		duplicate_count_by_type[type]++;
 		return 1;
-	} else if (find_sha1_pack(oid.hash, packed_git)) {
+	} else if (find_sha1_pack(oid.hash,
+				  get_packed_git(the_repository))) {
 		e->type = type;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
@@ -1305,7 +1308,8 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
 		duplicate_count_by_type[OBJ_BLOB]++;
 		truncate_pack(&checkpoint);
 
-	} else if (find_sha1_pack(oid.hash, packed_git)) {
+	} else if (find_sha1_pack(oid.hash,
+				  get_packed_git(the_repository))) {
 		e->type = OBJ_BLOB;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
http-backend.c +4 -2
index f3dc218b2a..64dde78c1b 100644
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "config.h"
+#include "repository.h"
 #include "refs.h"
 #include "pkt-line.h"
 #include "object.h"
@@ -10,6 +11,7 @@
 #include "url.h"
 #include "argv-array.h"
 #include "packfile.h"
+#include "object-store.h"
 
 static const char content_type[] = "Content-Type";
 static const char content_length[] = "Content-Length";
@@ -518,13 +520,13 @@ static void get_info_packs(struct strbuf *hdr, char *arg)
 
 	select_getanyfile(hdr);
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = get_packed_git(the_repository); p; p = p->next) {
 		if (p->pack_local)
 			cnt++;
 	}
 
 	strbuf_grow(&buf, cnt * 53 + 2);
-	for (p = packed_git; p; p = p->next) {
+	for (p = get_packed_git(the_repository); p; p = p->next) {
 		if (p->pack_local)
 			strbuf_addf(&buf, "P %s\n", p->pack_name + objdirlen + 6);
 	}
http-push.c +1 -0
index 0913f8ab86..97fe22a705 100644
@@ -12,6 +12,7 @@
 #include "sigchain.h"
 #include "argv-array.h"
 #include "packfile.h"
+#include "object-store.h"
 
 #ifdef EXPAT_NEEDS_XMLPARSE_H
 #include <xmlparse.h>
http-walker.c +1 -0
index 07c2b1af82..d850408d18 100644
@@ -5,6 +5,7 @@
 #include "list.h"
 #include "transport.h"
 #include "packfile.h"
+#include "object-store.h"
 
 struct alt_base {
 	char *base;
http.c +1 -0
index 31755023a4..efa977112a 100644
@@ -14,6 +14,7 @@
 #include "packfile.h"
 #include "protocol.h"
 #include "string-list.h"
+#include "object-store.h"
 
 static struct trace_key trace_curl = TRACE_KEY_INIT(CURL);
 static int trace_curl_data = 1;
object-store.h +34 -0
index 0b4db5867d..c687ab7587 100644
@@ -52,6 +52,30 @@ void add_to_alternates_memory(const char *dir);
  */
 struct strbuf *alt_scratch_buf(struct alternate_object_database *alt);
 
+struct packed_git {
+	struct packed_git *next;
+	struct list_head mru;
+	struct pack_window *windows;
+	off_t pack_size;
+	const void *index_data;
+	size_t index_size;
+	uint32_t num_objects;
+	uint32_t num_bad_objects;
+	unsigned char *bad_object_sha1;
+	int index_version;
+	time_t mtime;
+	int pack_fd;
+	unsigned pack_local:1,
+		 pack_keep:1,
+		 freshened:1,
+		 do_not_close:1,
+		 pack_promisor:1;
+	unsigned char sha1[20];
+	struct revindex_entry *revindex;
+	/* something like ".git/objects/pack/xxxxx.pack" */
+	char pack_name[FLEX_ARRAY]; /* more */
+};
+
 struct raw_object_store {
 	/*
 	 * Path to the repository's object store.
@@ -64,6 +88,16 @@ struct raw_object_store {
 
 	struct alternate_object_database *alt_odb_list;
 	struct alternate_object_database **alt_odb_tail;
+
+	/*
+	 * private data
+	 *
+	 * should only be accessed directly by packfile.c
+	 */
+
+	struct packed_git *packed_git;
+	/* A most-recently-used ordered version of the packed_git list. */
+	struct list_head packed_git_mru;
 };
 
 struct raw_object_store *raw_object_store_new(void);
object.c +7 -0
index 581347b535..04631ee841 100644
@@ -452,6 +452,7 @@ struct raw_object_store *raw_object_store_new(void)
 	struct raw_object_store *o = xmalloc(sizeof(*o));
 
 	memset(o, 0, sizeof(*o));
+	INIT_LIST_HEAD(&o->packed_git_mru);
 	return o;
 }
 
@@ -480,4 +481,10 @@ void raw_object_store_clear(struct raw_object_store *o)
 
 	free_alt_odbs(o);
 	o->alt_odb_tail = NULL;
+
+	INIT_LIST_HEAD(&o->packed_git_mru);
+	/*
+	 * TODO: call close_all_packs once migrated to
+	 * take an object store argument
+	 */
 }
pack-bitmap.c +3 -1
index 9270983e5f..22cd425788 100644
@@ -10,6 +10,8 @@
 #include "pack-revindex.h"
 #include "pack-objects.h"
 #include "packfile.h"
+#include "repository.h"
+#include "object-store.h"
 
 /*
  * An entry on the bitmap index, representing the bitmap for a given
@@ -335,7 +337,7 @@ static int open_pack_bitmap(void)
 	assert(!bitmap_git.map && !bitmap_git.loaded);
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = get_packed_git(the_repository); p; p = p->next) {
 		if (open_pack_bitmap_1(p) == 0)
 			ret = 0;
 	}
pack-check.c +1 -0
index 073c1fbd46..2378f25999 100644
@@ -3,6 +3,7 @@
 #include "pack-revindex.h"
 #include "progress.h"
 #include "packfile.h"
+#include "object-store.h"
 
 struct idx_entry {
 	off_t                offset;
pack-revindex.c +1 -0
index ff5f62c033..bb521cf7fb 100644
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "pack-revindex.h"
+#include "object-store.h"
 
 /*
  * Pack index for existing packs give us easy access to the offsets into
packfile.c +31 -20

140 lines not on this page · raw message

packfile.h +3 -0
index a7fca598d6..76496226bb 100644
@@ -38,6 +38,9 @@ extern void prepare_packed_git(void);
 extern void reprepare_packed_git(void);
 extern void install_packed_git(struct packed_git *pack);
 
+struct packed_git *get_packed_git(struct repository *r);
+struct list_head *get_packed_git_mru(struct repository *r);
+
 /*
  * Give a rough count of objects in the repository. This sacrifices accuracy
  * for speed.
reachable.c +1 -0
index 88d7d679da..25cfd99d1c 100644
@@ -11,6 +11,7 @@
 #include "list-objects.h"
 #include "packfile.h"
 #include "worktree.h"
+#include "object-store.h"
 
 struct connectivity_progress {
 	struct progress *progress;
server-info.c +4 -2
index 26a6c20b7d..75a8b65e47 100644
@@ -1,9 +1,11 @@
 #include "cache.h"
+#include "repository.h"
 #include "refs.h"
 #include "object.h"
 #include "commit.h"
 #include "tag.h"
 #include "packfile.h"
+#include "object-store.h"
 
 /*
  * Create the file "path" by writing to a temporary file and renaming
@@ -200,7 +202,7 @@ static void init_pack_info(const char *infofile, int force)
 	objdirlen = strlen(objdir);
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = get_packed_git(the_repository); p; p = p->next) {
 		/* we ignore things on alternate path since they are
 		 * not available to the pullers in general.
 		 */
@@ -210,7 +212,7 @@ static void init_pack_info(const char *infofile, int force)
 	}
 	num_pack = i;
 	info = xcalloc(num_pack, sizeof(struct pack_info *));
-	for (i = 0, p = packed_git; p; p = p->next) {
+	for (i = 0, p = get_packed_git(the_repository); p; p = p->next) {
 		if (!p->pack_local)
 			continue;
 		info[i] = xcalloc(1, sizeof(struct pack_info));
sha1_name.c +3 -2
index a22a29cda0..ffff7d8710 100644
@@ -197,7 +197,8 @@ static void find_short_packed_object(struct disambiguate_state *ds)
 	struct packed_git *p;
 
 	prepare_packed_git();
-	for (p = packed_git; p && !ds->ambiguous; p = p->next)
+	for (p = get_packed_git(the_repository); p && !ds->ambiguous;
+	     p = p->next)
 		unique_in_pack(p, ds);
 }
 
@@ -567,7 +568,7 @@ static void find_abbrev_len_packed(struct min_abbrev_data *mad)
 	struct packed_git *p;
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next)
+	for (p = get_packed_git(the_repository); p; p = p->next)
 		find_abbrev_len_for_pack(p, mad);
 }
 
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#90 lore raw

PATCH 6/27 object-store: close all packs upon clearing the object store

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

---
 builtin/am.c           | 2 +-
 builtin/clone.c        | 2 +-
 builtin/fetch.c        | 2 +-
 builtin/merge.c        | 2 +-
 builtin/receive-pack.c | 2 +-
 object.c               | 7 +++----
 packfile.c             | 4 ++--
 packfile.h             | 2 +-
 8 files changed, 11 insertions(+), 12 deletions(-)
builtin/am.c +1 -1
index 5bdd2d7578..47beddbe24 100644
@@ -1859,7 +1859,7 @@ static void am_run(struct am_state *state, int resume)
 	 */
 	if (!state->rebasing) {
 		am_destroy(state);
-		close_all_packs();
+		close_all_packs(the_repository->objects);
 		run_command_v_opt(argv_gc_auto, RUN_GIT_CMD);
 	}
 }
builtin/clone.c +1 -1
index 855947f1ab..7df5932b85 100644
@@ -1218,7 +1218,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 	transport_disconnect(transport);
 
 	if (option_dissociate) {
-		close_all_packs();
+		close_all_packs(the_repository->objects);
 		dissociate_from_references();
 	}
 
builtin/fetch.c +1 -1
index 8ee998ea2e..a39e9d7b15 100644
@@ -1478,7 +1478,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
 
 	string_list_clear(&list, 0);
 
-	close_all_packs();
+	close_all_packs(the_repository->objects);
 
 	argv_array_pushl(&argv_gc_auto, "gc", "--auto", NULL);
 	if (verbosity < 0)
builtin/merge.c +1 -1
index 30264cfd7c..96d56cbdd2 100644
@@ -411,7 +411,7 @@ static void finish(struct commit *head_commit,
 			 * We ignore errors in 'gc --auto', since the
 			 * user should see them.
 			 */
-			close_all_packs();
+			close_all_packs(the_repository->objects);
 			run_command_v_opt(argv_gc_auto, RUN_GIT_CMD);
 		}
 	}
builtin/receive-pack.c +1 -1
index b7ce7c7f52..1a298a6711 100644
@@ -2026,7 +2026,7 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)
 			proc.git_cmd = 1;
 			proc.argv = argv_gc_auto;
 
-			close_all_packs();
+			close_all_packs(the_repository->objects);
 			if (!start_command(&proc)) {
 				if (use_sideband)
 					copy_to_sideband(proc.err, -1, NULL);
object.c +3 -4
index 04631ee841..4c2cf7ff5d 100644
@@ -5,6 +5,7 @@
 #include "commit.h"
 #include "tag.h"
 #include "object-store.h"
+#include "packfile.h"
 
 static struct object **obj_hash;
 static int nr_objs, obj_hash_size;
@@ -483,8 +484,6 @@ void raw_object_store_clear(struct raw_object_store *o)
 	o->alt_odb_tail = NULL;
 
 	INIT_LIST_HEAD(&o->packed_git_mru);
-	/*
-	 * TODO: call close_all_packs once migrated to
-	 * take an object store argument
-	 */
+	close_all_packs(o);
+	o->packed_git = NULL;
 }
packfile.c +2 -2
index 39f4a85200..f2dc084745 100644
@@ -311,11 +311,11 @@ static void close_pack(struct packed_git *p)
 	close_pack_index(p);
 }
 
-void close_all_packs(void)
+void close_all_packs(struct raw_object_store *o)
 {
 	struct packed_git *p;
 
-	for (p = the_repository->objects->packed_git; p; p = p->next)
+	for (p = o->packed_git; p; p = p->next)
 		if (p->do_not_close)
 			die("BUG: want to close pack marked 'do-not-close'");
 		else
packfile.h +1 -1
index 76496226bb..5b1ce00f84 100644
@@ -66,7 +66,7 @@ extern void close_pack_index(struct packed_git *);
 
 extern unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t, unsigned long *);
 extern void close_pack_windows(struct packed_git *);
-extern void close_all_packs(void);
+extern void close_all_packs(struct raw_object_store *o);
 extern void unuse_pack(struct pack_window **);
 extern void clear_delta_base_cache(void);
 extern struct packed_git *add_packed_git(const char *path, size_t path_len, int local);
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#91 lore raw

PATCH 7/27 pack: move prepare_packed_git_run_once to object store

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Each repository's object store can be initialized independently, so they must not share a run_once variable.

---
 object-store.h | 6 ++++++
 packfile.c     | 7 +++----
 2 files changed, 9 insertions(+), 4 deletions(-)
object-store.h +6 -0
index c687ab7587..6a07a14d63 100644
@@ -98,6 +98,12 @@ struct raw_object_store {
 	struct packed_git *packed_git;
 	/* A most-recently-used ordered version of the packed_git list. */
 	struct list_head packed_git_mru;
+
+	/*
+	 * Whether packed_git has already been populated with this repository's
+	 * packs.
+	 */
+	unsigned packed_git_initialized : 1;
 };
 
 struct raw_object_store *raw_object_store_new(void);
packfile.c +3 -4
index f2dc084745..2a053711cf 100644
@@ -884,12 +884,11 @@ static void prepare_packed_git_mru(void)
 		list_add_tail(&p->mru, &the_repository->objects->packed_git_mru);
 }
 
-static int prepare_packed_git_run_once = 0;
 void prepare_packed_git(void)
 {
 	struct alternate_object_database *alt;
 
-	if (prepare_packed_git_run_once)
+	if (the_repository->objects->packed_git_initialized)
 		return;
 	prepare_packed_git_one(get_object_directory(), 1);
 	prepare_alt_odb();
@@ -897,13 +896,13 @@ void prepare_packed_git(void)
 		prepare_packed_git_one(alt->path, 0);
 	rearrange_packed_git();
 	prepare_packed_git_mru();
-	prepare_packed_git_run_once = 1;
+	the_repository->objects->packed_git_initialized = 1;
 }
 
 void reprepare_packed_git(void)
 {
 	approximate_object_count_valid = 0;
-	prepare_packed_git_run_once = 0;
+	the_repository->objects->packed_git_initialized = 0;
 	prepare_packed_git();
 }
 
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#92 lore raw

PATCH 8/27 pack: move approximate object count to object store

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

The approximate_object_count() function maintains a rough count of objects in a repository to estimate how long object name abbreviates should be. Object names are scoped to a repository and the appropriate length may differ by repository, so the object count should not be global.

---
 object-store.h |  8 ++++++++
 packfile.c     | 11 +++++------
 2 files changed, 13 insertions(+), 6 deletions(-)
object-store.h +8 -0
index 6a07a14d63..b53e125902 100644
@@ -99,6 +99,14 @@ struct raw_object_store {
 	/* A most-recently-used ordered version of the packed_git list. */
 	struct list_head packed_git_mru;
 
+	/*
+	 * A fast, rough count of the number of objects in the repository.
+	 * These two fields are not meant for direct access. Use
+	 * approximate_object_count() instead.
+	 */
+	unsigned long approximate_object_count;
+	unsigned approximate_object_count_valid : 1;
+
 	/*
 	 * Whether packed_git has already been populated with this repository's
 	 * packs.
packfile.c +5 -6
index 2a053711cf..b0b24ea9b8 100644
@@ -803,8 +803,6 @@ static void prepare_packed_git_one(char *objdir, int local)
 	strbuf_release(&path);
 }
 
-static int approximate_object_count_valid;
-
 /*
  * Give a fast, rough count of the number of objects in the repository. This
  * ignores loose objects completely. If you have a lot of them, then either
@@ -814,8 +812,8 @@ static int approximate_object_count_valid;
  */
 unsigned long approximate_object_count(void)
 {
-	static unsigned long count;
-	if (!approximate_object_count_valid) {
+	if (!the_repository->objects->approximate_object_count_valid) {
+		unsigned long count;
 		struct packed_git *p;
 
 		prepare_packed_git();
@@ -825,8 +823,9 @@ unsigned long approximate_object_count(void)
 				continue;
 			count += p->num_objects;
 		}
+		the_repository->objects->approximate_object_count = count;
 	}
-	return count;
+	return the_repository->objects->approximate_object_count;
 }
 
 static void *get_next_packed_git(const void *p)
@@ -901,7 +900,7 @@ void prepare_packed_git(void)
 
 void reprepare_packed_git(void)
 {
-	approximate_object_count_valid = 0;
+	the_repository->objects->approximate_object_count_valid = 0;
 	the_repository->objects->packed_git_initialized = 0;
 	prepare_packed_git();
 }
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#93 lore raw

PATCH 9/27 sha1_file: add raw_object_store argument to alt_odb_usable

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Add a raw_object_store to alt_odb_usable to be more specific about which repository to act on. The choice of the repository is delegated to its only caller link_alt_odb_entry.

---
 sha1_file.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
sha1_file.c +5 -3
index 84b361c125..097c372d03 100644
@@ -347,7 +347,9 @@ static const char *alt_sha1_path(struct alternate_object_database *alt,
 /*
  * Return non-zero iff the path is usable as an alternate object database.
  */
-static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)
+static int alt_odb_usable(struct raw_object_store *o,
+			  struct strbuf *path,
+			  const char *normalized_objdir)
 {
 	struct alternate_object_database *alt;
 
@@ -363,7 +365,7 @@ static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)
 	 * Prevent the common mistake of listing the same
 	 * thing twice, or object directory itself.
 	 */
-	for (alt = the_repository->objects->alt_odb_list; alt; alt = alt->next) {
+	for (alt = o->alt_odb_list; alt; alt = alt->next) {
 		if (!fspathcmp(path->buf, alt->path))
 			return 0;
 	}
@@ -415,7 +417,7 @@ static int link_alt_odb_entry(const char *entry, const char *relative_base,
 	while (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')
 		strbuf_setlen(&pathbuf, pathbuf.len - 1);
 
-	if (!alt_odb_usable(&pathbuf, normalized_objdir)) {
+	if (!alt_odb_usable(the_repository->objects, &pathbuf, normalized_objdir)) {
 		strbuf_release(&pathbuf);
 		return -1;
 	}
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#94 lore raw

PATCH 10/27 sha1_file: add repository argument to link_alt_odb_entry

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Add a repository argument to allow the link_alt_odb_entry caller to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

Since the implementation does not yet work with other repositories, use a wrapper macro to enforce that the caller passes in the_repository as the first argument. It would be more appealing to use BUILD_ASSERT_OR_ZERO to enforce this, but that doesn't work because it requires a compile-time constant and common compilers like gcc 4.8.4 do not consider "r == the_repository" a compile-time constant.

This and the following three patches add repository arguments to link_alt_odb_entry, read_info_alternates, link_alt_odb_entries and prepare_alt_odb. Three out of the four functions are found in a recursive call chain, calling each other, and one of them accesses the repositories objectdir (which was migrated; it was an obvious choice) and ignore_env (which we need to keep in the repository struct for clarify); hence we will pass through the repository unlike just the object store object + the ignore_env flag.

---
 sha1_file.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
sha1_file.c +5 -3
index 097c372d03..7c0ace646a 100644
@@ -391,8 +391,9 @@ static int alt_odb_usable(struct raw_object_store *o,
  * terminating NUL.
  */
 static void read_info_alternates(const char * relative_base, int depth);
-static int link_alt_odb_entry(const char *entry, const char *relative_base,
-	int depth, const char *normalized_objdir)
+#define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)
+static int link_alt_odb_entry_the_repository(const char *entry,
+	const char *relative_base, int depth, const char *normalized_objdir)
 {
 	struct alternate_object_database *ent;
 	struct strbuf pathbuf = STRBUF_INIT;
@@ -489,7 +490,8 @@ static void link_alt_odb_entries(const char *alt, int sep,
 		alt = parse_alt_odb_entry(alt, sep, &entry);
 		if (!entry.len)
 			continue;
-		link_alt_odb_entry(entry.buf, relative_base, depth, objdirbuf.buf);
+		link_alt_odb_entry(the_repository, entry.buf,
+				   relative_base, depth, objdirbuf.buf);
 	}
 	strbuf_release(&entry);
 	strbuf_release(&objdirbuf);
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#95 lore raw

PATCH 11/27 sha1_file: add repository argument to read_info_alternates

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

See previous patch for explanation.

---
 sha1_file.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
sha1_file.c +7 -4
index 7c0ace646a..81ad2a84f2 100644
@@ -390,7 +390,9 @@ static int alt_odb_usable(struct raw_object_store *o,
  * SHA1, an extra slash for the first level indirection, and the
  * terminating NUL.
  */
-static void read_info_alternates(const char * relative_base, int depth);
+#define read_info_alternates(r, rb, d) read_info_alternates_##r(rb, d)
+static void read_info_alternates_the_repository(const char *relative_base,
+						int depth);
 #define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)
 static int link_alt_odb_entry_the_repository(const char *entry,
 	const char *relative_base, int depth, const char *normalized_objdir)
@@ -431,7 +433,7 @@ static int link_alt_odb_entry_the_repository(const char *entry,
 	ent->next = NULL;
 
 	/* recursively add alternates */
-	read_info_alternates(pathbuf.buf, depth + 1);
+	read_info_alternates(the_repository, pathbuf.buf, depth + 1);
 
 	strbuf_release(&pathbuf);
 	return 0;
@@ -497,7 +499,8 @@ static void link_alt_odb_entries(const char *alt, int sep,
 	strbuf_release(&objdirbuf);
 }
 
-static void read_info_alternates(const char * relative_base, int depth)
+static void read_info_alternates_the_repository(const char *relative_base,
+						int depth)
 {
 	char *path;
 	struct strbuf buf = STRBUF_INIT;
@@ -678,7 +681,7 @@ void prepare_alt_odb(void)
 	link_alt_odb_entries(the_repository->objects->alternate_db,
 			     PATH_SEP, NULL, 0);
 
-	read_info_alternates(get_object_directory(), 0);
+	read_info_alternates(the_repository, get_object_directory(), 0);
 }
 
 /* Returns 1 if we have successfully freshened the file, 0 otherwise. */
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#96 lore raw

PATCH 12/27 sha1_file: add repository argument to link_alt_odb_entries

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

See previous patch for explanation.

---
 sha1_file.c | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)
sha1_file.c +13 -6
index 81ad2a84f2..ba4fc9103b 100644
@@ -468,8 +468,12 @@ static const char *parse_alt_odb_entry(const char *string,
 	return end;
 }
 
-static void link_alt_odb_entries(const char *alt, int sep,
-				 const char *relative_base, int depth)
+#define link_alt_odb_entries(r, a, s, rb, d) \
+	link_alt_odb_entries_##r(a, s, rb, d)
+static void link_alt_odb_entries_the_repository(const char *alt,
+						int sep,
+						const char *relative_base,
+						int depth)
 {
 	struct strbuf objdirbuf = STRBUF_INIT;
 	struct strbuf entry = STRBUF_INIT;
@@ -512,7 +516,7 @@ static void read_info_alternates_the_repository(const char *relative_base,
 		return;
 	}
 
-	link_alt_odb_entries(buf.buf, '\n', relative_base, depth);
+	link_alt_odb_entries(the_repository, buf.buf, '\n', relative_base, depth);
 	strbuf_release(&buf);
 	free(path);
 }
@@ -566,7 +570,8 @@ void add_to_alternates_file(const char *reference)
 		if (commit_lock_file(&lock))
 			die_errno("unable to move new alternates file into place");
 		if (the_repository->objects->alt_odb_tail)
-			link_alt_odb_entries(reference, '\n', NULL, 0);
+			link_alt_odb_entries(the_repository, reference,
+					     '\n', NULL, 0);
 	}
 	free(alts);
 }
@@ -579,7 +584,8 @@ void add_to_alternates_memory(const char *reference)
 	 */
 	prepare_alt_odb();
 
-	link_alt_odb_entries(reference, '\n', NULL, 0);
+	link_alt_odb_entries(the_repository, reference,
+			     '\n', NULL, 0);
 }
 
 /*
@@ -678,7 +684,8 @@ void prepare_alt_odb(void)
 
 	the_repository->objects->alt_odb_tail =
 			&the_repository->objects->alt_odb_list;
-	link_alt_odb_entries(the_repository->objects->alternate_db,
+	link_alt_odb_entries(the_repository,
+			     the_repository->objects->alternate_db,
 			     PATH_SEP, NULL, 0);
 
 	read_info_alternates(the_repository, get_object_directory(), 0);
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#97 lore raw

PATCH 13/27 sha1_file: add repository argument to prepare_alt_odb

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

See previous patch for explanation.

---
 builtin/fsck.c |  2 +-
 object-store.h |  3 ++-
 packfile.c     |  2 +-
 sha1_file.c    | 12 ++++++------
 sha1_name.c    |  2 +-
 5 files changed, 11 insertions(+), 10 deletions(-)
builtin/fsck.c +1 -1
index 7707407275..3ef25fab97 100644
@@ -719,7 +719,7 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 
 		fsck_object_dir(get_object_directory());
 
-		prepare_alt_odb();
+		prepare_alt_odb(the_repository);
 		alt_odb_list = the_repository->objects->alt_odb_list;
 		for (alt = alt_odb_list; alt; alt = alt->next)
 			fsck_object_dir(alt->path);
object-store.h +2 -1
index b53e125902..79de470639 100644
@@ -20,7 +20,8 @@ struct alternate_object_database {
 
 	char path[FLEX_ARRAY];
 };
-void prepare_alt_odb(void);
+#define prepare_alt_odb(r) prepare_alt_odb_##r()
+void prepare_alt_odb_the_repository(void);
 char *compute_alternate_path(const char *path, struct strbuf *err);
 typedef int alt_odb_fn(struct alternate_object_database *, void *);
 int foreach_alt_odb(alt_odb_fn, void*);
packfile.c +1 -1
index b0b24ea9b8..17170fc662 100644
@@ -890,7 +890,7 @@ void prepare_packed_git(void)
 	if (the_repository->objects->packed_git_initialized)
 		return;
 	prepare_packed_git_one(get_object_directory(), 1);
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	for (alt = the_repository->objects->alt_odb_list; alt; alt = alt->next)
 		prepare_packed_git_one(alt->path, 0);
 	rearrange_packed_git();
sha1_file.c +6 -6
index ba4fc9103b..0fac75bd31 100644
@@ -582,7 +582,7 @@ void add_to_alternates_memory(const char *reference)
 	 * Make sure alternates are initialized, or else our entry may be
 	 * overwritten when they are.
 	 */
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 
 	link_alt_odb_entries(the_repository, reference,
 			     '\n', NULL, 0);
@@ -668,7 +668,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 	struct alternate_object_database *ent;
 	int r = 0;
 
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	for (ent = the_repository->objects->alt_odb_list; ent; ent = ent->next) {
 		r = fn(ent, cb);
 		if (r)
@@ -677,7 +677,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 	return r;
 }
 
-void prepare_alt_odb(void)
+void prepare_alt_odb_the_repository(void)
 {
 	if (the_repository->objects->alt_odb_tail)
 		return;
@@ -728,7 +728,7 @@ static int check_and_freshen_local(const unsigned char *sha1, int freshen)
 static int check_and_freshen_nonlocal(const unsigned char *sha1, int freshen)
 {
 	struct alternate_object_database *alt;
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	for (alt = the_repository->objects->alt_odb_list; alt; alt = alt->next) {
 		const char *path = alt_sha1_path(alt, sha1);
 		if (check_and_freshen_file(path, freshen))
@@ -887,7 +887,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,
 	if (!lstat(*path, st))
 		return 0;
 
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	errno = ENOENT;
 	for (alt = the_repository->objects->alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
@@ -918,7 +918,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)
 		return fd;
 	most_interesting_errno = errno;
 
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	for (alt = the_repository->objects->alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		fd = git_open(*path);
sha1_name.c +1 -1
index ffff7d8710..4325f74e0c 100644
@@ -354,7 +354,7 @@ static int init_object_disambiguation(const char *name, int len,
 
 	ds->len = len;
 	ds->hex_pfx[len] = '\0';
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	return 0;
 }
 
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#98 lore raw

PATCH 14/27 sha1_file: allow link_alt_odb_entries to handle arbitrary repositories

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Actually this also allows read_info_alternates and link_alt_odb_entry to handle arbitrary repositories, but link_alt_odb_entries is the most interesting function in this set of functions, hence the commit subject.

These functions span a strongly connected component in the function graph, i.e. the recursive call chain might look like

-> link_alt_odb_entries

    -> link_alt_odb_entry
      -> read_info_alternates
        -> link_alt_odb_entries

That is why we need to convert all these functions at the same time.

---
 object-store.h |  4 ++++
 sha1_file.c    | 36 ++++++++++++++++--------------------
 2 files changed, 20 insertions(+), 20 deletions(-)
object-store.h +4 -0
index 79de470639..3fc461a463 100644
@@ -18,6 +18,10 @@ struct alternate_object_database {
 	char loose_objects_subdir_seen[256];
 	struct oid_array loose_objects_cache;
 
+	/*
+	 * Path to the alternative object store. If this is a relative path,
+	 * it is relative to the current working directory.
+	 */
 	char path[FLEX_ARRAY];
 };
 #define prepare_alt_odb(r) prepare_alt_odb_##r()
sha1_file.c +16 -20
index 0fac75bd31..d38f5cdb0e 100644
@@ -390,11 +390,10 @@ static int alt_odb_usable(struct raw_object_store *o,
  * SHA1, an extra slash for the first level indirection, and the
  * terminating NUL.
  */
-#define read_info_alternates(r, rb, d) read_info_alternates_##r(rb, d)
-static void read_info_alternates_the_repository(const char *relative_base,
-						int depth);
-#define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)
-static int link_alt_odb_entry_the_repository(const char *entry,
+static void read_info_alternates(struct repository *r,
+				 const char *relative_base,
+				 int depth);
+static int link_alt_odb_entry(struct repository *r, const char *entry,
 	const char *relative_base, int depth, const char *normalized_objdir)
 {
 	struct alternate_object_database *ent;
@@ -420,7 +419,7 @@ static int link_alt_odb_entry_the_repository(const char *entry,
 	while (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')
 		strbuf_setlen(&pathbuf, pathbuf.len - 1);
 
-	if (!alt_odb_usable(the_repository->objects, &pathbuf, normalized_objdir)) {
+	if (!alt_odb_usable(r->objects, &pathbuf, normalized_objdir)) {
 		strbuf_release(&pathbuf);
 		return -1;
 	}
@@ -428,12 +427,12 @@ static int link_alt_odb_entry_the_repository(const char *entry,
 	ent = alloc_alt_odb(pathbuf.buf);
 
 	/* add the alternate entry */
-	*the_repository->objects->alt_odb_tail = ent;
-	the_repository->objects->alt_odb_tail = &(ent->next);
+	*r->objects->alt_odb_tail = ent;
+	r->objects->alt_odb_tail = &(ent->next);
 	ent->next = NULL;
 
 	/* recursively add alternates */
-	read_info_alternates(the_repository, pathbuf.buf, depth + 1);
+	read_info_alternates(r, pathbuf.buf, depth + 1);
 
 	strbuf_release(&pathbuf);
 	return 0;
@@ -468,12 +467,8 @@ static const char *parse_alt_odb_entry(const char *string,
 	return end;
 }
 
-#define link_alt_odb_entries(r, a, s, rb, d) \
-	link_alt_odb_entries_##r(a, s, rb, d)
-static void link_alt_odb_entries_the_repository(const char *alt,
-						int sep,
-						const char *relative_base,
-						int depth)
+static void link_alt_odb_entries(struct repository *r, const char *alt,
+				 int sep, const char *relative_base, int depth)
 {
 	struct strbuf objdirbuf = STRBUF_INIT;
 	struct strbuf entry = STRBUF_INIT;
@@ -487,7 +482,7 @@ static void link_alt_odb_entries_the_repository(const char *alt,
 		return;
 	}
 
-	strbuf_add_absolute_path(&objdirbuf, get_object_directory());
+	strbuf_add_absolute_path(&objdirbuf, r->objects->objectdir);
 	if (strbuf_normalize_path(&objdirbuf) < 0)
 		die("unable to normalize object directory: %s",
 		    objdirbuf.buf);
@@ -496,15 +491,16 @@ static void link_alt_odb_entries_the_repository(const char *alt,
 		alt = parse_alt_odb_entry(alt, sep, &entry);
 		if (!entry.len)
 			continue;
-		link_alt_odb_entry(the_repository, entry.buf,
+		link_alt_odb_entry(r, entry.buf,
 				   relative_base, depth, objdirbuf.buf);
 	}
 	strbuf_release(&entry);
 	strbuf_release(&objdirbuf);
 }
 
-static void read_info_alternates_the_repository(const char *relative_base,
-						int depth)
+static void read_info_alternates(struct repository *r,
+				 const char *relative_base,
+				 int depth)
 {
 	char *path;
 	struct strbuf buf = STRBUF_INIT;
@@ -516,7 +512,7 @@ static void read_info_alternates_the_repository(const char *relative_base,
 		return;
 	}
 
-	link_alt_odb_entries(the_repository, buf.buf, '\n', relative_base, depth);
+	link_alt_odb_entries(r, buf.buf, '\n', relative_base, depth);
 	strbuf_release(&buf);
 	free(path);
 }
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#99 lore raw

PATCH 15/27 sha1_file: allow prepare_alt_odb to handle arbitrary repositories

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

---
 object-store.h |  3 +--
 sha1_file.c    | 13 +++++--------
 2 files changed, 6 insertions(+), 10 deletions(-)
object-store.h +1 -2
index 3fc461a463..f54bc0b951 100644
@@ -24,8 +24,7 @@ struct alternate_object_database {
 	 */
 	char path[FLEX_ARRAY];
 };
-#define prepare_alt_odb(r) prepare_alt_odb_##r()
-void prepare_alt_odb_the_repository(void);
+void prepare_alt_odb(struct repository *r);
 char *compute_alternate_path(const char *path, struct strbuf *err);
 typedef int alt_odb_fn(struct alternate_object_database *, void *);
 int foreach_alt_odb(alt_odb_fn, void*);
sha1_file.c +5 -8
index d38f5cdb0e..04118f331c 100644
@@ -673,18 +673,15 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 	return r;
 }
 
-void prepare_alt_odb_the_repository(void)
+void prepare_alt_odb(struct repository *r)
 {
-	if (the_repository->objects->alt_odb_tail)
+	if (r->objects->alt_odb_tail)
 		return;
 
-	the_repository->objects->alt_odb_tail =
-			&the_repository->objects->alt_odb_list;
-	link_alt_odb_entries(the_repository,
-			     the_repository->objects->alternate_db,
-			     PATH_SEP, NULL, 0);
+	r->objects->alt_odb_tail = &r->objects->alt_odb_list;
+	link_alt_odb_entries(r, r->objects->alternate_db, PATH_SEP, NULL, 0);
 
-	read_info_alternates(the_repository, get_object_directory(), 0);
+	read_info_alternates(r, r->objects->objectdir, 0);
 }
 
 /* Returns 1 if we have successfully freshened the file, 0 otherwise. */
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#100 lore raw

PATCH 16/27 sha1_file: add repository argument to sha1_file_name

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Add a repository argument to allow sha1_file_name callers to be more specific about which repository to handle. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

While at it, move the declaration to object-store.h, where it should be easier to find.

---
 cache.h        |  6 ------
 http-walker.c  |  3 ++-
 http.c         |  5 ++---
 object-store.h |  7 +++++++
 sha1_file.c    | 10 +++++-----
 5 files changed, 16 insertions(+), 15 deletions(-)
cache.h +0 -6
index d3429a0d48..cbec0ecd23 100644
@@ -961,12 +961,6 @@ extern void check_repository_format(void);
 #define DATA_CHANGED    0x0020
 #define TYPE_CHANGED    0x0040
 
-/*
- * Put in `buf` the name of the file in the local object database that
- * would be used to store a loose object with the specified sha1.
- */
-extern void sha1_file_name(struct strbuf *buf, const unsigned char *sha1);
-
 /*
  * Return an abbreviated sha1 unique within this repository's object database.
  * The result will be at least `len` characters long, and will be NUL
http-walker.c +2 -1
index d850408d18..75d55d42a9 100644
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "repository.h"
 #include "commit.h"
 #include "walker.h"
 #include "http.h"
@@ -546,7 +547,7 @@ static int fetch_object(struct walker *walker, unsigned char *sha1)
 		ret = error("File %s has bad hash", hex);
 	} else if (req->rename < 0) {
 		struct strbuf buf = STRBUF_INIT;
-		sha1_file_name(&buf, req->sha1);
+		sha1_file_name(the_repository, &buf, req->sha1);
 		ret = error("unable to write sha1 filename %s", buf.buf);
 		strbuf_release(&buf);
 	}
http.c +2 -3
index efa977112a..4d613d5f6b 100644
@@ -2247,7 +2247,7 @@ struct http_object_request *new_http_object_request(const char *base_url,
 	hashcpy(freq->sha1, sha1);
 	freq->localfile = -1;
 
-	sha1_file_name(&filename, sha1);
+	sha1_file_name(the_repository, &filename, sha1);
 	snprintf(freq->tmpfile, sizeof(freq->tmpfile),
 		 "%s.temp", filename.buf);
 
@@ -2396,8 +2396,7 @@ int finish_http_object_request(struct http_object_request *freq)
 		unlink_or_warn(freq->tmpfile);
 		return -1;
 	}
-
-	sha1_file_name(&filename, freq->sha1);
+	sha1_file_name(the_repository, &filename, freq->sha1);
 	freq->rename = finalize_object_file(freq->tmpfile, filename.buf);
 	strbuf_release(&filename);
 
object-store.h +7 -0
index f54bc0b951..3707f3c495 100644
@@ -121,4 +121,11 @@ struct raw_object_store {
 struct raw_object_store *raw_object_store_new(void);
 void raw_object_store_clear(struct raw_object_store *o);
 
+/*
+ * Put in `buf` the name of the file in the local object database that
+ * would be used to store a loose object with the specified sha1.
+ */
+#define sha1_file_name(r, b, s) sha1_file_name_##r(b, s)
+void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1);
+
 #endif /* OBJECT_STORE_H */
sha1_file.c +5 -5
index 04118f331c..fe16d30f66 100644
@@ -323,7 +323,7 @@ static void fill_sha1_path(struct strbuf *buf, const unsigned char *sha1)
 	}
 }
 
-void sha1_file_name(struct strbuf *buf, const unsigned char *sha1)
+void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1)
 {
 	strbuf_addstr(buf, get_object_directory());
 	strbuf_addch(buf, '/');
@@ -713,7 +713,7 @@ static int check_and_freshen_local(const unsigned char *sha1, int freshen)
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(&buf, sha1);
+	sha1_file_name(the_repository, &buf, sha1);
 
 	return check_and_freshen_file(buf.buf, freshen);
 }
@@ -874,7 +874,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(&buf, sha1);
+	sha1_file_name(the_repository, &buf, sha1);
 	*path = buf.buf;
 
 	if (!lstat(*path, st))
@@ -903,7 +903,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(&buf, sha1);
+	sha1_file_name(the_repository, &buf, sha1);
 	*path = buf.buf;
 
 	fd = git_open(*path);
@@ -1588,7 +1588,7 @@ static int write_loose_object(const unsigned char *sha1, char *hdr, int hdrlen,
 	static struct strbuf filename = STRBUF_INIT;
 
 	strbuf_reset(&filename);
-	sha1_file_name(&filename, sha1);
+	sha1_file_name(the_repository, &filename, sha1);
 
 	fd = create_tmpfile(&tmp_file, filename.buf);
 	if (fd < 0) {
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#101 lore raw

PATCH 17/27 sha1_file: add repository argument to stat_sha1_file

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Add a repository argument to allow the stat_sha1_file caller to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

---
 sha1_file.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
sha1_file.c +5 -4
index fe16d30f66..80bac89014 100644
@@ -867,8 +867,9 @@ int git_open_cloexec(const char *name, int flags)
  * Note that it may point to static storage and is only valid until another
  * call to sha1_file_name(), etc.
  */
-static int stat_sha1_file(const unsigned char *sha1, struct stat *st,
-			  const char **path)
+#define stat_sha1_file(r, s, st, p) stat_sha1_file_##r(s, st, p)
+static int stat_sha1_file_the_repository(const unsigned char *sha1,
+					 struct stat *st, const char **path)
 {
 	struct alternate_object_database *alt;
 	static struct strbuf buf = STRBUF_INIT;
@@ -1174,7 +1175,7 @@ static int sha1_loose_object_info(const unsigned char *sha1,
 	if (!oi->typep && !oi->typename && !oi->sizep && !oi->contentp) {
 		const char *path;
 		struct stat st;
-		if (stat_sha1_file(sha1, &st, &path) < 0)
+		if (stat_sha1_file(the_repository, sha1, &st, &path) < 0)
 			return -1;
 		if (oi->disk_sizep)
 			*oi->disk_sizep = st.st_size;
@@ -1388,7 +1389,7 @@ void *read_sha1_file_extended(const unsigned char *sha1,
 		die("replacement %s not found for %s",
 		    sha1_to_hex(repl), sha1_to_hex(sha1));
 
-	if (!stat_sha1_file(repl, &st, &path))
+	if (!stat_sha1_file(the_repository, repl, &st, &path))
 		die("loose object %s (stored in %s) is corrupt",
 		    sha1_to_hex(repl), path);
 
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#102 lore raw

PATCH 18/27 sha1_file: add repository argument to open_sha1_file

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Add a repository argument to allow the open_sha1_file caller to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

---
 sha1_file.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
sha1_file.c +4 -2
index 80bac89014..a2ab2b82c3 100644
@@ -896,7 +896,9 @@ static int stat_sha1_file_the_repository(const unsigned char *sha1,
  * Like stat_sha1_file(), but actually open the object and return the
  * descriptor. See the caveats on the "path" parameter above.
  */
-static int open_sha1_file(const unsigned char *sha1, const char **path)
+#define open_sha1_file(r, s, p) open_sha1_file_##r(s, p)
+static int open_sha1_file_the_repository(const unsigned char *sha1,
+					 const char **path)
 {
 	int fd;
 	struct alternate_object_database *alt;
@@ -939,7 +941,7 @@ static void *map_sha1_file_1(const char *path,
 	if (path)
 		fd = git_open(path);
 	else
-		fd = open_sha1_file(sha1, &path);
+		fd = open_sha1_file(the_repository, sha1, &path);
 	map = NULL;
 	if (fd >= 0) {
 		struct stat st;
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#103 lore raw

PATCH 19/27 sha1_file: add repository argument to map_sha1_file_1

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Add a repository argument to allow the map_sha1_file_1 caller to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

---
 sha1_file.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
sha1_file.c +6 -5
index a2ab2b82c3..4b6144b7cd 100644
@@ -931,9 +931,10 @@ static int open_sha1_file_the_repository(const unsigned char *sha1,
  * Map the loose object at "path" if it is not NULL, or the path found by
  * searching for a loose object named "sha1".
  */
-static void *map_sha1_file_1(const char *path,
-			     const unsigned char *sha1,
-			     unsigned long *size)
+#define map_sha1_file_1(r, p, s, si) map_sha1_file_1_##r(p, s, si)
+static void *map_sha1_file_1_the_repository(const char *path,
+					    const unsigned char *sha1,
+					    unsigned long *size)
 {
 	void *map;
 	int fd;
@@ -962,7 +963,7 @@ static void *map_sha1_file_1(const char *path,
 
 void *map_sha1_file(const unsigned char *sha1, unsigned long *size)
 {
-	return map_sha1_file_1(NULL, sha1, size);
+	return map_sha1_file_1(the_repository, NULL, sha1, size);
 }
 
 static int unpack_sha1_short_header(git_zstream *stream,
@@ -2192,7 +2193,7 @@ int read_loose_object(const char *path,
 
 	*contents = NULL;
 
-	map = map_sha1_file_1(path, NULL, &mapsize);
+	map = map_sha1_file_1(the_repository, path, NULL, &mapsize);
 	if (!map) {
 		error_errno("unable to mmap %s", path);
 		goto out;
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#104 lore raw

PATCH 20/27 sha1_file: add repository argument to map_sha1_file

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Add a repository argument to allow map_sha1_file callers to be more specific about which repository to handle. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

While at it, move the declaration to object-store.h, where it should be easier to find.

---
 cache.h        | 1 -
 object-store.h | 3 +++
 sha1_file.c    | 4 ++--
 streaming.c    | 5 ++++-
 4 files changed, 9 insertions(+), 4 deletions(-)
cache.h +0 -1
index cbec0ecd23..720664e394 100644
@@ -1242,7 +1242,6 @@ extern int pretend_sha1_file(void *, unsigned long, enum object_type, unsigned c
 extern int force_object_loose(const unsigned char *sha1, time_t mtime);
 extern int git_open_cloexec(const char *name, int flags);
 #define git_open(name) git_open_cloexec(name, O_RDONLY)
-extern void *map_sha1_file(const unsigned char *sha1, unsigned long *size);
 extern int unpack_sha1_header(git_zstream *stream, unsigned char *map, unsigned long mapsize, void *buffer, unsigned long bufsiz);
 extern int parse_sha1_header(const char *hdr, unsigned long *sizep);
 
object-store.h +3 -0
index 3707f3c495..03671745f4 100644
@@ -128,4 +128,7 @@ void raw_object_store_clear(struct raw_object_store *o);
 #define sha1_file_name(r, b, s) sha1_file_name_##r(b, s)
 void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1);
 
+#define map_sha1_file(r, s, sz) map_sha1_file_##r(s, sz)
+void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size);
+
 #endif /* OBJECT_STORE_H */
sha1_file.c +2 -2
index 4b6144b7cd..f968f67a14 100644
@@ -961,7 +961,7 @@ static void *map_sha1_file_1_the_repository(const char *path,
 	return map;
 }
 
-void *map_sha1_file(const unsigned char *sha1, unsigned long *size)
+void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size)
 {
 	return map_sha1_file_1(the_repository, NULL, sha1, size);
 }
@@ -1185,7 +1185,7 @@ static int sha1_loose_object_info(const unsigned char *sha1,
 		return 0;
 	}
 
-	map = map_sha1_file(sha1, &mapsize);
+	map = map_sha1_file(the_repository, sha1, &mapsize);
 	if (!map)
 		return -1;
 
streaming.c +4 -1
index 5892b50bd8..22d27df55e 100644
@@ -3,6 +3,8 @@
  */
 #include "cache.h"
 #include "streaming.h"
+#include "repository.h"
+#include "object-store.h"
 #include "packfile.h"
 
 enum input_source {
@@ -335,7 +337,8 @@ static struct stream_vtbl loose_vtbl = {
 
 static open_method_decl(loose)
 {
-	st->u.loose.mapped = map_sha1_file(sha1, &st->u.loose.mapsize);
+	st->u.loose.mapped = map_sha1_file(the_repository,
+					   sha1, &st->u.loose.mapsize);
 	if (!st->u.loose.mapped)
 		return -1;
 	if ((unpack_sha1_header(&st->z,
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#105 lore raw

PATCH 21/27 sha1_file: add repository argument to sha1_loose_object_info

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Add a repository argument to allow the sha1_loose_object_info caller to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

---
 sha1_file.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
sha1_file.c +5 -4
index f968f67a14..74c68ea776 100644
@@ -1152,9 +1152,10 @@ int parse_sha1_header(const char *hdr, unsigned long *sizep)
 	return parse_sha1_header_extended(hdr, &oi, 0);
 }
 
-static int sha1_loose_object_info(const unsigned char *sha1,
-				  struct object_info *oi,
-				  int flags)
+#define sha1_loose_object_info(r, s, o, f) sha1_loose_object_info_##r(s, o, f)
+static int sha1_loose_object_info_the_repository(const unsigned char *sha1,
+						 struct object_info *oi,
+						 int flags)
 {
 	int status = 0;
 	unsigned long mapsize;
@@ -1273,7 +1274,7 @@ int sha1_object_info_extended(const unsigned char *sha1, struct object_info *oi,
 			break;
 
 		/* Most likely it's a loose object. */
-		if (!sha1_loose_object_info(real, oi, flags))
+		if (!sha1_loose_object_info(the_repository, real, oi, flags))
 			return 0;
 
 		/* Not a loose object; someone else may have just packed it. */
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#106 lore raw

PATCH 22/27 sha1_file: allow sha1_file_name to handle arbitrary repositories

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

---
 object-store.h | 3 +--
 sha1_file.c    | 4 ++--
 2 files changed, 3 insertions(+), 4 deletions(-)
object-store.h +1 -2
index 03671745f4..d9cc875153 100644
@@ -125,8 +125,7 @@ void raw_object_store_clear(struct raw_object_store *o);
  * Put in `buf` the name of the file in the local object database that
  * would be used to store a loose object with the specified sha1.
  */
-#define sha1_file_name(r, b, s) sha1_file_name_##r(b, s)
-void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1);
+void sha1_file_name(struct repository *r, struct strbuf *buf, const unsigned char *sha1);
 
 #define map_sha1_file(r, s, sz) map_sha1_file_##r(s, sz)
 void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size);
sha1_file.c +2 -2
index 74c68ea776..aed9a558c5 100644
@@ -323,9 +323,9 @@ static void fill_sha1_path(struct strbuf *buf, const unsigned char *sha1)
 	}
 }
 
-void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1)
+void sha1_file_name(struct repository *r, struct strbuf *buf, const unsigned char *sha1)
 {
-	strbuf_addstr(buf, get_object_directory());
+	strbuf_addstr(buf, r->objects->objectdir);
 	strbuf_addch(buf, '/');
 	fill_sha1_path(buf, sha1);
 }
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#107 lore raw

PATCH 23/27 sha1_file: allow stat_sha1_file to handle arbitrary repositories

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

---
 sha1_file.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)
sha1_file.c +5 -6
index aed9a558c5..fc4f209981 100644
@@ -867,23 +867,22 @@ int git_open_cloexec(const char *name, int flags)
  * Note that it may point to static storage and is only valid until another
  * call to sha1_file_name(), etc.
  */
-#define stat_sha1_file(r, s, st, p) stat_sha1_file_##r(s, st, p)
-static int stat_sha1_file_the_repository(const unsigned char *sha1,
-					 struct stat *st, const char **path)
+static int stat_sha1_file(struct repository *r, const unsigned char *sha1,
+			  struct stat *st, const char **path)
 {
 	struct alternate_object_database *alt;
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(the_repository, &buf, sha1);
+	sha1_file_name(r, &buf, sha1);
 	*path = buf.buf;
 
 	if (!lstat(*path, st))
 		return 0;
 
-	prepare_alt_odb(the_repository);
+	prepare_alt_odb(r);
 	errno = ENOENT;
-	for (alt = the_repository->objects->alt_odb_list; alt; alt = alt->next) {
+	for (alt = r->objects->alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		if (!lstat(*path, st))
 			return 0;
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#108 lore raw

PATCH 24/27 sha1_file: allow open_sha1_file to handle arbitrary repositories

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

---
 sha1_file.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)
sha1_file.c +5 -6
index fc4f209981..1fa32c8a06 100644
@@ -895,9 +895,8 @@ static int stat_sha1_file(struct repository *r, const unsigned char *sha1,
  * Like stat_sha1_file(), but actually open the object and return the
  * descriptor. See the caveats on the "path" parameter above.
  */
-#define open_sha1_file(r, s, p) open_sha1_file_##r(s, p)
-static int open_sha1_file_the_repository(const unsigned char *sha1,
-					 const char **path)
+static int open_sha1_file(struct repository *r,
+			  const unsigned char *sha1, const char **path)
 {
 	int fd;
 	struct alternate_object_database *alt;
@@ -905,7 +904,7 @@ static int open_sha1_file_the_repository(const unsigned char *sha1,
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(the_repository, &buf, sha1);
+	sha1_file_name(r, &buf, sha1);
 	*path = buf.buf;
 
 	fd = git_open(*path);
@@ -913,8 +912,8 @@ static int open_sha1_file_the_repository(const unsigned char *sha1,
 		return fd;
 	most_interesting_errno = errno;
 
-	prepare_alt_odb(the_repository);
-	for (alt = the_repository->objects->alt_odb_list; alt; alt = alt->next) {
+	prepare_alt_odb(r);
+	for (alt = r->objects->alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		fd = git_open(*path);
 		if (fd >= 0)
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#109 lore raw

PATCH 25/27 sha1_file: allow map_sha1_file_1 to handle arbitrary repositories

In reply to Nguyễn Thái Ngọc Duy

From: Jonathan Nieder <[email protected]>

---
 sha1_file.c | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)
sha1_file.c +3 -5
index 1fa32c8a06..1d5d23dfe2 100644
@@ -929,10 +929,8 @@ static int open_sha1_file(struct repository *r,
  * Map the loose object at "path" if it is not NULL, or the path found by
  * searching for a loose object named "sha1".
  */
-#define map_sha1_file_1(r, p, s, si) map_sha1_file_1_##r(p, s, si)
-static void *map_sha1_file_1_the_repository(const char *path,
-					    const unsigned char *sha1,
-					    unsigned long *size)
+static void *map_sha1_file_1(struct repository *r, const char *path,
+			     const unsigned char *sha1, unsigned long *size)
 {
 	void *map;
 	int fd;
@@ -940,7 +938,7 @@ static void *map_sha1_file_1_the_repository(const char *path,
 	if (path)
 		fd = git_open(path);
 	else
-		fd = open_sha1_file(the_repository, sha1, &path);
+		fd = open_sha1_file(r, sha1, &path);
 	map = NULL;
 	if (fd >= 0) {
 		struct stat st;
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#110 lore raw

PATCH 26/27 sha1_file: allow map_sha1_file to handle arbitrary repositories

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

---
 object-store.h | 3 +--
 sha1_file.c    | 5 +++--
 2 files changed, 4 insertions(+), 4 deletions(-)
object-store.h +1 -2
index d9cc875153..fef33f345f 100644
@@ -127,7 +127,6 @@ void raw_object_store_clear(struct raw_object_store *o);
  */
 void sha1_file_name(struct repository *r, struct strbuf *buf, const unsigned char *sha1);
 
-#define map_sha1_file(r, s, sz) map_sha1_file_##r(s, sz)
-void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size);
+void *map_sha1_file(struct repository *r, const unsigned char *sha1, unsigned long *size);
 
 #endif /* OBJECT_STORE_H */
sha1_file.c +3 -2
index 1d5d23dfe2..9addad6887 100644
@@ -957,9 +957,10 @@ static void *map_sha1_file_1(struct repository *r, const char *path,
 	return map;
 }
 
-void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size)
+void *map_sha1_file(struct repository *r,
+		    const unsigned char *sha1, unsigned long *size)
 {
-	return map_sha1_file_1(the_repository, NULL, sha1, size);
+	return map_sha1_file_1(r, NULL, sha1, size);
 }
 
 static int unpack_sha1_short_header(git_zstream *stream,
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#111 lore raw

PATCH 27/27 sha1_file: allow sha1_loose_object_info to handle arbitrary repositories

In reply to Nguyễn Thái Ngọc Duy

From: Jonathan Nieder <[email protected]>

---
 sha1_file.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)
sha1_file.c +5 -6
index 9addad6887..0989bbd948 100644
@@ -1149,10 +1149,9 @@ int parse_sha1_header(const char *hdr, unsigned long *sizep)
 	return parse_sha1_header_extended(hdr, &oi, 0);
 }
 
-#define sha1_loose_object_info(r, s, o, f) sha1_loose_object_info_##r(s, o, f)
-static int sha1_loose_object_info_the_repository(const unsigned char *sha1,
-						 struct object_info *oi,
-						 int flags)
+static int sha1_loose_object_info(struct repository *r,
+				  const unsigned char *sha1,
+				  struct object_info *oi, int flags)
 {
 	int status = 0;
 	unsigned long mapsize;
@@ -1176,14 +1175,14 @@ static int sha1_loose_object_info_the_repository(const unsigned char *sha1,
 	if (!oi->typep && !oi->typename && !oi->sizep && !oi->contentp) {
 		const char *path;
 		struct stat st;
-		if (stat_sha1_file(the_repository, sha1, &st, &path) < 0)
+		if (stat_sha1_file(r, sha1, &st, &path) < 0)
 			return -1;
 		if (oi->disk_sizep)
 			*oi->disk_sizep = st.st_size;
 		return 0;
 	}
 
-	map = map_sha1_file(the_repository, sha1, &mapsize);
+	map = map_sha1_file(r, sha1, &mapsize);
 	if (!map)
 		return -1;
 
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#112 lore raw

Re: [PATCH 00/27] sb/object-store updates

In reply to Nguyễn Thái Ngọc Duy
Quoted 7 lines · Nguyễn Thái Ngọc Duy

I think I have addressed all comments I've received so far. What I decided not to do, I have responded individually. One comment I did not respond nor do is the approximate thing, which could be done later.

Interdiff is big due to the "objects." to "objects->" conversion (blame Brandon for his suggestion, don't blame me :D)

Haha, I'm guessing you prefer having a pointer too then? :P

The interdiff looks good, though I'll set some time aside today to go through the series as a whole again.

Signature · 1 line
Brandon Williams
#113 lore raw

Re: [PATCH 00/27] sb/object-store updates

Quoted 6 lines · Brandon Williams
Quoted 5 lines · Nguyễn Thái Ngọc Duy

I think I have addressed all comments I've received so far. What I decided not to do, I have responded individually. One comment I did not respond nor do is the approximate thing, which could be done later.

Quoted 7 lines
Quoted 2 lines

Interdiff is big due to the "objects." to "objects->" conversion (blame Brandon for his suggestion, don't blame me :D)

Haha, I'm guessing you prefer having a pointer too then? :P

The interdiff looks good, though I'll set some time aside today to go through the series as a whole again.

Had some more time than I thought; this series looks good! Thanks for take the time to keep this rolling. Hopefully we can see this merged soon :)

Signature · 1 line
Brandon Williams
#114 lore raw

Re: [PATCH 00/27] sb/object-store updates

Quoted 11 lines · Brandon Williams
Quoted 7 lines
Quoted 2 lines

Interdiff is big due to the "objects." to "objects->" conversion (blame Brandon for his suggestion, don't blame me :D)

Haha, I'm guessing you prefer having a pointer too then? :P

The interdiff looks good, though I'll set some time aside today to go through the series as a whole again.

Had some more time than I thought; this series looks good! Thanks for take the time to keep this rolling. Hopefully we can see this merged soon :)

Yeah, I definitely hope that this one is already 'next'-worthy; the micro-update to ignore-env we saw recently looked good, too.

Thanks.

author
#115 lore raw

Re: [PATCH 00/27] sb/object-store updates

In reply to Brandon Williams
Quoted 10 lines · Brandon Williams
Quoted 7 lines · Nguyễn Thái Ngọc Duy

I think I have addressed all comments I've received so far. What I decided not to do, I have responded individually. One comment I did not respond nor do is the approximate thing, which could be done later.

Interdiff is big due to the "objects." to "objects->" conversion (blame Brandon for his suggestion, don't blame me :D)

Haha, I'm guessing you prefer having a pointer too then? :P

That goes without saying or I would be arguing against that instead of spending time fixing a bunch of compile errors ;-)

Signature · 1 line
Duy
#116 lore raw

Re: [PATCH 00/27] sb/object-store updates

In reply to Nguyễn Thái Ngọc Duy

packfile.c static int add_promisor_object(const struct object_id *oid, · line 1942

Quoted 10 lines · Nguyễn Thái Ngọc Duy

Interdiff is big due to the "objects." to "objects->" conversion (blame Brandon for his suggestion, don't blame me :D)

packfile.c +1 -1
@@ -1938,7 +1939,7 @@ static int add_promisor_object(const struct object_id *oid,
        /*
         * If this is a tree, commit, or tag, the objects it refers
-        * to are also promisor objects. (Blobs refer to no objects.)
+        * to are also promisor objects-> (Blobs refer to no objects->)
         */

Meh.

author
#117 lore raw

Re: [PATCH 00/27] sb/object-store updates

Quoted 4 lines · Eric Sunshine
Quoted 3 lines · Nguyễn Thái Ngọc Duy

Interdiff is big due to the "objects." to "objects->" conversion (blame Brandon for his suggestion, don't blame me :D)

Quoted 9 lines

packfile.c static int add_promisor_object(const struct object_id *oid, · line 1942

Quoted 7 lines
packfile.c +1 -1
@@ -1938,7 +1939,7 @@ static int add_promisor_object(const struct object_id *oid,
        /*
         * If this is a tree, commit, or tag, the objects it refers
-        * to are also promisor objects. (Blobs refer to no objects.)
+        * to are also promisor objects-> (Blobs refer to no objects->)
         */

Meh.

I got too excited when searching and replacing. Here's the fixup patch.

-- 8< --
Subject: [PATCH] fixup! object-store: move packed_git and packed_git_mru to
object store

---
 packfile.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
packfile.c +1 -1
index 63c89ee31a..0906b8f741 100644
@@ -1937,7 +1937,7 @@ static int add_promisor_object(const struct object_id *oid,
 
 	/*
 	 * If this is a tree, commit, or tag, the objects it refers
-	 * to are also promisor objects-> (Blobs refer to no objects->)
+	 * to are also promisor objects. (Blobs refer to no objects.)
 	 */
 	if (obj->type == OBJ_TREE) {
 		struct tree *tree = (struct tree *)obj;
Signature · 3 lines
2.17.0.rc0.348.gd5a49e0b6f

-- 8< --
Reviewed-by
#118 lore raw

Re: [PATCH 00/27] sb/object-store updates

Quoted 2 lines · Duy Nguyen

I got too excited when searching and replacing. Here's the fixup patch.

Thanks for picking up the series that I left unattended over the last weeks. I have reviewed nd/remove-ignore-env-field as well as sb/object-store as currently queued and found them well done.

Thanks for driving this when I was away!

With the fixup patch, both series are

(No need to apply that as it already looks funny to have 3-5 sign offs including mine)

With these 2 series reviewed, I'll continue reviewing sb/pack-files-in-repository and the rest of my mail box.

Thanks,
Stefan

Quoted 2 lines

-- 8< --
Subject: [PATCH] fixup! object-store: move packed_git and packed_git_mru

to

packfile.c static int add_promisor_object(const struct

Quoted 11 lines

object store

---
  packfile.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
packfile.c +0 -0
index 63c89ee31a..0906b8f741 100644
@@ -1937,7 +1937,7 @@ static int add_promisor_object(const struct

object_id *oid,

Quoted 11 lines
packfile.c +1 -2
         /*
          * If this is a tree, commit, or tag, the objects it refers
-        * to are also promisor objects-> (Blobs refer to no objects->)
+        * to are also promisor objects. (Blobs refer to no objects.)
          */
         if (obj->type == OBJ_TREE) {
                 struct tree *tree = (struct tree *)obj;
--

2.17.0.rc0.348.gd5a49e0b6f

-- 8< --

#119 lore raw

Re: [PATCH 00/27] sb/object-store updates

Quoted 4 lines · Stefan Beller

Thanks for driving this when I was away!

With the fixup patch, both series are

I think everybody involved agrees that these two you cited above are already in good shape. Let's have them in 'next' for the remainder of this cycle and go incremental, and merge them to 'master' soon after 2.17 final is tagged.

Thanks all for working well together.

#120 lore raw

Re: [PATCH 00/27] sb/object-store updates

Quoted 7 lines · Junio C Hamano
Quoted 4 lines · Stefan Beller

Thanks for driving this when I was away!

With the fixup patch, both series are

Quoted 6 lines

I think everybody involved agrees that these two you cited above are already in good shape. Let's have them in 'next' for the remainder of this cycle and go incremental, and merge them to 'master' soon after 2.17 final is tagged.

Thanks all for working well together.

Just FYI, when merging the topic nd/pack-objects-pack-struct to a codebase with these two topics, because the former adds a handful of inline functions that wants to see packed_git but all existing sources have relied on the rule that cache.h is included early and the fact that packed_git was declared in there, the change in sb/object-store to move the structure declaration to object-store.h causes quite a trouble. I plan to resolve this with an evil merge to include object-store.h from pack-objects.h to resolve this with minimum damage to topics in flight.

#121 lore raw

cover sb/packfiles-in-repository updates

In reply to Nguyễn Thái Ngọc Duy

This is the rebased version on the updated sb/object-store I just sent out plus the fix for get_object_directory(). The interdiff (after rebased) looks small and nice

packfile.c +1 -1
index e02136bebb..63c89ee31a 100644
@@ -890,7 +890,7 @@ static void prepare_packed_git(struct repository *r)
 
 	if (r->objects->packed_git_initialized)
 		return;
-	prepare_packed_git_one(r, get_object_directory(), 1);
+	prepare_packed_git_one(r, r->objects->objectdir, 1);
 	prepare_alt_odb(r);
 	for (alt = r->objects->alt_odb_list; alt; alt = alt->next)
 		prepare_packed_git_one(r, alt->path, 0);

I notice there's still one get_object_directory() left in packfile.c but that should not cause problems with converted functions. That could be done in "phase 2".

Nguyễn Thái Ngọc Duy (1):
packfile: keep prepare_packed_git() private

Stefan Beller (11):
packfile: allow prepare_packed_git_mru to handle arbitrary
repositories
packfile: allow rearrange_packed_git to handle arbitrary repositories
packfile: allow install_packed_git to handle arbitrary repositories
packfile: add repository argument to prepare_packed_git_one
packfile: add repository argument to prepare_packed_git
packfile: add repository argument to reprepare_packed_git
packfile: allow prepare_packed_git_one to handle arbitrary
repositories
packfile: allow prepare_packed_git to handle arbitrary repositories
packfile: allow reprepare_packed_git to handle arbitrary repositories
packfile: add repository argument to find_pack_entry
packfile: allow find_pack_entry to handle arbitrary repositories

 builtin/count-objects.c  |  3 +-
 builtin/fsck.c           |  2 --
 builtin/gc.c             |  3 +-
 builtin/pack-objects.c   |  1 -
 builtin/pack-redundant.c |  2 --
 builtin/receive-pack.c   |  3 +-
 bulk-checkin.c           |  3 +-
 fast-import.c            |  3 +-
 fetch-pack.c             |  3 +-
 http-backend.c           |  1 -
 http.c                   |  2 +-
 pack-bitmap.c            |  1 -
 packfile.c               | 76 +++++++++++++++++++---------------------
 packfile.h               | 11 +++---
 server-info.c            |  1 -
 sha1_file.c              |  8 ++---
 sha1_name.c              |  2 --
 17 files changed, 58 insertions(+), 67 deletions(-)
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#122 lore raw

PATCH 1/12 packfile: allow prepare_packed_git_mru to handle arbitrary repositories

From: Stefan Beller <[email protected]>

This conversion was done without the #define trick used in the earlier series refactoring to have better repository access, because this function is easy to review, as all lines are converted and it has only one caller

---
 packfile.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
packfile.c +5 -5
index 17170fc662..8c335bdcd1 100644
@@ -873,14 +873,14 @@ static void rearrange_packed_git(void)
 		set_next_packed_git, sort_pack);
 }
 
-static void prepare_packed_git_mru(void)
+static void prepare_packed_git_mru(struct repository *r)
 {
 	struct packed_git *p;
 
-	INIT_LIST_HEAD(&the_repository->objects->packed_git_mru);
+	INIT_LIST_HEAD(&r->objects->packed_git_mru);
 
-	for (p = the_repository->objects->packed_git; p; p = p->next)
-		list_add_tail(&p->mru, &the_repository->objects->packed_git_mru);
+	for (p = r->objects->packed_git; p; p = p->next)
+		list_add_tail(&p->mru, &r->objects->packed_git_mru);
 }
 
 void prepare_packed_git(void)
@@ -894,7 +894,7 @@ void prepare_packed_git(void)
 	for (alt = the_repository->objects->alt_odb_list; alt; alt = alt->next)
 		prepare_packed_git_one(alt->path, 0);
 	rearrange_packed_git();
-	prepare_packed_git_mru();
+	prepare_packed_git_mru(the_repository);
 	the_repository->objects->packed_git_initialized = 1;
 }
 
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#123 lore raw

PATCH 2/12 packfile: allow rearrange_packed_git to handle arbitrary repositories

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

---
 packfile.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
packfile.c +4 -4
index 8c335bdcd1..bff22a8c81 100644
@@ -866,10 +866,10 @@ static int sort_pack(const void *a_, const void *b_)
 	return -1;
 }
 
-static void rearrange_packed_git(void)
+static void rearrange_packed_git(struct repository *r)
 {
-	the_repository->objects->packed_git = llist_mergesort(
-		the_repository->objects->packed_git, get_next_packed_git,
+	r->objects->packed_git = llist_mergesort(
+		r->objects->packed_git, get_next_packed_git,
 		set_next_packed_git, sort_pack);
 }
 
@@ -893,7 +893,7 @@ void prepare_packed_git(void)
 	prepare_alt_odb(the_repository);
 	for (alt = the_repository->objects->alt_odb_list; alt; alt = alt->next)
 		prepare_packed_git_one(alt->path, 0);
-	rearrange_packed_git();
+	rearrange_packed_git(the_repository);
 	prepare_packed_git_mru(the_repository);
 	the_repository->objects->packed_git_initialized = 1;
 }
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#124 lore raw

PATCH 3/12 packfile: allow install_packed_git to handle arbitrary repositories

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

This conversion was done without the #define trick used in the earlier series refactoring to have better repository access, because this function is easy to review, as it only has one caller and all lines but the first two are converted.

We must not convert 'pack_open_fds' to be a repository specific variable, as it is used to monitor resource usage of the machine that Git executes on.

---
 fast-import.c | 2 +-
 http.c        | 2 +-
 packfile.c    | 8 ++++----
 packfile.h    | 2 +-
 4 files changed, 7 insertions(+), 7 deletions(-)
fast-import.c +1 -1
index b3492fce5c..ae4ced3ae1 100644
@@ -1038,7 +1038,7 @@ static void end_packfile(void)
 		if (!new_p)
 			die("core git rejected index %s", idx_name);
 		all_packs[pack_id] = new_p;
-		install_packed_git(new_p);
+		install_packed_git(the_repository, new_p);
 		free(idx_name);
 
 		/* Print the boundary */
http.c +1 -1
index 4d613d5f6b..111e3c12c8 100644
@@ -2134,7 +2134,7 @@ int finish_http_pack_request(struct http_pack_request *preq)
 		return -1;
 	}
 
-	install_packed_git(p);
+	install_packed_git(the_repository, p);
 	free(tmp_idx);
 	return 0;
 }
packfile.c +4 -4
index bff22a8c81..ff302142c7 100644
@@ -680,13 +680,13 @@ struct packed_git *add_packed_git(const char *path, size_t path_len, int local)
 	return p;
 }
 
-void install_packed_git(struct packed_git *pack)
+void install_packed_git(struct repository *r, struct packed_git *pack)
 {
 	if (pack->pack_fd != -1)
 		pack_open_fds++;
 
-	pack->next = the_repository->objects->packed_git;
-	the_repository->objects->packed_git = pack;
+	pack->next = r->objects->packed_git;
+	r->objects->packed_git = pack;
 }
 
 void (*report_garbage)(unsigned seen_bits, const char *path);
@@ -782,7 +782,7 @@ static void prepare_packed_git_one(char *objdir, int local)
 			     * corresponding .pack file that we can map.
 			     */
 			    (p = add_packed_git(path.buf, path.len, local)) != NULL)
-				install_packed_git(p);
+				install_packed_git(the_repository, p);
 		}
 
 		if (!report_garbage)
packfile.h +1 -1
index 5b1ce00f84..77442172f0 100644
@@ -36,7 +36,7 @@ extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
 extern void prepare_packed_git(void);
 extern void reprepare_packed_git(void);
-extern void install_packed_git(struct packed_git *pack);
+extern void install_packed_git(struct repository *r, struct packed_git *pack);
 
 struct packed_git *get_packed_git(struct repository *r);
 struct list_head *get_packed_git_mru(struct repository *r);
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#125 lore raw

PATCH 4/12 packfile: add repository argument to prepare_packed_git_one

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

---
 packfile.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
packfile.c +4 -3
index ff302142c7..0e5fa67526 100644
@@ -735,7 +735,8 @@ static void report_pack_garbage(struct string_list *list)
 	report_helper(list, seen_bits, first, list->nr);
 }
 
-static void prepare_packed_git_one(char *objdir, int local)
+#define prepare_packed_git_one(r, o, l) prepare_packed_git_one_##r(o, l)
+static void prepare_packed_git_one_the_repository(char *objdir, int local)
 {
 	struct strbuf path = STRBUF_INIT;
 	size_t dirnamelen;
@@ -889,10 +890,10 @@ void prepare_packed_git(void)
 
 	if (the_repository->objects->packed_git_initialized)
 		return;
-	prepare_packed_git_one(get_object_directory(), 1);
+	prepare_packed_git_one(the_repository, get_object_directory(), 1);
 	prepare_alt_odb(the_repository);
 	for (alt = the_repository->objects->alt_odb_list; alt; alt = alt->next)
-		prepare_packed_git_one(alt->path, 0);
+		prepare_packed_git_one(the_repository, alt->path, 0);
 	rearrange_packed_git(the_repository);
 	prepare_packed_git_mru(the_repository);
 	the_repository->objects->packed_git_initialized = 1;
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#126 lore raw

PATCH 5/12 packfile: add repository argument to prepare_packed_git

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

Add a repository argument to allow prepare_packed_git callers to be more specific about which repository to handle. See commit "sha1_file:
add repository argument to link_alt_odb_entry" for an explanation of the #define trick.

---
 builtin/count-objects.c  |  2 +-
 builtin/fsck.c           |  2 +-
 builtin/gc.c             |  2 +-
 builtin/pack-objects.c   |  2 +-
 builtin/pack-redundant.c |  2 +-
 fast-import.c            |  2 +-
 http-backend.c           |  2 +-
 pack-bitmap.c            |  2 +-
 packfile.c               | 10 +++++-----
 packfile.h               |  3 ++-
 server-info.c            |  2 +-
 sha1_name.c              |  4 ++--
 12 files changed, 18 insertions(+), 17 deletions(-)
builtin/count-objects.c +1 -1
index b28ff00be2..ea8bd9e2e2 100644
@@ -123,7 +123,7 @@ int cmd_count_objects(int argc, const char **argv, const char *prefix)
 		struct strbuf pack_buf = STRBUF_INIT;
 		struct strbuf garbage_buf = STRBUF_INIT;
 		if (!get_packed_git(the_repository))
-			prepare_packed_git();
+			prepare_packed_git(the_repository);
 		for (p = get_packed_git(the_repository); p; p = p->next) {
 			if (!p->pack_local)
 				continue;
builtin/fsck.c +1 -1
index 3ef25fab97..d40a82b702 100644
@@ -729,7 +729,7 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 			uint32_t total = 0, count = 0;
 			struct progress *progress = NULL;
 
-			prepare_packed_git();
+			prepare_packed_git(the_repository);
 
 			if (show_progress) {
 				for (p = get_packed_git(the_repository); p;
builtin/gc.c +1 -1
index b00238cd5d..4c7409946e 100644
@@ -174,7 +174,7 @@ static int too_many_packs(void)
 	if (gc_auto_pack_limit <= 0)
 		return 0;
 
-	prepare_packed_git();
+	prepare_packed_git(the_repository);
 	for (cnt = 0, p = get_packed_git(the_repository); p; p = p->next) {
 		if (!p->pack_local)
 			continue;
builtin/pack-objects.c +1 -1
index 223f2d9fc0..491ce433da 100644
@@ -3152,7 +3152,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 	if (progress && all_progress_implied)
 		progress = 2;
 
-	prepare_packed_git();
+	prepare_packed_git(the_repository);
 	if (ignore_packed_keep) {
 		struct packed_git *p;
 		for (p = get_packed_git(the_repository); p; p = p->next)
builtin/pack-redundant.c +1 -1
index b5b007e706..da6637f7fc 100644
@@ -631,7 +631,7 @@ int cmd_pack_redundant(int argc, const char **argv, const char *prefix)
 			break;
 	}
 
-	prepare_packed_git();
+	prepare_packed_git(the_repository);
 
 	if (load_all_packs)
 		load_all();
fast-import.c +1 -1
index ae4ced3ae1..47981e6db7 100644
@@ -3473,7 +3473,7 @@ int cmd_main(int argc, const char **argv)
 		rc_free[i].next = &rc_free[i + 1];
 	rc_free[cmd_save - 1].next = NULL;
 
-	prepare_packed_git();
+	prepare_packed_git(the_repository);
 	start_packfile();
 	set_die_routine(die_nicely);
 	set_checkpoint_signal();
http-backend.c +1 -1
index 64dde78c1b..c1b1c2d557 100644
@@ -519,7 +519,7 @@ static void get_info_packs(struct strbuf *hdr, char *arg)
 	size_t cnt = 0;
 
 	select_getanyfile(hdr);
-	prepare_packed_git();
+	prepare_packed_git(the_repository);
 	for (p = get_packed_git(the_repository); p; p = p->next) {
 		if (p->pack_local)
 			cnt++;
pack-bitmap.c +1 -1
index 22cd425788..57fec38f3f 100644
@@ -336,7 +336,7 @@ static int open_pack_bitmap(void)
 
 	assert(!bitmap_git.map && !bitmap_git.loaded);
 
-	prepare_packed_git();
+	prepare_packed_git(the_repository);
 	for (p = get_packed_git(the_repository); p; p = p->next) {
 		if (open_pack_bitmap_1(p) == 0)
 			ret = 0;
packfile.c +5 -5
index 0e5fa67526..bb090f8a29 100644
@@ -817,7 +817,7 @@ unsigned long approximate_object_count(void)
 		unsigned long count;
 		struct packed_git *p;
 
-		prepare_packed_git();
+		prepare_packed_git(the_repository);
 		count = 0;
 		for (p = the_repository->objects->packed_git; p; p = p->next) {
 			if (open_pack_index(p))
@@ -884,7 +884,7 @@ static void prepare_packed_git_mru(struct repository *r)
 		list_add_tail(&p->mru, &r->objects->packed_git_mru);
 }
 
-void prepare_packed_git(void)
+void prepare_packed_git_the_repository(void)
 {
 	struct alternate_object_database *alt;
 
@@ -903,7 +903,7 @@ void reprepare_packed_git(void)
 {
 	the_repository->objects->approximate_object_count_valid = 0;
 	the_repository->objects->packed_git_initialized = 0;
-	prepare_packed_git();
+	prepare_packed_git(the_repository);
 }
 
 struct packed_git *get_packed_git(struct repository *r)
@@ -1854,7 +1854,7 @@ int find_pack_entry(const unsigned char *sha1, struct pack_entry *e)
 {
 	struct list_head *pos;
 
-	prepare_packed_git();
+	prepare_packed_git(the_repository);
 	if (!the_repository->objects->packed_git)
 		return 0;
 
@@ -1908,7 +1908,7 @@ int for_each_packed_object(each_packed_object_fn cb, void *data, unsigned flags)
 	int r = 0;
 	int pack_errors = 0;
 
-	prepare_packed_git();
+	prepare_packed_git(the_repository);
 	for (p = the_repository->objects->packed_git; p; p = p->next) {
 		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
 			continue;
packfile.h +2 -1
index 77442172f0..3f59456e7e 100644
@@ -34,7 +34,8 @@ extern struct packed_git *parse_pack_index(unsigned char *sha1, const char *idx_
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-extern void prepare_packed_git(void);
+#define prepare_packed_git(r) prepare_packed_git_##r()
+extern void prepare_packed_git_the_repository(void);
 extern void reprepare_packed_git(void);
 extern void install_packed_git(struct repository *r, struct packed_git *pack);
 
server-info.c +1 -1
index 75a8b65e47..bf5c325867 100644
@@ -201,7 +201,7 @@ static void init_pack_info(const char *infofile, int force)
 	objdir = get_object_directory();
 	objdirlen = strlen(objdir);
 
-	prepare_packed_git();
+	prepare_packed_git(the_repository);
 	for (p = get_packed_git(the_repository); p; p = p->next) {
 		/* we ignore things on alternate path since they are
 		 * not available to the pullers in general.
sha1_name.c +2 -2
index 4325f74e0c..e0b62f3c56 100644
@@ -196,7 +196,7 @@ static void find_short_packed_object(struct disambiguate_state *ds)
 {
 	struct packed_git *p;
 
-	prepare_packed_git();
+	prepare_packed_git(the_repository);
 	for (p = get_packed_git(the_repository); p && !ds->ambiguous;
 	     p = p->next)
 		unique_in_pack(p, ds);
@@ -567,7 +567,7 @@ static void find_abbrev_len_packed(struct min_abbrev_data *mad)
 {
 	struct packed_git *p;
 
-	prepare_packed_git();
+	prepare_packed_git(the_repository);
 	for (p = get_packed_git(the_repository); p; p = p->next)
 		find_abbrev_len_for_pack(p, mad);
 }
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#127 lore raw

PATCH 6/12 packfile: add repository argument to reprepare_packed_git

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

See previous patch for explanation.

---
 builtin/gc.c           | 2 +-
 builtin/receive-pack.c | 3 ++-
 bulk-checkin.c         | 3 ++-
 fetch-pack.c           | 3 ++-
 packfile.c             | 2 +-
 packfile.h             | 3 ++-
 sha1_file.c            | 2 +-
 7 files changed, 11 insertions(+), 7 deletions(-)
builtin/gc.c +1 -1
index 4c7409946e..a78dad51aa 100644
@@ -478,7 +478,7 @@ int cmd_gc(int argc, const char **argv, const char *prefix)
 		return error(FAILED_RUN, rerere.argv[0]);
 
 	report_garbage = report_pack_garbage;
-	reprepare_packed_git();
+	reprepare_packed_git(the_repository);
 	if (pack_garbage.nr > 0)
 		clean_pack_garbage();
 
builtin/receive-pack.c +2 -1
index 1a298a6711..469b916707 100644
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "repository.h"
 #include "config.h"
 #include "lockfile.h"
 #include "pack.h"
@@ -1777,7 +1778,7 @@ static const char *unpack(int err_fd, struct shallow_info *si)
 		status = finish_command(&child);
 		if (status)
 			return "index-pack abnormal exit";
-		reprepare_packed_git();
+		reprepare_packed_git(the_repository);
 	}
 	return NULL;
 }
bulk-checkin.c +2 -1
index 3310fd210a..eadc2d5172 100644
@@ -3,6 +3,7 @@
  */
 #include "cache.h"
 #include "bulk-checkin.h"
+#include "repository.h"
 #include "csum-file.h"
 #include "pack.h"
 #include "strbuf.h"
@@ -57,7 +58,7 @@ static void finish_bulk_checkin(struct bulk_checkin_state *state)
 
 	strbuf_release(&packname);
 	/* Make objects we just wrote available to ourselves */
-	reprepare_packed_git();
+	reprepare_packed_git(the_repository);
 }
 
 static int already_written(struct bulk_checkin_state *state, unsigned char sha1[])
fetch-pack.c +2 -1
index 8253d746e0..eac5928a27 100644
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "lockfile.h"
 #include "refs.h"
@@ -1192,7 +1193,7 @@ struct ref *fetch_pack(struct fetch_pack_args *args,
 	prepare_shallow_info(&si, shallow);
 	ref_cpy = do_fetch_pack(args, fd, ref, sought, nr_sought,
 				&si, pack_lockfile);
-	reprepare_packed_git();
+	reprepare_packed_git(the_repository);
 	update_shallow(args, sought, nr_sought, &si);
 	clear_shallow_info(&si);
 	return ref_cpy;
packfile.c +1 -1
index bb090f8a29..1b4296277a 100644
@@ -899,7 +899,7 @@ void prepare_packed_git_the_repository(void)
 	the_repository->objects->packed_git_initialized = 1;
 }
 
-void reprepare_packed_git(void)
+void reprepare_packed_git_the_repository(void)
 {
 	the_repository->objects->approximate_object_count_valid = 0;
 	the_repository->objects->packed_git_initialized = 0;
packfile.h +2 -1
index 3f59456e7e..ab5046938c 100644
@@ -36,7 +36,8 @@ extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
 #define prepare_packed_git(r) prepare_packed_git_##r()
 extern void prepare_packed_git_the_repository(void);
-extern void reprepare_packed_git(void);
+#define reprepare_packed_git(r) reprepare_packed_git_##r()
+extern void reprepare_packed_git_the_repository(void);
 extern void install_packed_git(struct repository *r, struct packed_git *pack);
 
 struct packed_git *get_packed_git(struct repository *r);
sha1_file.c +1 -1
index 0989bbd948..9c024cd957 100644
@@ -1274,7 +1274,7 @@ int sha1_object_info_extended(const unsigned char *sha1, struct object_info *oi,
 			return 0;
 
 		/* Not a loose object; someone else may have just packed it. */
-		reprepare_packed_git();
+		reprepare_packed_git(the_repository);
 		if (find_pack_entry(real, &e))
 			break;
 
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#128 lore raw

PATCH 7/12 packfile: allow prepare_packed_git_one to handle arbitrary repositories

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

---
 packfile.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
packfile.c +3 -4
index 1b4296277a..f49902539b 100644
@@ -735,8 +735,7 @@ static void report_pack_garbage(struct string_list *list)
 	report_helper(list, seen_bits, first, list->nr);
 }
 
-#define prepare_packed_git_one(r, o, l) prepare_packed_git_one_##r(o, l)
-static void prepare_packed_git_one_the_repository(char *objdir, int local)
+static void prepare_packed_git_one(struct repository *r, char *objdir, int local)
 {
 	struct strbuf path = STRBUF_INIT;
 	size_t dirnamelen;
@@ -769,7 +768,7 @@ static void prepare_packed_git_one_the_repository(char *objdir, int local)
 		base_len = path.len;
 		if (strip_suffix_mem(path.buf, &base_len, ".idx")) {
 			/* Don't reopen a pack we already have. */
-			for (p = the_repository->objects->packed_git; p;
+			for (p = r->objects->packed_git; p;
 			     p = p->next) {
 				size_t len;
 				if (strip_suffix(p->pack_name, ".pack", &len) &&
@@ -783,7 +782,7 @@ static void prepare_packed_git_one_the_repository(char *objdir, int local)
 			     * corresponding .pack file that we can map.
 			     */
 			    (p = add_packed_git(path.buf, path.len, local)) != NULL)
-				install_packed_git(the_repository, p);
+				install_packed_git(r, p);
 		}
 
 		if (!report_garbage)
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f
#129 lore raw

PATCH 8/12 packfile: allow prepare_packed_git to handle arbitrary repositories

In reply to Nguyễn Thái Ngọc Duy

From: Stefan Beller <[email protected]>

---
 packfile.c | 18 +++++++++---------
 packfile.h |  3 +--
 2 files changed, 10 insertions(+), 11 deletions(-)
packfile.c +9 -9
index f49902539b..ad90c61422 100644
@@ -883,19 +883,19 @@ static void prepare_packed_git_mru(struct repository *r)
 		list_add_tail(&p->mru, &r->objects->packed_git_mru);
 }
 
-void prepare_packed_git_the_repository(void)
+void prepare_packed_git(struct repository *r)
 {
 	struct alternate_object_database *alt;
 
-	if (the_repository->objects->packed_git_initialized)
+	if (r->objects->packed_git_initialized)
 		return;
-	prepare_packed_git_one(the_repository, get_object_directory(), 1);
-	prepare_alt_odb(the_repository);
-	for (alt = the_repository->objects->alt_odb_list; alt; alt = alt->next)
-		prepare_packed_git_one(the_repository, alt->path, 0);
-	rearrange_packed_git(the_repository);
-	prepare_packed_git_mru(the_repository);
-	the_repository->objects->packed_git_initialized = 1;
+	prepare_packed_git_one(r, r->objects->objectdir, 1);
+	prepare_alt_odb(r);
+	for (alt = r->objects->alt_odb_list; alt; alt = alt->next)
+		prepare_packed_git_one(r, alt->path, 0);
+	rearrange_packed_git(r);
+	prepare_packed_git_mru(r);
+	r->objects->packed_git_initialized = 1;
 }
 
 void reprepare_packed_git_the_repository(void)
packfile.h +1 -2
index ab5046938c..3fd9092472 100644
@@ -34,8 +34,7 @@ extern struct packed_git *parse_pack_index(unsigned char *sha1, const char *idx_
 #define PACKDIR_FILE_GARBAGE 4
 extern void (*report_garbage)(unsigned seen_bits, const char *path);
 
-#define prepare_packed_git(r) prepare_packed_git_##r()
-extern void prepare_packed_git_the_repository(void);
+extern void prepare_packed_git(struct repository *r);
 #define reprepare_packed_git(r) reprepare_packed_git_##r()
 extern void reprepare_packed_git_the_repository(void);
 extern void install_packed_git(struct repository *r, struct packed_git *pack);
Signature · 1 line
2.17.0.rc0.348.gd5a49e0b6f