Skip to contentSkip to conversation

Moving global state into the repository object (part 1)

PATCH Superseded Stefan Beller 9 participants 239 messages Feb 13, 2018 – Mar 3, 2018 Page 2 of 2

lore
Share & tools
lore
#152 lore raw

PATCH v3 5/27 object-store: move packed_git and packed_git_mru to object store

In reply to Stefan Beller

In a process with multiple repositories open, packfile accessors should be associated to a single repository and not shared globally. Move packed_git and packed_git_mru into the_repository and adjust callers to reflect this.

Patch generated by

1. Moving the struct packed_git declaration to object-store.h and packed_git, packed_git_mru globals to struct object_store.

2. Applying the semantic patch contrib/coccinelle/refactoring/packed_git.cocci to adjust callers. This semantic patch is placed in a sub directory of the coccinelle contrib dir, as this semantic patch is not expected to be of general usefulness; it is only useful during developing this series and merging it with other topics in flight. At a later date, just delete that semantic patch.

3. Applying line wrapping fixes from "make style" to break the resulting long lines.

4. Adding missing #includes of repository.h and object-store.h where needed.

5. As the packfiles are now owned by an objectstore/repository, which is ephemeral unlike globals, we introduce memory leaks. So address them in raw_object_store_clear().

---
 builtin/count-objects.c  |  6 ++++--
 builtin/fsck.c           |  7 +++++--
 builtin/gc.c             |  4 +++-
 builtin/index-pack.c     |  1 +
 builtin/pack-objects.c   | 20 +++++++++++---------
 builtin/pack-redundant.c |  6 ++++--
 builtin/receive-pack.c   |  1 +
 cache.h                  | 29 ----------------------------
 fast-import.c            |  6 ++++--
 http-backend.c           |  6 ++++--
 http-push.c              |  1 +
 http-walker.c            |  1 +
 http.c                   |  1 +
 object-store.h           | 41 +++++++++++++++++++++++++++++++++++++++-
 object.c                 |  7 +++++++
 pack-bitmap.c            |  4 +++-
 pack-check.c             |  1 +
 pack-revindex.c          |  1 +
 packfile.c               | 39 +++++++++++++++++++-------------------
 reachable.c              |  1 +
 repository.c             |  2 ++
 server-info.c            |  6 ++++--
 sha1_name.c              |  5 +++--
 23 files changed, 122 insertions(+), 74 deletions(-)
builtin/count-objects.c +4 -2
index 33343818c8..9334648dcc 100644
@@ -7,6 +7,8 @@
 #include "cache.h"
 #include "config.h"
 #include "dir.h"
+#include "repository.h"
+#include "object-store.h"
 #include "builtin.h"
 #include "parse-options.h"
 #include "quote.h"
@@ -120,9 +122,9 @@ int cmd_count_objects(int argc, const char **argv, const char *prefix)
 		struct strbuf loose_buf = STRBUF_INIT;
 		struct strbuf pack_buf = STRBUF_INIT;
 		struct strbuf garbage_buf = STRBUF_INIT;
-		if (!packed_git)
+		if (!the_repository->objects.packed_git)
 			prepare_packed_git();
-		for (p = packed_git; p; p = p->next) {
+		for (p = the_repository->objects.packed_git; p; p = p->next) {
 			if (!p->pack_local)
 				continue;
 			if (open_pack_index(p))
builtin/fsck.c +5 -2
index 908e4f092a..2e99e02128 100644
@@ -2,6 +2,7 @@
 #include "cache.h"
 #include "repository.h"
 #include "config.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tree.h"
 #include "blob.h"
@@ -729,7 +730,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 			prepare_packed_git();
 
 			if (show_progress) {
-				for (p = packed_git; p; p = p->next) {
+				for (p = the_repository->objects.packed_git; p;
+				     p = p->next) {
 					if (open_pack_index(p))
 						continue;
 					total += p->num_objects;
@@ -737,7 +739,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 
 				progress = start_progress(_("Checking objects"), total);
 			}
-			for (p = packed_git; p; p = p->next) {
+			for (p = the_repository->objects.packed_git; p;
+			     p = p->next) {
 				/* verify gives error messages itself */
 				if (verify_pack(p, fsck_obj_buffer,
 						progress, count))
builtin/gc.c +3 -1
index 77fa720bd0..96ff790867 100644
@@ -11,6 +11,7 @@
  */
 
 #include "builtin.h"
+#include "repository.h"
 #include "config.h"
 #include "tempfile.h"
 #include "lockfile.h"
@@ -18,6 +19,7 @@
 #include "run-command.h"
 #include "sigchain.h"
 #include "argv-array.h"
+#include "object-store.h"
 #include "commit.h"
 #include "packfile.h"
 
@@ -173,7 +175,7 @@ static int too_many_packs(void)
 		return 0;
 
 	prepare_packed_git();
-	for (cnt = 0, p = packed_git; p; p = p->next) {
+	for (cnt = 0, p = the_repository->objects.packed_git; p; p = p->next) {
 		if (!p->pack_local)
 			continue;
 		if (p->pack_keep)
builtin/index-pack.c +1 -0
index 5ebd370c56..e60cc3ed98 100644
@@ -12,6 +12,7 @@
 #include "exec_cmd.h"
 #include "streaming.h"
 #include "thread-utils.h"
+#include "object-store.h"
 #include "packfile.h"
 
 static const char index_pack_usage[] =
builtin/pack-objects.c +11 -9
index 83dcbc9773..7f376c2aef 100644
@@ -1,6 +1,8 @@
 #include "builtin.h"
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
+#include "object-store.h"
 #include "attr.h"
 #include "object.h"
 #include "blob.h"
@@ -1025,8 +1027,7 @@ static int want_object_in_pack(const struct object_id *oid,
 		if (want != -1)
 			return want;
 	}
-
-	list_for_each(pos, &packed_git_mru) {
+	list_for_each(pos, &the_repository->objects.packed_git_mru) {
 		struct packed_git *p = list_entry(pos, struct packed_git, mru);
 		off_t offset;
 
@@ -1044,7 +1045,7 @@ static int want_object_in_pack(const struct object_id *oid,
 			}
 			want = want_found_object(exclude, p);
 			if (!exclude && want > 0)
-				list_move(&p->mru, &packed_git_mru);
+				list_move(&p->mru, &the_repository->objects.packed_git_mru);
 			if (want != -1)
 				return want;
 		}
@@ -2673,7 +2674,7 @@ static void add_objects_in_unpacked_packs(struct rev_info *revs)
 
 	memset(&in_pack, 0, sizeof(in_pack));
 
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		struct object_id oid;
 		struct object *o;
 
@@ -2736,7 +2737,8 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 	static struct packed_git *last_found = (void *)1;
 	struct packed_git *p;
 
-	p = (last_found != (void *)1) ? last_found : packed_git;
+	p = (last_found != (void *)1) ? last_found :
+					the_repository->objects.packed_git;
 
 	while (p) {
 		if ((!p->pack_local || p->pack_keep) &&
@@ -2745,7 +2747,7 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 			return 1;
 		}
 		if (p == last_found)
-			p = packed_git;
+			p = the_repository->objects.packed_git;
 		else
 			p = p->next;
 		if (p == last_found)
@@ -2781,7 +2783,7 @@ static void loosen_unused_packed_objects(struct rev_info *revs)
 	uint32_t i;
 	struct object_id oid;
 
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if (!p->pack_local || p->pack_keep)
 			continue;
 
@@ -3152,7 +3154,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 	prepare_packed_git();
 	if (ignore_packed_keep) {
 		struct packed_git *p;
-		for (p = packed_git; p; p = p->next)
+		for (p = the_repository->objects.packed_git; p; p = p->next)
 			if (p->pack_local && p->pack_keep)
 				break;
 		if (!p) /* no keep-able packs found */
@@ -3165,7 +3167,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 		 * also covers non-local objects
 		 */
 		struct packed_git *p;
-		for (p = packed_git; p; p = p->next) {
+		for (p = the_repository->objects.packed_git; p; p = p->next) {
 			if (!p->pack_local) {
 				have_non_local_packs = 1;
 				break;
builtin/pack-redundant.c +4 -2
index aaa8136322..55462bc826 100644
@@ -7,6 +7,8 @@
 */
 
 #include "builtin.h"
+#include "repository.h"
+#include "object-store.h"
 #include "packfile.h"
 
 #define BLKSIZE 512
@@ -571,7 +573,7 @@ static struct pack_list * add_pack(struct packed_git *p)
 
 static struct pack_list * add_pack_file(const char *filename)
 {
-	struct packed_git *p = packed_git;
+	struct packed_git *p = the_repository->objects.packed_git;
 
 	if (strlen(filename) < 40)
 		die("Bad pack filename: %s", filename);
@@ -586,7 +588,7 @@ static struct pack_list * add_pack_file(const char *filename)
 
 static void load_all(void)
 {
-	struct packed_git *p = packed_git;
+	struct packed_git *p = the_repository->objects.packed_git;
 
 	while (p) {
 		add_pack(p);
builtin/receive-pack.c +1 -0
index b7ce7c7f52..b2eac79a6e 100644
@@ -7,6 +7,7 @@
 #include "sideband.h"
 #include "run-command.h"
 #include "exec_cmd.h"
+#include "object-store.h"
 #include "commit.h"
 #include "object.h"
 #include "remote.h"
cache.h +0 -29
index 964873faf2..3fd9dc52cd 100644
@@ -1585,35 +1585,6 @@ struct pack_window {
 	unsigned int inuse_cnt;
 };
 
-extern struct packed_git {
-	struct packed_git *next;
-	struct list_head mru;
-	struct pack_window *windows;
-	off_t pack_size;
-	const void *index_data;
-	size_t index_size;
-	uint32_t num_objects;
-	uint32_t num_bad_objects;
-	unsigned char *bad_object_sha1;
-	int index_version;
-	time_t mtime;
-	int pack_fd;
-	unsigned pack_local:1,
-		 pack_keep:1,
-		 freshened:1,
-		 do_not_close:1,
-		 pack_promisor:1;
-	unsigned char sha1[20];
-	struct revindex_entry *revindex;
-	/* something like ".git/objects/pack/xxxxx.pack" */
-	char pack_name[FLEX_ARRAY]; /* more */
-} *packed_git;
-
-/*
- * A most-recently-used ordered version of the packed_git list.
- */
-extern struct list_head packed_git_mru;
-
 struct pack_entry {
 	off_t offset;
 	unsigned char sha1[20];
fast-import.c +4 -2
index b70ac025e0..1685fe59a2 100644
@@ -154,8 +154,10 @@ Format of STDIN stream:
 
 #include "builtin.h"
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "lockfile.h"
+#include "object-store.h"
 #include "object.h"
 #include "blob.h"
 #include "tree.h"
@@ -1110,7 +1112,7 @@ static int store_object(
 	if (e->idx.offset) {
 		duplicate_count_by_type[type]++;
 		return 1;
-	} else if (find_sha1_pack(oid.hash, packed_git)) {
+	} else if (find_sha1_pack(oid.hash, the_repository->objects.packed_git)) {
 		e->type = type;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
@@ -1305,7 +1307,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
 		duplicate_count_by_type[OBJ_BLOB]++;
 		truncate_pack(&checkpoint);
 
-	} else if (find_sha1_pack(oid.hash, packed_git)) {
+	} else if (find_sha1_pack(oid.hash, the_repository->objects.packed_git)) {
 		e->type = OBJ_BLOB;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
http-backend.c +4 -2
index f3dc218b2a..4d93126c15 100644
@@ -1,7 +1,9 @@
 #include "cache.h"
 #include "config.h"
+#include "repository.h"
 #include "refs.h"
 #include "pkt-line.h"
+#include "object-store.h"
 #include "object.h"
 #include "tag.h"
 #include "exec_cmd.h"
@@ -518,13 +520,13 @@ static void get_info_packs(struct strbuf *hdr, char *arg)
 
 	select_getanyfile(hdr);
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if (p->pack_local)
 			cnt++;
 	}
 
 	strbuf_grow(&buf, cnt * 53 + 2);
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if (p->pack_local)
 			strbuf_addf(&buf, "P %s\n", p->pack_name + objdirlen + 6);
 	}
http-push.c +1 -0
index 0913f8ab86..1ca662390f 100644
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tag.h"
 #include "blob.h"
http-walker.c +1 -0
index 07c2b1af82..8bb5d991bb 100644
@@ -4,6 +4,7 @@
 #include "http.h"
 #include "list.h"
 #include "transport.h"
+#include "object-store.h"
 #include "packfile.h"
 
 struct alt_base {
http.c +1 -0
index 31755023a4..a4a9e583c7 100644
@@ -1,6 +1,7 @@
 #include "git-compat-util.h"
 #include "http.h"
 #include "config.h"
+#include "object-store.h"
 #include "pack.h"
 #include "sideband.h"
 #include "run-command.h"
object-store.h +40 -1

56 lines not on this page · raw message

object.c +7 -0
index 17b1da6d6b..c76b62572a 100644
@@ -466,4 +466,11 @@ void raw_object_store_clear(struct raw_object_store *o)
 
 	free_alt_odbs(o);
 	o->alt_odb_tail = NULL;
+
+	while (!list_empty(&o->packed_git_mru))
+		list_del(&o->packed_git_mru);
+	/*
+	 * TODO: call close_all_packs once migrated to
+	 * take an object store argument
+	 */
 }
pack-bitmap.c +3 -1
index 9270983e5f..d51172b1d5 100644
@@ -5,11 +5,13 @@
 #include "revision.h"
 #include "progress.h"
 #include "list-objects.h"
+#include "object-store.h"
 #include "pack.h"
 #include "pack-bitmap.h"
 #include "pack-revindex.h"
 #include "pack-objects.h"
 #include "packfile.h"
+#include "repository.h"
 
 /*
  * An entry on the bitmap index, representing the bitmap for a given
@@ -335,7 +337,7 @@ static int open_pack_bitmap(void)
 	assert(!bitmap_git.map && !bitmap_git.loaded);
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if (open_pack_bitmap_1(p) == 0)
 			ret = 0;
 	}
pack-check.c +1 -0
index 073c1fbd46..c80c3ed8fd 100644
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "pack.h"
+#include "object-store.h"
 #include "pack-revindex.h"
 #include "progress.h"
 #include "packfile.h"
pack-revindex.c +1 -0
index ff5f62c033..bb521cf7fb 100644
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "pack-revindex.h"
+#include "object-store.h"
 
 /*
  * Pack index for existing packs give us easy access to the offsets into
packfile.c +20 -19
index 216ea836ee..d41e4c83d0 100644
@@ -7,6 +7,7 @@
 #include "packfile.h"
 #include "delta.h"
 #include "list.h"
+#include "object-store.h"
 #include "streaming.h"
 #include "sha1-lookup.h"
 #include "commit.h"
@@ -45,8 +46,6 @@ static unsigned int pack_open_fds;
 static unsigned int pack_max_fds;
 static size_t peak_pack_mapped;
 static size_t pack_mapped;
-struct packed_git *packed_git;
-LIST_HEAD(packed_git_mru);
 
 #define SZ_FMT PRIuMAX
 static inline uintmax_t sz_fmt(size_t s) { return s; }
@@ -246,7 +245,7 @@ static int unuse_one_window(struct packed_git *current)
 
 	if (current)
 		scan_windows(current, &lru_p, &lru_w, &lru_l);
-	for (p = packed_git; p; p = p->next)
+	for (p = the_repository->objects.packed_git; p; p = p->next)
 		scan_windows(p, &lru_p, &lru_w, &lru_l);
 	if (lru_p) {
 		munmap(lru_w->base, lru_w->len);
@@ -316,7 +315,7 @@ void close_all_packs(void)
 {
 	struct packed_git *p;
 
-	for (p = packed_git; p; p = p->next)
+	for (p = the_repository->objects.packed_git; p; p = p->next)
 		if (p->do_not_close)
 			die("BUG: want to close pack marked 'do-not-close'");
 		else
@@ -384,7 +383,7 @@ static int close_one_pack(void)
 	struct pack_window *mru_w = NULL;
 	int accept_windows_inuse = 1;
 
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if (p->pack_fd == -1)
 			continue;
 		find_lru_pack(p, &lru_p, &mru_w, &accept_windows_inuse);
@@ -686,8 +685,8 @@ void install_packed_git(struct packed_git *pack)
 	if (pack->pack_fd != -1)
 		pack_open_fds++;
 
-	pack->next = packed_git;
-	packed_git = pack;
+	pack->next = the_repository->objects.packed_git;
+	the_repository->objects.packed_git = pack;
 }
 
 void (*report_garbage)(unsigned seen_bits, const char *path);
@@ -769,7 +768,8 @@ static void prepare_packed_git_one(char *objdir, int local)
 		base_len = path.len;
 		if (strip_suffix_mem(path.buf, &base_len, ".idx")) {
 			/* Don't reopen a pack we already have. */
-			for (p = packed_git; p; p = p->next) {
+			for (p = the_repository->objects.packed_git; p;
+			     p = p->next) {
 				size_t len;
 				if (strip_suffix(p->pack_name, ".pack", &len) &&
 				    len == base_len &&
@@ -820,7 +820,7 @@ unsigned long approximate_object_count(void)
 
 		prepare_packed_git();
 		count = 0;
-		for (p = packed_git; p; p = p->next) {
+		for (p = the_repository->objects.packed_git; p; p = p->next) {
 			if (open_pack_index(p))
 				continue;
 			count += p->num_objects;
@@ -869,18 +869,19 @@ static int sort_pack(const void *a_, const void *b_)
 
 static void rearrange_packed_git(void)
 {
-	packed_git = llist_mergesort(packed_git, get_next_packed_git,
-				     set_next_packed_git, sort_pack);
+	the_repository->objects.packed_git = llist_mergesort(
+		the_repository->objects.packed_git, get_next_packed_git,
+		set_next_packed_git, sort_pack);
 }
 
 static void prepare_packed_git_mru(void)
 {
 	struct packed_git *p;
 
-	INIT_LIST_HEAD(&packed_git_mru);
+	INIT_LIST_HEAD(&the_repository->objects.packed_git_mru);
 
-	for (p = packed_git; p; p = p->next)
-		list_add_tail(&p->mru, &packed_git_mru);
+	for (p = the_repository->objects.packed_git; p; p = p->next)
+		list_add_tail(&p->mru, &the_repository->objects.packed_git_mru);
 }
 
 static int prepare_packed_git_run_once = 0;
@@ -1014,7 +1015,7 @@ const struct packed_git *has_packed_and_bad(const unsigned char *sha1)
 	struct packed_git *p;
 	unsigned i;
 
-	for (p = packed_git; p; p = p->next)
+	for (p = the_repository->objects.packed_git; p; p = p->next)
 		for (i = 0; i < p->num_bad_objects; i++)
 			if (!hashcmp(sha1, p->bad_object_sha1 + 20 * i))
 				return p;
@@ -1845,13 +1846,13 @@ int find_pack_entry(const unsigned char *sha1, struct pack_entry *e)
 	struct list_head *pos;
 
 	prepare_packed_git();
-	if (!packed_git)
+	if (!the_repository->objects.packed_git)
 		return 0;
 
-	list_for_each(pos, &packed_git_mru) {
+	list_for_each(pos, &the_repository->objects.packed_git_mru) {
 		struct packed_git *p = list_entry(pos, struct packed_git, mru);
 		if (fill_pack_entry(sha1, e, p)) {
-			list_move(&p->mru, &packed_git_mru);
+			list_move(&p->mru, &the_repository->objects.packed_git_mru);
 			return 1;
 		}
 	}
@@ -1898,7 +1899,7 @@ int for_each_packed_object(each_packed_object_fn cb, void *data, unsigned flags)
 	int pack_errors = 0;
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
 			continue;
 		if ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&
reachable.c +1 -0
index 88d7d679da..59ed16ba76 100644
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "refs.h"
+#include "object-store.h"
 #include "tag.h"
 #include "commit.h"
 #include "blob.h"
repository.c +2 -0
index 2255ff657e..9798f4ffca 100644
@@ -142,6 +142,8 @@ int repo_init(struct repository *repo, const char *gitdir, const char *worktree)
 
 	repo->ignore_env = 1;
 
+	INIT_LIST_HEAD(&repo->objects.packed_git_mru);
+
 	if (repo_init_gitdir(repo, gitdir))
 		goto error;
 
server-info.c +4 -2
index 26a6c20b7d..96885c45f6 100644
@@ -1,5 +1,7 @@
 #include "cache.h"
+#include "repository.h"
 #include "refs.h"
+#include "object-store.h"
 #include "object.h"
 #include "commit.h"
 #include "tag.h"
@@ -200,7 +202,7 @@ static void init_pack_info(const char *infofile, int force)
 	objdirlen = strlen(objdir);
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		/* we ignore things on alternate path since they are
 		 * not available to the pullers in general.
 		 */
@@ -210,7 +212,7 @@ static void init_pack_info(const char *infofile, int force)
 	}
 	num_pack = i;
 	info = xcalloc(num_pack, sizeof(struct pack_info *));
-	for (i = 0, p = packed_git; p; p = p->next) {
+	for (i = 0, p = the_repository->objects.packed_git; p; p = p->next) {
 		if (!p->pack_local)
 			continue;
 		info[i] = xcalloc(1, sizeof(struct pack_info));
sha1_name.c +3 -2
index 957ce25680..016c883b5c 100644
@@ -196,7 +196,8 @@ static void find_short_packed_object(struct disambiguate_state *ds)
 	struct packed_git *p;
 
 	prepare_packed_git();
-	for (p = packed_git; p && !ds->ambiguous; p = p->next)
+	for (p = the_repository->objects.packed_git; p && !ds->ambiguous;
+	     p = p->next)
 		unique_in_pack(p, ds);
 }
 
@@ -566,7 +567,7 @@ static void find_abbrev_len_packed(struct min_abbrev_data *mad)
 	struct packed_git *p;
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next)
+	for (p = the_repository->objects.packed_git; p; p = p->next)
 		find_abbrev_len_for_pack(p, mad);
 }
 
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#153 lore raw

Re: [PATCH 05/27] object-store: move packed_git and packed_git_mru to object store

object-store.h struct raw_object_store { · line 104

Quoted 9 lines · Stefan Beller
object-store.h +9 -0
+
+/*
+ * The mru list_head is supposed to be initialized using
+ * the LIST_HEAD macro, assigning prev/next to itself.
+ * However this doesn't work in this case as some compilers dislike
+ * that macro on member variables. Use NULL instead as that is defined
+ * and accepted, deferring the real init to prepare_packed_git_mru(). */
+#define __MRU_INIT { NULL, NULL }
+#define RAW_OBJECT_STORE_INIT { NULL, NULL, __MRU_INIT, NULL, NULL }

I do not think it has to be this way to abuse two NULLs, if you faithfully mimicked how LIST_HEAD() macro is constructed. The reason why it does not try to introduce

	struct list_head x = LIST_HEAD_INIT;

and instead, uses

	LIST_HEAD(x);

is because of the need for self referral. If we learn from it, we can do the same, i.e. instead of doing

	struct raw_object_store x = RAW_OBJECT_STORE_INIT;

we can do

	RAW_OBJECT_STORE(x);

that expands to

	struct raw_object_store x = {
		...
		{ &x.packed_git_mru, &x.packed_git_mru },
		...
	};

no? Then we do not need such a lengthy comment that reads only like an excuse ;-)

#154 lore raw

Re: [PATCH 05/27] object-store: move packed_git and packed_git_mru to object store

Quoted 33 lines · Junio C Hamano

object-store.h struct raw_object_store { · line 104

Quoted 9 lines · Stefan Beller
object-store.h +9 -0
+
+/*
+ * The mru list_head is supposed to be initialized using
+ * the LIST_HEAD macro, assigning prev/next to itself.
+ * However this doesn't work in this case as some compilers dislike
+ * that macro on member variables. Use NULL instead as that is defined
+ * and accepted, deferring the real init to prepare_packed_git_mru(). */
+#define __MRU_INIT { NULL, NULL }
+#define RAW_OBJECT_STORE_INIT { NULL, NULL, __MRU_INIT, NULL, NULL }

I do not think it has to be this way to abuse two NULLs, if you faithfully mimicked how LIST_HEAD() macro is constructed. The reason why it does not try to introduce

        struct list_head x = LIST_HEAD_INIT;

and instead, uses

        LIST_HEAD(x);

is because of the need for self referral. If we learn from it, we can do the same, i.e. instead of doing

        struct raw_object_store x = RAW_OBJECT_STORE_INIT;

we can do

        RAW_OBJECT_STORE(x);

that expands to

Quoted 8 lines
        struct raw_object_store x = {
                ...
                { &x.packed_git_mru, &x.packed_git_mru },
                ...
        };

no? Then we do not need such a lengthy comment that reads only like an excuse ;-)

We cannot do this, because the object store is directly embedded into the the_repository (not as a pointer), which is a global on its own. So we'd have to do this at the repository level, which I would not want. I'd want to have the #defines where the structs are declared.

Any guidance on how to do that correctly with self referential definitions without making the object store a pointer then?

#155 lore raw

Re: [PATCH 05/27] object-store: move packed_git and packed_git_mru to object store

In reply to Stefan Beller

Hi,

Quoted 10 lines · Stefan Beller

In a process with multiple repositories open, packfile accessors should be associated to a single repository and not shared globally. Move packed_git and packed_git_mru into the_repository and adjust callers to reflect this.

Patch generated by

1. Moving the struct packed_git declaration to object-store.h and packed_git, packed_git_mru globals to struct object_store.

Quoted 7 lines

2. Applying the semantic patch contrib/coccinelle/refactoring/packed_git.cocci to adjust callers. This semantic patch is placed in a sub directory of the coccinelle contrib dir, as this semantic patch is not expected to be of general usefulness; it is only useful during developing this series and merging it with other topics in flight. At a later date, just delete that semantic patch.

Can the semantic patch go in the commit message instead? It is very brief.

Actually, I don't see this semantic patch in the diffstat. Is the commit message stale?

Quoted 5 lines

3. Applying line wrapping fixes from "make style" to break the resulting long lines.

4. Adding missing #includes of repository.h and object-store.h where needed.

Is there a way to automate this step? (I'm asking for my own reference when writing future patches, not because of any concern about the correctness of this one.)

Quoted 4 lines

5. As the packfiles are now owned by an objectstore/repository, which is ephemeral unlike globals, we introduce memory leaks. So address them in raw_object_store_clear().

The compound words are confusing me. What is an objectstore/repository? Are these referring to particular identifiers or something else?

Would some wording like the following work?

5. Freeing packed_git and packed_git_mru in raw_object_store_clear to avoid a per-repository memory leak. Previously they were global singletons, so code to free them did not exist.

[...]

builtin/index-pack.c line 18

Quoted 10 lines
builtin/index-pack.c +1 -0
@@ -12,6 +12,7 @@
 #include "exec_cmd.h"
 #include "streaming.h"
 #include "thread-utils.h"
+#include "object-store.h"
 #include "packfile.h"
 
 static const char index_pack_usage[] =

Change from a different patch leaked into this one?

[...]

Quoted 1 line

+++ b/builtin/pack-objects.c

[...]

builtin/index-pack.c static int want_object_in_pack(const struct object_id *oid, · line 1048

Quoted 6 lines
builtin/index-pack.c +1 -1
@@ -1044,7 +1045,7 @@ static int want_object_in_pack(const struct object_id *oid,
 			}
 			want = want_found_object(exclude, p);
 			if (!exclude && want > 0)
-				list_move(&p->mru, &packed_git_mru);
+				list_move(&p->mru, &the_repository->objects.packed_git_mru);

Long line. Can "make style" catch this?

[...]

builtin/index-pack.c line 13

Quoted 9 lines
builtin/index-pack.c +2 -0
+++ b/builtin/receive-pack.c
@@ -7,6 +7,7 @@
 #include "sideband.h"
 #include "run-command.h"
 #include "exec_cmd.h"
+#include "object-store.h"
 #include "commit.h"
 #include "object.h"
 #include "remote.h"

Another change leaked in?

[...]

Quoted 7 lines
cache.h +0 -1
@@ -1585,35 +1585,6 @@ struct pack_window {
 	unsigned int inuse_cnt;
 };
 
-extern struct packed_git {

[...]

Quoted 1 line

-} *packed_git;

Move detecting diff confirms that this wasn't modified. Thanks for creating it.

[...]

Quoted 1 line

+++ b/fast-import.c

[...]

cache.h static int store_object( · line 1115

Quoted 6 lines
cache.h +1 -1
@@ -1110,7 +1112,7 @@ static int store_object(
 	if (e->idx.offset) {
 		duplicate_count_by_type[type]++;
 		return 1;
-	} else if (find_sha1_pack(oid.hash, packed_git)) {
+	} else if (find_sha1_pack(oid.hash, the_repository->objects.packed_git)) {

Long line. (I'll refrain from commenting about any further ones.)

[...]

cache.h line 5

Quoted 7 lines
cache.h +2 -0
+++ b/http-push.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tag.h"
 #include "blob.h"

Stray change?

http-walker.c line 10

Quoted 12 lines
http-walker.c +1 -0
index 07c2b1af82..8bb5d991bb 100644
@@ -4,6 +4,7 @@
 #include "http.h"
 #include "list.h"
 #include "transport.h"
+#include "object-store.h"
 #include "packfile.h"
 
 struct alt_base {

Same question.

http.c line 7

Quoted 12 lines
http.c +1 -0
index 31755023a4..a4a9e583c7 100644
@@ -1,6 +1,7 @@
 #include "git-compat-util.h"
 #include "http.h"
 #include "config.h"
+#include "object-store.h"
 #include "pack.h"
 #include "sideband.h"
 #include "run-command.h"

Likewise.

Quoted 36 lines
object-store.h +24 -0
index e78eea1dde..1de9e07102 100644
@@ -52,6 +52,30 @@ void add_to_alternates_memory(const char *dir);
  */
 struct strbuf *alt_scratch_buf(struct alternate_object_database *alt);
 
+struct packed_git {
+	struct packed_git *next;
+	struct list_head mru;
+	struct pack_window *windows;
+	off_t pack_size;
+	const void *index_data;
+	size_t index_size;
+	uint32_t num_objects;
+	uint32_t num_bad_objects;
+	unsigned char *bad_object_sha1;
+	int index_version;
+	time_t mtime;
+	int pack_fd;
+	unsigned pack_local:1,
+		 pack_keep:1,
+		 freshened:1,
+		 do_not_close:1,
+		 pack_promisor:1;
+	unsigned char sha1[20];
+	struct revindex_entry *revindex;
+	/* something like ".git/objects/pack/xxxxx.pack" */
+	char pack_name[FLEX_ARRAY]; /* more */
+};
+
 struct raw_object_store {
 	/*
 	 * Path to the repository's object store.

object-store.h struct raw_object_store { · line 91

Quoted 9 lines
object-store.h +6 -0
@@ -59,10 +83,25 @@ struct raw_object_store {
 	 */
 	char *objectdir;
 
+	struct packed_git *packed_git;
+	/*
+	 * A most-recently-used ordered version of the packed_git list, which can
+	 * be iterated instead of packed_git (and marked via mru_mark).
+	 */
+	struct list_head packed_git_mru;

I don't understand the new part of the comment. Can you explain here, for me?

Is this meant as a list of related functions, an explanation of what the field is for, or something else?

object-store.h struct raw_object_store { · line 102

Quoted 12 lines
object-store.h +8 -1
+
 	struct alternate_object_database *alt_odb_list;
 	struct alternate_object_database **alt_odb_tail;
 };
-#define RAW_OBJECT_STORE_INIT { NULL, NULL, NULL }
+
+/*
+ * The mru list_head is supposed to be initialized using
+ * the LIST_HEAD macro, assigning prev/next to itself.
+ * However this doesn't work in this case as some compilers dislike
+ * that macro on member variables. Use NULL instead as that is defined
+ * and accepted, deferring the real init to prepare_packed_git_mru(). */

style nit: '*/' should be on its own line.

More importantly, we can avoid such an issue as described by Junio. :)

object-store.h struct raw_object_store { · line 103

Quoted 1 line
object-store.h +1 -0
+#define __MRU_INIT { NULL, NULL }

Identifiers with leading underscores like this are in a reserved namespace for the language implementation --- we can't count on them being available for our own use.

Quoted 1 line
object-store.h +1 -0
+#define RAW_OBJECT_STORE_INIT { NULL, NULL, __MRU_INIT, NULL, NULL }

[...]

object.c void raw_object_store_clear(struct raw_object_store *o) · line 475

Quoted 13 lines
object.c +7 -0
@@ -466,4 +466,11 @@ void raw_object_store_clear(struct raw_object_store *o)
 
 	free_alt_odbs(o);
 	o->alt_odb_tail = NULL;
+
+	while (!list_empty(&o->packed_git_mru))
+		list_del(&o->packed_git_mru);
+	/*
+	 * TODO: call close_all_packs once migrated to
+	 * take an object store argument
+	 */

Can you say more about this TODO? Does this mean the patches are out of order (i.e. that raw_object_store_clear leaves behind a leak until a later patch)?

[...]

pack-check.c line 6

Quoted 9 lines
pack-check.c +1 -0
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "pack.h"
+#include "object-store.h"
 #include "pack-revindex.h"
 #include "progress.h"
 #include "packfile.h"

Another unexplained #include (I'll refrain from pointing out later ones).

The rest looks good.

Thanks,
Jonathan

#156 lore raw

Re: [PATCH 05/27] object-store: move packed_git and packed_git_mru to object store

Quoted 10 lines
Quoted 7 lines

2. Applying the semantic patch contrib/coccinelle/refactoring/packed_git.cocci to adjust callers. This semantic patch is placed in a sub directory of the coccinelle contrib dir, as this semantic patch is not expected to be of general usefulness; it is only useful during developing this series and merging it with other topics in flight. At a later date, just delete that semantic patch.

Can the semantic patch go in the commit message instead? It is very brief.

done

Quoted 7 lines

Actually, I don't see this semantic patch in the diffstat. Is the commit message stale?

Quoted 3 lines

3. Applying line wrapping fixes from "make style" to break the resulting long lines.

Quoted 6 lines
Quoted 2 lines

4. Adding missing #includes of repository.h and object-store.h where needed.

Is there a way to automate this step? (I'm asking for my own reference when writing future patches, not because of any concern about the correctness of this one.)

no, for several reasons.
(a) I don't know how to write it in coccinelle, because
(b) I have not figured out the order in which we include headers, apart from "cache.h goes first", the rest of the includes sometimes looks like a random order, because different patches add new includes at different places. I have the impression, that (1) some add the include after all other existing includes or (2) try to figure out where to add the include to make sense in the existing include order or (3) sort alphabetically or (4) put it randomly, so the chance for a merge conflict with other series in flight decreases.
(c) I did it in a semi automated fashion:
while make fails:
add another include

The mess with including repository or object-store comes from the fact that I had v2 based on object-store, not the repository and cherry-picked this patch over to v3. Fixed all of the includes now.

Quoted 4 lines
Quoted 4 lines
object-store.h +0 -0
@@ -59,10 +83,25 @@ struct raw_object_store {
       */
      char *objectdir;
 
Quoted 9 lines

object-store.h struct raw_object_store { · line 91

Quoted 6 lines
object-store.h +6 -0
+     struct packed_git *packed_git;
+     /*
+      * A most-recently-used ordered version of the packed_git list, which can
+      * be iterated instead of packed_git (and marked via mru_mark).
+      */
+     struct list_head packed_git_mru;

I don't understand the new part of the comment. Can you explain here, for me?

cherrypicking error, fixed.

Quoted 11 lines

object-store.h struct raw_object_store { · line 102

Quoted 8 lines
object-store.h +7 -1
-#define RAW_OBJECT_STORE_INIT { NULL, NULL, NULL }
+
+/*
+ * The mru list_head is supposed to be initialized using
+ * the LIST_HEAD macro, assigning prev/next to itself.
+ * However this doesn't work in this case as some compilers dislike
+ * that macro on member variables. Use NULL instead as that is defined
+ * and accepted, deferring the real init to prepare_packed_git_mru(). */

style nit: '*/' should be on its own line.

Quoted 1 line

More importantly, we can avoid such an issue as described by Junio. :)

See reply to Junio, I am not quite sure I like that.

#157 lore raw

PATCH v3 6/27 object-store: close all packs upon clearing the object store

In reply to Stefan Beller
---
 builtin/am.c           | 2 +-
 builtin/clone.c        | 2 +-
 builtin/fetch.c        | 2 +-
 builtin/merge.c        | 2 +-
 builtin/receive-pack.c | 2 +-
 object.c               | 6 ++----
 packfile.c             | 4 ++--
 packfile.h             | 2 +-
 8 files changed, 10 insertions(+), 12 deletions(-)
builtin/am.c +1 -1
index 5bdd2d7578..4762a702e3 100644
@@ -1859,7 +1859,7 @@ static void am_run(struct am_state *state, int resume)
 	 */
 	if (!state->rebasing) {
 		am_destroy(state);
-		close_all_packs();
+		close_all_packs(&the_repository->objects);
 		run_command_v_opt(argv_gc_auto, RUN_GIT_CMD);
 	}
 }
builtin/clone.c +1 -1
index 101c27a593..13cfaa6488 100644
@@ -1217,7 +1217,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 	transport_disconnect(transport);
 
 	if (option_dissociate) {
-		close_all_packs();
+		close_all_packs(&the_repository->objects);
 		dissociate_from_references();
 	}
 
builtin/fetch.c +1 -1
index 8ee998ea2e..4d72efca78 100644
@@ -1478,7 +1478,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
 
 	string_list_clear(&list, 0);
 
-	close_all_packs();
+	close_all_packs(&the_repository->objects);
 
 	argv_array_pushl(&argv_gc_auto, "gc", "--auto", NULL);
 	if (verbosity < 0)
builtin/merge.c +1 -1
index 30264cfd7c..907ae44ab5 100644
@@ -411,7 +411,7 @@ static void finish(struct commit *head_commit,
 			 * We ignore errors in 'gc --auto', since the
 			 * user should see them.
 			 */
-			close_all_packs();
+			close_all_packs(&the_repository->objects);
 			run_command_v_opt(argv_gc_auto, RUN_GIT_CMD);
 		}
 	}
builtin/receive-pack.c +1 -1
index b2eac79a6e..954fc72c7c 100644
@@ -2027,7 +2027,7 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)
 			proc.git_cmd = 1;
 			proc.argv = argv_gc_auto;
 
-			close_all_packs();
+			close_all_packs(&the_repository->objects);
 			if (!start_command(&proc)) {
 				if (use_sideband)
 					copy_to_sideband(proc.err, -1, NULL);
object.c +2 -4
index c76b62572a..34daaf37b3 100644
@@ -4,6 +4,7 @@
 #include "tree.h"
 #include "commit.h"
 #include "tag.h"
+#include "packfile.h"
 
 static struct object **obj_hash;
 static int nr_objs, obj_hash_size;
@@ -469,8 +470,5 @@ void raw_object_store_clear(struct raw_object_store *o)
 
 	while (!list_empty(&o->packed_git_mru))
 		list_del(&o->packed_git_mru);
-	/*
-	 * TODO: call close_all_packs once migrated to
-	 * take an object store argument
-	 */
+	close_all_packs(o);
 }
packfile.c +2 -2
index d41e4c83d0..511a2b0cdf 100644
@@ -311,11 +311,11 @@ static void close_pack(struct packed_git *p)
 	close_pack_index(p);
 }
 
-void close_all_packs(void)
+void close_all_packs(struct raw_object_store *o)
 {
 	struct packed_git *p;
 
-	for (p = the_repository->objects.packed_git; p; p = p->next)
+	for (p = o->packed_git; p; p = p->next)
 		if (p->do_not_close)
 			die("BUG: want to close pack marked 'do-not-close'");
 		else
packfile.h +1 -1
index a7fca598d6..6a2c57045c 100644
@@ -63,7 +63,7 @@ extern void close_pack_index(struct packed_git *);
 
 extern unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t, unsigned long *);
 extern void close_pack_windows(struct packed_git *);
-extern void close_all_packs(void);
+extern void close_all_packs(struct raw_object_store *o);
 extern void unuse_pack(struct pack_window **);
 extern void clear_delta_base_cache(void);
 extern struct packed_git *add_packed_git(const char *path, size_t path_len, int local);
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#158 lore raw

Re: [PATCH 06/27] object-store: close all packs upon clearing the object store

Quoted 1 line · Stefan Beller

Straight forward change, looks good.

Quoted 132 lines
---
 builtin/am.c           | 2 +-
 builtin/clone.c        | 2 +-
 builtin/fetch.c        | 2 +-
 builtin/merge.c        | 2 +-
 builtin/receive-pack.c | 2 +-
 object.c               | 6 ++----
 packfile.c             | 4 ++--
 packfile.h             | 2 +-
 8 files changed, 10 insertions(+), 12 deletions(-)
builtin/am.c +1 -1
index 5bdd2d7578..4762a702e3 100644
@@ -1859,7 +1859,7 @@ static void am_run(struct am_state *state, int resume)
 	 */
 	if (!state->rebasing) {
 		am_destroy(state);
-		close_all_packs();
+		close_all_packs(&the_repository->objects);
 		run_command_v_opt(argv_gc_auto, RUN_GIT_CMD);
 	}
 }
builtin/clone.c +1 -1
index 101c27a593..13cfaa6488 100644
@@ -1217,7 +1217,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 	transport_disconnect(transport);
 
 	if (option_dissociate) {
-		close_all_packs();
+		close_all_packs(&the_repository->objects);
 		dissociate_from_references();
 	}
 
builtin/fetch.c +1 -1
index 8ee998ea2e..4d72efca78 100644
@@ -1478,7 +1478,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
 
 	string_list_clear(&list, 0);
 
-	close_all_packs();
+	close_all_packs(&the_repository->objects);
 
 	argv_array_pushl(&argv_gc_auto, "gc", "--auto", NULL);
 	if (verbosity < 0)
builtin/merge.c +1 -1
index 30264cfd7c..907ae44ab5 100644
@@ -411,7 +411,7 @@ static void finish(struct commit *head_commit,
 			 * We ignore errors in 'gc --auto', since the
 			 * user should see them.
 			 */
-			close_all_packs();
+			close_all_packs(&the_repository->objects);
 			run_command_v_opt(argv_gc_auto, RUN_GIT_CMD);
 		}
 	}
builtin/receive-pack.c +1 -1
index b2eac79a6e..954fc72c7c 100644
@@ -2027,7 +2027,7 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)
 			proc.git_cmd = 1;
 			proc.argv = argv_gc_auto;
 
-			close_all_packs();
+			close_all_packs(&the_repository->objects);
 			if (!start_command(&proc)) {
 				if (use_sideband)
 					copy_to_sideband(proc.err, -1, NULL);
object.c +2 -4
index c76b62572a..34daaf37b3 100644
@@ -4,6 +4,7 @@
 #include "tree.h"
 #include "commit.h"
 #include "tag.h"
+#include "packfile.h"
 
 static struct object **obj_hash;
 static int nr_objs, obj_hash_size;
@@ -469,8 +470,5 @@ void raw_object_store_clear(struct raw_object_store *o)
 
 	while (!list_empty(&o->packed_git_mru))
 		list_del(&o->packed_git_mru);
-	/*
-	 * TODO: call close_all_packs once migrated to
-	 * take an object store argument
-	 */
+	close_all_packs(o);
 }
packfile.c +2 -2
index d41e4c83d0..511a2b0cdf 100644
@@ -311,11 +311,11 @@ static void close_pack(struct packed_git *p)
 	close_pack_index(p);
 }
 
-void close_all_packs(void)
+void close_all_packs(struct raw_object_store *o)
 {
 	struct packed_git *p;
 
-	for (p = the_repository->objects.packed_git; p; p = p->next)
+	for (p = o->packed_git; p; p = p->next)
 		if (p->do_not_close)
 			die("BUG: want to close pack marked 'do-not-close'");
 		else
packfile.h +1 -1
index a7fca598d6..6a2c57045c 100644
@@ -63,7 +63,7 @@ extern void close_pack_index(struct packed_git *);
 
 extern unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t, unsigned long *);
 extern void close_pack_windows(struct packed_git *);
-extern void close_all_packs(void);
+extern void close_all_packs(struct raw_object_store *o);
 extern void unuse_pack(struct pack_window **);
 extern void clear_delta_base_cache(void);
 extern struct packed_git *add_packed_git(const char *path, size_t path_len, int local);
Signature · 1 line
2.16.1.291.g4437f3f132-goog
Signature · 1 line
Brandon Williams
#159 lore raw

PATCH v3 7/27 pack: move prepare_packed_git_run_once to object store

In reply to Stefan Beller

Each repository's object store can be initialized independently, so they must not share a run_once variable.

---
 object-store.h | 8 +++++++-
 packfile.c     | 7 +++----
 2 files changed, 10 insertions(+), 5 deletions(-)
object-store.h +7 -1
index 1de9e07102..6cecba3951 100644
@@ -92,6 +92,12 @@ struct raw_object_store {
 
 	struct alternate_object_database *alt_odb_list;
 	struct alternate_object_database **alt_odb_tail;
+
+	/*
+	 * Whether packed_git has already been populated with this repository's
+	 * packs.
+	 */
+	unsigned packed_git_initialized : 1;
 };
 
 /*
@@ -101,7 +107,7 @@ struct raw_object_store {
  * that macro on member variables. Use NULL instead as that is defined
  * and accepted, deferring the real init to prepare_packed_git_mru(). */
 #define __MRU_INIT { NULL, NULL }
-#define RAW_OBJECT_STORE_INIT { NULL, NULL, __MRU_INIT, NULL, NULL }
+#define RAW_OBJECT_STORE_INIT { NULL, NULL, __MRU_INIT, NULL, NULL, 0 }
 
 void raw_object_store_clear(struct raw_object_store *o);
 
packfile.c +3 -4
index 511a2b0cdf..a8a2e7fe43 100644
@@ -884,12 +884,11 @@ static void prepare_packed_git_mru(void)
 		list_add_tail(&p->mru, &the_repository->objects.packed_git_mru);
 }
 
-static int prepare_packed_git_run_once = 0;
 void prepare_packed_git(void)
 {
 	struct alternate_object_database *alt;
 
-	if (prepare_packed_git_run_once)
+	if (the_repository->objects.packed_git_initialized)
 		return;
 	prepare_packed_git_one(get_object_directory(), 1);
 	prepare_alt_odb();
@@ -897,13 +896,13 @@ void prepare_packed_git(void)
 		prepare_packed_git_one(alt->path, 0);
 	rearrange_packed_git();
 	prepare_packed_git_mru();
-	prepare_packed_git_run_once = 1;
+	the_repository->objects.packed_git_initialized = 1;
 }
 
 void reprepare_packed_git(void)
 {
 	approximate_object_count_valid = 0;
-	prepare_packed_git_run_once = 0;
+	the_repository->objects.packed_git_initialized = 0;
 	prepare_packed_git();
 }
 
Signature · 1 line
2.16.1.291.g4437f3f132-goog
Reviewed-by on v3 7/27
#160 lore raw

Re: [PATCH 07/27] pack: move prepare_packed_git_run_once to object store

Quoted 5 lines · On Tue, 20 Feb 2018 17:54:10 -0800 Stefan Beller <[email protected]> wrote:

Each repository's object store can be initialized independently, so they must not share a run_once variable.

object-store.h struct raw_object_store { · line 110

Quoted 2 lines
object-store.h +1 -1
-#define RAW_OBJECT_STORE_INIT { NULL, NULL, __MRU_INIT, NULL, NULL }
+#define RAW_OBJECT_STORE_INIT { NULL, NULL, __MRU_INIT, NULL, NULL, 0 }

Optional: Trailing zeros are not needed in struct initializers.

#161 lore raw

PATCH v3 8/27 pack: move approximate object count to object store

In reply to Stefan Beller

The approximate_object_count() function maintains a rough count of objects in a repository to estimate how long object name abbreviates should be. Object names are scoped to a repository and the appropriate length may differ by repository, so the object count should not be global.

---
 object-store.h | 10 +++++++++-
 packfile.c     | 11 +++++------
 2 files changed, 14 insertions(+), 7 deletions(-)
object-store.h +9 -1
index 6cecba3951..bd1e4fcd8b 100644
@@ -93,6 +93,14 @@ struct raw_object_store {
 	struct alternate_object_database *alt_odb_list;
 	struct alternate_object_database **alt_odb_tail;
 
+	/*
+	 * A fast, rough count of the number of objects in the repository.
+	 * These two fields are not meant for direct access. Use
+	 * approximate_object_count() instead.
+	 */
+	unsigned long approximate_object_count;
+	unsigned approximate_object_count_valid : 1;
+
 	/*
 	 * Whether packed_git has already been populated with this repository's
 	 * packs.
@@ -107,7 +115,7 @@ struct raw_object_store {
  * that macro on member variables. Use NULL instead as that is defined
  * and accepted, deferring the real init to prepare_packed_git_mru(). */
 #define __MRU_INIT { NULL, NULL }
-#define RAW_OBJECT_STORE_INIT { NULL, NULL, __MRU_INIT, NULL, NULL, 0 }
+#define RAW_OBJECT_STORE_INIT { NULL, NULL, __MRU_INIT, NULL, NULL, 0, 0, 0 }
 
 void raw_object_store_clear(struct raw_object_store *o);
 
packfile.c +5 -6
index a8a2e7fe43..693bafbc98 100644
@@ -803,8 +803,6 @@ static void prepare_packed_git_one(char *objdir, int local)
 	strbuf_release(&path);
 }
 
-static int approximate_object_count_valid;
-
 /*
  * Give a fast, rough count of the number of objects in the repository. This
  * ignores loose objects completely. If you have a lot of them, then either
@@ -814,8 +812,8 @@ static int approximate_object_count_valid;
  */
 unsigned long approximate_object_count(void)
 {
-	static unsigned long count;
-	if (!approximate_object_count_valid) {
+	if (!the_repository->objects.approximate_object_count_valid) {
+		unsigned long count;
 		struct packed_git *p;
 
 		prepare_packed_git();
@@ -825,8 +823,9 @@ unsigned long approximate_object_count(void)
 				continue;
 			count += p->num_objects;
 		}
+		the_repository->objects.approximate_object_count = count;
 	}
-	return count;
+	return the_repository->objects.approximate_object_count;
 }
 
 static void *get_next_packed_git(const void *p)
@@ -901,7 +900,7 @@ void prepare_packed_git(void)
 
 void reprepare_packed_git(void)
 {
-	approximate_object_count_valid = 0;
+	the_repository->objects.approximate_object_count_valid = 0;
 	the_repository->objects.packed_git_initialized = 0;
 	prepare_packed_git();
 }
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#162 lore raw

Re: [PATCH 08/27] pack: move approximate object count to object store

Quoted 18 lines · Stefan Beller

The approximate_object_count() function maintains a rough count of objects in a repository to estimate how long object name abbreviates should be. Object names are scoped to a repository and the appropriate length may differ by repository, so the object count should not be global.

---
 object-store.h | 10 +++++++++-
 packfile.c     | 11 +++++------
 2 files changed, 14 insertions(+), 7 deletions(-)
object-store.h +0 -0
index 6cecba3951..bd1e4fcd8b 100644

object-store.h struct raw_object_store { · line 102

Quoted 10 lines
object-store.h +7 -0
@@ -93,6 +93,14 @@ struct raw_object_store {
 	struct alternate_object_database *alt_odb_list;
 	struct alternate_object_database **alt_odb_tail;
 
+	/*
+	 * A fast, rough count of the number of objects in the repository.
+	 * These two fields are not meant for direct access. Use
+	 * approximate_object_count() instead.
+	 */
+	unsigned long approximate_object_count;
+	unsigned approximate_object_count_valid : 1;

Patch looks fine and is effectively a no-op, though what is the need for both of these variables? Maybe it can be simplified down to just use one? Just musing as its out of the scope of this patch and we probably shouldn't try to change that in this series.

Quoted 60 lines
object-store.h +2 -1
+
 	/*
 	 * Whether packed_git has already been populated with this repository's
 	 * packs.
@@ -107,7 +115,7 @@ struct raw_object_store {
  * that macro on member variables. Use NULL instead as that is defined
  * and accepted, deferring the real init to prepare_packed_git_mru(). */
 #define __MRU_INIT { NULL, NULL }
-#define RAW_OBJECT_STORE_INIT { NULL, NULL, __MRU_INIT, NULL, NULL, 0 }
+#define RAW_OBJECT_STORE_INIT { NULL, NULL, __MRU_INIT, NULL, NULL, 0, 0, 0 }
 
 void raw_object_store_clear(struct raw_object_store *o);
 
packfile.c +5 -6
index a8a2e7fe43..693bafbc98 100644
@@ -803,8 +803,6 @@ static void prepare_packed_git_one(char *objdir, int local)
 	strbuf_release(&path);
 }
 
-static int approximate_object_count_valid;
-
 /*
  * Give a fast, rough count of the number of objects in the repository. This
  * ignores loose objects completely. If you have a lot of them, then either
@@ -814,8 +812,8 @@ static int approximate_object_count_valid;
  */
 unsigned long approximate_object_count(void)
 {
-	static unsigned long count;
-	if (!approximate_object_count_valid) {
+	if (!the_repository->objects.approximate_object_count_valid) {
+		unsigned long count;
 		struct packed_git *p;
 
 		prepare_packed_git();
@@ -825,8 +823,9 @@ unsigned long approximate_object_count(void)
 				continue;
 			count += p->num_objects;
 		}
+		the_repository->objects.approximate_object_count = count;
 	}
-	return count;
+	return the_repository->objects.approximate_object_count;
 }
 
 static void *get_next_packed_git(const void *p)
@@ -901,7 +900,7 @@ void prepare_packed_git(void)
 
 void reprepare_packed_git(void)
 {
-	approximate_object_count_valid = 0;
+	the_repository->objects.approximate_object_count_valid = 0;
 	the_repository->objects.packed_git_initialized = 0;
 	prepare_packed_git();
 }
Signature · 1 line
2.16.1.291.g4437f3f132-goog
Signature · 1 line
Brandon Williams
#163 lore raw

Re: [PATCH 08/27] pack: move approximate object count to object store

Quoted 22 lines · Brandon Williams
Quoted 21 lines · Stefan Beller

The approximate_object_count() function maintains a rough count of objects in a repository to estimate how long object name abbreviates should be. Object names are scoped to a repository and the appropriate length may differ by repository, so the object count should not be global.

---
 object-store.h | 10 +++++++++-
 packfile.c     | 11 +++++------
 2 files changed, 14 insertions(+), 7 deletions(-)
object-store.h +0 -0
index 6cecba3951..bd1e4fcd8b 100644
@@ -93,6 +93,14 @@ struct raw_object_store {
      struct alternate_object_database *alt_odb_list;
      struct alternate_object_database **alt_odb_tail;
 
Quoted 12 lines

object-store.h struct raw_object_store { · line 102

Quoted 7 lines
object-store.h +7 -0
+     /*
+      * A fast, rough count of the number of objects in the repository.
+      * These two fields are not meant for direct access. Use
+      * approximate_object_count() instead.
+      */
+     unsigned long approximate_object_count;
+     unsigned approximate_object_count_valid : 1;

Patch looks fine and is effectively a no-op, though what is the need for both of these variables? Maybe it can be simplified down to just use one? Just musing as its out of the scope of this patch and we probably shouldn't try to change that in this series.

I agree we should. It was introduced in e323de4ad7f (sha1_file:
allow sha1_loose_object_info to handle arbitrary repositories, 2017-08-30) and I think it was seen as a clever optimization trick back then?

#164 lore raw

Re: [PATCH 08/27] pack: move approximate object count to object store

Quoted 8 lines · Stefan Beller
Quoted 8 lines
Quoted 7 lines
object-store.h +7 -0
+     /*
+      * A fast, rough count of the number of objects in the repository.
+      * These two fields are not meant for direct access. Use
+      * approximate_object_count() instead.
+      */
+     unsigned long approximate_object_count;
+     unsigned approximate_object_count_valid : 1;
Quoted 8 lines
Quoted 4 lines

Patch looks fine and is effectively a no-op, though what is the need for both of these variables? Maybe it can be simplified down to just use one? Just musing as its out of the scope of this patch and we probably shouldn't try to change that in this series.

I agree we should. It was introduced in e323de4ad7f (sha1_file:
allow sha1_loose_object_info to handle arbitrary repositories, 2017-08-30) and I think it was seen as a clever optimization trick back then?

I think you meant 8e3f52d778 (find_unique_abbrev: move logic out of get_short_sha1(), 2016-10-03)?

Yes, it was just to avoid the dual-meaning of "0" for "not set" and a repository with no packfiles. It would probably be fine to get rid of it. If you have no packfiles then you probably don't have enough objects to worry about micro-optimizing. And anyway, the "wasted" case wouldn't even make any syscalls (it would do a noop prepare_packed_git() and then realize the packed_git list is empty).

-Peff

#165 lore raw

Re: [PATCH 08/27] pack: move approximate object count to object store

Quoted 15 lines · Jeff King
Quoted 13 lines · Stefan Beller
Quoted 12 lines

object-store.h struct raw_object_store { · line 102

Quoted 7 lines
object-store.h +7 -0
+     /*
+      * A fast, rough count of the number of objects in the repository.
+      * These two fields are not meant for direct access. Use
+      * approximate_object_count() instead.
+      */
+     unsigned long approximate_object_count;
+     unsigned approximate_object_count_valid : 1;

Patch looks fine and is effectively a no-op, though what is the need for both of these variables? Maybe it can be simplified down to just use one? Just musing as its out of the scope of this patch and we probably shouldn't try to change that in this series.

Quoted 6 lines
Quoted 3 lines

I agree we should. It was introduced in e323de4ad7f (sha1_file:
allow sha1_loose_object_info to handle arbitrary repositories, 2017-08-30) and I think it was seen as a clever optimization trick back then?

I think you meant 8e3f52d778 (find_unique_abbrev: move logic out of get_short_sha1(), 2016-10-03)?

Yes, copy paste error.

Quoted 7 lines

Yes, it was just to avoid the dual-meaning of "0" for "not set" and a repository with no packfiles. It would probably be fine to get rid of it. If you have no packfiles then you probably don't have enough objects to worry about micro-optimizing. And anyway, the "wasted" case wouldn't even make any syscalls (it would do a noop prepare_packed_git() and then realize the packed_git list is empty).

Good point!

Quoted 2 lines

-Peff

#166 lore raw

PATCH v3 9/27 sha1_file: add raw_object_store argument to alt_odb_usable

In reply to Stefan Beller

Add a raw_object_store to alt_odb_usable to be more specific about which repository to act on. The choice of the repository is delegated to its only caller link_alt_odb_entry.

---
 sha1_file.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
sha1_file.c +5 -3
index 1348dce68f..a8e23bd2f8 100644
@@ -346,7 +346,9 @@ static const char *alt_sha1_path(struct alternate_object_database *alt,
 /*
  * Return non-zero iff the path is usable as an alternate object database.
  */
-static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)
+static int alt_odb_usable(struct raw_object_store *o,
+			  struct strbuf *path,
+			  const char *normalized_objdir)
 {
 	struct alternate_object_database *alt;
 
@@ -362,7 +364,7 @@ static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)
 	 * Prevent the common mistake of listing the same
 	 * thing twice, or object directory itself.
 	 */
-	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
+	for (alt = o->alt_odb_list; alt; alt = alt->next) {
 		if (!fspathcmp(path->buf, alt->path))
 			return 0;
 	}
@@ -414,7 +416,7 @@ static int link_alt_odb_entry(const char *entry, const char *relative_base,
 	while (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')
 		strbuf_setlen(&pathbuf, pathbuf.len - 1);
 
-	if (!alt_odb_usable(&pathbuf, normalized_objdir)) {
+	if (!alt_odb_usable(&the_repository->objects, &pathbuf, normalized_objdir)) {
 		strbuf_release(&pathbuf);
 		return -1;
 	}
Signature · 1 line
2.16.1.291.g4437f3f132-goog
Reviewed-by on v3 9/27
#167 lore raw

Re: [PATCH 09/27] sha1_file: add raw_object_store argument to alt_odb_usable

Quoted 5 lines · On Tue, 20 Feb 2018 17:54:12 -0800 Stefan Beller <[email protected]> wrote:

Add a raw_object_store to alt_odb_usable to be more specific about which repository to act on. The choice of the repository is delegated to its only caller link_alt_odb_entry.

I checked that alt_odb_usable no longer depends on any repository-like globals.

#168 lore raw

PATCH v3 10/27 sha1_file: add repository argument to link_alt_odb_entry

In reply to Stefan Beller

Add a repository argument to allow the link_alt_odb_entry caller to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

Since the implementation does not yet work with other repositories, use a wrapper macro to enforce that the caller passes in the_repository as the first argument. It would be more appealing to use BUILD_ASSERT_OR_ZERO to enforce this, but that doesn't work because it requires a compile-time constant and common compilers like gcc 4.8.4 do not consider "r == the_repository" a compile-time constant.

This and the following three patches add repository arguments to link_alt_odb_entry, read_info_alternates, link_alt_odb_entries and prepare_alt_odb. Three out of the four functions are found in a recursive call chain, calling each other, and one of them accesses the repositories objectdir (which was migrated; it was an obvious choice) and ignore_env (which we need to keep in the repository struct for clarify); hence we will pass through the repository unlike just the object store object + the ignore_env flag.

---
 sha1_file.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
sha1_file.c +5 -3
index a8e23bd2f8..aaa6276211 100644
@@ -390,8 +390,9 @@ static int alt_odb_usable(struct raw_object_store *o,
  * terminating NUL.
  */
 static void read_info_alternates(const char * relative_base, int depth);
-static int link_alt_odb_entry(const char *entry, const char *relative_base,
-	int depth, const char *normalized_objdir)
+#define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)
+static int link_alt_odb_entry_the_repository(const char *entry,
+	const char *relative_base, int depth, const char *normalized_objdir)
 {
 	struct alternate_object_database *ent;
 	struct strbuf pathbuf = STRBUF_INIT;
@@ -488,7 +489,8 @@ static void link_alt_odb_entries(const char *alt, int sep,
 		alt = parse_alt_odb_entry(alt, sep, &entry);
 		if (!entry.len)
 			continue;
-		link_alt_odb_entry(entry.buf, relative_base, depth, objdirbuf.buf);
+		link_alt_odb_entry(the_repository, entry.buf,
+				   relative_base, depth, objdirbuf.buf);
 	}
 	strbuf_release(&entry);
 	strbuf_release(&objdirbuf);
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#169 lore raw

PATCH v3 11/27 sha1_file: add repository argument to read_info_alternates

In reply to Stefan Beller

See previous patch for explanation.

---
 sha1_file.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
sha1_file.c +7 -4
index aaa6276211..6edcc3d3a7 100644
@@ -389,7 +389,9 @@ static int alt_odb_usable(struct raw_object_store *o,
  * SHA1, an extra slash for the first level indirection, and the
  * terminating NUL.
  */
-static void read_info_alternates(const char * relative_base, int depth);
+#define read_info_alternates(r, rb, d) read_info_alternates_##r(rb, d)
+static void read_info_alternates_the_repository(const char *relative_base,
+						int depth);
 #define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)
 static int link_alt_odb_entry_the_repository(const char *entry,
 	const char *relative_base, int depth, const char *normalized_objdir)
@@ -430,7 +432,7 @@ static int link_alt_odb_entry_the_repository(const char *entry,
 	ent->next = NULL;
 
 	/* recursively add alternates */
-	read_info_alternates(pathbuf.buf, depth + 1);
+	read_info_alternates(the_repository, pathbuf.buf, depth + 1);
 
 	strbuf_release(&pathbuf);
 	return 0;
@@ -496,7 +498,8 @@ static void link_alt_odb_entries(const char *alt, int sep,
 	strbuf_release(&objdirbuf);
 }
 
-static void read_info_alternates(const char * relative_base, int depth)
+static void read_info_alternates_the_repository(const char *relative_base,
+						int depth)
 {
 	char *path;
 	struct strbuf buf = STRBUF_INIT;
@@ -680,7 +683,7 @@ void prepare_alt_odb(void)
 			&the_repository->objects.alt_odb_list;
 	link_alt_odb_entries(alt, PATH_SEP, NULL, 0);
 
-	read_info_alternates(get_object_directory(), 0);
+	read_info_alternates(the_repository, get_object_directory(), 0);
 }
 
 /* Returns 1 if we have successfully freshened the file, 0 otherwise. */
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#170 lore raw

PATCH v3 12/27 sha1_file: add repository argument to link_alt_odb_entries

In reply to Stefan Beller

See previous patch for explanation.

---
 sha1_file.c | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)
sha1_file.c +13 -6
index 6edcc3d3a7..a0e9116318 100644
@@ -467,8 +467,12 @@ static const char *parse_alt_odb_entry(const char *string,
 	return end;
 }
 
-static void link_alt_odb_entries(const char *alt, int sep,
-				 const char *relative_base, int depth)
+#define link_alt_odb_entries(r, a, s, rb, d) \
+	link_alt_odb_entries_##r(a, s, rb, d)
+static void link_alt_odb_entries_the_repository(const char *alt,
+						int sep,
+						const char *relative_base,
+						int depth)
 {
 	struct strbuf objdirbuf = STRBUF_INIT;
 	struct strbuf entry = STRBUF_INIT;
@@ -511,7 +515,7 @@ static void read_info_alternates_the_repository(const char *relative_base,
 		return;
 	}
 
-	link_alt_odb_entries(buf.buf, '\n', relative_base, depth);
+	link_alt_odb_entries(the_repository, buf.buf, '\n', relative_base, depth);
 	strbuf_release(&buf);
 	free(path);
 }
@@ -565,7 +569,8 @@ void add_to_alternates_file(const char *reference)
 		if (commit_lock_file(&lock))
 			die_errno("unable to move new alternates file into place");
 		if (the_repository->objects.alt_odb_tail)
-			link_alt_odb_entries(reference, '\n', NULL, 0);
+			link_alt_odb_entries(the_repository, reference,
+					     '\n', NULL, 0);
 	}
 	free(alts);
 }
@@ -578,7 +583,8 @@ void add_to_alternates_memory(const char *reference)
 	 */
 	prepare_alt_odb();
 
-	link_alt_odb_entries(reference, '\n', NULL, 0);
+	link_alt_odb_entries(the_repository, reference,
+			     '\n', NULL, 0);
 }
 
 /*
@@ -681,7 +687,8 @@ void prepare_alt_odb(void)
 
 	the_repository->objects.alt_odb_tail =
 			&the_repository->objects.alt_odb_list;
-	link_alt_odb_entries(alt, PATH_SEP, NULL, 0);
+	link_alt_odb_entries(the_repository, alt,
+			     PATH_SEP, NULL, 0);
 
 	read_info_alternates(the_repository, get_object_directory(), 0);
 }
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#171 lore raw

PATCH v3 13/27 sha1_file: add repository argument to prepare_alt_odb

In reply to Stefan Beller

See previous patch for explanation.

While at it, move the declaration to object-store.h, where it should be easier to find.

---
 builtin/fsck.c |  2 +-
 object-store.h |  3 ++-
 packfile.c     |  2 +-
 sha1_file.c    | 13 +++++++------
 sha1_name.c    |  3 ++-
 5 files changed, 13 insertions(+), 10 deletions(-)
builtin/fsck.c +1 -1
index 2e99e02128..76c94e9b5a 100644
@@ -717,7 +717,7 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 	} else {
 		fsck_object_dir(get_object_directory());
 
-		prepare_alt_odb();
+		prepare_alt_odb(the_repository);
 		for (alt = the_repository->objects.alt_odb_list;
 				alt; alt = alt->next)
 			fsck_object_dir(alt->path);
object-store.h +2 -1
index bd1e4fcd8b..514ad94287 100644
@@ -20,7 +20,8 @@ struct alternate_object_database {
 
 	char path[FLEX_ARRAY];
 };
-void prepare_alt_odb(void);
+#define prepare_alt_odb(r) prepare_alt_odb_##r()
+void prepare_alt_odb_the_repository(void);
 char *compute_alternate_path(const char *path, struct strbuf *err);
 typedef int alt_odb_fn(struct alternate_object_database *, void *);
 int foreach_alt_odb(alt_odb_fn, void*);
packfile.c +1 -1
index 693bafbc98..ccfe084642 100644
@@ -890,7 +890,7 @@ void prepare_packed_git(void)
 	if (the_repository->objects.packed_git_initialized)
 		return;
 	prepare_packed_git_one(get_object_directory(), 1);
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next)
 		prepare_packed_git_one(alt->path, 0);
 	rearrange_packed_git();
sha1_file.c +7 -6
index a0e9116318..c26921df4a 100644
@@ -23,6 +23,7 @@
 #include "sha1-lookup.h"
 #include "bulk-checkin.h"
 #include "repository.h"
+#include "object-store.h"
 #include "streaming.h"
 #include "dir.h"
 #include "list.h"
@@ -581,7 +582,7 @@ void add_to_alternates_memory(const char *reference)
 	 * Make sure alternates are initialized, or else our entry may be
 	 * overwritten when they are.
 	 */
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 
 	link_alt_odb_entries(the_repository, reference,
 			     '\n', NULL, 0);
@@ -667,7 +668,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 	struct alternate_object_database *ent;
 	int r = 0;
 
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	for (ent = the_repository->objects.alt_odb_list; ent; ent = ent->next) {
 		r = fn(ent, cb);
 		if (r)
@@ -676,7 +677,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 	return r;
 }
 
-void prepare_alt_odb(void)
+void prepare_alt_odb_the_repository(void)
 {
 	const char *alt;
 
@@ -730,7 +731,7 @@ static int check_and_freshen_local(const unsigned char *sha1, int freshen)
 static int check_and_freshen_nonlocal(const unsigned char *sha1, int freshen)
 {
 	struct alternate_object_database *alt;
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		const char *path = alt_sha1_path(alt, sha1);
 		if (check_and_freshen_file(path, freshen))
@@ -889,7 +890,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,
 	if (!lstat(*path, st))
 		return 0;
 
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	errno = ENOENT;
 	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
@@ -920,7 +921,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)
 		return fd;
 	most_interesting_errno = errno;
 
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		fd = git_open(*path);
sha1_name.c +2 -1
index 016c883b5c..3e490ee8f6 100644
@@ -11,6 +11,7 @@
 #include "sha1-array.h"
 #include "packfile.h"
 #include "repository.h"
+#include "object-store.h"
 
 static int get_oid_oneline(const char *, struct object_id *, struct commit_list *);
 
@@ -353,7 +354,7 @@ static int init_object_disambiguation(const char *name, int len,
 
 	ds->len = len;
 	ds->hex_pfx[len] = '\0';
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	return 0;
 }
 
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#172 lore raw

Re: [PATCH 13/27] sha1_file: add repository argument to prepare_alt_odb

Hi,

Quoted 4 lines · Stefan Beller

See previous patch for explanation.

While at it, move the declaration to object-store.h, where it should be easier to find.

Which declaration? It looks like prepare_alt_odb is already in object-store.h.

[...]

builtin/fsck.c int cmd_fsck(int argc, const char **argv, const char *prefix) · line 720

Quoted 8 lines
builtin/fsck.c +1 -1
@@ -717,7 +717,7 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 	} else {
 		fsck_object_dir(get_object_directory());
 
-		prepare_alt_odb();
+		prepare_alt_odb(the_repository);

Patch 2 added a #include of "repository.h". Good.

(I checked because with the definition of prepare_alt_odb as a macro, the function call would compile correctly even if the_repository weren't in scope, but we want to include what we use as a matter of style/maintainability.)

[...]

packfile.c void prepare_packed_git(void) · line 893

Quoted 8 lines
packfile.c +1 -1
@@ -890,7 +890,7 @@ void prepare_packed_git(void)
 	if (the_repository->objects.packed_git_initialized)
 		return;
 	prepare_packed_git_one(get_object_directory(), 1);
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);

Also good, since patch 3 added a #include of "repository.h".

[...]

sha1_file.c line 26

Quoted 7 lines
sha1_file.c +1 -0
@@ -23,6 +23,7 @@
 #include "sha1-lookup.h"
 #include "bulk-checkin.h"
 #include "repository.h"
+#include "object-store.h"

Should this #include have been added in an earlier patch, since the file both uses and defines prepare_alt_odb, which is declared there?

The rest looks good.

Thanks,
Jonathan

#173 lore raw

Re: [PATCH 13/27] sha1_file: add repository argument to prepare_alt_odb

Quoted 10 lines

sha1_file.c line 26

Quoted 7 lines
sha1_file.c +1 -0
@@ -23,6 +23,7 @@
 #include "sha1-lookup.h"
 #include "bulk-checkin.h"
 #include "repository.h"
+#include "object-store.h"

Should this #include have been added in an earlier patch, since the file both uses and defines prepare_alt_odb, which is declared there?

This is another superflous include, dropped it. Thanks for catching.

#174 lore raw

PATCH v3 14/27 sha1_file: allow link_alt_odb_entries to handle arbitrary repositories

In reply to Stefan Beller

Actually this also allows read_info_alternates and link_alt_odb_entry to handle arbitrary repositories, but link_alt_odb_entries is the most interesting function in this set of functions, hence the commit subject.

These functions span a strongly connected component in the function graph, i.e. the recursive call chain might look like

-> link_alt_odb_entries

    -> link_alt_odb_entry
      -> read_info_alternates
        -> link_alt_odb_entries

That is why we need to convert all these functions at the same time.

---
 object-store.h |  4 ++++
 sha1_file.c    | 36 ++++++++++++++++--------------------
 2 files changed, 20 insertions(+), 20 deletions(-)
object-store.h +4 -0
index 514ad94287..f283fbdba9 100644
@@ -18,6 +18,10 @@ struct alternate_object_database {
 	char loose_objects_subdir_seen[256];
 	struct oid_array loose_objects_cache;
 
+	/*
+	 * Path to the alternative object store. If this is a relative path,
+	 * it is relative to the current working directory.
+	 */
 	char path[FLEX_ARRAY];
 };
 #define prepare_alt_odb(r) prepare_alt_odb_##r()
sha1_file.c +16 -20
index c26921df4a..6e5105a252 100644
@@ -390,11 +390,10 @@ static int alt_odb_usable(struct raw_object_store *o,
  * SHA1, an extra slash for the first level indirection, and the
  * terminating NUL.
  */
-#define read_info_alternates(r, rb, d) read_info_alternates_##r(rb, d)
-static void read_info_alternates_the_repository(const char *relative_base,
-						int depth);
-#define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)
-static int link_alt_odb_entry_the_repository(const char *entry,
+static void read_info_alternates(struct repository *r,
+				 const char *relative_base,
+				 int depth);
+static int link_alt_odb_entry(struct repository *r, const char *entry,
 	const char *relative_base, int depth, const char *normalized_objdir)
 {
 	struct alternate_object_database *ent;
@@ -420,7 +419,7 @@ static int link_alt_odb_entry_the_repository(const char *entry,
 	while (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')
 		strbuf_setlen(&pathbuf, pathbuf.len - 1);
 
-	if (!alt_odb_usable(&the_repository->objects, &pathbuf, normalized_objdir)) {
+	if (!alt_odb_usable(&r->objects, &pathbuf, normalized_objdir)) {
 		strbuf_release(&pathbuf);
 		return -1;
 	}
@@ -428,12 +427,12 @@ static int link_alt_odb_entry_the_repository(const char *entry,
 	ent = alloc_alt_odb(pathbuf.buf);
 
 	/* add the alternate entry */
-	*the_repository->objects.alt_odb_tail = ent;
-	the_repository->objects.alt_odb_tail = &(ent->next);
+	*r->objects.alt_odb_tail = ent;
+	r->objects.alt_odb_tail = &(ent->next);
 	ent->next = NULL;
 
 	/* recursively add alternates */
-	read_info_alternates(the_repository, pathbuf.buf, depth + 1);
+	read_info_alternates(r, pathbuf.buf, depth + 1);
 
 	strbuf_release(&pathbuf);
 	return 0;
@@ -468,12 +467,8 @@ static const char *parse_alt_odb_entry(const char *string,
 	return end;
 }
 
-#define link_alt_odb_entries(r, a, s, rb, d) \
-	link_alt_odb_entries_##r(a, s, rb, d)
-static void link_alt_odb_entries_the_repository(const char *alt,
-						int sep,
-						const char *relative_base,
-						int depth)
+static void link_alt_odb_entries(struct repository *r, const char *alt,
+				 int sep, const char *relative_base, int depth)
 {
 	struct strbuf objdirbuf = STRBUF_INIT;
 	struct strbuf entry = STRBUF_INIT;
@@ -487,7 +482,7 @@ static void link_alt_odb_entries_the_repository(const char *alt,
 		return;
 	}
 
-	strbuf_add_absolute_path(&objdirbuf, get_object_directory());
+	strbuf_add_absolute_path(&objdirbuf, r->objects.objectdir);
 	if (strbuf_normalize_path(&objdirbuf) < 0)
 		die("unable to normalize object directory: %s",
 		    objdirbuf.buf);
@@ -496,15 +491,16 @@ static void link_alt_odb_entries_the_repository(const char *alt,
 		alt = parse_alt_odb_entry(alt, sep, &entry);
 		if (!entry.len)
 			continue;
-		link_alt_odb_entry(the_repository, entry.buf,
+		link_alt_odb_entry(r, entry.buf,
 				   relative_base, depth, objdirbuf.buf);
 	}
 	strbuf_release(&entry);
 	strbuf_release(&objdirbuf);
 }
 
-static void read_info_alternates_the_repository(const char *relative_base,
-						int depth)
+static void read_info_alternates(struct repository *r,
+				 const char *relative_base,
+				 int depth)
 {
 	char *path;
 	struct strbuf buf = STRBUF_INIT;
@@ -516,7 +512,7 @@ static void read_info_alternates_the_repository(const char *relative_base,
 		return;
 	}
 
-	link_alt_odb_entries(the_repository, buf.buf, '\n', relative_base, depth);
+	link_alt_odb_entries(r, buf.buf, '\n', relative_base, depth);
 	strbuf_release(&buf);
 	free(path);
 }
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#175 lore raw

Re: [PATCH 14/27] sha1_file: allow link_alt_odb_entries to handle arbitrary repositories

Quoted 12 lines · Stefan Beller

Actually this also allows read_info_alternates and link_alt_odb_entry to handle arbitrary repositories, but link_alt_odb_entries is the most interesting function in this set of functions, hence the commit subject.

These functions span a strongly connected component in the function graph, i.e. the recursive call chain might look like

-> link_alt_odb_entries

    -> link_alt_odb_entry
      -> read_info_alternates
        -> link_alt_odb_entries
Quoted 5 lines

That is why we need to convert all these functions at the same time.

---

Looks good.

Quoted 9 lines
 object-store.h |  4 ++++
 sha1_file.c    | 36 ++++++++++++++++--------------------
 2 files changed, 20 insertions(+), 20 deletions(-)
object-store.h +0 -0
index 514ad94287..f283fbdba9 100644

object-store.h struct alternate_object_database { · line 24

Quoted 7 lines
object-store.h +4 -0
@@ -18,6 +18,10 @@ struct alternate_object_database {
 	char loose_objects_subdir_seen[256];
 	struct oid_array loose_objects_cache;
 
+	/*
+	 * Path to the alternative object store. If this is a relative path,
+	 * it is relative to the current working directory.
+	 */

Thanks for adding the comment.

Quoted 104 lines
object-store.h +0 -0
 	char path[FLEX_ARRAY];
 };
 #define prepare_alt_odb(r) prepare_alt_odb_##r()
sha1_file.c +16 -20
index c26921df4a..6e5105a252 100644
@@ -390,11 +390,10 @@ static int alt_odb_usable(struct raw_object_store *o,
  * SHA1, an extra slash for the first level indirection, and the
  * terminating NUL.
  */
-#define read_info_alternates(r, rb, d) read_info_alternates_##r(rb, d)
-static void read_info_alternates_the_repository(const char *relative_base,
-						int depth);
-#define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)
-static int link_alt_odb_entry_the_repository(const char *entry,
+static void read_info_alternates(struct repository *r,
+				 const char *relative_base,
+				 int depth);
+static int link_alt_odb_entry(struct repository *r, const char *entry,
 	const char *relative_base, int depth, const char *normalized_objdir)
 {
 	struct alternate_object_database *ent;
@@ -420,7 +419,7 @@ static int link_alt_odb_entry_the_repository(const char *entry,
 	while (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')
 		strbuf_setlen(&pathbuf, pathbuf.len - 1);
 
-	if (!alt_odb_usable(&the_repository->objects, &pathbuf, normalized_objdir)) {
+	if (!alt_odb_usable(&r->objects, &pathbuf, normalized_objdir)) {
 		strbuf_release(&pathbuf);
 		return -1;
 	}
@@ -428,12 +427,12 @@ static int link_alt_odb_entry_the_repository(const char *entry,
 	ent = alloc_alt_odb(pathbuf.buf);
 
 	/* add the alternate entry */
-	*the_repository->objects.alt_odb_tail = ent;
-	the_repository->objects.alt_odb_tail = &(ent->next);
+	*r->objects.alt_odb_tail = ent;
+	r->objects.alt_odb_tail = &(ent->next);
 	ent->next = NULL;
 
 	/* recursively add alternates */
-	read_info_alternates(the_repository, pathbuf.buf, depth + 1);
+	read_info_alternates(r, pathbuf.buf, depth + 1);
 
 	strbuf_release(&pathbuf);
 	return 0;
@@ -468,12 +467,8 @@ static const char *parse_alt_odb_entry(const char *string,
 	return end;
 }
 
-#define link_alt_odb_entries(r, a, s, rb, d) \
-	link_alt_odb_entries_##r(a, s, rb, d)
-static void link_alt_odb_entries_the_repository(const char *alt,
-						int sep,
-						const char *relative_base,
-						int depth)
+static void link_alt_odb_entries(struct repository *r, const char *alt,
+				 int sep, const char *relative_base, int depth)
 {
 	struct strbuf objdirbuf = STRBUF_INIT;
 	struct strbuf entry = STRBUF_INIT;
@@ -487,7 +482,7 @@ static void link_alt_odb_entries_the_repository(const char *alt,
 		return;
 	}
 
-	strbuf_add_absolute_path(&objdirbuf, get_object_directory());
+	strbuf_add_absolute_path(&objdirbuf, r->objects.objectdir);
 	if (strbuf_normalize_path(&objdirbuf) < 0)
 		die("unable to normalize object directory: %s",
 		    objdirbuf.buf);
@@ -496,15 +491,16 @@ static void link_alt_odb_entries_the_repository(const char *alt,
 		alt = parse_alt_odb_entry(alt, sep, &entry);
 		if (!entry.len)
 			continue;
-		link_alt_odb_entry(the_repository, entry.buf,
+		link_alt_odb_entry(r, entry.buf,
 				   relative_base, depth, objdirbuf.buf);
 	}
 	strbuf_release(&entry);
 	strbuf_release(&objdirbuf);
 }
 
-static void read_info_alternates_the_repository(const char *relative_base,
-						int depth)
+static void read_info_alternates(struct repository *r,
+				 const char *relative_base,
+				 int depth)
 {
 	char *path;
 	struct strbuf buf = STRBUF_INIT;
@@ -516,7 +512,7 @@ static void read_info_alternates_the_repository(const char *relative_base,
 		return;
 	}
 
-	link_alt_odb_entries(the_repository, buf.buf, '\n', relative_base, depth);
+	link_alt_odb_entries(r, buf.buf, '\n', relative_base, depth);
 	strbuf_release(&buf);
 	free(path);
 }
Signature · 1 line
2.16.1.291.g4437f3f132-goog
Signature · 1 line
Brandon Williams
#176 lore raw

PATCH v3 15/27 sha1_file: allow prepare_alt_odb to handle arbitrary repositories

In reply to Stefan Beller
---
 object-store.h |  3 +--
 sha1_file.c    | 21 +++++++++++----------
 2 files changed, 12 insertions(+), 12 deletions(-)
object-store.h +1 -2
index f283fbdba9..873b341774 100644
@@ -24,8 +24,7 @@ struct alternate_object_database {
 	 */
 	char path[FLEX_ARRAY];
 };
-#define prepare_alt_odb(r) prepare_alt_odb_##r()
-void prepare_alt_odb_the_repository(void);
+void prepare_alt_odb(struct repository *r);
 char *compute_alternate_path(const char *path, struct strbuf *err);
 typedef int alt_odb_fn(struct alternate_object_database *, void *);
 int foreach_alt_odb(alt_odb_fn, void*);
sha1_file.c +11 -10
index 6e5105a252..9cf3fffbeb 100644
@@ -673,21 +673,22 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 	return r;
 }
 
-void prepare_alt_odb_the_repository(void)
+void prepare_alt_odb(struct repository *r)
 {
-	const char *alt;
-
-	if (the_repository->objects.alt_odb_tail)
+	if (r->objects.alt_odb_tail)
 		return;
 
-	alt = getenv(ALTERNATE_DB_ENVIRONMENT);
+	r->objects.alt_odb_tail = &r->objects.alt_odb_list;
+
+	if (!r->ignore_env) {
+		const char *alt = getenv(ALTERNATE_DB_ENVIRONMENT);
+		if (!alt)
+			alt = "";
 
-	the_repository->objects.alt_odb_tail =
-			&the_repository->objects.alt_odb_list;
-	link_alt_odb_entries(the_repository, alt,
-			     PATH_SEP, NULL, 0);
+		link_alt_odb_entries(r, alt, PATH_SEP, NULL, 0);
+	}
 
-	read_info_alternates(the_repository, get_object_directory(), 0);
+	read_info_alternates(r, r->objects.objectdir, 0);
 }
 
 /* Returns 1 if we have successfully freshened the file, 0 otherwise. */
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#177 lore raw

Re: [PATCH 15/27] sha1_file: allow prepare_alt_odb to handle arbitrary repositories

Quoted 6 lines · On Tue, 20 Feb 2018 17:54:18 -0800 Stefan Beller <[email protected]> wrote:
sha1_file.c +1 -4
-void prepare_alt_odb_the_repository(void)
+void prepare_alt_odb(struct repository *r)
 {
-	const char *alt;
-
-	if (the_repository->objects.alt_odb_tail)

sha1_file.c int foreach_alt_odb(alt_odb_fn fn, void *cb) · line 686

Quoted 10 lines
sha1_file.c +7 -1
+	if (r->objects.alt_odb_tail)
 		return;
 
-	alt = getenv(ALTERNATE_DB_ENVIRONMENT);
+	r->objects.alt_odb_tail = &r->objects.alt_odb_list;
+
+	if (!r->ignore_env) {
+		const char *alt = getenv(ALTERNATE_DB_ENVIRONMENT);
+		if (!alt)
+			alt = "";

alt can be NULL, just like in the existing code, so these 2 lines are unnecessary. (link_alt_odb_entries() will just do nothing in either case.)

(I also think that the check of absence of alt should be done by the caller of link_alt_odb_entries(), not by link_alt_odb_entries() itself, but that is much beyond the scope of this patch set.)

Quoted 11 lines
sha1_file.c +3 -5
-	the_repository->objects.alt_odb_tail =
-			&the_repository->objects.alt_odb_list;
-	link_alt_odb_entries(the_repository, alt,
-			     PATH_SEP, NULL, 0);
+		link_alt_odb_entries(r, alt, PATH_SEP, NULL, 0);
+	}
 
-	read_info_alternates(the_repository, get_object_directory(), 0);
+	read_info_alternates(r, r->objects.objectdir, 0);
 }
#178 lore raw

Re: [PATCH 15/27] sha1_file: allow prepare_alt_odb to handle arbitrary repositories

Quoted 24 lines · Jonathan Tan
Quoted 6 lines · On Tue, 20 Feb 2018 17:54:18 -0800 Stefan Beller <[email protected]> wrote:
sha1_file.c +1 -4
-void prepare_alt_odb_the_repository(void)
+void prepare_alt_odb(struct repository *r)
 {
-     const char *alt;
-
-     if (the_repository->objects.alt_odb_tail)

sha1_file.c int foreach_alt_odb(alt_odb_fn fn, void *cb) · line 686

Quoted 10 lines
sha1_file.c +7 -1
+     if (r->objects.alt_odb_tail)
              return;
 
-     alt = getenv(ALTERNATE_DB_ENVIRONMENT);
+     r->objects.alt_odb_tail = &r->objects.alt_odb_list;
+
+     if (!r->ignore_env) {
+             const char *alt = getenv(ALTERNATE_DB_ENVIRONMENT);
+             if (!alt)
+                     alt = "";

alt can be NULL, just like in the existing code, so these 2 lines are unnecessary. (link_alt_odb_entries() will just do nothing in either case.)

Quoted 4 lines

(I also think that the check of absence of alt should be done by the caller of link_alt_odb_entries(), not by link_alt_odb_entries() itself, but that is much beyond the scope of this patch set.)

reverted to a literal translation of s/the_repository/r/, I forget why I implemented this change in the first place. dropped for now.

Thanks,
Stefan

#179 lore raw

PATCH v3 16/27 sha1_file: add repository argument to sha1_file_name

In reply to Stefan Beller

Add a repository argument to allow sha1_file_name callers to be more specific about which repository to handle. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

While at it, move the declaration to object-store.h, where it should be easier to find.

---
 cache.h        |  6 ------
 http-walker.c  |  3 ++-
 http.c         |  5 ++---
 object-store.h |  7 +++++++
 sha1_file.c    | 10 +++++-----
 5 files changed, 16 insertions(+), 15 deletions(-)
cache.h +0 -6
index 3fd9dc52cd..e641c23b0a 100644
@@ -961,12 +961,6 @@ extern void check_repository_format(void);
 #define DATA_CHANGED    0x0020
 #define TYPE_CHANGED    0x0040
 
-/*
- * Put in `buf` the name of the file in the local object database that
- * would be used to store a loose object with the specified sha1.
- */
-extern void sha1_file_name(struct strbuf *buf, const unsigned char *sha1);
-
 /*
  * Return an abbreviated sha1 unique within this repository's object database.
  * The result will be at least `len` characters long, and will be NUL
http-walker.c +2 -1
index 8bb5d991bb..76bfdbca76 100644
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "repository.h"
 #include "commit.h"
 #include "walker.h"
 #include "http.h"
@@ -546,7 +547,7 @@ static int fetch_object(struct walker *walker, unsigned char *sha1)
 		ret = error("File %s has bad hash", hex);
 	} else if (req->rename < 0) {
 		struct strbuf buf = STRBUF_INIT;
-		sha1_file_name(&buf, req->sha1);
+		sha1_file_name(the_repository, &buf, req->sha1);
 		ret = error("unable to write sha1 filename %s", buf.buf);
 		strbuf_release(&buf);
 	}
http.c +2 -3
index a4a9e583c7..afe2707e86 100644
@@ -2247,7 +2247,7 @@ struct http_object_request *new_http_object_request(const char *base_url,
 	hashcpy(freq->sha1, sha1);
 	freq->localfile = -1;
 
-	sha1_file_name(&filename, sha1);
+	sha1_file_name(the_repository, &filename, sha1);
 	snprintf(freq->tmpfile, sizeof(freq->tmpfile),
 		 "%s.temp", filename.buf);
 
@@ -2396,8 +2396,7 @@ int finish_http_object_request(struct http_object_request *freq)
 		unlink_or_warn(freq->tmpfile);
 		return -1;
 	}
-
-	sha1_file_name(&filename, freq->sha1);
+	sha1_file_name(the_repository, &filename, freq->sha1);
 	freq->rename = finalize_object_file(freq->tmpfile, filename.buf);
 	strbuf_release(&filename);
 
object-store.h +7 -0
index 873b341774..4f5060e768 100644
@@ -123,4 +123,11 @@ struct raw_object_store {
 
 void raw_object_store_clear(struct raw_object_store *o);
 
+/*
+ * Put in `buf` the name of the file in the local object database that
+ * would be used to store a loose object with the specified sha1.
+ */
+#define sha1_file_name(r, b, s) sha1_file_name_##r(b, s)
+void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1);
+
 #endif /* OBJECT_STORE_H */
sha1_file.c +5 -5
index 9cf3fffbeb..811ed9e53f 100644
@@ -323,7 +323,7 @@ static void fill_sha1_path(struct strbuf *buf, const unsigned char *sha1)
 	}
 }
 
-void sha1_file_name(struct strbuf *buf, const unsigned char *sha1)
+void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1)
 {
 	strbuf_addstr(buf, get_object_directory());
 	strbuf_addch(buf, '/');
@@ -720,7 +720,7 @@ static int check_and_freshen_local(const unsigned char *sha1, int freshen)
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(&buf, sha1);
+	sha1_file_name(the_repository, &buf, sha1);
 
 	return check_and_freshen_file(buf.buf, freshen);
 }
@@ -881,7 +881,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(&buf, sha1);
+	sha1_file_name(the_repository, &buf, sha1);
 	*path = buf.buf;
 
 	if (!lstat(*path, st))
@@ -910,7 +910,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(&buf, sha1);
+	sha1_file_name(the_repository, &buf, sha1);
 	*path = buf.buf;
 
 	fd = git_open(*path);
@@ -1595,7 +1595,7 @@ static int write_loose_object(const unsigned char *sha1, char *hdr, int hdrlen,
 	static struct strbuf filename = STRBUF_INIT;
 
 	strbuf_reset(&filename);
-	sha1_file_name(&filename, sha1);
+	sha1_file_name(the_repository, &filename, sha1);
 
 	fd = create_tmpfile(&tmp_file, filename.buf);
 	if (fd < 0) {
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#180 lore raw

Re: [PATCH 16/27] sha1_file: add repository argument to sha1_file_name

Quoted 10 lines · Stefan Beller

Add a repository argument to allow sha1_file_name callers to be more specific about which repository to handle. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

While at it, move the declaration to object-store.h, where it should be easier to find.

Seems like we may want to make a sha1-file.h or an oid-file.h or something like that at some point as that seems like a better place for the function than in the object-store.h file?

Signature · 1 line
Brandon Williams
#181 lore raw

Re: [PATCH 16/27] sha1_file: add repository argument to sha1_file_name

Quoted 9 lines · Brandon Williams
Quoted 8 lines · Stefan Beller

Add a repository argument to allow sha1_file_name callers to be more specific about which repository to handle. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

Quoted 6 lines
Quoted 2 lines

While at it, move the declaration to object-store.h, where it should be easier to find.

Seems like we may want to make a sha1-file.h or an oid-file.h or something like that at some point as that seems like a better place for the function than in the object-store.h file?

It depends what our long term goal is.
Do we want header and source file name to match for each function? Or do we want a coarser set of headers, such that we have a broad object-store.h, but that is implemented in sha1_file.c, packfile.c for the parts of the raw_objectstore and other .c files for the higher levels of the object store?

For now I'd just keep it in object-store.h as moving out just a couple functions seems less consistent?

Stefan

#182 lore raw

Re: [PATCH 16/27] sha1_file: add repository argument to sha1_file_name

Quoted 24 lines · Stefan Beller
Quoted 9 lines · Brandon Williams
Quoted 8 lines · Stefan Beller

Add a repository argument to allow sha1_file_name callers to be more specific about which repository to handle. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

Quoted 6 lines
Quoted 2 lines

While at it, move the declaration to object-store.h, where it should be easier to find.

Seems like we may want to make a sha1-file.h or an oid-file.h or something like that at some point as that seems like a better place for the function than in the object-store.h file?

It depends what our long term goal is.
Do we want header and source file name to match for each function? Or do we want a coarser set of headers, such that we have a broad object-store.h, but that is implemented in sha1_file.c, packfile.c for the parts of the raw_objectstore and other .c files for the higher levels of the object store?

Quoted 2 lines

For now I'd just keep it in object-store.h as moving out just a couple functions seems less consistent?

Fair enough :)

Signature · 1 line
Brandon Williams
#183 lore raw

PATCH v3 17/27 sha1_file: add repository argument to stat_sha1_file

In reply to Stefan Beller

Add a repository argument to allow the stat_sha1_file caller to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

---
 sha1_file.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
sha1_file.c +5 -4
index 811ed9e53f..ed297773e5 100644
@@ -874,8 +874,9 @@ int git_open_cloexec(const char *name, int flags)
  * Note that it may point to static storage and is only valid until another
  * call to sha1_file_name(), etc.
  */
-static int stat_sha1_file(const unsigned char *sha1, struct stat *st,
-			  const char **path)
+#define stat_sha1_file(r, s, st, p) stat_sha1_file_##r(s, st, p)
+static int stat_sha1_file_the_repository(const unsigned char *sha1,
+					 struct stat *st, const char **path)
 {
 	struct alternate_object_database *alt;
 	static struct strbuf buf = STRBUF_INIT;
@@ -1181,7 +1182,7 @@ static int sha1_loose_object_info(const unsigned char *sha1,
 	if (!oi->typep && !oi->typename && !oi->sizep && !oi->contentp) {
 		const char *path;
 		struct stat st;
-		if (stat_sha1_file(sha1, &st, &path) < 0)
+		if (stat_sha1_file(the_repository, sha1, &st, &path) < 0)
 			return -1;
 		if (oi->disk_sizep)
 			*oi->disk_sizep = st.st_size;
@@ -1395,7 +1396,7 @@ void *read_sha1_file_extended(const unsigned char *sha1,
 		die("replacement %s not found for %s",
 		    sha1_to_hex(repl), sha1_to_hex(sha1));
 
-	if (!stat_sha1_file(repl, &st, &path))
+	if (!stat_sha1_file(the_repository, repl, &st, &path))
 		die("loose object %s (stored in %s) is corrupt",
 		    sha1_to_hex(repl), path);
 
Signature · 1 line
2.16.1.291.g4437f3f132-goog
Reviewed-by on v3 17/27
#184 lore raw

Re: [PATCH 17/27] sha1_file: add repository argument to stat_sha1_file

Hi,

Quoted 5 lines · Stefan Beller

Add a repository argument to allow the stat_sha1_file caller to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

Quoted 8 lines

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

---
 sha1_file.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

Simple and obviously correct.

#185 lore raw

PATCH v3 18/27 sha1_file: add repository argument to open_sha1_file

In reply to Stefan Beller

Add a repository argument to allow the open_sha1_file caller to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

---
 sha1_file.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
sha1_file.c +4 -2
index ed297773e5..68aff90792 100644
@@ -903,7 +903,9 @@ static int stat_sha1_file_the_repository(const unsigned char *sha1,
  * Like stat_sha1_file(), but actually open the object and return the
  * descriptor. See the caveats on the "path" parameter above.
  */
-static int open_sha1_file(const unsigned char *sha1, const char **path)
+#define open_sha1_file(r, s, p) open_sha1_file_##r(s, p)
+static int open_sha1_file_the_repository(const unsigned char *sha1,
+					 const char **path)
 {
 	int fd;
 	struct alternate_object_database *alt;
@@ -946,7 +948,7 @@ static void *map_sha1_file_1(const char *path,
 	if (path)
 		fd = git_open(path);
 	else
-		fd = open_sha1_file(sha1, &path);
+		fd = open_sha1_file(the_repository, sha1, &path);
 	map = NULL;
 	if (fd >= 0) {
 		struct stat st;
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#186 lore raw

PATCH v3 19/27 sha1_file: add repository argument to map_sha1_file_1

In reply to Stefan Beller

Add a repository argument to allow the map_sha1_file_1 caller to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

---
 sha1_file.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
sha1_file.c +6 -5
index 68aff90792..7fa9ed2f25 100644
@@ -938,9 +938,10 @@ static int open_sha1_file_the_repository(const unsigned char *sha1,
  * Map the loose object at "path" if it is not NULL, or the path found by
  * searching for a loose object named "sha1".
  */
-static void *map_sha1_file_1(const char *path,
-			     const unsigned char *sha1,
-			     unsigned long *size)
+#define map_sha1_file_1(r, p, s, si) map_sha1_file_1_##r(p, s, si)
+static void *map_sha1_file_1_the_repository(const char *path,
+					    const unsigned char *sha1,
+					    unsigned long *size)
 {
 	void *map;
 	int fd;
@@ -969,7 +970,7 @@ static void *map_sha1_file_1(const char *path,
 
 void *map_sha1_file(const unsigned char *sha1, unsigned long *size)
 {
-	return map_sha1_file_1(NULL, sha1, size);
+	return map_sha1_file_1(the_repository, NULL, sha1, size);
 }
 
 static int unpack_sha1_short_header(git_zstream *stream,
@@ -2199,7 +2200,7 @@ int read_loose_object(const char *path,
 
 	*contents = NULL;
 
-	map = map_sha1_file_1(path, NULL, &mapsize);
+	map = map_sha1_file_1(the_repository, path, NULL, &mapsize);
 	if (!map) {
 		error_errno("unable to mmap %s", path);
 		goto out;
Signature · 1 line
2.16.1.291.g4437f3f132-goog
Reviewed-by on v3 19/27
#187 lore raw

Re: [PATCH 19/27] sha1_file: add repository argument to map_sha1_file_1

Quoted 10 lines · On Tue, 20 Feb 2018 17:54:22 -0800 Stefan Beller <[email protected]> wrote:

Add a repository argument to allow the map_sha1_file_1 caller to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

#188 lore raw

PATCH v3 20/27 sha1_file: add repository argument to map_sha1_file

In reply to Stefan Beller

Add a repository argument to allow map_sha1_file callers to be more specific about which repository to handle. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

While at it, move the declaration to object-store.h, where it should be easier to find.

---
 cache.h        | 1 -
 object-store.h | 3 +++
 sha1_file.c    | 4 ++--
 streaming.c    | 5 ++++-
 4 files changed, 9 insertions(+), 4 deletions(-)
cache.h +0 -1
index e641c23b0a..d338b65d75 100644
@@ -1242,7 +1242,6 @@ extern int pretend_sha1_file(void *, unsigned long, enum object_type, unsigned c
 extern int force_object_loose(const unsigned char *sha1, time_t mtime);
 extern int git_open_cloexec(const char *name, int flags);
 #define git_open(name) git_open_cloexec(name, O_RDONLY)
-extern void *map_sha1_file(const unsigned char *sha1, unsigned long *size);
 extern int unpack_sha1_header(git_zstream *stream, unsigned char *map, unsigned long mapsize, void *buffer, unsigned long bufsiz);
 extern int parse_sha1_header(const char *hdr, unsigned long *sizep);
 
object-store.h +3 -0
index 4f5060e768..a37d74a0c5 100644
@@ -130,4 +130,7 @@ void raw_object_store_clear(struct raw_object_store *o);
 #define sha1_file_name(r, b, s) sha1_file_name_##r(b, s)
 void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1);
 
+#define map_sha1_file(r, s, sz) map_sha1_file_##r(s, sz)
+void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size);
+
 #endif /* OBJECT_STORE_H */
sha1_file.c +2 -2
index 7fa9ed2f25..0d3ff99cf2 100644
@@ -968,7 +968,7 @@ static void *map_sha1_file_1_the_repository(const char *path,
 	return map;
 }
 
-void *map_sha1_file(const unsigned char *sha1, unsigned long *size)
+void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size)
 {
 	return map_sha1_file_1(the_repository, NULL, sha1, size);
 }
@@ -1192,7 +1192,7 @@ static int sha1_loose_object_info(const unsigned char *sha1,
 		return 0;
 	}
 
-	map = map_sha1_file(sha1, &mapsize);
+	map = map_sha1_file(the_repository, sha1, &mapsize);
 	if (!map)
 		return -1;
 
streaming.c +4 -1
index 5892b50bd8..22d27df55e 100644
@@ -3,6 +3,8 @@
  */
 #include "cache.h"
 #include "streaming.h"
+#include "repository.h"
+#include "object-store.h"
 #include "packfile.h"
 
 enum input_source {
@@ -335,7 +337,8 @@ static struct stream_vtbl loose_vtbl = {
 
 static open_method_decl(loose)
 {
-	st->u.loose.mapped = map_sha1_file(sha1, &st->u.loose.mapsize);
+	st->u.loose.mapped = map_sha1_file(the_repository,
+					   sha1, &st->u.loose.mapsize);
 	if (!st->u.loose.mapped)
 		return -1;
 	if ((unpack_sha1_header(&st->z,
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#189 lore raw

PATCH v3 21/27 sha1_file: add repository argument to sha1_loose_object_info

In reply to Stefan Beller

Add a repository argument to allow the sha1_loose_object_info caller to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

---
 sha1_file.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
sha1_file.c +5 -4
index 0d3ff99cf2..d1f77a4c80 100644
@@ -1159,9 +1159,10 @@ int parse_sha1_header(const char *hdr, unsigned long *sizep)
 	return parse_sha1_header_extended(hdr, &oi, 0);
 }
 
-static int sha1_loose_object_info(const unsigned char *sha1,
-				  struct object_info *oi,
-				  int flags)
+#define sha1_loose_object_info(r, s, o, f) sha1_loose_object_info_##r(s, o, f)
+static int sha1_loose_object_info_the_repository(const unsigned char *sha1,
+						 struct object_info *oi,
+						 int flags)
 {
 	int status = 0;
 	unsigned long mapsize;
@@ -1280,7 +1281,7 @@ int sha1_object_info_extended(const unsigned char *sha1, struct object_info *oi,
 			break;
 
 		/* Most likely it's a loose object. */
-		if (!sha1_loose_object_info(real, oi, flags))
+		if (!sha1_loose_object_info(the_repository, real, oi, flags))
 			return 0;
 
 		/* Not a loose object; someone else may have just packed it. */
Signature · 1 line
2.16.1.291.g4437f3f132-goog
Reviewed-by on v3 21/27
#190 lore raw

Re: [PATCH 21/27] sha1_file: add repository argument to sha1_loose_object_info

Hi,

Quoted 5 lines · Stefan Beller

Add a repository argument to allow the sha1_loose_object_info caller to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

Quoted 8 lines

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

---
 sha1_file.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

Simple and obviously correct.

#191 lore raw

PATCH v3 22/27 sha1_file: allow sha1_file_name to handle arbitrary repositories

In reply to Stefan Beller
---
 object-store.h | 3 +--
 sha1_file.c    | 4 ++--
 2 files changed, 3 insertions(+), 4 deletions(-)
object-store.h +1 -2
index a37d74a0c5..146358fe33 100644
@@ -127,8 +127,7 @@ void raw_object_store_clear(struct raw_object_store *o);
  * Put in `buf` the name of the file in the local object database that
  * would be used to store a loose object with the specified sha1.
  */
-#define sha1_file_name(r, b, s) sha1_file_name_##r(b, s)
-void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1);
+void sha1_file_name(struct repository *r, struct strbuf *buf, const unsigned char *sha1);
 
 #define map_sha1_file(r, s, sz) map_sha1_file_##r(s, sz)
 void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size);
sha1_file.c +2 -2
index d1f77a4c80..8fad523d85 100644
@@ -323,9 +323,9 @@ static void fill_sha1_path(struct strbuf *buf, const unsigned char *sha1)
 	}
 }
 
-void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1)
+void sha1_file_name(struct repository *r, struct strbuf *buf, const unsigned char *sha1)
 {
-	strbuf_addstr(buf, get_object_directory());
+	strbuf_addstr(buf, r->objects.objectdir);
 	strbuf_addch(buf, '/');
 	fill_sha1_path(buf, sha1);
 }
Signature · 1 line
2.16.1.291.g4437f3f132-goog
Reviewed-by on v3 22/27
#192 lore raw

Re: [PATCH 22/27] sha1_file: allow sha1_file_name to handle arbitrary repositories

Quoted 2 lines · On Tue, 20 Feb 2018 17:54:25 -0800 Stefan Beller <[email protected]> wrote:

sha1_file.c static void fill_sha1_path(struct strbuf *buf, const unsigned char *sha1) · line 331

Quoted 8 lines
sha1_file.c +2 -2
-void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1)
+void sha1_file_name(struct repository *r, struct strbuf *buf, const unsigned char *sha1)
 {
-	strbuf_addstr(buf, get_object_directory());
+	strbuf_addstr(buf, r->objects.objectdir);
 	strbuf_addch(buf, '/');
 	fill_sha1_path(buf, sha1);
 }

In the future, we should probably have:
- a function to get the object store out of a repo (so that it can lazily initialize the object store struct if necessary)
- when the object store is obtained, its objectdir field is guaranteed to be populated
- sha1_file_name should take the object store struct, not the repo struct

but this is outside the scope of this patch.

#193 lore raw

Re: [PATCH 22/27] sha1_file: allow sha1_file_name to handle arbitrary repositories

Quoted 8 lines · Jonathan Tan
Quoted 2 lines · On Tue, 20 Feb 2018 17:54:25 -0800 Stefan Beller <[email protected]> wrote:
Quoted 16 lines

sha1_file.c static void fill_sha1_path(struct strbuf *buf, const unsigned char *sha1) · line 331

Quoted 8 lines
sha1_file.c +2 -2
-void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1)
+void sha1_file_name(struct repository *r, struct strbuf *buf, const unsigned char *sha1)
 {
-     strbuf_addstr(buf, get_object_directory());
+     strbuf_addstr(buf, r->objects.objectdir);
      strbuf_addch(buf, '/');
      fill_sha1_path(buf, sha1);
 }

In the future, we should probably have:
- a function to get the object store out of a repo (so that it can lazily initialize the object store struct if necessary)
- when the object store is obtained, its objectdir field is guaranteed to be populated
- sha1_file_name should take the object store struct, not the repo struct

We had that in v2, but I reverted it as the consensus seemed that we prefer the_repository to be passed around.

Quoted 2 lines

but this is outside the scope of this patch.

#194 lore raw

PATCH v3 23/27 sha1_file: allow stat_sha1_file to handle arbitrary repositories

In reply to Stefan Beller
---
 sha1_file.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)
sha1_file.c +5 -6
index 8fad523d85..6879af8993 100644
@@ -874,23 +874,22 @@ int git_open_cloexec(const char *name, int flags)
  * Note that it may point to static storage and is only valid until another
  * call to sha1_file_name(), etc.
  */
-#define stat_sha1_file(r, s, st, p) stat_sha1_file_##r(s, st, p)
-static int stat_sha1_file_the_repository(const unsigned char *sha1,
-					 struct stat *st, const char **path)
+static int stat_sha1_file(struct repository *r, const unsigned char *sha1,
+			  struct stat *st, const char **path)
 {
 	struct alternate_object_database *alt;
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(the_repository, &buf, sha1);
+	sha1_file_name(r, &buf, sha1);
 	*path = buf.buf;
 
 	if (!lstat(*path, st))
 		return 0;
 
-	prepare_alt_odb(the_repository);
+	prepare_alt_odb(r);
 	errno = ENOENT;
-	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
+	for (alt = r->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		if (!lstat(*path, st))
 			return 0;
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#195 lore raw

PATCH v3 24/27 sha1_file: allow open_sha1_file to handle arbitrary repositories

In reply to Stefan Beller
---
 sha1_file.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)
sha1_file.c +5 -6
index 6879af8993..598acc5410 100644
@@ -902,9 +902,8 @@ static int stat_sha1_file(struct repository *r, const unsigned char *sha1,
  * Like stat_sha1_file(), but actually open the object and return the
  * descriptor. See the caveats on the "path" parameter above.
  */
-#define open_sha1_file(r, s, p) open_sha1_file_##r(s, p)
-static int open_sha1_file_the_repository(const unsigned char *sha1,
-					 const char **path)
+static int open_sha1_file(struct repository *r,
+			  const unsigned char *sha1, const char **path)
 {
 	int fd;
 	struct alternate_object_database *alt;
@@ -912,7 +911,7 @@ static int open_sha1_file_the_repository(const unsigned char *sha1,
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(the_repository, &buf, sha1);
+	sha1_file_name(r, &buf, sha1);
 	*path = buf.buf;
 
 	fd = git_open(*path);
@@ -920,8 +919,8 @@ static int open_sha1_file_the_repository(const unsigned char *sha1,
 		return fd;
 	most_interesting_errno = errno;
 
-	prepare_alt_odb(the_repository);
-	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
+	prepare_alt_odb(r);
+	for (alt = r->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		fd = git_open(*path);
 		if (fd >= 0)
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#197 lore raw

PATCH v3 25/27 sha1_file: allow map_sha1_file_1 to handle arbitrary repositories

In reply to Stefan Beller

From: Jonathan Nieder <[email protected]>

---
 sha1_file.c | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)
sha1_file.c +3 -5
index 598acc5410..522804d9f0 100644
@@ -936,10 +936,8 @@ static int open_sha1_file(struct repository *r,
  * Map the loose object at "path" if it is not NULL, or the path found by
  * searching for a loose object named "sha1".
  */
-#define map_sha1_file_1(r, p, s, si) map_sha1_file_1_##r(p, s, si)
-static void *map_sha1_file_1_the_repository(const char *path,
-					    const unsigned char *sha1,
-					    unsigned long *size)
+static void *map_sha1_file_1(struct repository *r, const char *path,
+			     const unsigned char *sha1, unsigned long *size)
 {
 	void *map;
 	int fd;
@@ -947,7 +945,7 @@ static void *map_sha1_file_1_the_repository(const char *path,
 	if (path)
 		fd = git_open(path);
 	else
-		fd = open_sha1_file(the_repository, sha1, &path);
+		fd = open_sha1_file(r, sha1, &path);
 	map = NULL;
 	if (fd >= 0) {
 		struct stat st;
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#198 lore raw

PATCH v3 26/27 sha1_file: allow map_sha1_file to handle arbitrary repositories

In reply to Stefan Beller
---
 object-store.h | 3 +--
 sha1_file.c    | 5 +++--
 2 files changed, 4 insertions(+), 4 deletions(-)
object-store.h +1 -2
index 146358fe33..5d93677783 100644
@@ -129,7 +129,6 @@ void raw_object_store_clear(struct raw_object_store *o);
  */
 void sha1_file_name(struct repository *r, struct strbuf *buf, const unsigned char *sha1);
 
-#define map_sha1_file(r, s, sz) map_sha1_file_##r(s, sz)
-void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size);
+void *map_sha1_file(struct repository *r, const unsigned char *sha1, unsigned long *size);
 
 #endif /* OBJECT_STORE_H */
sha1_file.c +3 -2
index 522804d9f0..a20f811a07 100644
@@ -964,9 +964,10 @@ static void *map_sha1_file_1(struct repository *r, const char *path,
 	return map;
 }
 
-void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size)
+void *map_sha1_file(struct repository *r,
+		    const unsigned char *sha1, unsigned long *size)
 {
-	return map_sha1_file_1(the_repository, NULL, sha1, size);
+	return map_sha1_file_1(r, NULL, sha1, size);
 }
 
 static int unpack_sha1_short_header(git_zstream *stream,
Signature · 1 line
2.16.1.291.g4437f3f132-goog
Reviewed-by on v3 26/27
#199 lore raw

Re: [PATCH 26/27] sha1_file: allow map_sha1_file to handle arbitrary repositories

Quoted 6 lines · Stefan Beller
---
 object-store.h | 3 +--
 sha1_file.c    | 5 +++--
 2 files changed, 4 insertions(+), 4 deletions(-)

Thanks: this is short and simple, making my life as a reviewer very easy.

#200 lore raw

PATCH v3 27/27 sha1_file: allow sha1_loose_object_info to handle arbitrary repositories

In reply to Stefan Beller

From: Jonathan Nieder <[email protected]>

---
 sha1_file.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)
sha1_file.c +5 -6
index a20f811a07..36282acb1a 100644
@@ -1156,10 +1156,9 @@ int parse_sha1_header(const char *hdr, unsigned long *sizep)
 	return parse_sha1_header_extended(hdr, &oi, 0);
 }
 
-#define sha1_loose_object_info(r, s, o, f) sha1_loose_object_info_##r(s, o, f)
-static int sha1_loose_object_info_the_repository(const unsigned char *sha1,
-						 struct object_info *oi,
-						 int flags)
+static int sha1_loose_object_info(struct repository *r,
+				  const unsigned char *sha1,
+				  struct object_info *oi, int flags)
 {
 	int status = 0;
 	unsigned long mapsize;
@@ -1183,14 +1182,14 @@ static int sha1_loose_object_info_the_repository(const unsigned char *sha1,
 	if (!oi->typep && !oi->typename && !oi->sizep && !oi->contentp) {
 		const char *path;
 		struct stat st;
-		if (stat_sha1_file(the_repository, sha1, &st, &path) < 0)
+		if (stat_sha1_file(r, sha1, &st, &path) < 0)
 			return -1;
 		if (oi->disk_sizep)
 			*oi->disk_sizep = st.st_size;
 		return 0;
 	}
 
-	map = map_sha1_file(the_repository, sha1, &mapsize);
+	map = map_sha1_file(r, sha1, &mapsize);
 	if (!map)
 		return -1;
 
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#201 lore raw
In reply to Stefan Beller
Quoted 8 lines · Stefan Beller

v3:
* reverted back to use the repository for most of the functions (including unduplicating 'ignore_env')
* rebased on master again (I lost that state when doing v2, as I did both rebase as well as conversion to object store in one go)
* one additional patch, that moves the alternates related things to object-store.h, such that inclusion of cache.h (in object-store.h) is not required any more.

This is also available at
https://github.com/stefanbeller/git/tree/object-store-part1-v3

#202 lore raw

v4 cover Moving global state into the repository object (part 1)

In reply to Stefan Beller

v4:
* addressed feedback from the single patches (mostly nits)
* rebased on latest master

v3:
* reverted back to use the repository for most of the functions (including unduplicating 'ignore_env')
* rebased on master again (I lost that state when doing v2, as I did both rebase as well as conversion to object store in one go)
* one additional patch, that moves the alternates related things to object-store.h, such that inclusion of cache.h (in object-store.h) is not required any more.

v2:
* duplicated the 'ignore_env' bit into the object store as well
* the #define trick no longer works as we do not have a "the_objectstore" global, which means there is just one patch per function that is converted. As this follows the same structure of the previous series, I am still confident there is no hidden dependencies to globals outside the object store in these converted functions.
* rebased on top of current master, resolving the merge conflicts.
I think I used the list.h APIs right, but please double check.

Thanks,
Stefan

v1:
This is a real take on the first part of the recent RFC[1].

Jonathan Tan suggested[2] that "sha1_loose_object_info to handle arbitrary repositories" might be a good breaking point for a first part at that RFC at patch 38. This series is smaller and contains only 26 patches as the patches in the big RFC were slightly out of order.

I developed this series partly by writing patches, but mostly by cherrypicking from that RFC on top of current master. I noticed no external conflicts apart from one addition to the repositories _INIT macro, which was easy to resolve.

Comments in the early range of that RFC were on 003 where Junio pointed out that the coccinelle patch ought to be not in contrib/coccinelle, so I put it in a sub directory there, as 'make coccicheck' doesn't traverse subdirs.

brian had a questoin on patch 25 in the RFC, but that seemed to resolve itself without any suggestion to include into this series[3].

Duy suggested that we shall not use the repository blindly, but should carefully examine whether to pass on an object store or the refstore or such[4], which I agree with if it makes sense. This series unfortunately has an issue with that as I would not want to pass down the ignore_env flag separately from the object store, so I made all functions that only take the object store to have the raw object store as the first parameter, and others using the full repository.

Eric Sunshine brought up memory leaks with the RFC, and I would think to have plugged all holes.

[1] https://public-inbox.org/git/[email protected]/
[2] https://public-inbox.org/git/[email protected]/
[3] https://public-inbox.org/git/[email protected]/
[4] https://public-inbox.org/git/CACsJy8CGgekpX4cZkyyTSPrj87uQVKZSOL7fyT__P2dh_1LmVQ@mail.gmail.com/

Thanks,
Stefan

Jonathan Nieder (2):
sha1_file: allow map_sha1_file_1 to handle arbitrary repositories
sha1_file: allow sha1_loose_object_info to handle arbitrary
repositories

Stefan Beller (25):
repository: introduce raw object store field
object-store: migrate alternates struct and functions from cache.h
object-store: move alt_odb_list and alt_odb_tail to object store
object-store: free alt_odb_list
object-store: move packed_git and packed_git_mru to object store
object-store: close all packs upon clearing the object store
pack: move prepare_packed_git_run_once to object store
pack: move approximate object count to object store
sha1_file: add raw_object_store argument to alt_odb_usable
sha1_file: add repository argument to link_alt_odb_entry
sha1_file: add repository argument to read_info_alternates
sha1_file: add repository argument to link_alt_odb_entries
sha1_file: add repository argument to prepare_alt_odb
sha1_file: allow link_alt_odb_entries to handle arbitrary repositories
sha1_file: allow prepare_alt_odb to handle arbitrary repositories
sha1_file: add repository argument to sha1_file_name
sha1_file: add repository argument to stat_sha1_file
sha1_file: add repository argument to open_sha1_file
sha1_file: add repository argument to map_sha1_file_1
sha1_file: add repository argument to map_sha1_file
sha1_file: add repository argument to sha1_loose_object_info
sha1_file: allow sha1_file_name to handle arbitrary repositories
sha1_file: allow stat_sha1_file to handle arbitrary repositories
sha1_file: allow open_sha1_file to handle arbitrary repositories
sha1_file: allow map_sha1_file to handle arbitrary repositories

 builtin/am.c             |   2 +-
 builtin/clone.c          |   2 +-
 builtin/count-objects.c  |   5 +-
 builtin/fetch.c          |   2 +-
 builtin/fsck.c           |  12 ++--
 builtin/gc.c             |   3 +-
 builtin/grep.c           |   2 +-
 builtin/merge.c          |   2 +-
 builtin/pack-objects.c   |  20 ++++---
 builtin/pack-redundant.c |   5 +-
 builtin/receive-pack.c   |   2 +-
 cache.h                  |  87 ---------------------------
 environment.c            |   5 +-
 fast-import.c            |   7 ++-
 http-backend.c           |   5 +-
 http-walker.c            |   3 +-
 http.c                   |   5 +-
 object-store.h           | 124 ++++++++++++++++++++++++++++++++++++++
 object.c                 |  27 +++++++++
 pack-bitmap.c            |   3 +-
 packfile.c               |  64 ++++++++++----------
 packfile.h               |   2 +-
 path.c                   |   2 +-
 repository.c             |  21 +++++--
 repository.h             |   7 ++-
 server-info.c            |   5 +-
 sha1_file.c              | 125 +++++++++++++++++++++------------------
 sha1_name.c              |  10 ++--
 streaming.c              |   5 +-
 29 files changed, 335 insertions(+), 229 deletions(-)
 create mode 100644 object-store.h
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#203 lore raw

PATCH v4 1/27 repository: introduce raw object store field

The raw object store field will contain any objects needed for access to objects in a given repository.

This patch introduces the raw object store and populates it with the objectdir, which used to be part of the repository struct.

As the struct gains members, we'll also populate the function to clear the memory for these members.

In a later we'll introduce a struct object_parser, that will complement the object parsing in a repository struct: The raw object parser is the layer that will provide access to raw object content, while the higher level object parser code will parse raw objects and keeps track of parenthood and other object relationships using 'struct object'. For now only add the lower level to the repository struct.

---
 builtin/grep.c |  2 +-
 environment.c  |  5 +++--
 object-store.h | 15 +++++++++++++++
 object.c       |  5 +++++
 path.c         |  2 +-
 repository.c   | 19 ++++++++++++++-----
 repository.h   |  7 ++++---
 7 files changed, 43 insertions(+), 12 deletions(-)
 create mode 100644 object-store.h
builtin/grep.c +1 -1
index 3ca4ac80d8c..0f0c195705f 100644
@@ -432,7 +432,7 @@ static int grep_submodule(struct grep_opt *opt, struct repository *superproject,
 	 * object.
 	 */
 	grep_read_lock();
-	add_to_alternates_memory(submodule.objectdir);
+	add_to_alternates_memory(submodule.objects.objectdir);
 	grep_read_unlock();
 
 	if (oid) {
environment.c +3 -2
index de8431e01e7..ec10b062e68 100644
@@ -13,6 +13,7 @@
 #include "refs.h"
 #include "fmt-merge-msg.h"
 #include "commit.h"
+#include "object-store.h"
 
 int trust_executable_bit = 1;
 int trust_ctime = 1;
@@ -244,9 +245,9 @@ const char *get_git_work_tree(void)
 
 char *get_object_directory(void)
 {
-	if (!the_repository->objectdir)
+	if (!the_repository->objects.objectdir)
 		BUG("git environment hasn't been setup");
-	return the_repository->objectdir;
+	return the_repository->objects.objectdir;
 }
 
 int odb_mkstemp(struct strbuf *template, const char *pattern)
object-store.h new file +15 -0
new file mode 100644
index 00000000000..cf35760ceb6
@@ -0,0 +1,15 @@
+#ifndef OBJECT_STORE_H
+#define OBJECT_STORE_H
+
+struct raw_object_store {
+	/*
+	 * Path to the repository's object store.
+	 * Cannot be NULL after initialization.
+	 */
+	char *objectdir;
+};
+#define RAW_OBJECT_STORE_INIT { NULL }
+
+void raw_object_store_clear(struct raw_object_store *o);
+
+#endif /* OBJECT_STORE_H */
object.c +5 -0
index 9e6f9ff20b0..a2acdee1405 100644
@@ -445,3 +445,8 @@ void clear_commit_marks_all(unsigned int flags)
 			obj->flags &= ~flags;
 	}
 }
+
+void raw_object_store_clear(struct raw_object_store *o)
+{
+	FREE_AND_NULL(o->objectdir);
+}
path.c +1 -1
index da8b655730d..81a42d91155 100644
@@ -382,7 +382,7 @@ static void adjust_git_path(const struct repository *repo,
 		strbuf_splice(buf, 0, buf->len,
 			      repo->index_file, strlen(repo->index_file));
 	else if (dir_prefix(base, "objects"))
-		replace_dir(buf, git_dir_len + 7, repo->objectdir);
+		replace_dir(buf, git_dir_len + 7, repo->objects.objectdir);
 	else if (git_hooks_path && dir_prefix(base, "hooks"))
 		replace_dir(buf, git_dir_len + 5, git_hooks_path);
 	else if (repo->different_commondir)
repository.c +14 -5
index 4ffbe9bc94e..fb65e8072d5 100644
@@ -1,11 +1,18 @@
 #include "cache.h"
 #include "repository.h"
+#include "object-store.h"
 #include "config.h"
 #include "submodule-config.h"
 
 /* The main repository */
 static struct repository the_repo = {
-	NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &the_index, &hash_algos[GIT_HASH_SHA1], 0, 0
+	NULL, NULL,
+	RAW_OBJECT_STORE_INIT,
+	NULL, NULL, NULL,
+	NULL, NULL, NULL,
+	&the_index,
+	&hash_algos[GIT_HASH_SHA1],
+	0, 0
 };
 struct repository *the_repository = &the_repo;
 
@@ -42,9 +49,10 @@ static void repo_setup_env(struct repository *repo)
 						    !repo->ignore_env);
 	free(repo->commondir);
 	repo->commondir = strbuf_detach(&sb, NULL);
-	free(repo->objectdir);
-	repo->objectdir = git_path_from_env(DB_ENVIRONMENT, repo->commondir,
-					    "objects", !repo->ignore_env);
+	raw_object_store_clear(&repo->objects);
+	repo->objects.objectdir =
+		git_path_from_env(DB_ENVIRONMENT, repo->commondir,
+				  "objects", !repo->ignore_env);
 	free(repo->graft_file);
 	repo->graft_file = git_path_from_env(GRAFT_ENVIRONMENT, repo->commondir,
 					     "info/grafts", !repo->ignore_env);
@@ -209,12 +217,13 @@ void repo_clear(struct repository *repo)
 {
 	FREE_AND_NULL(repo->gitdir);
 	FREE_AND_NULL(repo->commondir);
-	FREE_AND_NULL(repo->objectdir);
 	FREE_AND_NULL(repo->graft_file);
 	FREE_AND_NULL(repo->index_file);
 	FREE_AND_NULL(repo->worktree);
 	FREE_AND_NULL(repo->submodule_prefix);
 
+	raw_object_store_clear(&repo->objects);
+
 	if (repo->config) {
 		git_configset_clear(repo->config);
 		FREE_AND_NULL(repo->config);
repository.h +4 -3
index 0329e40c7f5..fa73ab8e938 100644
@@ -1,6 +1,8 @@
 #ifndef REPOSITORY_H
 #define REPOSITORY_H
 
+#include "object-store.h"
+
 struct config_set;
 struct index_state;
 struct submodule_cache;
@@ -21,10 +23,9 @@ struct repository {
 	char *commondir;
 
 	/*
-	 * Path to the repository's object store.
-	 * Cannot be NULL after initialization.
+	 * Holds any information related to accessing the raw object content.
 	 */
-	char *objectdir;
+	struct raw_object_store objects;
 
 	/*
 	 * Path to the repository's graft file.
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#204 lore raw

Re: [PATCHv4 01/27] repository: introduce raw object store field

Review

Quoted 12 lines · Stefan Beller

/* The main repository */
static struct repository the_repo = {

(quoted diff) +7 -1
-	NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &the_index, &hash_algos[GIT_HASH_SHA1], 0, 0
+	NULL, NULL,
+	RAW_OBJECT_STORE_INIT,
+	NULL, NULL, NULL,
+	NULL, NULL, NULL,
+	&the_index,
+	&hash_algos[GIT_HASH_SHA1],
+	0, 0
 };
 struct repository *the_repository = &the_repo;

I wonder if we should do something like this. It makes the_repo initialization easier to read and it helps unify the init code with submodule.

No don't reroll. If you think it's a good idea, you can do something like this in the next series instead.

-- 8< --

check-racy.c +2 -0
index 24b6542352..47cbb4eb6d 100644
@@ -1,10 +1,12 @@
 #include "cache.h"
+#include "repository.h"
 
 int main(int ac, char **av)
 {
 	int i;
 	int dirty, clean, racy;
 
+	init_the_repository();
 	dirty = clean = racy = 0;
 	read_cache();
 	for (i = 0; i < active_nr; i++) {
common-main.c +3 -0
index 6a689007e7..a13ab981aa 100644
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "exec_cmd.h"
 #include "attr.h"
+#include "repository.h"
 
 /*
  * Many parts of Git have subprograms communicate via pipe, expect the
@@ -32,6 +33,8 @@ int main(int argc, const char **argv)
 	 */
 	sanitize_stdfds();
 
+	init_the_repository();
+
 	git_setup_gettext();
 
 	attr_start();
object-store.h +1 -1
index cf35760ceb..c3253ebc59 100644
@@ -8,8 +8,8 @@ struct raw_object_store {
 	 */
 	char *objectdir;
 };
-#define RAW_OBJECT_STORE_INIT { NULL }
 
+void raw_object_store_init(struct raw_object_store *o);
 void raw_object_store_clear(struct raw_object_store *o);
 
 #endif /* OBJECT_STORE_H */
object.c +5 -0
index 11d904c033..8a4d01dd5f 100644
@@ -446,6 +446,11 @@ void clear_commit_marks_all(unsigned int flags)
 	}
 }
 
+void raw_object_store_init(struct raw_object_store *o)
+{
+	memset(o, 0, sizeof(*o));
+}
+
 void raw_object_store_clear(struct raw_object_store *o)
 {
 	free(o->objectdir);
repository.c +18 -11
index 2255ff657e..0ebcca8539 100644
@@ -5,16 +5,22 @@
 #include "submodule-config.h"
 
 /* The main repository */
-static struct repository the_repo = {
-	NULL, NULL,
-	RAW_OBJECT_STORE_INIT,
-	NULL, NULL, NULL,
-	NULL, NULL, NULL,
-	&the_index,
-	&hash_algos[GIT_HASH_SHA1],
-	0, 0
-};
-struct repository *the_repository = &the_repo;
+static struct repository the_repo;
+struct repository *the_repository;
+
+static void repo_pre_init(struct repository *repo)
+{
+	memset(repo, 0, sizeof(*repo));
+	raw_object_store_init(&repo->objects);
+}
+
+void init_the_repository(void)
+{
+	the_repository = &the_repo;
+	repo_pre_init(the_repository);
+	the_repository->index = &the_index;
+	repo_set_hash_algo(the_repository, GIT_HASH_SHA1);
+}
 
 static char *git_path_from_env(const char *envvar, const char *git_dir,
 			       const char *path, int fromenv)
@@ -138,7 +144,8 @@ static int read_and_verify_repository_format(struct repository_format *format,
 int repo_init(struct repository *repo, const char *gitdir, const char *worktree)
 {
 	struct repository_format format;
-	memset(repo, 0, sizeof(*repo));
+
+	repo_pre_init(repo);
 
 	repo->ignore_env = 1;
 
repository.h +1 -0
index 1f8bc7a7cf..da6a8f9af9 100644
@@ -93,6 +93,7 @@ extern void repo_set_gitdir(struct repository *repo, const char *path);
 extern void repo_set_worktree(struct repository *repo, const char *path);
 extern void repo_set_hash_algo(struct repository *repo, int algo);
 extern int repo_init(struct repository *repo, const char *gitdir, const char *worktree);
+extern void init_the_repository(void);
 extern int repo_submodule_init(struct repository *submodule,
 			       struct repository *superproject,
 			       const char *path);

-- 8< --

#205 lore raw

Re: [PATCHv4 01/27] repository: introduce raw object store field

Quoted 17 lines · Duy Nguyen
common-main.c +2 -0
index 6a689007e7..a13ab981aa 100644
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "exec_cmd.h"
 #include "attr.h"
+#include "repository.h"
 
 /*
  * Many parts of Git have subprograms communicate via pipe, expect the
@@ -32,6 +33,8 @@ int main(int argc, const char **argv)
 	 */
 	sanitize_stdfds();
 
+	init_the_repository();

common-main.c int main(int argc, const char **argv)

Quoted 10 lines
common-main.c +8 -0
+
 	git_setup_gettext();
...
+void init_the_repository(void)
+{
+	the_repository = &the_repo;
+	repo_pre_init(the_repository);
+	the_repository->index = &the_index;
+	repo_set_hash_algo(the_repository, GIT_HASH_SHA1);
+}

I see what you did here, and I like it.

#206 lore raw

Re: [PATCHv4 01/27] repository: introduce raw object store field

Quoted 18 lines · Junio C Hamano
Quoted 16 lines · Duy Nguyen
common-main.c +1 -0
index 6a689007e7..a13ab981aa 100644
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "exec_cmd.h"
 #include "attr.h"
+#include "repository.h"
 
 /*
  * Many parts of Git have subprograms communicate via pipe, expect the
@@ -32,6 +33,8 @@ int main(int argc, const char **argv)
 	 */
 	sanitize_stdfds();
 
Quoted 13 lines

common-main.c int main(int argc, const char **argv)

Quoted 11 lines
common-main.c +9 -0
+	init_the_repository();
+
 	git_setup_gettext();
...
+void init_the_repository(void)
+{
+	the_repository = &the_repo;
+	repo_pre_init(the_repository);
+	the_repository->index = &the_index;
+	repo_set_hash_algo(the_repository, GIT_HASH_SHA1);
+}

I see what you did here, and I like it.

I thought this would be a good idea to do eventually but back when I first introduced struct repository there wasn't enough to justify it till now. This definitely makes it much easier to read the initialization and I prefer this over the initializer. Thanks for working on this :)

Signature · 1 line
Brandon Williams
#207 lore raw

Re: [PATCHv4 01/27] repository: introduce raw object store field

In reply to Duy Nguyen

check-racy.c

Quoted 26 lines · Duy Nguyen

Review

Quoted 12 lines · Stefan Beller

/* The main repository */
static struct repository the_repo = {

(quoted diff) +7 -1
-     NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &the_index, &hash_algos[GIT_HASH_SHA1], 0, 0
+     NULL, NULL,
+     RAW_OBJECT_STORE_INIT,
+     NULL, NULL, NULL,
+     NULL, NULL, NULL,
+     &the_index,
+     &hash_algos[GIT_HASH_SHA1],
+     0, 0
 };
 struct repository *the_repository = &the_repo;

I wonder if we should do something like this. It makes the_repo initialization easier to read and it helps unify the init code with submodule.

No don't reroll. If you think it's a good idea, you can do something like this in the next series instead.

-- 8< --

check-racy.c +0 -0
index 24b6542352..47cbb4eb6d 100644

totally offtopic: Do we want to move this file into t/helper?

Quoted 11 lines
common-main.c +1 -0
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "exec_cmd.h"
 #include "attr.h"
+#include "repository.h"
 
 /*
  * Many parts of Git have subprograms communicate via pipe, expect the

common-main.c int main(int argc, const char **argv) · line 37

Quoted 5 lines
common-main.c +2 -0
@@ -32,6 +33,8 @@ int main(int argc, const char **argv)
         */
        sanitize_stdfds();
 
+       init_the_repository();
+

So this is the real deal.

Quoted 11 lines
common-main.c +1 -1
-#define RAW_OBJECT_STORE_INIT { NULL }
 
+void raw_object_store_init(struct raw_object_store *o);
 void raw_object_store_clear(struct raw_object_store *o);

#endif /* OBJECT_STORE_H */

object.c +0 -0
index 11d904c033..8a4d01dd5f 100644

object.c void clear_commit_marks_all(unsigned int flags) · line 452

Quoted 7 lines
object.c +4 -0
@@ -446,6 +446,11 @@ void clear_commit_marks_all(unsigned int flags)
        }
 }
 
+void raw_object_store_init(struct raw_object_store *o)
+{
+       memset(o, 0, sizeof(*o));
+}

We'll have to adapt this for the list that we use for packed_git_mru, but that should be no problem.

Quoted 18 lines
object.c +14 -0
+
+static void repo_pre_init(struct repository *repo)
+{
+       memset(repo, 0, sizeof(*repo));
+       raw_object_store_init(&repo->objects);
+}
+
+void init_the_repository(void)
+{
+       the_repository = &the_repo;
+       repo_pre_init(the_repository);
+       the_repository->index = &the_index;
+       repo_set_hash_algo(the_repository, GIT_HASH_SHA1);
+}
 
 static char *git_path_from_env(const char *envvar, const char *git_dir,
                               const char *path, int fromenv)

object.c static int read_and_verify_repository_format(struct repository_format *format, · line 151

Quoted 9 lines
object.c +2 -1
@@ -138,7 +144,8 @@ static int read_and_verify_repository_format(struct repository_format *format,
 int repo_init(struct repository *repo, const char *gitdir, const char *worktree)
 {
        struct repository_format format;
-       memset(repo, 0, sizeof(*repo));
+
+       repo_pre_init(repo);
 
        repo->ignore_env = 1;
 

I like the approach. Though as you say, let's have it in the next series.

Thanks for the thoughts and guidance,
Stefan

#208 lore raw

Re: [PATCHv4 01/27] repository: introduce raw object store field

Quoted 22 lines · Stefan Beller
Quoted 21 lines · Duy Nguyen

Review

Quoted 12 lines · Stefan Beller

/* The main repository */
static struct repository the_repo = {

(quoted diff) +7 -1
-     NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &the_index, &hash_algos[GIT_HASH_SHA1], 0, 0
+     NULL, NULL,
+     RAW_OBJECT_STORE_INIT,
+     NULL, NULL, NULL,
+     NULL, NULL, NULL,
+     &the_index,
+     &hash_algos[GIT_HASH_SHA1],
+     0, 0
 };
 struct repository *the_repository = &the_repo;

I wonder if we should do something like this. It makes the_repo initialization easier to read and it helps unify the init code with submodule.

No don't reroll. If you think it's a good idea, you can do something like this in the next series instead.

Quoted 7 lines

check-racy.c

Quoted 5 lines

-- 8< --

check-racy.c +0 -0
index 24b6542352..47cbb4eb6d 100644

totally offtopic: Do we want to move this file into t/helper?

No wonder both Jeff and I missed this program (he didn't convert it to use cmd_main, and I didn't move it to t/helper). This git-check-racy is added in 42f774063d (Add check program "git-check-racy" - 2006-08-15) and is not part of the default build. You need to manually update Makefile first to build it.

Right now it's broken (multiple definition of 'main'). If you add init_the_repository() or something similar, just leave this file untouched. Maybe I'll fix it later, separately. Or perhaps I'll move this functionality to git-update-index if this is still worth keeping.

Signature · 1 line
Duy
#209 lore raw

PATCH v4 2/27 object-store: migrate alternates struct and functions from cache.h

In reply to Stefan Beller

Migrate the struct alternate_object_database and all its related functions to the object store as these functions are easier found in that header. The migration is just a verbatim copy, no need to include the object store header at any C file, because cache.h includes repository.h which in turn includes the object-store.h

---
 cache.h        | 51 --------------------------------------------------
 object-store.h | 51 ++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 51 insertions(+), 51 deletions(-)
cache.h +0 -51
index 21fbcc24149..99eb2bdb893 100644
@@ -1560,57 +1560,6 @@ extern int has_dirs_only_path(const char *name, int len, int prefix_len);
 extern void schedule_dir_for_removal(const char *name, int len);
 extern void remove_scheduled_dirs(void);
 
-extern struct alternate_object_database {
-	struct alternate_object_database *next;
-
-	/* see alt_scratch_buf() */
-	struct strbuf scratch;
-	size_t base_len;
-
-	/*
-	 * Used to store the results of readdir(3) calls when searching
-	 * for unique abbreviated hashes.  This cache is never
-	 * invalidated, thus it's racy and not necessarily accurate.
-	 * That's fine for its purpose; don't use it for tasks requiring
-	 * greater accuracy!
-	 */
-	char loose_objects_subdir_seen[256];
-	struct oid_array loose_objects_cache;
-
-	char path[FLEX_ARRAY];
-} *alt_odb_list;
-extern void prepare_alt_odb(void);
-extern char *compute_alternate_path(const char *path, struct strbuf *err);
-typedef int alt_odb_fn(struct alternate_object_database *, void *);
-extern int foreach_alt_odb(alt_odb_fn, void*);
-
-/*
- * Allocate a "struct alternate_object_database" but do _not_ actually
- * add it to the list of alternates.
- */
-struct alternate_object_database *alloc_alt_odb(const char *dir);
-
-/*
- * Add the directory to the on-disk alternates file; the new entry will also
- * take effect in the current process.
- */
-extern void add_to_alternates_file(const char *dir);
-
-/*
- * Add the directory to the in-memory list of alternates (along with any
- * recursive alternates it points to), but do not modify the on-disk alternates
- * file.
- */
-extern void add_to_alternates_memory(const char *dir);
-
-/*
- * Returns a scratch strbuf pre-filled with the alternate object directory,
- * including a trailing slash, which can be used to access paths in the
- * alternate. Always use this over direct access to alt->scratch, as it
- * cleans up any previous use of the scratch buffer.
- */
-extern struct strbuf *alt_scratch_buf(struct alternate_object_database *alt);
-
 struct pack_window {
 	struct pack_window *next;
 	unsigned char *base;
object-store.h +51 -0

57 lines not on this page · raw message

Signature · 1 line
2.16.1.291.g4437f3f132-goog
#210 lore raw

PATCH v4 3/27 object-store: move alt_odb_list and alt_odb_tail to object store

In reply to Stefan Beller

In a process with multiple repositories open, alternates should be associated to a single repository and not shared globally. Move alt_odb_list and alt_odb_tail into the_repository and adjust callers to reflect this.

Now that the alternative object data base is per repository, we're leaking its memory upon freeing a repository. The next patch plugs this hole.

No functional change intended.

---
 builtin/fsck.c |  4 +++-
 object-store.h |  9 ++++++---
 packfile.c     |  3 ++-
 sha1_file.c    | 25 ++++++++++++-------------
 sha1_name.c    |  3 ++-
 5 files changed, 25 insertions(+), 19 deletions(-)
builtin/fsck.c +3 -1
index 9981db22637..78edcb75c45 100644
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "commit.h"
 #include "tree.h"
@@ -722,7 +723,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 		fsck_object_dir(get_object_directory());
 
 		prepare_alt_odb();
-		for (alt = alt_odb_list; alt; alt = alt->next)
+		for (alt = the_repository->objects.alt_odb_list;
+		     alt; alt = alt->next)
 			fsck_object_dir(alt->path);
 
 		if (check_full) {
object-store.h +6 -3
index 5678aa1136c..e78eea1dde3 100644
@@ -1,7 +1,7 @@
 #ifndef OBJECT_STORE_H
 #define OBJECT_STORE_H
 
-extern struct alternate_object_database {
+struct alternate_object_database {
 	struct alternate_object_database *next;
 
 	/* see alt_scratch_buf() */
@@ -19,7 +19,7 @@ extern struct alternate_object_database {
 	struct oid_array loose_objects_cache;
 
 	char path[FLEX_ARRAY];
-} *alt_odb_list;
+};
 void prepare_alt_odb(void);
 char *compute_alternate_path(const char *path, struct strbuf *err);
 typedef int alt_odb_fn(struct alternate_object_database *, void *);
@@ -58,8 +58,11 @@ struct raw_object_store {
 	 * Cannot be NULL after initialization.
 	 */
 	char *objectdir;
+
+	struct alternate_object_database *alt_odb_list;
+	struct alternate_object_database **alt_odb_tail;
 };
-#define RAW_OBJECT_STORE_INIT { NULL }
+#define RAW_OBJECT_STORE_INIT { NULL, NULL, NULL }
 
 void raw_object_store_clear(struct raw_object_store *o);
 
packfile.c +2 -1
index 7dbe8739d17..216ea836ee3 100644
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "list.h"
 #include "pack.h"
+#include "repository.h"
 #include "dir.h"
 #include "mergesort.h"
 #include "packfile.h"
@@ -891,7 +892,7 @@ void prepare_packed_git(void)
 		return;
 	prepare_packed_git_one(get_object_directory(), 1);
 	prepare_alt_odb();
-	for (alt = alt_odb_list; alt; alt = alt->next)
+	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next)
 		prepare_packed_git_one(alt->path, 0);
 	rearrange_packed_git();
 	prepare_packed_git_mru();
sha1_file.c +12 -13
index 826d7a0ae37..ac25146076e 100644
@@ -22,6 +22,7 @@
 #include "pack-revindex.h"
 #include "sha1-lookup.h"
 #include "bulk-checkin.h"
+#include "repository.h"
 #include "streaming.h"
 #include "dir.h"
 #include "list.h"
@@ -340,9 +341,6 @@ static const char *alt_sha1_path(struct alternate_object_database *alt,
 	return buf->buf;
 }
 
-struct alternate_object_database *alt_odb_list;
-static struct alternate_object_database **alt_odb_tail;
-
 /*
  * Return non-zero iff the path is usable as an alternate object database.
  */
@@ -362,7 +360,7 @@ static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)
 	 * Prevent the common mistake of listing the same
 	 * thing twice, or object directory itself.
 	 */
-	for (alt = alt_odb_list; alt; alt = alt->next) {
+	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		if (!fspathcmp(path->buf, alt->path))
 			return 0;
 	}
@@ -422,8 +420,8 @@ static int link_alt_odb_entry(const char *entry, const char *relative_base,
 	ent = alloc_alt_odb(pathbuf.buf);
 
 	/* add the alternate entry */
-	*alt_odb_tail = ent;
-	alt_odb_tail = &(ent->next);
+	*the_repository->objects.alt_odb_tail = ent;
+	the_repository->objects.alt_odb_tail = &(ent->next);
 	ent->next = NULL;
 
 	/* recursively add alternates */
@@ -557,7 +555,7 @@ void add_to_alternates_file(const char *reference)
 		fprintf_or_die(out, "%s\n", reference);
 		if (commit_lock_file(&lock))
 			die_errno("unable to move new alternates file into place");
-		if (alt_odb_tail)
+		if (the_repository->objects.alt_odb_tail)
 			link_alt_odb_entries(reference, '\n', NULL, 0);
 	}
 	free(alts);
@@ -655,7 +653,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 	int r = 0;
 
 	prepare_alt_odb();
-	for (ent = alt_odb_list; ent; ent = ent->next) {
+	for (ent = the_repository->objects.alt_odb_list; ent; ent = ent->next) {
 		r = fn(ent, cb);
 		if (r)
 			break;
@@ -667,12 +665,13 @@ void prepare_alt_odb(void)
 {
 	const char *alt;
 
-	if (alt_odb_tail)
+	if (the_repository->objects.alt_odb_tail)
 		return;
 
 	alt = getenv(ALTERNATE_DB_ENVIRONMENT);
 
-	alt_odb_tail = &alt_odb_list;
+	the_repository->objects.alt_odb_tail =
+			&the_repository->objects.alt_odb_list;
 	link_alt_odb_entries(alt, PATH_SEP, NULL, 0);
 
 	read_info_alternates(get_object_directory(), 0);
@@ -716,7 +715,7 @@ static int check_and_freshen_nonlocal(const unsigned char *sha1, int freshen)
 {
 	struct alternate_object_database *alt;
 	prepare_alt_odb();
-	for (alt = alt_odb_list; alt; alt = alt->next) {
+	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		const char *path = alt_sha1_path(alt, sha1);
 		if (check_and_freshen_file(path, freshen))
 			return 1;
@@ -876,7 +875,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,
 
 	prepare_alt_odb();
 	errno = ENOENT;
-	for (alt = alt_odb_list; alt; alt = alt->next) {
+	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		if (!lstat(*path, st))
 			return 0;
@@ -906,7 +905,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)
 	most_interesting_errno = errno;
 
 	prepare_alt_odb();
-	for (alt = alt_odb_list; alt; alt = alt->next) {
+	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		fd = git_open(*path);
 		if (fd >= 0)
sha1_name.c +2 -1
index 611c7d24dde..957ce25680b 100644
@@ -10,6 +10,7 @@
 #include "dir.h"
 #include "sha1-array.h"
 #include "packfile.h"
+#include "repository.h"
 
 static int get_oid_oneline(const char *, struct object_id *, struct commit_list *);
 
@@ -104,7 +105,7 @@ static void find_short_object_filename(struct disambiguate_state *ds)
 		 */
 		fakeent = alloc_alt_odb(get_object_directory());
 	}
-	fakeent->next = alt_odb_list;
+	fakeent->next = the_repository->objects.alt_odb_list;
 
 	for (alt = fakeent; alt && !ds->ambiguous; alt = alt->next) {
 		int pos;
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#211 lore raw

PATCH v4 4/27 object-store: free alt_odb_list

In reply to Stefan Beller

Free the memory and reset alt_odb_{list, tail} to NULL.

---
 object.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
object.c +17 -0
index a2acdee1405..920dc4463fa 100644
@@ -446,7 +446,24 @@ void clear_commit_marks_all(unsigned int flags)
 	}
 }
 
+static void free_alt_odb(struct alternate_object_database *alt)
+{
+	strbuf_release(&alt->scratch);
+	oid_array_clear(&alt->loose_objects_cache);
+}
+
+static void free_alt_odbs(struct raw_object_store *o)
+{
+	while (o->alt_odb_list) {
+		free_alt_odb(o->alt_odb_list);
+		o->alt_odb_list = o->alt_odb_list->next;
+	}
+}
+
 void raw_object_store_clear(struct raw_object_store *o)
 {
 	FREE_AND_NULL(o->objectdir);
+
+	free_alt_odbs(o);
+	o->alt_odb_tail = NULL;
 }
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#212 lore raw

PATCH v4 5/27 object-store: move packed_git and packed_git_mru to object store

In reply to Stefan Beller

In a process with multiple repositories open, packfile accessors should be associated to a single repository and not shared globally. Move packed_git and packed_git_mru into the_repository and adjust callers to reflect this.

Patch generated by

1. Moving the struct packed_git declaration to object-store.h and packed_git, packed_git_mru globals to struct object_store.

2. Apply the following semantic patch to adjust callers:
@@ @@
- packed_git
+ the_repository->objects.packed_git

@@ @@
- packed_git_mru
+ the_repository->objects.packed_git_mru

3. Applying line wrapping fixes from "make style" to break the resulting long lines.

4. Adding missing #includes of repository.h where needed.

5. As the packfiles are now owned by an objectstore, which is ephemeral unlike globals, we introduce memory leaks. So address them in raw_object_store_clear(). Defer freeing packed_git to the next patch due to the size of this patch.

---
 builtin/count-objects.c  |  5 +++--
 builtin/fsck.c           |  6 ++++--
 builtin/gc.c             |  3 ++-
 builtin/pack-objects.c   | 20 +++++++++++---------
 builtin/pack-redundant.c |  5 +++--
 cache.h                  | 29 -----------------------------
 fast-import.c            |  7 +++++--
 http-backend.c           |  5 +++--
 object-store.h           | 31 ++++++++++++++++++++++++++++++-
 object.c                 |  6 ++++++
 pack-bitmap.c            |  3 ++-
 packfile.c               | 39 ++++++++++++++++++++-------------------
 repository.c             |  4 +++-
 server-info.c            |  5 +++--
 sha1_name.c              |  5 +++--
 15 files changed, 98 insertions(+), 75 deletions(-)
builtin/count-objects.c +3 -2
index 33343818c83..6aab8d08fcc 100644
@@ -7,6 +7,7 @@
 #include "cache.h"
 #include "config.h"
 #include "dir.h"
+#include "repository.h"
 #include "builtin.h"
 #include "parse-options.h"
 #include "quote.h"
@@ -120,9 +121,9 @@ int cmd_count_objects(int argc, const char **argv, const char *prefix)
 		struct strbuf loose_buf = STRBUF_INIT;
 		struct strbuf pack_buf = STRBUF_INIT;
 		struct strbuf garbage_buf = STRBUF_INIT;
-		if (!packed_git)
+		if (!the_repository->objects.packed_git)
 			prepare_packed_git();
-		for (p = packed_git; p; p = p->next) {
+		for (p = the_repository->objects.packed_git; p; p = p->next) {
 			if (!p->pack_local)
 				continue;
 			if (open_pack_index(p))
builtin/fsck.c +4 -2
index 78edcb75c45..c3a339193c8 100644
@@ -735,7 +735,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 			prepare_packed_git();
 
 			if (show_progress) {
-				for (p = packed_git; p; p = p->next) {
+				for (p = the_repository->objects.packed_git; p;
+				     p = p->next) {
 					if (open_pack_index(p))
 						continue;
 					total += p->num_objects;
@@ -743,7 +744,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 
 				progress = start_progress(_("Checking objects"), total);
 			}
-			for (p = packed_git; p; p = p->next) {
+			for (p = the_repository->objects.packed_git; p;
+			     p = p->next) {
 				/* verify gives error messages itself */
 				if (verify_pack(p, fsck_obj_buffer,
 						progress, count))
builtin/gc.c +2 -1
index 77fa720bd0b..59bf5c2ea0b 100644
@@ -11,6 +11,7 @@
  */
 
 #include "builtin.h"
+#include "repository.h"
 #include "config.h"
 #include "tempfile.h"
 #include "lockfile.h"
@@ -173,7 +174,7 @@ static int too_many_packs(void)
 		return 0;
 
 	prepare_packed_git();
-	for (cnt = 0, p = packed_git; p; p = p->next) {
+	for (cnt = 0, p = the_repository->objects.packed_git; p; p = p->next) {
 		if (!p->pack_local)
 			continue;
 		if (p->pack_keep)
builtin/pack-objects.c +11 -9
index 5c674b2843c..a64005760be 100644
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "attr.h"
 #include "object.h"
@@ -1025,8 +1026,7 @@ static int want_object_in_pack(const struct object_id *oid,
 		if (want != -1)
 			return want;
 	}
-
-	list_for_each(pos, &packed_git_mru) {
+	list_for_each(pos, &the_repository->objects.packed_git_mru) {
 		struct packed_git *p = list_entry(pos, struct packed_git, mru);
 		off_t offset;
 
@@ -1044,7 +1044,8 @@ static int want_object_in_pack(const struct object_id *oid,
 			}
 			want = want_found_object(exclude, p);
 			if (!exclude && want > 0)
-				list_move(&p->mru, &packed_git_mru);
+				list_move(&p->mru,
+					  &the_repository->objects.packed_git_mru);
 			if (want != -1)
 				return want;
 		}
@@ -2673,7 +2674,7 @@ static void add_objects_in_unpacked_packs(struct rev_info *revs)
 
 	memset(&in_pack, 0, sizeof(in_pack));
 
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		struct object_id oid;
 		struct object *o;
 
@@ -2736,7 +2737,8 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 	static struct packed_git *last_found = (void *)1;
 	struct packed_git *p;
 
-	p = (last_found != (void *)1) ? last_found : packed_git;
+	p = (last_found != (void *)1) ? last_found :
+					the_repository->objects.packed_git;
 
 	while (p) {
 		if ((!p->pack_local || p->pack_keep) &&
@@ -2745,7 +2747,7 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 			return 1;
 		}
 		if (p == last_found)
-			p = packed_git;
+			p = the_repository->objects.packed_git;
 		else
 			p = p->next;
 		if (p == last_found)
@@ -2781,7 +2783,7 @@ static void loosen_unused_packed_objects(struct rev_info *revs)
 	uint32_t i;
 	struct object_id oid;
 
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if (!p->pack_local || p->pack_keep)
 			continue;
 
@@ -3152,7 +3154,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 	prepare_packed_git();
 	if (ignore_packed_keep) {
 		struct packed_git *p;
-		for (p = packed_git; p; p = p->next)
+		for (p = the_repository->objects.packed_git; p; p = p->next)
 			if (p->pack_local && p->pack_keep)
 				break;
 		if (!p) /* no keep-able packs found */
@@ -3165,7 +3167,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 		 * also covers non-local objects
 		 */
 		struct packed_git *p;
-		for (p = packed_git; p; p = p->next) {
+		for (p = the_repository->objects.packed_git; p; p = p->next) {
 			if (!p->pack_local) {
 				have_non_local_packs = 1;
 				break;
builtin/pack-redundant.c +3 -2
index aaa81363222..a196fae79d4 100644
@@ -7,6 +7,7 @@
 */
 
 #include "builtin.h"
+#include "repository.h"
 #include "packfile.h"
 
 #define BLKSIZE 512
@@ -571,7 +572,7 @@ static struct pack_list * add_pack(struct packed_git *p)
 
 static struct pack_list * add_pack_file(const char *filename)
 {
-	struct packed_git *p = packed_git;
+	struct packed_git *p = the_repository->objects.packed_git;
 
 	if (strlen(filename) < 40)
 		die("Bad pack filename: %s", filename);
@@ -586,7 +587,7 @@ static struct pack_list * add_pack_file(const char *filename)
 
 static void load_all(void)
 {
-	struct packed_git *p = packed_git;
+	struct packed_git *p = the_repository->objects.packed_git;
 
 	while (p) {
 		add_pack(p);
cache.h +0 -29
index 99eb2bdb893..63889acb591 100644
@@ -1569,35 +1569,6 @@ struct pack_window {
 	unsigned int inuse_cnt;
 };
 
-extern struct packed_git {
-	struct packed_git *next;
-	struct list_head mru;
-	struct pack_window *windows;
-	off_t pack_size;
-	const void *index_data;
-	size_t index_size;
-	uint32_t num_objects;
-	uint32_t num_bad_objects;
-	unsigned char *bad_object_sha1;
-	int index_version;
-	time_t mtime;
-	int pack_fd;
-	unsigned pack_local:1,
-		 pack_keep:1,
-		 freshened:1,
-		 do_not_close:1,
-		 pack_promisor:1;
-	unsigned char sha1[20];
-	struct revindex_entry *revindex;
-	/* something like ".git/objects/pack/xxxxx.pack" */
-	char pack_name[FLEX_ARRAY]; /* more */
-} *packed_git;
-
-/*
- * A most-recently-used ordered version of the packed_git list.
- */
-extern struct list_head packed_git_mru;
-
 struct pack_entry {
 	off_t offset;
 	unsigned char sha1[20];
fast-import.c +5 -2
index 0f818cd9328..3d7002d1cd6 100644
@@ -154,6 +154,7 @@ Format of STDIN stream:
 
 #include "builtin.h"
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "lockfile.h"
 #include "object.h"
@@ -1110,7 +1111,8 @@ static int store_object(
 	if (e->idx.offset) {
 		duplicate_count_by_type[type]++;
 		return 1;
-	} else if (find_sha1_pack(oid.hash, packed_git)) {
+	} else if (find_sha1_pack(oid.hash,
+				  the_repository->objects.packed_git)) {
 		e->type = type;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
@@ -1307,7 +1309,8 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
 		duplicate_count_by_type[OBJ_BLOB]++;
 		truncate_pack(&checkpoint);
 
-	} else if (find_sha1_pack(oid.hash, packed_git)) {
+	} else if (find_sha1_pack(oid.hash,
+				  the_repository->objects.packed_git)) {
 		e->type = OBJ_BLOB;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
http-backend.c +3 -2
index f3dc218b2a3..85b6a97923f 100644
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "config.h"
+#include "repository.h"
 #include "refs.h"
 #include "pkt-line.h"
 #include "object.h"
@@ -518,13 +519,13 @@ static void get_info_packs(struct strbuf *hdr, char *arg)
 
 	select_getanyfile(hdr);
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if (p->pack_local)
 			cnt++;
 	}
 
 	strbuf_grow(&buf, cnt * 53 + 2);
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if (p->pack_local)
 			strbuf_addf(&buf, "P %s\n", p->pack_name + objdirlen + 6);
 	}
object-store.h +30 -1
index e78eea1dde3..4f768465a14 100644
@@ -52,6 +52,30 @@ void add_to_alternates_memory(const char *dir);
  */
 struct strbuf *alt_scratch_buf(struct alternate_object_database *alt);
 
+struct packed_git {
+	struct packed_git *next;
+	struct list_head mru;
+	struct pack_window *windows;
+	off_t pack_size;
+	const void *index_data;
+	size_t index_size;
+	uint32_t num_objects;
+	uint32_t num_bad_objects;
+	unsigned char *bad_object_sha1;
+	int index_version;
+	time_t mtime;
+	int pack_fd;
+	unsigned pack_local:1,
+		 pack_keep:1,
+		 freshened:1,
+		 do_not_close:1,
+		 pack_promisor:1;
+	unsigned char sha1[20];
+	struct revindex_entry *revindex;
+	/* something like ".git/objects/pack/xxxxx.pack" */
+	char pack_name[FLEX_ARRAY]; /* more */
+};
+
 struct raw_object_store {
 	/*
 	 * Path to the repository's object store.
@@ -59,10 +83,15 @@ struct raw_object_store {
 	 */
 	char *objectdir;
 
+	struct packed_git *packed_git;
+	/* A most-recently-used ordered version of the packed_git list. */
+	struct list_head packed_git_mru;
+
 	struct alternate_object_database *alt_odb_list;
 	struct alternate_object_database **alt_odb_tail;
 };
-#define RAW_OBJECT_STORE_INIT { NULL, NULL, NULL }
+
+#define RAW_OBJECT_STORE_INIT(o) { NULL, NULL, LIST_HEAD_INIT(o.packed_git_mru), NULL, NULL }
 
 void raw_object_store_clear(struct raw_object_store *o);
 
object.c +6 -0
index 920dc4463fa..367441efa94 100644
@@ -466,4 +466,10 @@ void raw_object_store_clear(struct raw_object_store *o)
 
 	free_alt_odbs(o);
 	o->alt_odb_tail = NULL;
+
+	INIT_LIST_HEAD(&o->packed_git_mru);
+	/*
+	 * TODO: call close_all_packs once migrated to
+	 * take an object store argument
+	 */
 }
pack-bitmap.c +2 -1
index 9270983e5f5..bbede32589a 100644
@@ -10,6 +10,7 @@
 #include "pack-revindex.h"
 #include "pack-objects.h"
 #include "packfile.h"
+#include "repository.h"
 
 /*
  * An entry on the bitmap index, representing the bitmap for a given
@@ -335,7 +336,7 @@ static int open_pack_bitmap(void)
 	assert(!bitmap_git.map && !bitmap_git.loaded);
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if (open_pack_bitmap_1(p) == 0)
 			ret = 0;
 	}
packfile.c +20 -19
index 216ea836ee3..59210deaaf7 100644
@@ -45,8 +45,6 @@ static unsigned int pack_open_fds;
 static unsigned int pack_max_fds;
 static size_t peak_pack_mapped;
 static size_t pack_mapped;
-struct packed_git *packed_git;
-LIST_HEAD(packed_git_mru);
 
 #define SZ_FMT PRIuMAX
 static inline uintmax_t sz_fmt(size_t s) { return s; }
@@ -246,7 +244,7 @@ static int unuse_one_window(struct packed_git *current)
 
 	if (current)
 		scan_windows(current, &lru_p, &lru_w, &lru_l);
-	for (p = packed_git; p; p = p->next)
+	for (p = the_repository->objects.packed_git; p; p = p->next)
 		scan_windows(p, &lru_p, &lru_w, &lru_l);
 	if (lru_p) {
 		munmap(lru_w->base, lru_w->len);
@@ -316,7 +314,7 @@ void close_all_packs(void)
 {
 	struct packed_git *p;
 
-	for (p = packed_git; p; p = p->next)
+	for (p = the_repository->objects.packed_git; p; p = p->next)
 		if (p->do_not_close)
 			die("BUG: want to close pack marked 'do-not-close'");
 		else
@@ -384,7 +382,7 @@ static int close_one_pack(void)
 	struct pack_window *mru_w = NULL;
 	int accept_windows_inuse = 1;
 
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if (p->pack_fd == -1)
 			continue;
 		find_lru_pack(p, &lru_p, &mru_w, &accept_windows_inuse);
@@ -686,8 +684,8 @@ void install_packed_git(struct packed_git *pack)
 	if (pack->pack_fd != -1)
 		pack_open_fds++;
 
-	pack->next = packed_git;
-	packed_git = pack;
+	pack->next = the_repository->objects.packed_git;
+	the_repository->objects.packed_git = pack;
 }
 
 void (*report_garbage)(unsigned seen_bits, const char *path);
@@ -769,7 +767,8 @@ static void prepare_packed_git_one(char *objdir, int local)
 		base_len = path.len;
 		if (strip_suffix_mem(path.buf, &base_len, ".idx")) {
 			/* Don't reopen a pack we already have. */
-			for (p = packed_git; p; p = p->next) {
+			for (p = the_repository->objects.packed_git; p;
+			     p = p->next) {
 				size_t len;
 				if (strip_suffix(p->pack_name, ".pack", &len) &&
 				    len == base_len &&
@@ -820,7 +819,7 @@ unsigned long approximate_object_count(void)
 
 		prepare_packed_git();
 		count = 0;
-		for (p = packed_git; p; p = p->next) {
+		for (p = the_repository->objects.packed_git; p; p = p->next) {
 			if (open_pack_index(p))
 				continue;
 			count += p->num_objects;
@@ -869,18 +868,19 @@ static int sort_pack(const void *a_, const void *b_)
 
 static void rearrange_packed_git(void)
 {
-	packed_git = llist_mergesort(packed_git, get_next_packed_git,
-				     set_next_packed_git, sort_pack);
+	the_repository->objects.packed_git = llist_mergesort(
+		the_repository->objects.packed_git, get_next_packed_git,
+		set_next_packed_git, sort_pack);
 }
 
 static void prepare_packed_git_mru(void)
 {
 	struct packed_git *p;
 
-	INIT_LIST_HEAD(&packed_git_mru);
+	INIT_LIST_HEAD(&the_repository->objects.packed_git_mru);
 
-	for (p = packed_git; p; p = p->next)
-		list_add_tail(&p->mru, &packed_git_mru);
+	for (p = the_repository->objects.packed_git; p; p = p->next)
+		list_add_tail(&p->mru, &the_repository->objects.packed_git_mru);
 }
 
 static int prepare_packed_git_run_once = 0;
@@ -1014,7 +1014,7 @@ const struct packed_git *has_packed_and_bad(const unsigned char *sha1)
 	struct packed_git *p;
 	unsigned i;
 
-	for (p = packed_git; p; p = p->next)
+	for (p = the_repository->objects.packed_git; p; p = p->next)
 		for (i = 0; i < p->num_bad_objects; i++)
 			if (!hashcmp(sha1, p->bad_object_sha1 + 20 * i))
 				return p;
@@ -1845,13 +1845,14 @@ int find_pack_entry(const unsigned char *sha1, struct pack_entry *e)
 	struct list_head *pos;
 
 	prepare_packed_git();
-	if (!packed_git)
+	if (!the_repository->objects.packed_git)
 		return 0;
 
-	list_for_each(pos, &packed_git_mru) {
+	list_for_each(pos, &the_repository->objects.packed_git_mru) {
 		struct packed_git *p = list_entry(pos, struct packed_git, mru);
 		if (fill_pack_entry(sha1, e, p)) {
-			list_move(&p->mru, &packed_git_mru);
+			list_move(&p->mru,
+				  &the_repository->objects.packed_git_mru);
 			return 1;
 		}
 	}
@@ -1898,7 +1899,7 @@ int for_each_packed_object(each_packed_object_fn cb, void *data, unsigned flags)
 	int pack_errors = 0;
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
 			continue;
 		if ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&
repository.c +3 -1
index fb65e8072d5..a069b1b6404 100644
@@ -7,7 +7,7 @@
 /* The main repository */
 static struct repository the_repo = {
 	NULL, NULL,
-	RAW_OBJECT_STORE_INIT,
+	RAW_OBJECT_STORE_INIT(the_repo.objects),
 	NULL, NULL, NULL,
 	NULL, NULL, NULL,
 	&the_index,
@@ -143,6 +143,8 @@ int repo_init(struct repository *repo, const char *gitdir, const char *worktree)
 
 	repo->ignore_env = 1;
 
+	INIT_LIST_HEAD(&repo->objects.packed_git_mru);
+
 	if (repo_init_gitdir(repo, gitdir))
 		goto error;
 
server-info.c +3 -2
index 26a6c20b7d4..c369ac27cbc 100644
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "repository.h"
 #include "refs.h"
 #include "object.h"
 #include "commit.h"
@@ -200,7 +201,7 @@ static void init_pack_info(const char *infofile, int force)
 	objdirlen = strlen(objdir);
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		/* we ignore things on alternate path since they are
 		 * not available to the pullers in general.
 		 */
@@ -210,7 +211,7 @@ static void init_pack_info(const char *infofile, int force)
 	}
 	num_pack = i;
 	info = xcalloc(num_pack, sizeof(struct pack_info *));
-	for (i = 0, p = packed_git; p; p = p->next) {
+	for (i = 0, p = the_repository->objects.packed_git; p; p = p->next) {
 		if (!p->pack_local)
 			continue;
 		info[i] = xcalloc(1, sizeof(struct pack_info));
sha1_name.c +3 -2
index 957ce25680b..016c883b5c7 100644
@@ -196,7 +196,8 @@ static void find_short_packed_object(struct disambiguate_state *ds)
 	struct packed_git *p;
 
 	prepare_packed_git();
-	for (p = packed_git; p && !ds->ambiguous; p = p->next)
+	for (p = the_repository->objects.packed_git; p && !ds->ambiguous;
+	     p = p->next)
 		unique_in_pack(p, ds);
 }
 
@@ -566,7 +567,7 @@ static void find_abbrev_len_packed(struct min_abbrev_data *mad)
 	struct packed_git *p;
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next)
+	for (p = the_repository->objects.packed_git; p; p = p->next)
 		find_abbrev_len_for_pack(p, mad);
 }
 
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#213 lore raw

PATCH v4 6/27 object-store: close all packs upon clearing the object store

In reply to Stefan Beller
---
 builtin/am.c           | 2 +-
 builtin/clone.c        | 2 +-
 builtin/fetch.c        | 2 +-
 builtin/merge.c        | 2 +-
 builtin/receive-pack.c | 2 +-
 object.c               | 7 +++----
 packfile.c             | 4 ++--
 packfile.h             | 2 +-
 8 files changed, 11 insertions(+), 12 deletions(-)
builtin/am.c +1 -1
index 6661edc162b..fc1d9b43c51 100644
@@ -1859,7 +1859,7 @@ static void am_run(struct am_state *state, int resume)
 	 */
 	if (!state->rebasing) {
 		am_destroy(state);
-		close_all_packs();
+		close_all_packs(&the_repository->objects);
 		run_command_v_opt(argv_gc_auto, RUN_GIT_CMD);
 	}
 }
builtin/clone.c +1 -1
index 101c27a593f..13cfaa6488a 100644
@@ -1217,7 +1217,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 	transport_disconnect(transport);
 
 	if (option_dissociate) {
-		close_all_packs();
+		close_all_packs(&the_repository->objects);
 		dissociate_from_references();
 	}
 
builtin/fetch.c +1 -1
index 8ee998ea2ee..4d72efca781 100644
@@ -1478,7 +1478,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
 
 	string_list_clear(&list, 0);
 
-	close_all_packs();
+	close_all_packs(&the_repository->objects);
 
 	argv_array_pushl(&argv_gc_auto, "gc", "--auto", NULL);
 	if (verbosity < 0)
builtin/merge.c +1 -1
index 92ba99a1a5e..0af0c53a632 100644
@@ -411,7 +411,7 @@ static void finish(struct commit *head_commit,
 			 * We ignore errors in 'gc --auto', since the
 			 * user should see them.
 			 */
-			close_all_packs();
+			close_all_packs(&the_repository->objects);
 			run_command_v_opt(argv_gc_auto, RUN_GIT_CMD);
 		}
 	}
builtin/receive-pack.c +1 -1
index 75e7f18acef..8e25aae54c8 100644
@@ -2027,7 +2027,7 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)
 			proc.git_cmd = 1;
 			proc.argv = argv_gc_auto;
 
-			close_all_packs();
+			close_all_packs(&the_repository->objects);
 			if (!start_command(&proc)) {
 				if (use_sideband)
 					copy_to_sideband(proc.err, -1, NULL);
object.c +3 -4
index 367441efa94..a7c238339bd 100644
@@ -4,6 +4,7 @@
 #include "tree.h"
 #include "commit.h"
 #include "tag.h"
+#include "packfile.h"
 
 static struct object **obj_hash;
 static int nr_objs, obj_hash_size;
@@ -468,8 +469,6 @@ void raw_object_store_clear(struct raw_object_store *o)
 	o->alt_odb_tail = NULL;
 
 	INIT_LIST_HEAD(&o->packed_git_mru);
-	/*
-	 * TODO: call close_all_packs once migrated to
-	 * take an object store argument
-	 */
+	close_all_packs(o);
+	o->packed_git = NULL;
 }
packfile.c +2 -2
index 59210deaaf7..65d9a4f6c61 100644
@@ -310,11 +310,11 @@ static void close_pack(struct packed_git *p)
 	close_pack_index(p);
 }
 
-void close_all_packs(void)
+void close_all_packs(struct raw_object_store *o)
 {
 	struct packed_git *p;
 
-	for (p = the_repository->objects.packed_git; p; p = p->next)
+	for (p = o->packed_git; p; p = p->next)
 		if (p->do_not_close)
 			die("BUG: want to close pack marked 'do-not-close'");
 		else
packfile.h +1 -1
index a7fca598d67..6a2c57045ca 100644
@@ -63,7 +63,7 @@ extern void close_pack_index(struct packed_git *);
 
 extern unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t, unsigned long *);
 extern void close_pack_windows(struct packed_git *);
-extern void close_all_packs(void);
+extern void close_all_packs(struct raw_object_store *o);
 extern void unuse_pack(struct pack_window **);
 extern void clear_delta_base_cache(void);
 extern struct packed_git *add_packed_git(const char *path, size_t path_len, int local);
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#214 lore raw

PATCH v4 7/27 pack: move prepare_packed_git_run_once to object store

In reply to Stefan Beller

Each repository's object store can be initialized independently, so they must not share a run_once variable.

---
 object-store.h | 8 +++++++-
 packfile.c     | 7 +++----
 2 files changed, 10 insertions(+), 5 deletions(-)
object-store.h +7 -1
index 4f768465a14..bd6441e525f 100644
@@ -89,9 +89,15 @@ struct raw_object_store {
 
 	struct alternate_object_database *alt_odb_list;
 	struct alternate_object_database **alt_odb_tail;
+
+	/*
+	 * Whether packed_git has already been populated with this repository's
+	 * packs.
+	 */
+	unsigned packed_git_initialized : 1;
 };
 
-#define RAW_OBJECT_STORE_INIT(o) { NULL, NULL, LIST_HEAD_INIT(o.packed_git_mru), NULL, NULL }
+#define RAW_OBJECT_STORE_INIT(o) { NULL, NULL, LIST_HEAD_INIT(o.packed_git_mru), NULL, NULL, 0 }
 
 void raw_object_store_clear(struct raw_object_store *o);
 
packfile.c +3 -4
index 65d9a4f6c61..3ee349ab1bd 100644
@@ -883,12 +883,11 @@ static void prepare_packed_git_mru(void)
 		list_add_tail(&p->mru, &the_repository->objects.packed_git_mru);
 }
 
-static int prepare_packed_git_run_once = 0;
 void prepare_packed_git(void)
 {
 	struct alternate_object_database *alt;
 
-	if (prepare_packed_git_run_once)
+	if (the_repository->objects.packed_git_initialized)
 		return;
 	prepare_packed_git_one(get_object_directory(), 1);
 	prepare_alt_odb();
@@ -896,13 +895,13 @@ void prepare_packed_git(void)
 		prepare_packed_git_one(alt->path, 0);
 	rearrange_packed_git();
 	prepare_packed_git_mru();
-	prepare_packed_git_run_once = 1;
+	the_repository->objects.packed_git_initialized = 1;
 }
 
 void reprepare_packed_git(void)
 {
 	approximate_object_count_valid = 0;
-	prepare_packed_git_run_once = 0;
+	the_repository->objects.packed_git_initialized = 0;
 	prepare_packed_git();
 }
 
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#215 lore raw

PATCH v4 8/27 pack: move approximate object count to object store

In reply to Stefan Beller

The approximate_object_count() function maintains a rough count of objects in a repository to estimate how long object name abbreviates should be. Object names are scoped to a repository and the appropriate length may differ by repository, so the object count should not be global.

---
 object-store.h | 10 +++++++++-
 packfile.c     | 11 +++++------
 2 files changed, 14 insertions(+), 7 deletions(-)
object-store.h +9 -1
index bd6441e525f..31ca4d3f340 100644
@@ -90,6 +90,14 @@ struct raw_object_store {
 	struct alternate_object_database *alt_odb_list;
 	struct alternate_object_database **alt_odb_tail;
 
+	/*
+	 * A fast, rough count of the number of objects in the repository.
+	 * These two fields are not meant for direct access. Use
+	 * approximate_object_count() instead.
+	 */
+	unsigned long approximate_object_count;
+	unsigned approximate_object_count_valid : 1;
+
 	/*
 	 * Whether packed_git has already been populated with this repository's
 	 * packs.
@@ -97,7 +105,7 @@ struct raw_object_store {
 	unsigned packed_git_initialized : 1;
 };
 
-#define RAW_OBJECT_STORE_INIT(o) { NULL, NULL, LIST_HEAD_INIT(o.packed_git_mru), NULL, NULL, 0 }
+#define RAW_OBJECT_STORE_INIT(o) { NULL, NULL, LIST_HEAD_INIT(o.packed_git_mru), NULL, NULL, 0, 0, 0 }
 
 void raw_object_store_clear(struct raw_object_store *o);
 
packfile.c +5 -6
index 3ee349ab1bd..8d7b1dd5f9c 100644
@@ -802,8 +802,6 @@ static void prepare_packed_git_one(char *objdir, int local)
 	strbuf_release(&path);
 }
 
-static int approximate_object_count_valid;
-
 /*
  * Give a fast, rough count of the number of objects in the repository. This
  * ignores loose objects completely. If you have a lot of them, then either
@@ -813,8 +811,8 @@ static int approximate_object_count_valid;
  */
 unsigned long approximate_object_count(void)
 {
-	static unsigned long count;
-	if (!approximate_object_count_valid) {
+	if (!the_repository->objects.approximate_object_count_valid) {
+		unsigned long count;
 		struct packed_git *p;
 
 		prepare_packed_git();
@@ -824,8 +822,9 @@ unsigned long approximate_object_count(void)
 				continue;
 			count += p->num_objects;
 		}
+		the_repository->objects.approximate_object_count = count;
 	}
-	return count;
+	return the_repository->objects.approximate_object_count;
 }
 
 static void *get_next_packed_git(const void *p)
@@ -900,7 +899,7 @@ void prepare_packed_git(void)
 
 void reprepare_packed_git(void)
 {
-	approximate_object_count_valid = 0;
+	the_repository->objects.approximate_object_count_valid = 0;
 	the_repository->objects.packed_git_initialized = 0;
 	prepare_packed_git();
 }
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#216 lore raw

PATCH v4 9/27 sha1_file: add raw_object_store argument to alt_odb_usable

In reply to Stefan Beller

Add a raw_object_store to alt_odb_usable to be more specific about which repository to act on. The choice of the repository is delegated to its only caller link_alt_odb_entry.

---
 sha1_file.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
sha1_file.c +5 -3
index ac25146076e..8e6f213f9d9 100644
@@ -344,7 +344,9 @@ static const char *alt_sha1_path(struct alternate_object_database *alt,
 /*
  * Return non-zero iff the path is usable as an alternate object database.
  */
-static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)
+static int alt_odb_usable(struct raw_object_store *o,
+			  struct strbuf *path,
+			  const char *normalized_objdir)
 {
 	struct alternate_object_database *alt;
 
@@ -360,7 +362,7 @@ static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)
 	 * Prevent the common mistake of listing the same
 	 * thing twice, or object directory itself.
 	 */
-	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
+	for (alt = o->alt_odb_list; alt; alt = alt->next) {
 		if (!fspathcmp(path->buf, alt->path))
 			return 0;
 	}
@@ -412,7 +414,7 @@ static int link_alt_odb_entry(const char *entry, const char *relative_base,
 	while (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')
 		strbuf_setlen(&pathbuf, pathbuf.len - 1);
 
-	if (!alt_odb_usable(&pathbuf, normalized_objdir)) {
+	if (!alt_odb_usable(&the_repository->objects, &pathbuf, normalized_objdir)) {
 		strbuf_release(&pathbuf);
 		return -1;
 	}
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#217 lore raw

PATCH v4 10/27 sha1_file: add repository argument to link_alt_odb_entry

In reply to Stefan Beller

Add a repository argument to allow the link_alt_odb_entry caller to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

Since the implementation does not yet work with other repositories, use a wrapper macro to enforce that the caller passes in the_repository as the first argument. It would be more appealing to use BUILD_ASSERT_OR_ZERO to enforce this, but that doesn't work because it requires a compile-time constant and common compilers like gcc 4.8.4 do not consider "r == the_repository" a compile-time constant.

This and the following three patches add repository arguments to link_alt_odb_entry, read_info_alternates, link_alt_odb_entries and prepare_alt_odb. Three out of the four functions are found in a recursive call chain, calling each other, and one of them accesses the repositories objectdir (which was migrated; it was an obvious choice) and ignore_env (which we need to keep in the repository struct for clarify); hence we will pass through the repository unlike just the object store object + the ignore_env flag.

---
 sha1_file.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
sha1_file.c +5 -3
index 8e6f213f9d9..e5fe0aa04ef 100644
@@ -388,8 +388,9 @@ static int alt_odb_usable(struct raw_object_store *o,
  * terminating NUL.
  */
 static void read_info_alternates(const char * relative_base, int depth);
-static int link_alt_odb_entry(const char *entry, const char *relative_base,
-	int depth, const char *normalized_objdir)
+#define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)
+static int link_alt_odb_entry_the_repository(const char *entry,
+	const char *relative_base, int depth, const char *normalized_objdir)
 {
 	struct alternate_object_database *ent;
 	struct strbuf pathbuf = STRBUF_INIT;
@@ -486,7 +487,8 @@ static void link_alt_odb_entries(const char *alt, int sep,
 		alt = parse_alt_odb_entry(alt, sep, &entry);
 		if (!entry.len)
 			continue;
-		link_alt_odb_entry(entry.buf, relative_base, depth, objdirbuf.buf);
+		link_alt_odb_entry(the_repository, entry.buf,
+				   relative_base, depth, objdirbuf.buf);
 	}
 	strbuf_release(&entry);
 	strbuf_release(&objdirbuf);
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#218 lore raw

PATCH v4 11/27 sha1_file: add repository argument to read_info_alternates

In reply to Stefan Beller

See previous patch for explanation.

---
 sha1_file.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
sha1_file.c +7 -4
index e5fe0aa04ef..f93d3b95b54 100644
@@ -387,7 +387,9 @@ static int alt_odb_usable(struct raw_object_store *o,
  * SHA1, an extra slash for the first level indirection, and the
  * terminating NUL.
  */
-static void read_info_alternates(const char * relative_base, int depth);
+#define read_info_alternates(r, rb, d) read_info_alternates_##r(rb, d)
+static void read_info_alternates_the_repository(const char *relative_base,
+						int depth);
 #define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)
 static int link_alt_odb_entry_the_repository(const char *entry,
 	const char *relative_base, int depth, const char *normalized_objdir)
@@ -428,7 +430,7 @@ static int link_alt_odb_entry_the_repository(const char *entry,
 	ent->next = NULL;
 
 	/* recursively add alternates */
-	read_info_alternates(pathbuf.buf, depth + 1);
+	read_info_alternates(the_repository, pathbuf.buf, depth + 1);
 
 	strbuf_release(&pathbuf);
 	return 0;
@@ -494,7 +496,8 @@ static void link_alt_odb_entries(const char *alt, int sep,
 	strbuf_release(&objdirbuf);
 }
 
-static void read_info_alternates(const char * relative_base, int depth)
+static void read_info_alternates_the_repository(const char *relative_base,
+						int depth)
 {
 	char *path;
 	struct strbuf buf = STRBUF_INIT;
@@ -678,7 +681,7 @@ void prepare_alt_odb(void)
 			&the_repository->objects.alt_odb_list;
 	link_alt_odb_entries(alt, PATH_SEP, NULL, 0);
 
-	read_info_alternates(get_object_directory(), 0);
+	read_info_alternates(the_repository, get_object_directory(), 0);
 }
 
 /* Returns 1 if we have successfully freshened the file, 0 otherwise. */
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#219 lore raw

PATCH v4 12/27 sha1_file: add repository argument to link_alt_odb_entries

In reply to Stefan Beller

See previous patch for explanation.

---
 sha1_file.c | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)
sha1_file.c +13 -6
index f93d3b95b54..8fb56ca0362 100644
@@ -465,8 +465,12 @@ static const char *parse_alt_odb_entry(const char *string,
 	return end;
 }
 
-static void link_alt_odb_entries(const char *alt, int sep,
-				 const char *relative_base, int depth)
+#define link_alt_odb_entries(r, a, s, rb, d) \
+	link_alt_odb_entries_##r(a, s, rb, d)
+static void link_alt_odb_entries_the_repository(const char *alt,
+						int sep,
+						const char *relative_base,
+						int depth)
 {
 	struct strbuf objdirbuf = STRBUF_INIT;
 	struct strbuf entry = STRBUF_INIT;
@@ -509,7 +513,7 @@ static void read_info_alternates_the_repository(const char *relative_base,
 		return;
 	}
 
-	link_alt_odb_entries(buf.buf, '\n', relative_base, depth);
+	link_alt_odb_entries(the_repository, buf.buf, '\n', relative_base, depth);
 	strbuf_release(&buf);
 	free(path);
 }
@@ -563,7 +567,8 @@ void add_to_alternates_file(const char *reference)
 		if (commit_lock_file(&lock))
 			die_errno("unable to move new alternates file into place");
 		if (the_repository->objects.alt_odb_tail)
-			link_alt_odb_entries(reference, '\n', NULL, 0);
+			link_alt_odb_entries(the_repository, reference,
+					     '\n', NULL, 0);
 	}
 	free(alts);
 }
@@ -576,7 +581,8 @@ void add_to_alternates_memory(const char *reference)
 	 */
 	prepare_alt_odb();
 
-	link_alt_odb_entries(reference, '\n', NULL, 0);
+	link_alt_odb_entries(the_repository, reference,
+			     '\n', NULL, 0);
 }
 
 /*
@@ -679,7 +685,8 @@ void prepare_alt_odb(void)
 
 	the_repository->objects.alt_odb_tail =
 			&the_repository->objects.alt_odb_list;
-	link_alt_odb_entries(alt, PATH_SEP, NULL, 0);
+	link_alt_odb_entries(the_repository, alt,
+			     PATH_SEP, NULL, 0);
 
 	read_info_alternates(the_repository, get_object_directory(), 0);
 }
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#220 lore raw

PATCH v4 13/27 sha1_file: add repository argument to prepare_alt_odb

In reply to Stefan Beller

See previous patch for explanation.

---
 builtin/fsck.c |  2 +-
 object-store.h |  3 ++-
 packfile.c     |  2 +-
 sha1_file.c    | 13 +++++++------
 sha1_name.c    |  2 +-
 5 files changed, 12 insertions(+), 10 deletions(-)
builtin/fsck.c +1 -1
index c3a339193c8..cdcf2fa926c 100644
@@ -722,7 +722,7 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 	} else {
 		fsck_object_dir(get_object_directory());
 
-		prepare_alt_odb();
+		prepare_alt_odb(the_repository);
 		for (alt = the_repository->objects.alt_odb_list;
 		     alt; alt = alt->next)
 			fsck_object_dir(alt->path);
object-store.h +2 -1
index 31ca4d3f340..95f328482aa 100644
@@ -20,7 +20,8 @@ struct alternate_object_database {
 
 	char path[FLEX_ARRAY];
 };
-void prepare_alt_odb(void);
+#define prepare_alt_odb(r) prepare_alt_odb_##r()
+void prepare_alt_odb_the_repository(void);
 char *compute_alternate_path(const char *path, struct strbuf *err);
 typedef int alt_odb_fn(struct alternate_object_database *, void *);
 int foreach_alt_odb(alt_odb_fn, void*);
packfile.c +1 -1
index 8d7b1dd5f9c..1e02fe3b0bb 100644
@@ -889,7 +889,7 @@ void prepare_packed_git(void)
 	if (the_repository->objects.packed_git_initialized)
 		return;
 	prepare_packed_git_one(get_object_directory(), 1);
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next)
 		prepare_packed_git_one(alt->path, 0);
 	rearrange_packed_git();
sha1_file.c +7 -6
index 8fb56ca0362..381eaf8221d 100644
@@ -23,6 +23,7 @@
 #include "sha1-lookup.h"
 #include "bulk-checkin.h"
 #include "repository.h"
+#include "object-store.h"
 #include "streaming.h"
 #include "dir.h"
 #include "list.h"
@@ -579,7 +580,7 @@ void add_to_alternates_memory(const char *reference)
 	 * Make sure alternates are initialized, or else our entry may be
 	 * overwritten when they are.
 	 */
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 
 	link_alt_odb_entries(the_repository, reference,
 			     '\n', NULL, 0);
@@ -665,7 +666,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 	struct alternate_object_database *ent;
 	int r = 0;
 
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	for (ent = the_repository->objects.alt_odb_list; ent; ent = ent->next) {
 		r = fn(ent, cb);
 		if (r)
@@ -674,7 +675,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 	return r;
 }
 
-void prepare_alt_odb(void)
+void prepare_alt_odb_the_repository(void)
 {
 	const char *alt;
 
@@ -728,7 +729,7 @@ static int check_and_freshen_local(const unsigned char *sha1, int freshen)
 static int check_and_freshen_nonlocal(const unsigned char *sha1, int freshen)
 {
 	struct alternate_object_database *alt;
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		const char *path = alt_sha1_path(alt, sha1);
 		if (check_and_freshen_file(path, freshen))
@@ -887,7 +888,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,
 	if (!lstat(*path, st))
 		return 0;
 
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	errno = ENOENT;
 	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
@@ -918,7 +919,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)
 		return fd;
 	most_interesting_errno = errno;
 
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		fd = git_open(*path);
sha1_name.c +1 -1
index 016c883b5c7..627e93f1447 100644
@@ -353,7 +353,7 @@ static int init_object_disambiguation(const char *name, int len,
 
 	ds->len = len;
 	ds->hex_pfx[len] = '\0';
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	return 0;
 }
 
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#221 lore raw

PATCH v4 14/27 sha1_file: allow link_alt_odb_entries to handle arbitrary repositories

In reply to Stefan Beller

Actually this also allows read_info_alternates and link_alt_odb_entry to handle arbitrary repositories, but link_alt_odb_entries is the most interesting function in this set of functions, hence the commit subject.

These functions span a strongly connected component in the function graph, i.e. the recursive call chain might look like

-> link_alt_odb_entries

    -> link_alt_odb_entry
      -> read_info_alternates
        -> link_alt_odb_entries

That is why we need to convert all these functions at the same time.

---
 object-store.h |  4 ++++
 sha1_file.c    | 36 ++++++++++++++++--------------------
 2 files changed, 20 insertions(+), 20 deletions(-)
object-store.h +4 -0
index 95f328482aa..08cd48ade11 100644
@@ -18,6 +18,10 @@ struct alternate_object_database {
 	char loose_objects_subdir_seen[256];
 	struct oid_array loose_objects_cache;
 
+	/*
+	 * Path to the alternative object store. If this is a relative path,
+	 * it is relative to the current working directory.
+	 */
 	char path[FLEX_ARRAY];
 };
 #define prepare_alt_odb(r) prepare_alt_odb_##r()
sha1_file.c +16 -20
index 381eaf8221d..ab42d430b8a 100644
@@ -388,11 +388,10 @@ static int alt_odb_usable(struct raw_object_store *o,
  * SHA1, an extra slash for the first level indirection, and the
  * terminating NUL.
  */
-#define read_info_alternates(r, rb, d) read_info_alternates_##r(rb, d)
-static void read_info_alternates_the_repository(const char *relative_base,
-						int depth);
-#define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)
-static int link_alt_odb_entry_the_repository(const char *entry,
+static void read_info_alternates(struct repository *r,
+				 const char *relative_base,
+				 int depth);
+static int link_alt_odb_entry(struct repository *r, const char *entry,
 	const char *relative_base, int depth, const char *normalized_objdir)
 {
 	struct alternate_object_database *ent;
@@ -418,7 +417,7 @@ static int link_alt_odb_entry_the_repository(const char *entry,
 	while (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')
 		strbuf_setlen(&pathbuf, pathbuf.len - 1);
 
-	if (!alt_odb_usable(&the_repository->objects, &pathbuf, normalized_objdir)) {
+	if (!alt_odb_usable(&r->objects, &pathbuf, normalized_objdir)) {
 		strbuf_release(&pathbuf);
 		return -1;
 	}
@@ -426,12 +425,12 @@ static int link_alt_odb_entry_the_repository(const char *entry,
 	ent = alloc_alt_odb(pathbuf.buf);
 
 	/* add the alternate entry */
-	*the_repository->objects.alt_odb_tail = ent;
-	the_repository->objects.alt_odb_tail = &(ent->next);
+	*r->objects.alt_odb_tail = ent;
+	r->objects.alt_odb_tail = &(ent->next);
 	ent->next = NULL;
 
 	/* recursively add alternates */
-	read_info_alternates(the_repository, pathbuf.buf, depth + 1);
+	read_info_alternates(r, pathbuf.buf, depth + 1);
 
 	strbuf_release(&pathbuf);
 	return 0;
@@ -466,12 +465,8 @@ static const char *parse_alt_odb_entry(const char *string,
 	return end;
 }
 
-#define link_alt_odb_entries(r, a, s, rb, d) \
-	link_alt_odb_entries_##r(a, s, rb, d)
-static void link_alt_odb_entries_the_repository(const char *alt,
-						int sep,
-						const char *relative_base,
-						int depth)
+static void link_alt_odb_entries(struct repository *r, const char *alt,
+				 int sep, const char *relative_base, int depth)
 {
 	struct strbuf objdirbuf = STRBUF_INIT;
 	struct strbuf entry = STRBUF_INIT;
@@ -485,7 +480,7 @@ static void link_alt_odb_entries_the_repository(const char *alt,
 		return;
 	}
 
-	strbuf_add_absolute_path(&objdirbuf, get_object_directory());
+	strbuf_add_absolute_path(&objdirbuf, r->objects.objectdir);
 	if (strbuf_normalize_path(&objdirbuf) < 0)
 		die("unable to normalize object directory: %s",
 		    objdirbuf.buf);
@@ -494,15 +489,16 @@ static void link_alt_odb_entries_the_repository(const char *alt,
 		alt = parse_alt_odb_entry(alt, sep, &entry);
 		if (!entry.len)
 			continue;
-		link_alt_odb_entry(the_repository, entry.buf,
+		link_alt_odb_entry(r, entry.buf,
 				   relative_base, depth, objdirbuf.buf);
 	}
 	strbuf_release(&entry);
 	strbuf_release(&objdirbuf);
 }
 
-static void read_info_alternates_the_repository(const char *relative_base,
-						int depth)
+static void read_info_alternates(struct repository *r,
+				 const char *relative_base,
+				 int depth)
 {
 	char *path;
 	struct strbuf buf = STRBUF_INIT;
@@ -514,7 +510,7 @@ static void read_info_alternates_the_repository(const char *relative_base,
 		return;
 	}
 
-	link_alt_odb_entries(the_repository, buf.buf, '\n', relative_base, depth);
+	link_alt_odb_entries(r, buf.buf, '\n', relative_base, depth);
 	strbuf_release(&buf);
 	free(path);
 }
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#222 lore raw

PATCH v4 15/27 sha1_file: allow prepare_alt_odb to handle arbitrary repositories

In reply to Stefan Beller
---
 object-store.h |  3 +--
 sha1_file.c    | 12 +++++-------
 2 files changed, 6 insertions(+), 9 deletions(-)
object-store.h +1 -2
index 08cd48ade11..24b9a750aef 100644
@@ -24,8 +24,7 @@ struct alternate_object_database {
 	 */
 	char path[FLEX_ARRAY];
 };
-#define prepare_alt_odb(r) prepare_alt_odb_##r()
-void prepare_alt_odb_the_repository(void);
+void prepare_alt_odb(struct repository *r);
 char *compute_alternate_path(const char *path, struct strbuf *err);
 typedef int alt_odb_fn(struct alternate_object_database *, void *);
 int foreach_alt_odb(alt_odb_fn, void*);
sha1_file.c +5 -7
index ab42d430b8a..03b9bbe8bb7 100644
@@ -671,21 +671,19 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 	return r;
 }
 
-void prepare_alt_odb_the_repository(void)
+void prepare_alt_odb(struct repository *r)
 {
 	const char *alt;
 
-	if (the_repository->objects.alt_odb_tail)
+	if (r->objects.alt_odb_tail)
 		return;
 
 	alt = getenv(ALTERNATE_DB_ENVIRONMENT);
 
-	the_repository->objects.alt_odb_tail =
-			&the_repository->objects.alt_odb_list;
-	link_alt_odb_entries(the_repository, alt,
-			     PATH_SEP, NULL, 0);
+	r->objects.alt_odb_tail = &r->objects.alt_odb_list;
+	link_alt_odb_entries(r, alt, PATH_SEP, NULL, 0);
 
-	read_info_alternates(the_repository, get_object_directory(), 0);
+	read_info_alternates(r, r->objects.objectdir, 0);
 }
 
 /* Returns 1 if we have successfully freshened the file, 0 otherwise. */
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#223 lore raw

PATCH v4 16/27 sha1_file: add repository argument to sha1_file_name

In reply to Stefan Beller

Add a repository argument to allow sha1_file_name callers to be more specific about which repository to handle. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

While at it, move the declaration to object-store.h, where it should be easier to find.

---
 cache.h        |  6 ------
 http-walker.c  |  3 ++-
 http.c         |  5 ++---
 object-store.h |  7 +++++++
 sha1_file.c    | 10 +++++-----
 5 files changed, 16 insertions(+), 15 deletions(-)
cache.h +0 -6
index 63889acb591..94d85a5c692 100644
@@ -936,12 +936,6 @@ extern void check_repository_format(void);
 #define DATA_CHANGED    0x0020
 #define TYPE_CHANGED    0x0040
 
-/*
- * Put in `buf` the name of the file in the local object database that
- * would be used to store a loose object with the specified sha1.
- */
-extern void sha1_file_name(struct strbuf *buf, const unsigned char *sha1);
-
 /*
  * Return an abbreviated sha1 unique within this repository's object database.
  * The result will be at least `len` characters long, and will be NUL
http-walker.c +2 -1
index 07c2b1af826..2c33969123a 100644
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "repository.h"
 #include "commit.h"
 #include "walker.h"
 #include "http.h"
@@ -545,7 +546,7 @@ static int fetch_object(struct walker *walker, unsigned char *sha1)
 		ret = error("File %s has bad hash", hex);
 	} else if (req->rename < 0) {
 		struct strbuf buf = STRBUF_INIT;
-		sha1_file_name(&buf, req->sha1);
+		sha1_file_name(the_repository, &buf, req->sha1);
 		ret = error("unable to write sha1 filename %s", buf.buf);
 		strbuf_release(&buf);
 	}
http.c +2 -3
index 31755023a4f..df9dbea59c5 100644
@@ -2246,7 +2246,7 @@ struct http_object_request *new_http_object_request(const char *base_url,
 	hashcpy(freq->sha1, sha1);
 	freq->localfile = -1;
 
-	sha1_file_name(&filename, sha1);
+	sha1_file_name(the_repository, &filename, sha1);
 	snprintf(freq->tmpfile, sizeof(freq->tmpfile),
 		 "%s.temp", filename.buf);
 
@@ -2395,8 +2395,7 @@ int finish_http_object_request(struct http_object_request *freq)
 		unlink_or_warn(freq->tmpfile);
 		return -1;
 	}
-
-	sha1_file_name(&filename, freq->sha1);
+	sha1_file_name(the_repository, &filename, freq->sha1);
 	freq->rename = finalize_object_file(freq->tmpfile, filename.buf);
 	strbuf_release(&filename);
 
object-store.h +7 -0
index 24b9a750aef..c0d1292cfc2 100644
@@ -113,4 +113,11 @@ struct raw_object_store {
 
 void raw_object_store_clear(struct raw_object_store *o);
 
+/*
+ * Put in `buf` the name of the file in the local object database that
+ * would be used to store a loose object with the specified sha1.
+ */
+#define sha1_file_name(r, b, s) sha1_file_name_##r(b, s)
+void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1);
+
 #endif /* OBJECT_STORE_H */
sha1_file.c +5 -5
index 03b9bbe8bb7..8ef2f856035 100644
@@ -321,7 +321,7 @@ static void fill_sha1_path(struct strbuf *buf, const unsigned char *sha1)
 	}
 }
 
-void sha1_file_name(struct strbuf *buf, const unsigned char *sha1)
+void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1)
 {
 	strbuf_addstr(buf, get_object_directory());
 	strbuf_addch(buf, '/');
@@ -715,7 +715,7 @@ static int check_and_freshen_local(const unsigned char *sha1, int freshen)
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(&buf, sha1);
+	sha1_file_name(the_repository, &buf, sha1);
 
 	return check_and_freshen_file(buf.buf, freshen);
 }
@@ -876,7 +876,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(&buf, sha1);
+	sha1_file_name(the_repository, &buf, sha1);
 	*path = buf.buf;
 
 	if (!lstat(*path, st))
@@ -905,7 +905,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(&buf, sha1);
+	sha1_file_name(the_repository, &buf, sha1);
 	*path = buf.buf;
 
 	fd = git_open(*path);
@@ -1591,7 +1591,7 @@ static int write_loose_object(const struct object_id *oid, char *hdr,
 	static struct strbuf filename = STRBUF_INIT;
 
 	strbuf_reset(&filename);
-	sha1_file_name(&filename, oid->hash);
+	sha1_file_name(the_repository, &filename, oid->hash);
 
 	fd = create_tmpfile(&tmp_file, filename.buf);
 	if (fd < 0) {
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#224 lore raw

PATCH v4 17/27 sha1_file: add repository argument to stat_sha1_file

In reply to Stefan Beller

Add a repository argument to allow the stat_sha1_file caller to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

---
 sha1_file.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
sha1_file.c +5 -4
index 8ef2f856035..b7e6e4a9bfc 100644
@@ -869,8 +869,9 @@ int git_open_cloexec(const char *name, int flags)
  * Note that it may point to static storage and is only valid until another
  * call to sha1_file_name(), etc.
  */
-static int stat_sha1_file(const unsigned char *sha1, struct stat *st,
-			  const char **path)
+#define stat_sha1_file(r, s, st, p) stat_sha1_file_##r(s, st, p)
+static int stat_sha1_file_the_repository(const unsigned char *sha1,
+					 struct stat *st, const char **path)
 {
 	struct alternate_object_database *alt;
 	static struct strbuf buf = STRBUF_INIT;
@@ -1176,7 +1177,7 @@ static int sha1_loose_object_info(const unsigned char *sha1,
 	if (!oi->typep && !oi->typename && !oi->sizep && !oi->contentp) {
 		const char *path;
 		struct stat st;
-		if (stat_sha1_file(sha1, &st, &path) < 0)
+		if (stat_sha1_file(the_repository, sha1, &st, &path) < 0)
 			return -1;
 		if (oi->disk_sizep)
 			*oi->disk_sizep = st.st_size;
@@ -1390,7 +1391,7 @@ void *read_sha1_file_extended(const unsigned char *sha1,
 		die("replacement %s not found for %s",
 		    sha1_to_hex(repl), sha1_to_hex(sha1));
 
-	if (!stat_sha1_file(repl, &st, &path))
+	if (!stat_sha1_file(the_repository, repl, &st, &path))
 		die("loose object %s (stored in %s) is corrupt",
 		    sha1_to_hex(repl), path);
 
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#225 lore raw

PATCH v4 18/27 sha1_file: add repository argument to open_sha1_file

In reply to Stefan Beller

Add a repository argument to allow the open_sha1_file caller to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

---
 sha1_file.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
sha1_file.c +4 -2
index b7e6e4a9bfc..7e471e0cf4b 100644
@@ -898,7 +898,9 @@ static int stat_sha1_file_the_repository(const unsigned char *sha1,
  * Like stat_sha1_file(), but actually open the object and return the
  * descriptor. See the caveats on the "path" parameter above.
  */
-static int open_sha1_file(const unsigned char *sha1, const char **path)
+#define open_sha1_file(r, s, p) open_sha1_file_##r(s, p)
+static int open_sha1_file_the_repository(const unsigned char *sha1,
+					 const char **path)
 {
 	int fd;
 	struct alternate_object_database *alt;
@@ -941,7 +943,7 @@ static void *map_sha1_file_1(const char *path,
 	if (path)
 		fd = git_open(path);
 	else
-		fd = open_sha1_file(sha1, &path);
+		fd = open_sha1_file(the_repository, sha1, &path);
 	map = NULL;
 	if (fd >= 0) {
 		struct stat st;
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#226 lore raw

PATCH v4 19/27 sha1_file: add repository argument to map_sha1_file_1

In reply to Stefan Beller

Add a repository argument to allow the map_sha1_file_1 caller to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

---
 sha1_file.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
sha1_file.c +6 -5
index 7e471e0cf4b..d4af8f56814 100644
@@ -933,9 +933,10 @@ static int open_sha1_file_the_repository(const unsigned char *sha1,
  * Map the loose object at "path" if it is not NULL, or the path found by
  * searching for a loose object named "sha1".
  */
-static void *map_sha1_file_1(const char *path,
-			     const unsigned char *sha1,
-			     unsigned long *size)
+#define map_sha1_file_1(r, p, s, si) map_sha1_file_1_##r(p, s, si)
+static void *map_sha1_file_1_the_repository(const char *path,
+					    const unsigned char *sha1,
+					    unsigned long *size)
 {
 	void *map;
 	int fd;
@@ -964,7 +965,7 @@ static void *map_sha1_file_1(const char *path,
 
 void *map_sha1_file(const unsigned char *sha1, unsigned long *size)
 {
-	return map_sha1_file_1(NULL, sha1, size);
+	return map_sha1_file_1(the_repository, NULL, sha1, size);
 }
 
 static int unpack_sha1_short_header(git_zstream *stream,
@@ -2200,7 +2201,7 @@ int read_loose_object(const char *path,
 
 	*contents = NULL;
 
-	map = map_sha1_file_1(path, NULL, &mapsize);
+	map = map_sha1_file_1(the_repository, path, NULL, &mapsize);
 	if (!map) {
 		error_errno("unable to mmap %s", path);
 		goto out;
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#227 lore raw

PATCH v4 20/27 sha1_file: add repository argument to map_sha1_file

In reply to Stefan Beller

Add a repository argument to allow map_sha1_file callers to be more specific about which repository to handle. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

While at it, move the declaration to object-store.h, where it should be easier to find.

---
 cache.h        | 1 -
 object-store.h | 3 +++
 sha1_file.c    | 4 ++--
 streaming.c    | 5 ++++-
 4 files changed, 9 insertions(+), 4 deletions(-)
cache.h +0 -1
index 94d85a5c692..57173991839 100644
@@ -1226,7 +1226,6 @@ extern int force_object_loose(const struct object_id *oid, time_t mtime);
 
 extern int git_open_cloexec(const char *name, int flags);
 #define git_open(name) git_open_cloexec(name, O_RDONLY)
-extern void *map_sha1_file(const unsigned char *sha1, unsigned long *size);
 extern int unpack_sha1_header(git_zstream *stream, unsigned char *map, unsigned long mapsize, void *buffer, unsigned long bufsiz);
 extern int parse_sha1_header(const char *hdr, unsigned long *sizep);
 
object-store.h +3 -0
index c0d1292cfc2..dbd589e0083 100644
@@ -120,4 +120,7 @@ void raw_object_store_clear(struct raw_object_store *o);
 #define sha1_file_name(r, b, s) sha1_file_name_##r(b, s)
 void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1);
 
+#define map_sha1_file(r, s, sz) map_sha1_file_##r(s, sz)
+void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size);
+
 #endif /* OBJECT_STORE_H */
sha1_file.c +2 -2
index d4af8f56814..6ec43e745d7 100644
@@ -963,7 +963,7 @@ static void *map_sha1_file_1_the_repository(const char *path,
 	return map;
 }
 
-void *map_sha1_file(const unsigned char *sha1, unsigned long *size)
+void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size)
 {
 	return map_sha1_file_1(the_repository, NULL, sha1, size);
 }
@@ -1187,7 +1187,7 @@ static int sha1_loose_object_info(const unsigned char *sha1,
 		return 0;
 	}
 
-	map = map_sha1_file(sha1, &mapsize);
+	map = map_sha1_file(the_repository, sha1, &mapsize);
 	if (!map)
 		return -1;
 
streaming.c +4 -1
index 5892b50bd89..22d27df55eb 100644
@@ -3,6 +3,8 @@
  */
 #include "cache.h"
 #include "streaming.h"
+#include "repository.h"
+#include "object-store.h"
 #include "packfile.h"
 
 enum input_source {
@@ -335,7 +337,8 @@ static struct stream_vtbl loose_vtbl = {
 
 static open_method_decl(loose)
 {
-	st->u.loose.mapped = map_sha1_file(sha1, &st->u.loose.mapsize);
+	st->u.loose.mapped = map_sha1_file(the_repository,
+					   sha1, &st->u.loose.mapsize);
 	if (!st->u.loose.mapped)
 		return -1;
 	if ((unpack_sha1_header(&st->z,
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#228 lore raw

PATCH v4 21/27 sha1_file: add repository argument to sha1_loose_object_info

In reply to Stefan Beller

Add a repository argument to allow the sha1_loose_object_info caller to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time.

---
 sha1_file.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
sha1_file.c +5 -4
index 6ec43e745d7..831e2da3572 100644
@@ -1154,9 +1154,10 @@ int parse_sha1_header(const char *hdr, unsigned long *sizep)
 	return parse_sha1_header_extended(hdr, &oi, 0);
 }
 
-static int sha1_loose_object_info(const unsigned char *sha1,
-				  struct object_info *oi,
-				  int flags)
+#define sha1_loose_object_info(r, s, o, f) sha1_loose_object_info_##r(s, o, f)
+static int sha1_loose_object_info_the_repository(const unsigned char *sha1,
+						 struct object_info *oi,
+						 int flags)
 {
 	int status = 0;
 	unsigned long mapsize;
@@ -1275,7 +1276,7 @@ int sha1_object_info_extended(const unsigned char *sha1, struct object_info *oi,
 			break;
 
 		/* Most likely it's a loose object. */
-		if (!sha1_loose_object_info(real, oi, flags))
+		if (!sha1_loose_object_info(the_repository, real, oi, flags))
 			return 0;
 
 		/* Not a loose object; someone else may have just packed it. */
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#229 lore raw

PATCH v4 22/27 sha1_file: allow sha1_file_name to handle arbitrary repositories

In reply to Stefan Beller
---
 object-store.h | 3 +--
 sha1_file.c    | 4 ++--
 2 files changed, 3 insertions(+), 4 deletions(-)
object-store.h +1 -2
index dbd589e0083..8b38330d256 100644
@@ -117,8 +117,7 @@ void raw_object_store_clear(struct raw_object_store *o);
  * Put in `buf` the name of the file in the local object database that
  * would be used to store a loose object with the specified sha1.
  */
-#define sha1_file_name(r, b, s) sha1_file_name_##r(b, s)
-void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1);
+void sha1_file_name(struct repository *r, struct strbuf *buf, const unsigned char *sha1);
 
 #define map_sha1_file(r, s, sz) map_sha1_file_##r(s, sz)
 void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size);
sha1_file.c +2 -2
index 831e2da3572..59993a0878d 100644
@@ -321,9 +321,9 @@ static void fill_sha1_path(struct strbuf *buf, const unsigned char *sha1)
 	}
 }
 
-void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1)
+void sha1_file_name(struct repository *r, struct strbuf *buf, const unsigned char *sha1)
 {
-	strbuf_addstr(buf, get_object_directory());
+	strbuf_addstr(buf, r->objects.objectdir);
 	strbuf_addch(buf, '/');
 	fill_sha1_path(buf, sha1);
 }
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#230 lore raw

PATCH v4 23/27 sha1_file: allow stat_sha1_file to handle arbitrary repositories

In reply to Stefan Beller
---
 sha1_file.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)
sha1_file.c +5 -6
index 59993a0878d..952deb02195 100644
@@ -869,23 +869,22 @@ int git_open_cloexec(const char *name, int flags)
  * Note that it may point to static storage and is only valid until another
  * call to sha1_file_name(), etc.
  */
-#define stat_sha1_file(r, s, st, p) stat_sha1_file_##r(s, st, p)
-static int stat_sha1_file_the_repository(const unsigned char *sha1,
-					 struct stat *st, const char **path)
+static int stat_sha1_file(struct repository *r, const unsigned char *sha1,
+			  struct stat *st, const char **path)
 {
 	struct alternate_object_database *alt;
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(the_repository, &buf, sha1);
+	sha1_file_name(r, &buf, sha1);
 	*path = buf.buf;
 
 	if (!lstat(*path, st))
 		return 0;
 
-	prepare_alt_odb(the_repository);
+	prepare_alt_odb(r);
 	errno = ENOENT;
-	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
+	for (alt = r->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		if (!lstat(*path, st))
 			return 0;
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#231 lore raw

PATCH v4 24/27 sha1_file: allow open_sha1_file to handle arbitrary repositories

In reply to Stefan Beller
---
 sha1_file.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)
sha1_file.c +5 -6
index 952deb02195..73bbe3a3e88 100644
@@ -897,9 +897,8 @@ static int stat_sha1_file(struct repository *r, const unsigned char *sha1,
  * Like stat_sha1_file(), but actually open the object and return the
  * descriptor. See the caveats on the "path" parameter above.
  */
-#define open_sha1_file(r, s, p) open_sha1_file_##r(s, p)
-static int open_sha1_file_the_repository(const unsigned char *sha1,
-					 const char **path)
+static int open_sha1_file(struct repository *r,
+			  const unsigned char *sha1, const char **path)
 {
 	int fd;
 	struct alternate_object_database *alt;
@@ -907,7 +906,7 @@ static int open_sha1_file_the_repository(const unsigned char *sha1,
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(the_repository, &buf, sha1);
+	sha1_file_name(r, &buf, sha1);
 	*path = buf.buf;
 
 	fd = git_open(*path);
@@ -915,8 +914,8 @@ static int open_sha1_file_the_repository(const unsigned char *sha1,
 		return fd;
 	most_interesting_errno = errno;
 
-	prepare_alt_odb(the_repository);
-	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
+	prepare_alt_odb(r);
+	for (alt = r->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		fd = git_open(*path);
 		if (fd >= 0)
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#232 lore raw

PATCH v4 25/27 sha1_file: allow map_sha1_file_1 to handle arbitrary repositories

In reply to Stefan Beller

From: Jonathan Nieder <[email protected]>

---
 sha1_file.c | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)
sha1_file.c +3 -5
index 73bbe3a3e88..e32f1da6b69 100644
@@ -931,10 +931,8 @@ static int open_sha1_file(struct repository *r,
  * Map the loose object at "path" if it is not NULL, or the path found by
  * searching for a loose object named "sha1".
  */
-#define map_sha1_file_1(r, p, s, si) map_sha1_file_1_##r(p, s, si)
-static void *map_sha1_file_1_the_repository(const char *path,
-					    const unsigned char *sha1,
-					    unsigned long *size)
+static void *map_sha1_file_1(struct repository *r, const char *path,
+			     const unsigned char *sha1, unsigned long *size)
 {
 	void *map;
 	int fd;
@@ -942,7 +940,7 @@ static void *map_sha1_file_1_the_repository(const char *path,
 	if (path)
 		fd = git_open(path);
 	else
-		fd = open_sha1_file(the_repository, sha1, &path);
+		fd = open_sha1_file(r, sha1, &path);
 	map = NULL;
 	if (fd >= 0) {
 		struct stat st;
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#233 lore raw

PATCH v4 26/27 sha1_file: allow map_sha1_file to handle arbitrary repositories

In reply to Stefan Beller
---
 object-store.h | 3 +--
 sha1_file.c    | 5 +++--
 2 files changed, 4 insertions(+), 4 deletions(-)
object-store.h +1 -2
index 8b38330d256..afe2f93459b 100644
@@ -119,7 +119,6 @@ void raw_object_store_clear(struct raw_object_store *o);
  */
 void sha1_file_name(struct repository *r, struct strbuf *buf, const unsigned char *sha1);
 
-#define map_sha1_file(r, s, sz) map_sha1_file_##r(s, sz)
-void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size);
+void *map_sha1_file(struct repository *r, const unsigned char *sha1, unsigned long *size);
 
 #endif /* OBJECT_STORE_H */
sha1_file.c +3 -2
index e32f1da6b69..902cba42106 100644
@@ -959,9 +959,10 @@ static void *map_sha1_file_1(struct repository *r, const char *path,
 	return map;
 }
 
-void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size)
+void *map_sha1_file(struct repository *r,
+		    const unsigned char *sha1, unsigned long *size)
 {
-	return map_sha1_file_1(the_repository, NULL, sha1, size);
+	return map_sha1_file_1(r, NULL, sha1, size);
 }
 
 static int unpack_sha1_short_header(git_zstream *stream,
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#234 lore raw

PATCH v4 27/27 sha1_file: allow sha1_loose_object_info to handle arbitrary repositories

In reply to Stefan Beller

From: Jonathan Nieder <[email protected]>

---
 sha1_file.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)
sha1_file.c +5 -6
index 902cba42106..dfc8deec38e 100644
@@ -1151,10 +1151,9 @@ int parse_sha1_header(const char *hdr, unsigned long *sizep)
 	return parse_sha1_header_extended(hdr, &oi, 0);
 }
 
-#define sha1_loose_object_info(r, s, o, f) sha1_loose_object_info_##r(s, o, f)
-static int sha1_loose_object_info_the_repository(const unsigned char *sha1,
-						 struct object_info *oi,
-						 int flags)
+static int sha1_loose_object_info(struct repository *r,
+				  const unsigned char *sha1,
+				  struct object_info *oi, int flags)
 {
 	int status = 0;
 	unsigned long mapsize;
@@ -1178,14 +1177,14 @@ static int sha1_loose_object_info_the_repository(const unsigned char *sha1,
 	if (!oi->typep && !oi->typename && !oi->sizep && !oi->contentp) {
 		const char *path;
 		struct stat st;
-		if (stat_sha1_file(the_repository, sha1, &st, &path) < 0)
+		if (stat_sha1_file(r, sha1, &st, &path) < 0)
 			return -1;
 		if (oi->disk_sizep)
 			*oi->disk_sizep = st.st_size;
 		return 0;
 	}
 
-	map = map_sha1_file(the_repository, sha1, &mapsize);
+	map = map_sha1_file(r, sha1, &mapsize);
 	if (!map)
 		return -1;
 
Signature · 1 line
2.16.1.291.g4437f3f132-goog
#235 lore raw
In reply to Stefan Beller

I notice that there are a few global state (or configuration rather) left after this: packed_git_window_size, packed_git_limit and delta_base_cache_limit. These can be changed by $GIT_DIR/config, but since it's still global, a submodule repository will use the same settings of its super repository. If $SUBMODULE/config changes any of these, they are ignored.

The natural thing to do is move these to raw_object_store too (and repo_submodule_init needs to check submodule's config file). But one may argue these should be per-process instead of per-repo though. I don't know. But I thought I should mention this.

Signature · 1 line
Duy
#236 lore raw
Quoted 6 lines · Duy Nguyen

I notice that there are a few global state (or configuration rather) left after this: packed_git_window_size, packed_git_limit and delta_base_cache_limit. These can be changed by $GIT_DIR/config, but since it's still global, a submodule repository will use the same settings of its super repository. If $SUBMODULE/config changes any of these, they are ignored.

That sounds all packing related, which I plan on working on next.

Quoted 4 lines

The natural thing to do is move these to raw_object_store too (and repo_submodule_init needs to check submodule's config file). But one may argue these should be per-process instead of per-repo though. I don't know. But I thought I should mention this.

For now a process and a repository is the same as git-gc or git-repack doesn't know about the --recurse-submodules flag, yet.
I wonder if we ever want to teach those commands the submodule recursion, because of the issue you bring up, which settings do we apply for a submodule? Traditionally we'd just have the command line override the configuration, which I don't know if it is a good idea for these settings.

Thanks,
Stefan

#237 lore raw
Quoted 6 lines · Stefan Beller
Quoted 4 lines

The natural thing to do is move these to raw_object_store too (and repo_submodule_init needs to check submodule's config file). But one may argue these should be per-process instead of per-repo though. I don't know. But I thought I should mention this.

For now a process and a repository is the same as git-gc or git-repack

I think you're thinking about the pack writing part, but these are configuration for pack reading (aka "object store"). If you read a blob from a submodule (e.g. git-grep), you'll use these configurations at some point.

There are of course more configuration for pack writing (like zlib_compression_level) which I deliberately avoided to mention because I don't even know where they belong.

Quoted 5 lines

doesn't know about the --recurse-submodules flag, yet.
I wonder if we ever want to teach those commands the submodule recursion, because of the issue you bring up, which settings do we apply for a submodule? Traditionally we'd just have the command line override the configuration, which I don't know if it is a good idea for these settings.

Signature · 1 line
Duy
#238 lore raw
In reply to Stefan Beller
Quoted 3 lines · On Fri, 23 Feb 2018 16:47:27 -0800 Stefan Beller <[email protected]> wrote:

v4:
* addressed feedback from the single patches (mostly nits)
* rebased on latest master

The patches I looked at previously (patches 7, 15, 19, 22, and 24) look good to me.