Skip to contentSkip to conversation

Optimize sha1_object_info for loose objects, not concurrent repacks

PATCH In review Steven Grimm 2 participants 2 messages Jun 15, 2016

lore
Share & tools
lore

Steven Grimm

author
#0 lore raw

PATCH 1/1 Optimize sha1_object_info for loose objects, not concurrent repacks

When dealing with a repository with lots of loose objects, sha1_object_info would rescan the packs directory every time an unpacked object was referenced before finally giving up and looking for the loose object. This caused a lot of extra unnecessary system calls during git pack-objects; the code was rereading the entire pack directory once for each loose object file.

This patch looks for a loose object before falling back to rescanning the pack directory, rather than the other way around.

Signed-off-by Steven Grimm <[email protected]>

---

I discovered this by running strace on a pack-objects that was taking especially long to run; it was making more system calls to scan the pack directory than to do stuff with the loose objects, which didn't seem right.

 sha1_file.c |    9 ++++++++-
 1 files changed, 8 insertions(+), 1 deletions(-)
sha1_file.c +8 -1
index e281c14..32e4664 100644
@@ -1929,11 +1929,18 @@ static int sha1_loose_object_info(const unsigned char *sha1, unsigned long *size
 int sha1_object_info(const unsigned char *sha1, unsigned long *sizep)
 {
 	struct pack_entry e;
+	int status;
 
 	if (!find_pack_entry(sha1, &e, NULL)) {
+		/* Most likely it's a loose object. */
+		status = sha1_loose_object_info(sha1, sizep);
+		if (status >= 0)
+			return status;
+
+		/* Not a loose object; someone else may have just packed it. */
 		reprepare_packed_git();
 		if (!find_pack_entry(sha1, &e, NULL))
-			return sha1_loose_object_info(sha1, sizep);
+			return status;
 	}
 	return packed_object_info(e.p, e.offset, sizep);
 }
Signature · 1 line
1.6.0.rc1.66.gc78d7
#1 lore raw
Quoted 10 lines · Steven Grimm

When dealing with a repository with lots of loose objects, sha1_object_info would rescan the packs directory every time an unpacked object was referenced before finally giving up and looking for the loose object. This caused a lot of extra unnecessary system calls during git pack-objects; the code was rereading the entire pack directory once for each loose object file.

This patch looks for a loose object before falling back to rescanning the pack directory, rather than the other way around.

Signed-off-by Steven Grimm <[email protected]>

Heh. Cute bug.

ACK.

Quoted 24 lines
sha1_file.c +8 -1
index e281c14..32e4664 100644
@@ -1929,11 +1929,18 @@ static int sha1_loose_object_info(const unsigned char *sha1, unsigned long *size
 int sha1_object_info(const unsigned char *sha1, unsigned long *sizep)
 {
 	struct pack_entry e;
+	int status;
 
 	if (!find_pack_entry(sha1, &e, NULL)) {
+		/* Most likely it's a loose object. */
+		status = sha1_loose_object_info(sha1, sizep);
+		if (status >= 0)
+			return status;
+
+		/* Not a loose object; someone else may have just packed it. */
 		reprepare_packed_git();
 		if (!find_pack_entry(sha1, &e, NULL))
-			return sha1_loose_object_info(sha1, sizep);
+			return status;
 	}
 	return packed_object_info(e.p, e.offset, sizep);
 }
Signature · 1 line
Shawn.