Skip to contentSkip to conversation

git-repack --max-pack-size: Add new file statics and struct fields

PATCH In review Dana How 2 participants 3 messages Jun 15, 2016

lore
Share & tools
lore

Dana How

author
#0 lore raw

PATCH 1/8 git-repack --max-pack-size: Add new file statics and struct fields

This adds "offset_limit", which will contain the limit specified by --max-pack-size, "written_list", the actual list of objects written to the current pack, and "nr_written", the number of objects in written_list. "prev_pack" is added to struct object_entry to indicate when an object has already been written but to a previous pack. The fields in object_entry are re-arranged & shrunk to save memory.

---
 builtin-pack-objects.c |   21 ++++++++++++---------
 1 files changed, 12 insertions(+), 9 deletions(-)
builtin-pack-objects.c +12 -9
index 45ac3e4..64318b3 100644
@@ -21,18 +21,15 @@ git-pack-objects [{ -q | --progress | --all-progress }] \n\
 	[<ref-list | <object-list]";
 
 struct object_entry {
-	unsigned char sha1[20];
 	unsigned long size;	/* uncompressed size */
+	unsigned long delta_size;	/* delta data size (uncompressed) */
+#define in_pack_header_size delta_size	/* only when reusing pack data */
 	off_t offset;	/* offset into the final pack file;
 				 * nonzero if already written.
 				 */
-	unsigned int depth;	/* delta depth */
-	unsigned int delta_limit;	/* base adjustment for in-pack delta */
 	unsigned int hash;	/* name hint hash */
-	enum object_type type;
-	enum object_type in_pack_type;	/* could be delta */
-	unsigned long delta_size;	/* delta data size (uncompressed) */
-#define in_pack_header_size delta_size	/* only when reusing pack data */
+	unsigned short depth;	/* delta depth */
+	unsigned short delta_limit;	/* base adjustment for in-pack delta */
 	struct object_entry *delta;	/* delta base object */
 	struct packed_git *in_pack; 	/* already in pack */
 	off_t in_pack_offset;
@@ -40,10 +37,14 @@ struct object_entry {
 	struct object_entry *delta_sibling; /* other deltified objects who
 					     * uses the same base as me
 					     */
-	int preferred_base;	/* we do not pack this, but is encouraged to
+	enum object_type type;
+	enum object_type in_pack_type;	/* could be delta */
+	char preferred_base;	/* we do not pack this, but is encouraged to
 				 * be used as the base objectto delta huge
 				 * objects against.
 				 */
+	char prev_pack;		/* written to previous pack? */
+	unsigned char sha1[20];
 };
 
 /*
@@ -66,9 +67,11 @@ static int local;
 static int incremental;
 static int allow_ofs_delta;
 
+static struct object_entry **written_list;
 static struct object_entry **sorted_by_sha, **sorted_by_type;
 static struct object_entry *objects;
-static uint32_t nr_objects, nr_alloc, nr_result;
+static uint32_t nr_objects, nr_alloc, nr_result, nr_written;
+static uint32_t offset_limit;
 static const char *base_name;
 static unsigned char pack_file_sha1[20];
 static int progress = 1;
Signature · 1 line
1.5.1.89.g8abf0
#1 lore raw
Quoted 2 lines · Dana How

... The fields
in object_entry are re-arranged & shrunk to save memory.

Is the driving principle "place fields with coarser alignment requirements first in the struct"? I noticed you have a handful pointers and an off_t after two short fields. Two shorts would be likely to make the next field aligned suitable for an int, but (1) if we ever add another short later that would not be true anymore, and (2) I suspect a pointer and an off_t can be longer than an int but int would never be longer than them.

author
#2 lore raw
Quoted 5 lines · Junio C Hamano
Quoted 2 lines · Dana How

... The fields
in object_entry are re-arranged & shrunk to save memory.

Quoted 7 lines

Is the driving principle "place fields with coarser alignment requirements first in the struct"? I noticed you have a handful pointers and an off_t after two short fields. Two shorts would be likely to make the next field aligned suitable for an int, but (1) if we ever add another short later that would not be true anymore, and (2) I suspect a pointer and an off_t can be longer than an int but int would never be longer than them.

All true. My objective was to save more memory than I had "wasted" by adding the prev_pack field. Since it appears I'll be submitting the patchset yet again, I'll make this re-arrangement complete next time.

Thanks,

Signature · 1 line
Dana L. How  [email protected]  +1 650 804 5991 cell