nd/pack-objects-pack-struct updates
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v8 Latest15 patches 4 repliesIn review
- v713 patches 27 repliesSuperseded
- v611 patches 38 repliesSuperseded
Earlier versions · 5
| Version | Posted | Patches | Replies | Status |
|---|---|---|---|---|
| v5 | 11 patches | 8 replies | Superseded | |
| v4 | 11 patches | 10 replies | Superseded | |
| v3 | 12 patches | 6 replies | Superseded | |
| v2 | 9 patches | 1 reply | Superseded | |
| v1 | 11 patches | 9 replies | Superseded |
Changes in v8 from the cover letter
- prefer BUG() over die()
- do "1U <<" instead of "1 << " to avoid undefined behavior with signed shifting.
- add more comments based on Jeff's feedback
- plug a leak in try_delta() when delta_size is too large
- be kind and set depth/cache_max_small_delta_size to max limit instead of dying when the user gives a value over limit
- make travis execute pack-objects uncommon code
- use git_env_*() instead of manually handling getenv() values
- fallback code for when a new pack is added when pack-objects is running
- Compressed cached delta size limit is increased from 64k to 1MB
- Cached delta size limit is decreased from 2G to 1MB
Patches in v8 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/15 | t/README: mention about running the test suite in special modest/ | +10 | 0 replies | |
| 2/15 | pack-objects: a bit of document about struct object_entrypack-objects.h | +45 | 0 replies | |
| 3/15 | pack-objects: turn type and in_pack_type to bitfieldsbuiltin/ cache.h object.h +2 more | +58 | 0 replies | |
| 4/15 | pack-objects: use bitfield for object_entry::dfs_statebuiltin/ pack-objects.h | +20 | 0 replies | |
| 5/15 | pack-objects: use bitfield for object_entry::depthDocumentation/ Documentation/ Documentation/ +2 more | +15 | 0 replies | |
| 6/15 | pack-objects: move in_pack_pos out of struct object_entrybuiltin/ pack-bitmap-write.c pack-bitmap.c +2 more | +26 | 0 replies | |
| 7/15 | pack-objects: move in_pack out of struct object_entrybuiltin/ cache.h pack-objects.c +3 more | +128 | 0 replies | |
| 8/15 | pack-objects: refer to delta objects by index instead of pointerbuiltin/ pack-objects.h | +124 | 0 replies | |
| 9/15 | pack-objects: shrink z_delta_size field in struct object_entryDocumentation/ builtin/ pack-objects.h | +22 | 0 replies | |
| 10/15 | pack-objects: don't check size when the object is badbuiltin/ | +2 | 0 replies | |
| 11/15 | pack-objects: clarify the use of object_entry::sizebuiltin/ pack-objects.h | +11 | 0 replies | |
| 12/15 | pack-objects: shrink size field in struct object_entrybuiltin/ pack-objects.c pack-objects.h +1 more | +146 | 0 replies | |
| 13/15 | pack-objects: shrink delta_size field in struct object_entrybuiltin/ pack-objects.h | +38 | 0 replies | |
| 14/15 | pack-objects: reorder members to shrink struct object_entrypack-objects.h | +21 | 0 replies | |
| 15/15 | ci: exercise the whole test suite with uncommon code in pack-objectsci/ | +4 | 0 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v8 yet.
Cover letter [PATCH v8 00/15] nd/pack-objects-pack-struct updates
v8 changes
- prefer BUG() over die()
- do "1U <<" instead of "1 << " to avoid undefined behavior with signed shifting.
- add more comments based on Jeff's feedback
- plug a leak in try_delta() when delta_size is too large
- be kind and set depth/cache_max_small_delta_size to max limit instead of dying when the user gives a value over limit
- make travis execute pack-objects uncommon code
- use git_env_*() instead of manually handling getenv() values
- fallback code for when a new pack is added when pack-objects is running
- Compressed cached delta size limit is increased from 64k to 1MB
- Cached delta size limit is decreased from 2G to 1MB
Interdiff
builtin/pack-objects.c +24 -11
@@ -1439,7 +1439,7 @@ static void check_object(struct object_entry *entry) goto give_up;if (type <0 ) -die ("BUG: invalid type %d" , type); +BUG ("invalid type %d" , type); entry->in_pack_type = type;/*
@@ -1861,6 +1861,11 @@ static pthread_mutex_t progress_mutex;#endif +/* +* Return the size of the object without doing any delta +* reconstruction (so non-deltas are true object sizes, but deltas +* return the size of the delta data). +*/ unsigned long oe_get_size_slow (struct packing_data *pack,const struct object_entry *e) {
@@ -1881,7 +1886,7 @@ unsigned long oe_get_size_slow(struct packing_data *pack, p =oe_in_pack (pack, e);if (!p) -die ("BUG: when e->type is a delta, it must belong to a pack" ); +BUG ("when e->type is a delta, it must belong to a pack" );read_lock (); w_curs =NULL ;
@@ -2006,8 +2011,10 @@ static int try_delta(struct unpacked *trg, struct unpacked *src, delta_buf =create_delta (src->index, trg->data, trg_size, &delta_size, max_size);if (!delta_buf)return 0 ; -if (delta_size >= (1 <<OE_DELTA_SIZE_BITS )) +if (delta_size >= (1U <<OE_DELTA_SIZE_BITS )) { +free (delta_buf);return 0 ; + }if (DELTA (trg_entry)) {/* Prefer only shallower same-sized deltas. */
@@ -2163,7 +2170,7 @@ static void find_deltas(struct object_entry **list, unsigned *list_size,unsigned long size; size =do_compress (&entry->delta_data,DELTA_SIZE (entry)); -if (size < (1 <<OE_Z_DELTA_BITS )) { +if (size < (1U <<OE_Z_DELTA_BITS )) { entry->z_delta_size = size;cache_lock (); delta_cache_size -=DELTA_SIZE (entry);
@@ -3131,7 +3138,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix) };if (DFS_NUM_STATES > (1 <<OE_DFS_STATE_BITS )) -die ("BUG: too many dfs states, increase OE_DFS_STATE_BITS" ); +BUG ("too many dfs states, increase OE_DFS_STATE_BITS" ); check_replace_refs =0 ;
@@ -3149,12 +3156,16 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)if (pack_to_stdout != !base_name || argc)usage_with_options (pack_usage, pack_objects_options); -if (depth >= (1 <<OE_DEPTH_BITS )) -die (_ ("delta chain depth %d is greater than maximum limit %d" ), - depth, (1 <<OE_DEPTH_BITS ) -1 ); -if (cache_max_small_delta_size >= (1 <<OE_Z_DELTA_BITS )) -die (_ ("pack.deltaCacheLimit is greater than maximum limit %d" ), - (1 <<OE_Z_DELTA_BITS ) -1 ); +if (depth >= (1 <<OE_DEPTH_BITS )) { +warning (_ ("delta chain depth %d is too deep, forcing %d" ), + depth, (1 <<OE_DEPTH_BITS ) -1 ); + depth = (1 <<OE_DEPTH_BITS ) -1 ; + } +if (cache_max_small_delta_size >= (1U <<OE_Z_DELTA_BITS )) { +warning (_ ("pack.deltaCacheLimit is too high, forcing %d" ), + (1U <<OE_Z_DELTA_BITS ) -1 ); + cache_max_small_delta_size = (1U <<OE_Z_DELTA_BITS ) -1 ; + }argv_array_push (&rp,"pack-objects" );if (thin) {
@@ -3274,6 +3285,8 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix) } } +prepare_packing_data (&to_pack); +if (progress) progress_state =start_progress (_ ("Counting objects" ),0 );if (!use_internal_rev_list)
ci/run-tests.sh +4 -1
@@ -10,7 +10,10 @@ ln -s "$cache_dir/.prove" t/.prove make --quiet test if test "$jobname" = "linux-gcc" then - GIT_TEST_SPLIT_INDEX=YesPlease make --quiet test + export GIT_TEST_SPLIT_INDEX=YesPlease + export GIT_TEST_FULL_IN_PACK_ARRAY=true + export GIT_TEST_OE_SIZE=10 + make --quiet test fi check_unignored_build_artifacts
pack-objects.c +45 -19
90 lines not on this page · raw message
pack-objects.h +29 -17
115 lines not on this page · raw message
t/README +16 -6
@@ -291,16 +291,26 @@ expect the rest to function correctly. and know what setup is needed for it. Or when you want to run everything up to a certain test. + +Running tests with special setups +--------------------------------- + +The whole test suite could be run to test some special features +that cannot be easily covered by a few specific test cases. These +could be enabled by running the test suite with correct GIT_TEST_ +environment set. + +GIT_TEST_SPLIT_INDEX forces split-index mode on the whole test suite. + GIT_TEST_FULL_IN_PACK_ARRAY exercises the uncommon pack-objects code path where there are more than 1024 packs even if the actual number of packs in repository is below this limit. -GIT_TEST_OE_SIZE_BITS=<bits> exercises the uncommon pack-objects -code path where we do not cache objecct size in memory and read it -from existing packs on demand. This normally only happens when the -object size is over 2GB. This variable forces the code path on any -object larger than 2^<bits> bytes. - +GIT_TEST_OE_SIZE=<n> exercises the uncommon pack-objects code path +where we do not cache object size in memory and read it from existing +packs on demand. This normally only happens when the object size is +over 2GB. This variable forces the code path on any object larger than +<n> bytes. Naming Tests ------------
Nguyễn Thái Ngọc Duy (15):
t/README: mention about running the test suite in special modes
pack-objects: a bit of document about struct object_entry
pack-objects: turn type and in_pack_type to bitfields
pack-objects: use bitfield for object_entry::dfs_state
pack-objects: use bitfield for object_entry::depth
pack-objects: move in_pack_pos out of struct object_entry
pack-objects: move in_pack out of struct object_entry
pack-objects: refer to delta objects by index instead of pointer
pack-objects: shrink z_delta_size field in struct object_entry
pack-objects: don't check size when the object is bad
pack-objects: clarify the use of object_entry::size
pack-objects: shrink size field in struct object_entry
pack-objects: shrink delta_size field in struct object_entry
pack-objects: reorder members to shrink struct object_entry
ci: exercise the whole test suite with uncommon code in pack-objects
Documentation/config.txt | 4 +- Documentation/git-pack-objects.txt | 4 +- Documentation/git-repack.txt | 4 +- builtin/pack-objects.c | 366 +++++++++++++++++++---------- cache.h | 3 + ci/run-tests.sh | 5 +- object.h | 1 - pack-bitmap-write.c | 14 +- pack-bitmap.c | 2 +- pack-bitmap.h | 4 +- pack-objects.c | 69 ++++++ pack-objects.h | 312 ++++++++++++++++++++++-- t/README | 20 ++ t/t5300-pack-object.sh | 5 + 14 files changed, 650 insertions(+), 163 deletions(-)
Signature · 1 line
2.17.0.rc2.515.g4feb9b7923