Avoid expensive 'repack -ad' in gc --auto
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v2 Latest5 patches 20 repliesIn review
- v11 patch 3 repliesSuperseded
Changes in v2 from the cover letter
- the core idea remains the same, mem_have is increased to half total memory though. I figure including the whole mmap'd pack in the memory estimation may be a bit much, which is why I make this change.
- no creating .keep files temporarily
- the config key is renamed gc.bigBasePackThreshold (named after core.bigFileThreshold)
- note that if you set gc.bigFileThreshold, then normal gc (without --auto) can trigger this mode too.
- you can also control this with --[no-]keep-base-pack
- documents and tests
- some more progress output improvements
Patches in v2 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/5 | fixup! Add a test showing that 'git repack' throws away grafted-away parentst/ | +1 | 1 reply | |
| 2/5 | repack: add --keep-pack optionDocumentation/ Documentation/ builtin/ +2 more | +78 | 2 replies | |
| 3/5 | gc --auto: exclude base pack if not enough mem to "repack -ad"Documentation/ Documentation/ builtin/ +5 more | +204 | 13 replies | |
| 4/5 | pack-objects: show some progress when counting kept objectsbuiltin/ | +6 | 3 replies | |
| 5/5 | pack-objects: display progress in get_object_details()builtin/ | +6 | 0 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v2 yet.
Cover letter [PATCH v2 0/5] Avoid expensive 'repack -ad' in gc --auto
I'm pretty happy with this now. Like v1, this is about not touching the giant base pack when doing background gc. This saves about 2/3 of memory, which in turn should improve performance if you're under memory pressure.
v2 changes:
- the core idea remains the same, mem_have is increased to half total memory though. I figure including the whole mmap'd pack in the memory estimation may be a bit much, which is why I make this change.
- no creating .keep files temporarily
- the config key is renamed gc.bigBasePackThreshold (named after core.bigFileThreshold)
- note that if you set gc.bigFileThreshold, then normal gc (without --auto) can trigger this mode too.
- you can also control this with --[no-]keep-base-pack
- documents and tests
- some more progress output improvements
I'm _not_ doing external rev-list in this series though. I found out that we have added more and more stuff in the internal rev-list code path over the year and simply running
git rev-list .... | git pack-objects
will break stuff (the test suite first for example). I will do it because it does help. But it will take some time.
PS. This conflicts with sb/packfiles-in-repository on 'pu' because I introduced new references to the global variable "packed_git" and prepare_packed_git(). Resolving this should be simple though:
- drop prepare_packed_git()
- replace packed_git with get_packed_git(the_repository)
Nguyễn Thái Ngọc Duy (5):
fixup! Add a test showing that 'git repack' throws away grafted-away
parents
repack: add --keep-pack option
gc --auto: exclude base pack if not enough mem to "repack -ad"
pack-objects: show some progress when counting kept objects
pack-objects: display progress in get_object_details()
Documentation/config.txt | 7 ++ Documentation/git-gc.txt | 13 +++ Documentation/git-pack-objects.txt | 4 + Documentation/git-repack.txt | 4 + builtin/gc.c | 153 +++++++++++++++++++++++++++-- builtin/pack-objects.c | 51 ++++++++-- builtin/repack.c | 23 ++++- config.mak.uname | 1 + git-compat-util.h | 4 + pack-objects.h | 2 + t/t6500-gc.sh | 29 ++++++ t/t7700-repack.sh | 21 +++- 12 files changed, 295 insertions(+), 17 deletions(-)
Signature · 1 line
2.16.2.784.gb291bd247e