RFC repack: combine '--geometric' and '--cruft'

First, a short note. This series is an RFC because I have not had the chance to review and test it as thoroughly as I normally would, and because we are deep in the -rc phase.

I wanted to get this series off my backlog since I have decided to leave GitHub at the end of the month for a new role. I will still be contributing to Git in my new role (which I will start in the early part of July), but wanted to get this off my backlog nonetheless.

This series teaches git repack how to combine --geometric and --cruft.

Today these two modes are mutually exclusive, since --cruft implies -a, and -a is fundamentally incompatible with --geometric. As a result, repositories have to choose between keeping reachable objects in a geometric progression of packs and collecting unreachable objects into cruft packs.

The goal of this series is to to be able to do both simultaneously. When both options are given, 'git repack' rolls up the selected non-cruft packs as usual while collecting unreachable objects separately into a cruft pack. That means a command like

    $ git repack -d --geometric=2 --cruft --combine-cruft-below-size=1G

will keep reachable non-cruft packs in a geometric progression, while combining sufficiently-small cruft packs (along with newly-discovered unreachable objects) into a fresh cruft pack.

The series is structured roughly as follows:

* The first two patches prepare the cruft pack machinery for the later changes by making non-kept pack exclusion unconditional and extracting a helper for looking up packs in an existing_packs list.

* The next four patches route geometric pack deletion through the common existing_packs machinery. They mark packs above the geometric split as retained, teach incremental-MIDX retention not to keep packs that are being rolled up, switch geometric repacks over to the common deletion path, and then remove the old geometry-specific deletion helper.

* The next three patches teach pack-objects the new pieces needed by this mode. The main addition is --stdin-packs=follow-reachable, which walks from reference tips and includes only reachable objects from the selected packs, while still allowing traversal through excluded-open packs and stopping at excluded-closed ones. The following patch teaches that mode to use --refs-snapshot, so that pack-objects and the MIDX bitmap writer can agree on the same set of tips.

* The final patch wires everything together in git repack, including teaching the cruft writer how to interpret the geometric split when choosing which packs to include or exclude.

Thanks in advance for your review!

Taylor Blau (10):
repack: unconditionally exclude non-kept packs
repack: extract locate_existing_pack() helper
repack: mark geometric progression of packs as retained
repack: teach MIDX retention about geometric rollups
repack: delete geometric packs via existing_packs
repack-geometry: drop unused redundant-pack removal
pack-objects: extract stdin_packs_add_all_pack_entries()
pack-objects: introduce '--stdin-packs=follow-reachable'
pack-objects: support '--refs-snapshot' with 'follow-reachable'
repack: support combining '--geometric' with '--cruft'

 Documentation/git-pack-objects.adoc |  25 +++
 Documentation/git-repack.adoc       |  11 ++
 builtin/pack-objects.c              | 276 ++++++++++++++++++++++++----
 builtin/repack.c                    |  38 ++--
 repack-cruft.c                      |  29 ++-
 repack-geometry.c                   |  44 -----
 repack.c                            | 101 +++++++++-
 repack.h                            |  15 +-
 t/t5331-pack-objects-stdin.sh       | 201 ++++++++++++++++++++
 t/t7704-repack-cruft.sh             | 251 +++++++++++++++++++++++++
 10 files changed, 878 insertions(+), 113 deletions(-)

base-commit: ab776a62a78576513ee121424adb19597fbb7613

2.55.0.rc2.10.g29e31820dce