Refactor chunk-format into an API

I was thinking about file formats recently and realized that the "chunks" that are common to the commit-graph and multi-pack-index could inform future file formats. To make that process easier, let's combine the process of writing and reading chunks into a common API that both of these existing formats use.

There is some extra benefit immediately: the writing and reading code for each gets a bit cleaner. Also, there were different checks in each that made the process more robust. Now, these share a common set of checks.

In particular, Szeder made some updates to the commit-graph writing process that forms the model for this API.

Thanks, -Stolee

Derrick Stolee (15):
commit-graph: anonymize data in chunk_write_fn
chunk-format: add API for writing table of contents
midx: rename pack_info to write_midx_context
midx: use context in write_midx_pack_names()
midx: add entries to write_midx_context
midx: add pack_perm to write_midx_context
midx: add num_large_offsets to write_midx_context
midx: convert chunk write methods to return int
midx: drop chunk progress during write
midx: use chunk-format API in write_midx_internal()
midx: use 64-bit multiplication for chunk sizes
chunk-format: create write_chunks()
chunk-format: create chunk reading API
commit-graph: restore duplicate chunk checks
chunk-format: add technical docs

 Documentation/technical/chunk-format.txt      |  54 ++
 .../technical/commit-graph-format.txt         |   3 +
 Documentation/technical/pack-format.txt       |   3 +
 Makefile                                      |   1 +
 chunk-format.c                                | 105 ++++
 chunk-format.h                                |  69 +++
 commit-graph.c                                | 298 ++++++-----
 midx.c                                        | 466 ++++++++----------
 t/t5318-commit-graph.sh                       |   2 +-
 t/t5319-multi-pack-index.sh                   |   6 +-
 10 files changed, 623 insertions(+), 384 deletions(-)
 create mode 100644 Documentation/technical/chunk-format.txt
 create mode 100644 chunk-format.c
 create mode 100644 chunk-format.h

base-commit: 72ffeb997eaf999f6938b2a7e0d9a75dcceaa311
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-804%2Fderrickstolee%2Fchunk-format%2Frefactor-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-804/derrickstolee/chunk-format/refactor-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/804

gitgitgadget