Skip to content

midx: verify MIDX checksum before reusing

Taylor Blau git 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest4 patches 8 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/4csum-file: introduce checksum_valid()csum-file.c csum-file.h pack-check.c+20 -101 reply
2/4commit-graph: rewrite to use checksum_valid()commit-graph.c+6 -81 reply
3/4midx: don't reuse corrupt MIDXs when writingmidx.c t/t5319-multi-pack-index.sh+18 -01 reply
4/4midx: report checksum mismatches during 'verify'midx.c t/t5319-multi-pack-index.sh+8 -08 replies

Reviews trailers per patch

1 of 4 patches has a Reviewed-by or Acked-by.

PatchJeff KingTaylor Blau
1/4Helped-by
2/4
3/4
4/4R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v1 thread

Cover letter [PATCH 0/4] midx: verify MIDX checksum before reusing

Here is a small series that came out of a corruption we noticed at GitHub. The corruption is described in the third patch, but the gist is this:

- After writing a MIDX, we experience a disk failure that corrupts the contents of the repository's MIDX.

- We later go to write another MIDX, notice that one is already there, and then blindly reuse its data without checking its validity, resulting in a MIDX that contains corrupt data, but whose trailing checksum matches the preceding contents.

This series checks the validity of the trailing checksum when reusing a MIDX while writing a new one, as well as a few other related fix-ups. That should be sufficient to avoid propagating corrupt data forward while writing a MIDX.

We could go further here and run 'git multi-pack-index verify' on the existing MIDX while writing a new one, but the point of this reuse is to avoid having to re-compute/verify the contents of an existing MIDX. So a more lightweight option is pursued here, but alternatives are discussed in 3/4.

Thanks in advance for your review.

Taylor Blau (4):
csum-file: introduce checksum_valid()
commit-graph: rewrite to use checksum_valid()
midx: don't reuse corrupt MIDXs when writing
midx: report checksum mismatches during 'verify'

 commit-graph.c              | 14 ++++++--------
 csum-file.c                 | 16 ++++++++++++++++
 csum-file.h                 |  3 +++
 midx.c                      | 13 +++++++++++++
 pack-check.c                | 11 +----------
 t/t5319-multi-pack-index.sh | 13 +++++++++++++
 6 files changed, 52 insertions(+), 18 deletions(-)
Signature · 1 line
2.31.1.163.ga65ce7f831