un-breaking pack-objects with bitmaps
Get series with b4
b4 am [email protected]Versions newest first
- v1 Latest4 patches 13 repliesIn review
Patches in v1 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/4 | bitmap_has_sha1_in_uninteresting(): drop BUG checkpack-bitmap.c pack-bitmap.h | +1 | 0 replies | |
| 2/4 | t5310: test delta reuse with bitmapst/ | +93 | 7 replies | |
| 3/4 | traverse_bitmap_commit_list(): don't free resultpack-bitmap.c | +0 | 0 replies | |
| 4/4 | pack-bitmap: drop "loaded" flagpack-bitmap.c | +3 | 0 replies |
Reviews trailers per patch
None of the 4 patches has a Reviewed-by or Acked-by of their own; the series has one for all of them.
| Patch | Stefan Beller |
|---|---|
| Series | R-b |
| 1/4 | |
| 2/4 | |
| 3/4 | |
| 4/4 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [PATCH 0/4] un-breaking pack-objects with bitmaps
Quoted 18 lines · Jeff King
Quoted 15 lines · Ævar Arnfjörð Bjarmason
Review
Quoted 9 lines · Jeff King
(quoted diff) +9
-0+int bitmap_has_sha1_in_uninteresting(struct bitmap_index *bitmap_git, + const unsigned char *sha1) +{ + int pos; + + if (!bitmap_git) + return 0; /* no bitmap loaded */ + if (!bitmap_git->result) + BUG("failed to perform bitmap walk before querying");Some part of what calls this completely breaks pushing from the "next" branch when you have local bitmaps (we *really* should have some tests for this...).
Quoted 1 line
Yikes, thanks for reporting. I agree we need better tests here.
OK, here is the fix. Since the problem is in 'next', this is done as a patch on top of jk/pack-delta-reuse-with-bitmap. But since we're set to rewind 'next' post-release anyway, we could squash it directly into 30cdc33fba from the original series. That would help later bisections from running into it, which may be worth it as it's a pretty severe breakage. Or maybe not:
1. The test suite doesn't actually fail, because it's toy repos are too small.
2. It only triggers in the real-world if you have bitmaps turned on, which are not the default.
So it may not be that likely in practice to bother a hypothetical future bisecting developer.
Quoted 3 lines
[1] Actually, there is also
prepare_bitmap_git(), but it is not really for general use by callers. It should be made static, or better yet, I suspect it can be folded into its callers.
This actually turned out not to work. There's a caller over in pack-bitmap-write.c, and it makes things worse to try to expand the logic there. So it technically _is_ possible to have a bitmap_index without a "have" field, but it also doesn't make sense to ask about "uninteresting" objects there. You haven't done (and cannot do) a traversal on such an object.
Which I think goes back to Stefan's original question: is this just a crappy API? And the answer is "yes, to some degree". There are really two uses of bitmaps:
- you want to do a traverse_commit_list() walk, but faster
- you want to selectively query the on-disk bitmaps (e.g., you are walking for --contains and want to ask "do we have a bitmap for this object?"
Those currently use the same struct bitmap_index, but with two different constructors (prepare_bitmap_git and prepare_bitmap_walk). It probably ought to be two different ones (with the "walk" variant using the "query" variant under the hood). I've punted on that full conversion for now, but did clean up a few confusing bits.
[1/4]: bitmap_has_sha1_in_uninteresting(): drop BUG check
The actual fix. This should get merged to next ASAP (or the original topic just reverted).
[2/4]: t5310: test delta reuse with bitmaps
I did this separately to give us flexibility to squash or merge quickly. But it does find Ævar's bug on a git without patch 1.
[3/4]: traverse_bitmap_commit_list(): don't free result
The original assert should have simply been useless, but it was the surprising behavior of this function that turned it into a bug.
[4/4]: pack-bitmap: drop "loaded" flag
And this is just an annoyance I ran into, which is a fallout from our conversion to using an allocated bitmap_index struct.
pack-bitmap.c | 14 ++----- pack-bitmap.h | 2 +- t/t5310-pack-bitmaps.sh | 93 +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 97 insertions(+), 12 deletions(-)
-Peff