Skip to content

reftable: memory optimizations for reflog iteration

Patrick Steinhardt git 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest7 patches 12 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/7refs/reftable: reload correct stack when creating reflog iterrefs/reftable-backend.c+1 -15 replies
2/7reftable/record: convert old and new object IDs to arraysrefs/reftable-backend.c reftable/merged_test.c reftable/readwrite_test.c +4 more+61 -1510 replies
3/7reftable/record: avoid copying author inforeftable/record.c+21 -82 replies
4/7reftable/record: reuse refnames when decoding log recordsreftable/record.c reftable/reftable-record.h+2 -10 replies
5/7reftable/record: reuse message when decoding log recordsreftable/record.c reftable/reftable-record.h+4 -20 replies
6/7reftable/record: use scratch buffer when decoding recordsreftable/block.c reftable/block.h reftable/record.c +2 more+68 -523 replies
7/7refs/reftable: track last log record name via strbufrefs/reftable-backend.c+6 -50 replies

Reviews trailers per patch

None of the 7 patches has a Reviewed-by or Acked-by of their own; the series has one for all of them.

PatchJosh Steadmon
SeriesR-b
1/7
2/7
3/7
4/7
5/7
6/7
7/7

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v1 thread

Cover letter [PATCH 0/7] reftable: memory optimizations for reflog iteration

Hi,

this patch series does the same as all the preceding patch series that optimized how the reftable library iterates through refs, but for reflogs instead.

The goal of this patch series is to arrive at a constant number of allocations when iterating refs. This is achieved in mostly the same way we did it for ref iteration, namely by reusing already-allocated memory. Overall, this brings us down from 8 allocations per reflog record to essentially 0 allocations per reflog. Iterating through 1 million reflogs with git reflog list thus goes down from 8.068m allocations to only around 68.5k.

This series is built on top of "master" at b387623c12 (The third batch, 2024-03-01) with Junio's "ps/reftable-iteration-perf-part2" at 43f70eaea0 (refs/reftable: precompute prefix length, 2024-03-04) merged into it.

Patrick

Patrick Steinhardt (7):
refs/reftable: reload correct stack when creating reflog iter reftable/record: convert old and new object IDs to arrays reftable/record: avoid copying author info reftable/record: reuse refnames when decoding log records reftable/record: reuse message when decoding log records reftable/record: use scratch buffer when decoding records refs/reftable: track last log record name via strbuf

 refs/reftable-backend.c    |  52 +++++----------
 reftable/block.c           |   4 +-
 reftable/block.h           |   2 +
 reftable/merged_test.c     |  11 ++--
 reftable/readwrite_test.c  |  62 +++++++-----------
 reftable/record.c          | 129 ++++++++++++++-----------------------
 reftable/record.h          |   5 +-
 reftable/record_test.c     |  68 ++++++++++---------
 reftable/reftable-record.h |   6 +-
 reftable/stack_test.c      |  26 ++++----
 10 files changed, 154 insertions(+), 211 deletions(-)
Signature · 1 line
2.44.0