Skip to content

commit-graph: handle file descriptor exhaustion

Taylor Blau git 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest4 patches 9 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/4commit-graph.c: don't use discarded graph_name in errorcommit-graph.c+1 -10 replies
2/4t/test-lib.sh: make ULIMIT_FILE_DESCRIPTORS available to testst/t1400-update-ref.sh t/test-lib.sh+9 -90 replies
3/4commit-graph.c: gracefully handle file descriptor exhaustioncommit-graph.c t/t5324-split-commit-graph.sh+15 -22 replies
4/4commit-graph: close descriptors after mmapcommit-graph.c commit-graph.h fuzz-commit-graph.c+8 -156 replies

Reviews trailers per patch

1 of 4 patches has a Reviewed-by or Acked-by.

PatchJeff KingDerrick StoleeTaylor Blau
1/4
2/4
3/4Helped-by Helped-by
4/4R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 3 reviewers.

Read the v1 thread

Cover letter [PATCH 0/4] commit-graph: handle file descriptor exhaustion

Hi,

This is another series to handle an issue we started seeing at GitHub on repositories that have too many commit-graph layers.

This issue evaded me for several days, but the gist is that having too many descriptors open caused us to fail 'git_mkstemp_mode', which in turn left some values in 'ctx->commit_graph_filenames_after' uninitialized. This in turn caused a general protection fault when trying to free these uninitialized entries. Frustratingly, it causes stray 'commit-graph-chain.lock' files to stick around even after the process has gone away, causing all subsequent commit-graph write operations to fail.

The first two patches are semi-related cleanups and other preparation. The third patch fixes the main issue described here. Peff contributed the fourth patch which teaches the commit-graph machinery to be more willing to close file descriptors.

These patches are very much owed to the help that Stolee and Peff provided while writing them and for their willingness to let me bounce some ideas off of them.

Thanks in advance for your review.

Jeff King (1):
commit-graph: close descriptors after mmap

Taylor Blau (3):
commit-graph.c: don't use discarded graph_name in error t/test-lib.sh: make ULIMIT_FILE_DESCRIPTORS available to tests commit-graph.c: gracefully handle file descriptor exhaustion

 commit-graph.c                | 21 ++++++++-------------
 commit-graph.h                |  3 +--
 fuzz-commit-graph.c           |  5 ++---
 t/t1400-update-ref.sh         |  9 ---------
 t/t5324-split-commit-graph.sh | 13 +++++++++++++
 t/test-lib.sh                 |  9 +++++++++
 6 files changed, 33 insertions(+), 27 deletions(-)

--
2.26.0.113.ge9739cdccc