Skip to content

object-file: fix packfile flush during transaction commit

Justin Tobler git 2 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v2 Latest2 patches 2 repliesIn review
  2. v12 patches 10 repliesSuperseded

Changes in v2 from the cover letter

- Updated a commit message of first patch.
- Improved test readability in second patch.

Patches in v2 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/2object-file: lift ODB reprepare out of packfile flushobject-file.c+7 -31 reply
2/2object-file: flush transaction packfile before migrating objectsobject-file.c t/t1050-large.sh+19 -20 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v2 yet.

Read the v2 thread

Cover letter [PATCH v2 0/2] object-file: fix packfile flush during transaction commit

Greetings,

This short series fixes a bug I found related to committing an ODB transaction that contains both a loose object and "large" blob when also configured to batch fsync loose objects. The issue can be reproduced with the following:

git init
git config core.fsync loose-object
git config core.fsyncMethod batch
git config core.bigFileThreshold 5
echo foo >1-foo && echo foobar >2-foobar
git add 1-foo 2-foobar

and produces the following error:

        error: unable to write file .git/objects/pack/pack-2b7c2470289822070687e8d64186093a710eaed3.pack: No such file or directory
        fatal: unable to rename temporary file to '.git/objects/pack/pack-2b7c2470289822070687e8d64186093a710eaed3.pack'

If a "large" blob packfile is written to the transaction temporary directory, it is unable to be flushed during transaction commit because the underlying transaction is migrated to the main ODB before the packfile is finalized. To avoid this, this series ensures any pending packfile in the transaction is flushed first.

Changes since V1:
- Updated a commit message of first patch.
- Improved test readability in second patch.

Thanks,
-Justin

Justin Tobler (2):
object-file: lift ODB reprepare out of packfile flush
object-file: flush transaction packfile before migrating objects

 object-file.c    | 12 ++++++++----
 t/t1050-large.sh | 17 +++++++++++++++++
 2 files changed, 25 insertions(+), 4 deletions(-)

Range-diff against v1:
1: cf14416f22 ! 1: 6f74391ae8 object-file: lift ODB reprepare out of packfile flush
@@ Commit message

When flushing a packfile via flush_packfile_transaction(), odb_reprepare() is invoked so the written packfile becomes visible in
- the current process. In a subsequent commit, repreparing the ODB is
- slightly deferred when committing a "files" ODB transaction.
+ the current process. In a subsequent commit, flushing the packfile is
+ performed earlier when committing a "files" ODB transaction, but the ODB
+ reprepare needs to remain the last step.

Lift ODB reprepare out of flush_packfile_transaction() and instead require callers to explicitly invoke odb_reprepare() if required.
2: 18a1798d95 ! 2: ad2fa8ee3f object-file: flush transaction packfile before migrating objects

    @@ t/t1050-large.sh: test_expect_success 'add a large file or two' '
     +test_expect_success 'add large file with loose object in batch fsync' '
     +	test_when_finished "rm -rf batch" &&
     +	git init batch &&
    ++	(
    ++		cd batch &&
    ++		git config core.bigFileThreshold 5 &&
    ++		echo foo >1-small &&
    ++		echo foobar >2-large &&
     +
    -+	git -C batch config core.bigFileThreshold 5 &&
    -+	echo foo >batch/1-small &&
    -+	echo foobar >batch/2-large &&
    ++		git -c core.fsync=loose-object -c core.fsyncMethod=batch \
    ++			add 1-small 2-large &&
     +
    -+	git -C batch -c core.fsync=loose-object -c core.fsyncMethod=batch \
    -+		add 1-small 2-large &&
    -+
    -+	# Neither object may be left behind in a temporary location.
    -+	git -C batch cat-file -e :1-small &&
    -+	git -C batch cat-file -e :2-large
    ++		git cat-file -e :1-small &&
    ++		git cat-file -e :2-large
    ++	)
     +'
     +
      test_expect_success 'checkout a large file' '

base-commit: 47ce80527c56f462cb97db4ca8125342204d3783

Signature · 1 line
2.55.0.424.g13c7afec21