receive-pack: use ODB transactions to stage object writes
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v4 Latest11 patches 1 replyIn review
- v311 patches 9 repliesSuperseded
- v211 patches 16 repliesSuperseded
Earlier versions · 1
| Version | Posted | Patches | Replies | Status |
|---|---|---|---|---|
| v1 | 6 patches | 21 replies | Superseded |
Changes in v4 from the cover letter
- Removed ugly line break in commit message to prevent eye strain.
- odb_transaction_begin() now only sets the repository transaction on success.
- odb_transaction_env() now bubbles up error when failing to create the temporary directory.
Patches in v4 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/11 | object-file: rename files transaction prepare functionobject-file.c | +3 | 0 replies | |
| 2/11 | object-file: rename files transaction fsync functionobject-file.c | +10 | 0 replies | |
| 3/11 | object-file: embed transaction flush logic in commit functionobject-file.c | +28 | 0 replies | |
| 4/11 | object-file: drop check for inflight transactionsobject-file.c object-file.h | +1 | 0 replies | |
| 5/11 | object-file: propagate files transaction errorsobject-file.c object-file.h odb/ +1 more | +26 | 0 replies | |
| 6/11 | odb/transaction: propagate begin errorsbuiltin/ builtin/ builtin/ +5 more | +45 | 0 replies | |
| 7/11 | odb/transaction: propagate commit errorsodb/ odb/ | +11 | 0 replies | |
| 8/11 | odb/transaction: add transaction env interfaceobject-file.c odb/ odb/ | +41 | 0 replies | |
| 9/11 | odb/transaction: introduce ODB transaction flagsbuiltin/ builtin/ builtin/ +10 more | +61 | 0 replies | |
| 10/11 | builtin/receive-pack: drop redundant tmpdir envbuiltin/ | +0 | 0 replies | |
| 11/11 | builtin/receive-pack: stage incoming objects via ODB transactionsbuiltin/ | +35 | 0 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v4 yet.
Cover letter [PATCH v4 00/11] receive-pack: use ODB transactions to stage object writes
Greetings,
This patch series replaces direct usage of the tmp_objdir interfaces in git-receive-pack(1) to instead use the odb_transaction interfaces to create/manage a staging area to write objects to. The purpose of this change is to get git-receive-pack(1) one step closer to being ODB backend agnostic. For now, the object writes themselves are still "files" backend specific due to being handled by the git-index-pack(1) and git-unpack-objects(1) child processes. This will be tackled in a separate series though.
Changes since V3:
- Removed ugly line break in commit message to prevent eye strain.
- odb_transaction_begin() now only sets the repository transaction on success.
- odb_transaction_env() now bubbles up error when failing to create the temporary directory.
Changes since V2:
- Clarified commit log reasoning for embedding flush_loose_object_transaction() logic in commit function.
- Started printed some error messages on transaction errors.
- Removed include statement.
- Fixed transaction leak on odb_transaction_commit() error.
Changes since V1:
- Adapted other "file" ODB transaction helpers to be more consistent with current naming scheme.
- Removed redundant NULL transaction handling from odb_transaction_files_begin().
- odb_transaction_begin() now returns an error if there is already an inflight transaction pending instead of setting the out pointer to NULL.
- Updated odb_transaction_env() to return an error code and append environment variables to a strvec provided as an argument.
- Removed redundant setting of tmpdir environment variables for child processes after tmpdir has been migrated.
- Split changes adding ODB transaction flags into a separate commit.
- Consistently wire the ODB transaction throughout git-receive-pack code instead of reading it from the_repository.
- Updated user facing error message.
- Updated some comments to better document functions/flags.
- Clarified some commit messages.
- Fixed typos.
Thanks,
-Justin
Justin Tobler (11):
object-file: rename files transaction prepare function
object-file: rename files transaction fsync function
object-file: embed transaction flush logic in commit function
object-file: drop check for inflight transactions
object-file: propagate files transaction errors odb/transaction: propagate begin errors odb/transaction: propagate commit errors odb/transaction: add transaction env interface odb/transaction: introduce ODB transaction flags builtin/receive-pack: drop redundant tmpdir env builtin/receive-pack: stage incoming objects via ODB transactions
builtin/add.c | 2 +- builtin/receive-pack.c | 69 ++++++++--------- builtin/unpack-objects.c | 2 +- builtin/update-index.c | 2 +- cache-tree.c | 7 +- object-file.c | 161 +++++++++++++++++++++++++-------------- object-file.h | 8 +- odb/source-files.c | 9 +-- odb/source-inmemory.c | 3 +- odb/source-loose.c | 3 +- odb/source.h | 9 ++- odb/transaction.c | 33 ++++++-- odb/transaction.h | 59 +++++++++++--- read-cache.c | 7 +- 14 files changed, 244 insertions(+), 130 deletions(-)
Range-diff against v3:
1: 9c14b219ad = 1: 9c14b219ad object-file: rename files transaction prepare function 2: 5703a9e93b = 2: 5703a9e93b object-file: rename files transaction fsync function 3: 76204847f2 ! 3: 70267741b0 object-file: embed transaction flush logic in commit function
@@ Commit message
subsequent commit, the transaction temporary directory is used to stage packfiles and not just loose objects anymore.
- Lift the helper function logic directly into
- odb_transaction_files_commit() to more accurately signal to readers
- the operation being performed.
+ Lift the helper function logic into odb_transaction_files_commit() to
+ more accurately signal to readers the operation being performed.
4: c97eb7763f = 4: 34cd3822c5 object-file: drop check for inflight transactions
5: 1f3a1f7714 = 5: 240aa3475f object-file: propagate files transaction errors
6: 09d13272d5 ! 6: 0d91310fac odb/transaction: propagate begin errors
@@ odb/transaction.c
- odb_source_begin_transaction(odb->sources, &odb->transaction);
+ ret = odb_source_begin_transaction(odb->sources, out);
-+ odb->transaction = *out;
++ if (!ret)
++ odb->transaction = *out;
- return odb->transaction;
+ return ret;
7: 12833d6773 = 7: 5e4680ed75 odb/transaction: propagate commit errors
8: f2586f2f34 ! 8: babcf6b156 odb/transaction: add transaction env interface
@@ object-file.c: static int odb_transaction_files_commit(struct odb_transaction *b
+{
+ struct odb_transaction_files *transaction =
+ container_of(base, struct odb_transaction_files, base);
++ int ret;
+
-+ odb_transaction_files_prepare(&transaction->base);
-+ strvec_pushv(env, tmp_objdir_env(transaction->objdir));
++ ret = odb_transaction_files_prepare(&transaction->base);
++ if (!ret)
++ strvec_pushv(env, tmp_objdir_env(transaction->objdir));
+
-+ return 0;
++ return ret;
+}
+
int odb_transaction_files_begin(struct odb_source *source, 9: 9d082b5e47 ! 9: 96f2a21eec odb/transaction: introduce ODB transaction flags
@@ odb/transaction.c
- ret = odb_source_begin_transaction(odb->sources, out);
+ ret = odb_source_begin_transaction(odb->sources, out, flags);
- odb->transaction = *out;
+ if (!ret)
+ odb->transaction = *out;
- return ret;
## odb/transaction.h ##
@@
10: e11d8a6676 = 10: 56718f1190 builtin/receive-pack: drop redundant tmpdir env 11: fee57c2817 = 11: 5197a19fbf builtin/receive-pack: stage incoming objects via ODB transactions
base-commit: ab776a62a78576513ee121424adb19597fbb7613
Signature · 1 line
2.55.0.122.gf85a7e6620