propose config-based hooks
Subsystems (2)
Re: [PATCH v4 00/36] Run hooks via "git run hook" & hook library
In reply to Emily ShafferOn Thu, Aug 19 2021, Emily Shaffer wrote:
I'm submitting a re-roll of this soon, just some notes.
Quoted 88 lines · Ævar Arnfjörð Bjarmason
Quoted 7 lines
This is a v4 re-roll of the "Base for "config-based-hooks" topic.
* Almost all the callers were just "one-shot" callers, I introduced a newrun_hooks_oneshot()function for those, which gets rid of the verbosity around memory management, see e.g. the "builtin/gc.c" in the range-diff below. Thatrun_hooks_oneshot()can also take a NULL set of options.I discussed this elswhere, but just to make doubly sure that comment doesn't get lost, here's a link to my mail:
https://lore.kernel.org/git/YR2jLdYQA5CVzX5h%40google.comQuoted 8 lines
Range-diff against v3:
1: 27c94247f87 = 1: 81fe1ed90d5 Makefile: mark "check" target as .PHONY 2: 6e164edb0b0 ! 2: 0f749530777 Makefile: stop hardcoding {command,config}-list.h 3: ddae86802e2 ! 3: 644b31fe281 Makefile: remove an out-of-date comment 4: 58c37e4f06e = 4: 89c4d44b0c3 hook.[ch]: move find_hook() to this new library 5: 0cf7e078ef4 = 5: 3514e0c0251 hook.c: add a hook_exists() wrapper and use it in bugreport.c -: ----------- > 6: d5ef40f77dc hook.c users: use "hook_exists()" insted of "find_hook()" 6: f343fc7ae66 ! 7: 4cfd72722c1 hook-list.h: add a generated list of hooks, like config-list.hSince these 6 were broken out and then brought back into this series, I'll review them individually downthread.
Quoted 8 lines
7:
cf4b06bfdf8! 8:7cb4a4cb69ehook: add 'run' subcommand@@ builtin/hook.c (new) +#include "strbuf.h" +#include "strvec.h" + ++#define BUILTIN_HOOK_RUN_USAGE \ ++ N_("git hook run <hook-name> [-- <hook-args>]") ++Nice, now we avoid string duplication here...
Quoted 2 lines
+static const char * const builtin_hook_usage[] = { -+ N_("git hook <command> [...]"),...and there is no point including this vague thing that has a more specific description right after. Ok.
Quoted 4 lines
-+ N_("git hook run <hook-name> [-- <hook-args>]"), ++ BUILTIN_HOOK_RUN_USAGE, + NULL +};[...]
Quoted 19 lines
@@ builtin/hook.c (new) +{ + int i; + struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT; -+ int rc = 0; + const char *hook_name; + const char *hook_path; -+ + struct option run_options[] = { + OPT_END(), + }; ++ int ret; + + argc = parse_options(argc, argv, prefix, run_options, + builtin_hook_run_usage, -+ PARSE_OPT_KEEP_UNKNOWN | PARSE_OPT_KEEP_DASHDASH); -+ -+ if (argc > 1) { -+ if (strcmp(argv[1], "--") &&Quoted 8 lines
-+ strcmp(argv[1], "--end-of-options")) -+ /* Having a -- for "run" is mandatory */ -+ usage_with_options(builtin_hook_usage, run_options); -+ /* Add our arguments, start after -- */ -+ for (i = 2 ; i < argc; i++) -+ strvec_push(&opt.args, argv[i]); -+ } ++ PARSE_OPT_KEEP_DASHDASH);Nice - this code is being broken up and moved later.
Quoted 10 lines
+ -+ /* Need to take into account core.hooksPath */ -+ git_config(git_default_config, NULL); ++ if (!argc) ++ goto usage; + + /* -+ * We are not using run_hooks() because we'd like to detect -+ * missing hooks. Let's find it ourselves and call -+ * run_found_hooks() instead.
Quoted 3 lines
This comment disappears entirely. I am not too terribly upset about it, since this behavior changes in my series anyway, but it seems like it might have been an accident?
I removed it intentionally, I thought it was clear enough from the control flow, but added it back in. Thanks.
Quoted 266 lines
Quoted 43 lines
++ * Having a -- for "run" when providing <hook-args> is ++ * mandatory. + */ ++ if (argc > 1 && strcmp(argv[1], "--") && ++ strcmp(argv[1], "--end-of-options")) ++ goto usage; ++ ++ /* Add our arguments, start after -- */ ++ for (i = 2 ; i < argc; i++) ++ strvec_push(&opt.args, argv[i]); ++ ++ /* Need to take into account core.hooksPath */ ++ git_config(git_default_config, NULL); ++ + hook_name = argv[0]; + hook_path = find_hook(hook_name); + if (!hook_path) { + error("cannot find a hook named %s", hook_name); + return 1; + } -+ rc = run_found_hooks(hook_name, hook_path, &opt); + ++ ret = run_hooks(hook_name, hook_path, &opt); + run_hooks_opt_clear(&opt); -+ -+ return rc; ++ return ret; ++usage: ++ usage_with_options(builtin_hook_run_usage, run_options); +} + +int cmd_hook(int argc, const char **argv, const char *prefix) @@ builtin/hook.c (new) + struct option builtin_hook_options[] = { + OPT_END(), + }; ++ + argc = parse_options(argc, argv, NULL, builtin_hook_options, + builtin_hook_usage, PARSE_OPT_STOP_AT_NON_OPTION); + if (!argc) -+ usage_with_options(builtin_hook_usage, builtin_hook_options); ++ goto usage; +Quoted 8 lines
+ if (!strcmp(argv[0], "run")) + return run(argc, argv, prefix); -+ else -+ usage_with_options(builtin_hook_usage, builtin_hook_options); ++ ++usage: ++ usage_with_options(builtin_hook_usage, builtin_hook_options); +}The goto pattern seems readable enough elsewhere, though.
Quoted 8 lines
## command-list.txt ## @@ hook.c: int hook_exists(const char *name) + struct hook_cb_data *hook_cb = pp_cb; + struct hook *run_me = hook_cb->run_me; + ++ if (!run_me) ++ return 0; ++Now we protect ourselves from the repeated calls to
pick_next_hook(). Good.Quoted 6 lines
+ cp->no_stdin = 1; + cp->env = hook_cb->options->env.v; + cp->stdout_to_stderr = 1; @@ hook.c: int hook_exists(const char *name) + /* Provide context for errors if necessary */ + *pp_task_cb = run_me;Quoted 8 lines
+ ++ /* ++ * This pick_next_hook() will be called again, we're only ++ * running one hook, so indicate that no more work will be ++ * done. ++ */ ++ hook_cb->run_me = NULL; ++And clearly explaining the caller's behavior here. Sure.
Quoted 9 lines
+ return 1; +} + @@ hook.c: int hook_exists(const char *name) + + hook_cb->rc |= result; + -+ return 1; ++ return 0;And finally, report "everything is fine" on task finished. Thanks.
Quoted 6 lines
+}
+
-+int run_found_hooks(const char*hook_name, const char*hook_path,
-+ structrun_hooks_opt*options)
++int run_hooks(const char*hook_name, const char*hook_path,
++ structrun_hooks_opt*options)Ok. With the rename, we only have "verbose way to call it" and "speedy way to call it". No more "
run_found_hooks()". Thanks, I think I like this better - I found "run_found_hooks()" to be a little ambiguous.Quoted 16 lines
@@ hook.c: int hook_exists(const char *name) + hook_name); + + return cb_data.rc; -+} -+ -+int run_hooks(const char *hook_name, struct run_hooks_opt *options) -+{ -+ const char *hook_path; -+ int ret; -+ if (!options) -+ BUG("a struct run_hooks_opt must be provided to run_hooks"); -+ -+ hook_path = find_hook(hook_name); -+ -+ /*Quoted 8 lines
-+ * If you need to act on a missing hook, use run_found_hooks() -+ * instead -+ */ -+ if (!hook_path) -+ return 0; -+ -+ ret = run_found_hooks(hook_name, hook_path, options); -+ return ret;Ok - so we wait for another patch to introduce
run_hooks_oneshot. Fine.Quoted 7 lines
+} ## hook.h ## @@ hook.h: const char *find_hook(const char *name); + + /* Args to be passed to each hook */ + struct strvec args;Quoted 8 lines
-+ -+ /* -+ * Number of threads to parallelize across, currently a stub, -+ * we use the parallel API for future-proofing, but we always -+ * have one hook of a given name, so this is always an -+ * implicit 1 for now. -+ */ -+ int jobs;Thanks. I think this makes more sense than the weird "futureproofing" state it was in before.
Quoted 4 lines
## t/t1800-hook.sh (new) ## @@ t/t1800-hook.sh (new) + +test_expect_success 'git hook usage' 'Quoted 8 lines
+ test_expect_code 129 git hook && -+ test_expect_code 129 git hook -h && -+ test_expect_code 129 git hook run -h ++ test_expect_code 129 git hook run && ++ test_expect_code 129 git hook run -h && ++ test_expect_code 129 git hook run --unknown 2>err && ++ grep "unknown option" err +'Nice. I was able to use this test myself when I added 'git hook list', too.
The changes to this patch look good to me.
Quoted 5 lines
8:
7209f73f281! 9:2b8500aa675gc: use hook library for pre-auto-gc hook
@@ Metadata
## Commit message ##
gc: use hook library for pre-auto-gc hookQuoted 10 lines
- Using the
hook.hlibrary instead of therun-command.hlibrary to run
- pre-auto-gc means that those hooks can be set up in config files, as
- well as in the hookdir. pre-auto-gc is called only frombuiltin/gc.c.
+ Move the pre-auto-gc hook away fromrun-command.hto and over to the
+ newhook.hlibrary.
+
+ To do this introduce a simplerun_hooks_oneshot()wrapper, we'll be
+ using it extensively for these simple cases of wanting to run a single
+ hook under a given name, and having it free the memory we allocate for
+ us.Cool, so we have an excuse for introducing the oneshot wrapper right away. Thanks.
Quoted 14 lines
-@@ builtin/gc.c: static void add_repack_incremental_option(void) - - static int need_to_gc(void) - { -+ struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT; -+ - /* - * Setting gc.auto to 0 or negative can disable the - * automatic gc. @@ builtin/gc.c: static int need_to_gc(void) else return 0; - if (run_hook_le(NULL, "pre-auto-gc", NULL))Quoted 8 lines
-+ if (run_hooks("pre-auto-gc", &hook_opt)) { -+ run_hooks_opt_clear(&hook_opt); ++ if (run_hooks_oneshot("pre-auto-gc", NULL)) return 0; -+ } -+ run_hooks_opt_clear(&hook_opt); return 1; }Nice - the callsite looks much tidier now.
Quoted 7 lines
+ + ## hook.c ## +@@ hook.c: int run_hooks(const char *hook_name, const char *hook_path, + return cb_data.rc; + } ++Quoted 8 lines
++int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options) ++{ ++ const char *hook_path; ++ int ret; ++ struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT; ++ ++ if (!options) ++ options = &hook_opt_scratch;Ah, it still takes an options (optionally, ha ha) because
run_hooks()would require the caller to provide the hook list/hook path. Ok. Seems fine, and I like that the '_oneshot' naming change makes it less surprising thatrun_hooks_opt_clear()will get called for you.Quoted 23 lines
++ ++ hook_path = find_hook(hook_name); ++ if (!hook_path) { ++ ret = 0; ++ goto cleanup; ++ } ++ ++ ret = run_hooks(hook_name, hook_path, options); ++cleanup: ++ run_hooks_opt_clear(options); ++ return ret; ++} + + ## hook.h ## +@@ hook.h: void run_hooks_opt_clear(struct run_hooks_opt *o); + /** + * Takes an already resolved hook found via find_hook() and runs + * it. Does not call run_hooks_opt_clear() for you. ++ * ++ * See run_hooks_oneshot() for the simpler one-shot API. + */ + int run_hooks(const char *hookname, const char *hook_path, + struct run_hooks_opt *options);Quoted 8 lines
++ ++/** ++ * Calls find_hook() on your "hook_name" and runs the hooks (if any) ++ * with run_hooks(). ++ * ++ * If "options" is provided calls run_hooks_opt_clear() on it for ++ * you. If "options" is NULL a scratch one will be provided for you ++ * before calling run_hooks()."A scratch one will be provided for you" doesn't sound quite right - it's not exposed to the caller at all, but the comment sounds like you're handing the caller this scratch struct. Maybe it's better to say "the default options from
RUN_HOOKS_OPT_INITwill be used"?
Quoted 4 lines
As a bonus, if you directly reference the initter macro, then it will automatically become clear to folks reading the documentation what the expected parallelism is on their hook, as the parallelism (
_SERIESor_PARALLEL) is included on the name of the macro in my change later.
*Nod*, did that.
Quoted 66 lines
Quoted 13 lines
++ */ ++int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options); ++ + #endif9:
e9a1e7cf61e! 10:3ee55d2c10frebase: teach pre-rebase to usehook.h
@@ Metadata
Author: Emily Shaffer <[email protected]>## Commit message ##
- rebase: teach pre-rebase to usehook.h
+ rebase: convert pre-rebase to usehook.hQuoted 9 lines
Move the pre-rebase hook away from
run-command.hto and over to the newhook.hlibrary.
@@builtin/rebase.c: int cmd_rebase(int argc, const char **argv, const char *prefix
if (!ok_to_skip_pre_rebase &&
- run_hook_le(NULL, "pre-rebase", options.upstream_arg,
- argc ? argv[0] : NULL, NULL))
-+run_hooks("pre-rebase", &hook_opt)) {
-+run_hooks_opt_clear(&hook_opt);
++run_hooks_oneshot("pre-rebase", &hook_opt))Ok, it just uses the oneshot call instead. Looks good.
Quoted 3 lines
10: 1d087269303 ! 11: 050f20d14f0 am: convert applypatch hooks to use config 11: 32eec5dc2f0 ! 12: ac875d284da hooks: convert 'post-checkout' hook to hook library 12: e9fa3f67593 ! 13: 69763bc2255 merge: use config-based hooks for post-merge hookThe above all just use the oneshot instead, and have some minor tweaks to commit messages to lose references to the config. Looks good.
Quoted 11 lines
13:
12347d901bb! 14:2ca1ca1b8e4git hook run: add an --ignore-missing flag@@ Documentation/git-hook.txt: optional `--` (or `--end-of-options`, see linkgit:gi linkgit:githooks[5] ## builtin/hook.c ## +@@ + #include "strvec.h" + + #define BUILTIN_HOOK_RUN_USAGE \ +- N_("git hook run <hook-name> [-- <hook-args>]") ++ N_("git hook run [--ignore-missing] <hook-name> [-- <hook-args>]")Ok. We update the usage in the macro here instead. Nice.
Quoted 12 lines
@@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix) - /* - * We are not using run_hooks() because we'd like to detect - * missing hooks. Let's find it ourselves and call -- * run_found_hooks() instead. -+ * run_found_hooks() instead... - */ + git_config(git_default_config, NULL); + hook_name = argv[0]; ++ if (ignore_missing) ++ return run_hooks_oneshot(hook_name, &opt);Quoted 8 lines
hook_path = find_hook(hook_name); if (!hook_path) { -+ /* ... act like run_hooks() under --ignore-missing */ -+ if (ignore_missing) -+ return 0; error("cannot find a hook named %s", hook_name); - return 1; - }
Quoted 6 lines
Hm. I actually think this was clearer before. I *think* that is because you and I have different opinions on the clarity of multiple returns :)
But I think it is a little confusing to say "ok, we will call an entirely different entry point in case of this one thing which we will have to check anyways".
Tried to reword this per the above..
Quoted 12 lines
Quoted 11 lines
14:
71d209b4077! 15:5b66b04bec7send-email: use 'git hook run' for 'sendemail-validate'@@ git-send-email.perl: sub validate_patch { if ($repo) { + my $hook_name = 'sendemail-validate'; my $hooks_path = $repo->command_oneline('rev-parse', '--git-path', 'hooks'); -- my $validate_hook = catfile($hooks_path, + require File::Spec; +- my $validate_hook = File::Spec->catfile($hooks_path, - 'sendemail-validate'); -+ my $validate_hook = catfile($hooks_path, $hook_name); ++ my $validate_hook = File::Spec->catfile($hooks_path, $hook_name);
Quoted 32 lines
Ok, we are adapting to some other change around
File::Specthat happened to git-send-email.perl elsewhere. The change to this patch looks good to me.Quoted 28 lines
my $hook_error; if (-x $validate_hook) { - my $target = abs_path($fn); + require Cwd; @@ git-send-email.perl: sub validate_patch { chdir($repo->wc_path() or $repo->repo_path()) or die("chdir: $!"); 15: 246a82b55b2 = 16: 14a37a43db2 git-p4: use 'git hook' to run hooks 16: e3f8482d803 ! 17: ad5d0e0e7de commit: use hook.h to execute hooks@@ Metadata
Author: Emily Shaffer <[email protected]>## Commit message ##
- commit: usehook.hto execute hooks
+ commit: convert {pre-commit,prepare-commit-msg} hook tohook.h- Teach
run_commit_hook()to callhook.hinstead ofrun-command.h. This
- covers 'pre-commit', 'commit-msg', and
- 'prepare-commit-msg'.
-
- Additionally, ask the hook library - not run-command - whether any
- hooks will be run, as it's possible hooks may exist in the config but
- not the hookdir.
-
- Because all but 'post-commit' hooks are expected to make some state
- change, force all but 'post-commit' hook to run in series. 'post-commit'
- "is meant primarily for notification, and cannot affect the outcome of
-git commit," so it is fine to run in parallel.
I tried to address all of the above.
Quoted 3 lines
I am a little bummed that the patch reordering effectively removes the justification for parallel/serial execution for each hook from 'git blame'. Oh well.
But it'll be there if you add it to your commit to migrate them to the parallel macros, and that'll be easier to find with "blame" than having it tangled up in the previous "do all the hook things" changes.
So a note to reword/rebase that into your series...
Quoted 26 lines
Quoted 7 lines
+ Move these hooks hook away from
run-command.hto and over to the new
+hook.hlibrary.Signed-off-by Emily Shaffer <[email protected]>Signed-off-by Ævar Arnfjörð Bjarmason <[email protected]>##
commit.c##Otherwise the main change here is dealing with other improvements earlier in the series, like '
hook_exists()' and 'run_hooks_oneshot()'. Looks fine.Quoted 8 lines
17:
6ed61071c5e! 18:3d3a33e2674read-cache: convert post-index-change hook to use config
@@ Metadata
Author: Emily Shaffer <[email protected]>## Commit message ##
- read-cache: convert post-index-change hook to use config
+ read-cache: convert post-index-change to usehook.hQuoted 5 lines
- By using
hook.hinstead ofrun-command.hto run, post-index-change hooks
- can now be specified in the config in addition to the hookdir.
- post-index-change is not run anywhere besides inread-cache.c.
+ Move the post-index-change hook away fromrun-command.hto and over to
+ the newhook.hlibrary.
Quoted 3 lines
I do not think it is necessary to drop the mention about 'post-index-change' being run anywhere else, but that's just a nit, not worth fixing :)
I removed this in rewording/removing now-false mentions of "this does config".
I didn't add the post-index-change mentiond back in, I figured to do it consistently I'd need to start with pre-auto-gc saying it's only handled in builtin/gc.c & all the way down the line...
Quoted 21 lines
Quoted 11 lines
This removes the last direct user of run_hook_ve(), so we can make the function static now. It'll be removed entirely soon. @@ read-cache.c: static int do_write_locked_index(struct index_state *istate, struc + istate->updated_workdir ? "1" : "0", + istate->updated_skipworktree ? "1" : "0", + NULL); -+ run_hooks("post-index-change", &hook_opt); -+ run_hooks_opt_clear(&hook_opt); ++ run_hooks_oneshot("post-index-change", &hook_opt); +Mechanical change to using
run_hooks_oneshot(). OK.Quoted 7 lines
istate->updated_workdir = 0; istate->updated_skipworktree = 0;18:
e4ef3f4548a! 19:893f8666301receive-pack: convert push-to-checkout hook tohook.h
@@ Metadata
## Commit message ##
receive-pack: convert push-to-checkout hook tohook.h
Quoted 8 lines
Quoted 5 lines
- By using hook.h instead of run-command.h to invoke push-to-checkout, - hooks can now be specified in the config as well as in the hookdir. - push-to-checkout is not called anywhere but in builtin/receive-pack.c. + Move the push-to-checkout hook away from run-command.h to and over to + the new hook.h library.You made a comment in the commit before about the final instance of
run_hook_ve; do you want to make a similar one here aboutrun_hook_le?
Willd.o
Quoted 295 lines
The range-diff here is mechanical so looks fine to me.
Quoted 9 lines
19: e3dda367ec9 = 20: 070433deba5 run-command: remove old run_hook_{le,ve}() hook API 20: 477d75bf579 = 21: 1028e0c1667 run-command: allow stdin for run_processes_parallel 21: b7c0ee9719a ! 22: 639e59e9ed0 hook: support passing stdin to hooks@@
Documentation/git-hook.txt: what those are.##
builtin/hook.c##
@@
+ #include "strvec.h"Quoted 16 lines
- static const char * const builtin_hook_usage[] = {
-N_("git hook <command> [...]"),
--N_("git hook run <hook-name> [-- <hook-args>]"),
-+N_("git hook run [<args>] <hook-name> [-- <hook-args>]"),
- NULL
- };
-
- static const char * const builtin_hook_run_usage[] = {
-N_("git hook run <hook-name> [-- <hook-args>]"),
-+N_("git hook run [--to-stdin=<path>] <hook-name> [-- <hook-args>]"),
- NULL
- };
+ #defineBUILTIN_HOOK_RUN_USAGE\
+-N_("git hook run [--ignore-missing] <hook-name> [-- <hook-args>]")
++N_("git hook run [--ignore-missing] [--to-stdin=<path>] <hook-name> [-- <hook-args>]")Just coping with the macro instead of the duplicated strings. Ok.
The range-diff looks fine to me.
Quoted 26 lines
22:
4035069a98c! 23:7d1925cca48am: convert 'post-rewrite' hook tohook.h@@ builtin/am.c: static int run_applypatch_msg_hook(struct am_state *state) { - struct child_process cp = CHILD_PROCESS_INIT; - const char *hook = find_hook("post-rewrite"); -+ struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT; - int ret; - +- int ret; +- - if (!hook) - return 0; - @@ builtin/am.c: static int run_applypatch_msg_hook(struct am_state *state) - cp.in = xopen(am_path(state, "rewritten"), O_RDONLY); - cp.stdout_to_stderr = 1; - cp.trace2_hook_name = "post-rewrite"; -+ strvec_push(&opt.args, "rebase"); -+ opt.path_to_stdin = am_path(state, "rewritten"); ++ struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT; - ret = run_command(&cp); -+ ret = run_hooks("post-rewrite", &opt); ++ strvec_push(&opt.args, "rebase"); ++ opt.path_to_stdin = am_path(state, "rewritten");Quoted 8 lines
- close(cp.in); -+ run_hooks_opt_clear(&opt); - return ret; +- return ret; ++ return run_hooks_oneshot("post-rewrite", &opt); } + /**Man, the range-diff did a terrible job of summarizing this :)
The patch itself looks great, though, with the switch to using
run_hooks_oneshot. Thanks.Quoted 14 lines
24:
da46c859c1c! 25:05d1085f7ebhook: provide stdin bystring_listor callback
@@hook.c: static int pick_next_hook(structchild_process*cp,
} else {
cp->no_stdin= 1;
}
-@@hook.c: int run_found_hooks(const char*hook_name, const char*hook_path,
- run_processes_parallel_tr2(options->jobs,
+@@hook.c: int run_hooks(const char*hook_name, const char*hook_path,
+ run_processes_parallel_tr2(jobs,
pick_next_hook,notify_start_failure,
- NULL,
@@hook.c: int run_found_hooks(const char*hook_name, const char*hook_path,
notify_hook_finished,Quoted 8 lines
&cb_data,
"hook",
-@@hook.c: intrun_hooks(const char *hook_name, struct run_hooks_opt *options)+@@hook.c: int run_hooks(const char*hook_name, const char*hook_path, +
+ if (options->absolute_path)
+strbuf_release(&abs_path);
++free(my_hook.feed_pipe_cb_data);Nice catch. Thanks.
Quoted 33 lines
25: 7343be28ef4 ! 26: 4b7175af2e5 hook: convert 'post-rewrite' hook in sequencer.c to hook.h 26: 85bf13a0835 ! 27: 3f24e056410 transport: convert pre-push hook to use config @@ Metadata Author: Emily Shaffer <[email protected]> ## Commit message ## - transport: convert pre-push hook to use config + transport: convert pre-push hook to hook.h - By using the hook.h:run_hooks API, pre-push hooks can be specified in - the config as well as in the hookdir. + Move the pre-push hook away from run-command.h to and over to the new + hook.h library. Signed-off-by: Emily Shaffer <[email protected]> Signed-off-by: Ævar Arnfjörð Bjarmason <[email protected]> @@ transport.c: static void die_with_unpushed_submodules(struct string_list *needs_ - int ret = 0, x; + int ret = 0; + struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT; -+ struct strbuf tmp = STRBUF_INIT; struct ref *r; - struct child_process proc = CHILD_PROCESS_INIT; - struct strbuf buf; @@ transport.c: static void die_with_unpushed_submodules(struct string_list *needs_ - finish_command(&proc); - return -1; - } -- -- sigchain_push(SIGPIPE, SIG_IGN); -+ struct string_list to_stdin = STRING_LIST_INIT_DUP; ++ struct string_list to_stdin = STRING_LIST_INIT_NODUP;Quoted 8 lines
+- sigchain_push(SIGPIPE, SIG_IGN); +- - strbuf_init(&buf, 256); + strvec_push(&opt.args, transport->remote->name); + strvec_push(&opt.args, transport->url); for (r = remote_refs; r; r = r->next) { ++ struct strbuf buf = STRBUF_INIT;Ah. It is not being freed because its lifetime is being managed by
to_stdininstead. Ok, cool!Quoted 26 lines
++ if (!r->peer_ref) continue; -@@ transport.c: static int run_pre_push_hook(struct transport *transport, + if (r->status == REF_STATUS_REJECT_NONFASTFORWARD) continue; + if (r->status == REF_STATUS_REJECT_STALE) continue; if (r->status == REF_STATUS_REJECT_REMOTE_UPDATED) continue; if (r->status == REF_STATUS_UPTODATE) continue; - strbuf_reset(&buf); - strbuf_addf( &buf, "%s %s %s %s\n", -+ strbuf_reset(&tmp); -+ strbuf_addf(&tmp, "%s %s %s %s", ++ strbuf_addf(&buf, "%s %s %s %s", r->peer_ref->name, oid_to_hex(&r->new_oid), r->name, oid_to_hex(&r->old_oid)); - @@ transport.c: static int run_pre_push_hook(struct transport *transport, - ret = -1; - break; - } -+ string_list_append(&to_stdin, tmp.buf); ++ string_list_append(&to_stdin, strbuf_detach(&buf, NULL)); } - strbuf_release(&buf); @@ transport.c: static int run_pre_push_hook(struct transport *transport,Quoted 8 lines
- x = finish_command(&proc); - if (!ret) - ret = x; -+ ret = run_hooks("pre-push", &opt); -+ run_hooks_opt_clear(&opt); -+ strbuf_release(&tmp); ++ ret = run_hooks_oneshot("pre-push", &opt); ++ to_stdin.strdup_strings = 1;And this is a typical change to the oneshot function. Cool.
Quoted 5 lines
+ string_list_clear(&to_stdin, 0); return ret; -: ----------- > 28: ecf75f33233 hook tests: test for exact "pre-push" hook input -: ----------- > 29: 2c961be94b4 hook tests: use a modern style for "pre-push" testsI'll take a look at these separately.
Quoted 33 lines
27:
331014bad17! 30:1ce456f9d9dreference-transaction: usehook.hto run hooks@@ refs.c: int ref_update_reject_duplicates(struct string_list *refnames, const char *state) { - struct child_process proc = CHILD_PROCESS_INIT; - struct strbuf buf = STRBUF_INIT; +- struct strbuf buf = STRBUF_INIT; - const char *hook; + struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT; -+ struct string_list to_stdin = STRING_LIST_INIT_DUP; ++ struct string_list to_stdin = STRING_LIST_INIT_NODUP; int ret = 0, i; -+ char o[GIT_MAX_HEXSZ + 1], n[GIT_MAX_HEXSZ + 1]; - hook = find_hook("reference-transaction"); - if (!hook) -+ if (!hook_exists("reference-transaction")) - return ret; - +- return ret; +- - strvec_pushl(&proc.args, hook, state, NULL); - proc.in = -1; - proc.stdout_to_stderr = 1; @@ refs.c: int ref_update_reject_duplicates(struct string_list *refnames, - - ret = start_command(&proc); - if (ret) -- return ret; -- ++ if (!hook_exists("reference-transaction")) + return ret; +Quoted 8 lines
- sigchain_push(SIGPIPE, SIG_IGN); + strvec_push(&opt.args, state); for (i = 0; i < transaction->nr; i++) { struct ref_update *update = transaction->updates[i]; -+ oid_to_hex_r(o, &update->old_oid); -+ oid_to_hex_r(n, &update->new_oid); ++ struct strbuf buf = STRBUF_INIT;Ah, this is doing the same thing as with 'pre-push'. Cool.
Quoted 25 lines
- strbuf_reset(&buf); +- strbuf_reset(&buf); - strbuf_addf(&buf, "%s %s %s\n", -- oid_to_hex(&update->old_oid), -- oid_to_hex(&update->new_oid), -- update->refname); ++ strbuf_addf(&buf, "%s %s %s", + oid_to_hex(&update->old_oid), + oid_to_hex(&update->new_oid), + update->refname); - - if (write_in_full(proc.in, buf.buf, buf.len) < 0) { - if (errno != EPIPE) - ret = -1; - break; - } -+ strbuf_addf(&buf, "%s %s %s", o, n, update->refname); -+ string_list_append(&to_stdin, buf.buf); ++ string_list_append(&to_stdin, strbuf_detach(&buf, NULL)); } - close(proc.in); - sigchain_pop(SIGPIPE); +- strbuf_release(&buf);Quoted 8 lines
+ opt.feed_pipe = pipe_from_string_list; + opt.feed_pipe_ctx = &to_stdin; + -+ ret = run_hooks("reference-transaction", &opt); -+ run_hooks_opt_clear(&opt); - strbuf_release(&buf); ++ ret = run_hooks_oneshot("reference-transaction", &opt); ++ to_stdin.strdup_strings = 1;And turning on
strdup_stringshere means we will free them when we callstring_list_clear. Ok.Quoted 12 lines
+ string_list_clear(&to_stdin, 0); - ret |= finish_command(&proc); 32: db70b59b3bd ! 35: ceef2f3e804 receive-pack: convert receive hooks to hook.h @@ builtin/receive-pack.c: static void hook_output_to_sideband(struct strbuf *outpu +{ + struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT; + struct receive_hook_feed_context ctx; -+ int rc; + struct command *iter = commands; + + /* if there are no valid commands, don't invoke the hook at all. */Quoted 8 lines
@@ builtin/receive-pack.c: static void hook_output_to_sideband(struct strbuf *outpu + if (!iter) + return 0; + -+ /* pre-receive hooks should run in series as the hook updates refs */ -+ if (!strcmp(hook_name, "pre-receive")) -+ opt.jobs = 1; -+Hm, interesting. I'll note this as a case to watch for when I add the .jobs option back in.
Thanks. I'll take a look at the newly added/copied over patches next.
- Emily
v5 cover Run hooks via "git run hook" & hook library
In reply to Ævar Arnfjörð BjarmasonThis topic refactors our hook execution to use a new hook.[ch] API and rew "git hook run" tool in the case of the Perl & Python scripts that execute hooks. It should contain no functional changes in how hooks are run.
It's mainly authored by Emily Schaffer & is followed-up by her series to make hooks configurable via normal "git config": [1]
See [2] for the v4 of this. A range-diff follows, but some of the main changes:
* The part where we die() if we're asked about a hook not listed in githooks(5) is gone. This makes things in Emily's follow-on topic much easier.
* Various whitespace/small refactoring changes designed to make Emily's follow-on topic smaller.
* Changed "/*" to "/**" comments in hook.h as appropriate (API comments).
* I also re-arranged some functions in the header file to make those API comments easier to follow. I.e. we start with structs, and then various functions grouped by their respective functionality.
* Various small commit message / comment rewording etc., some in response to Emily's feedback, others that I found myself.
[The rest here is a summary of how this topic interacts with a re-roll of Emily's topic coming after this, feel free to skip it if only reading this topic]:
I have my own version of Emily's series rebased on top of this with some significant changes / fixes, which I mainly rebased/hacked up to validate that all the changes here made sense.
That version is an extensive edit not of her v3 (which I believe maps to her own 30ffe98601e) but the cf1f8e34a34 tip I found at the tip of her "config-based-hooks-restart" a couple of days ago)[3], which already incorporated some of my own feedback. That version is at my avar-nasamuffin/config-based-hooks-restart-3[4].
Emily: You should be able to run something like this to get a sensible range-diff of it v.s. what you have now (probably with s/nasamuffin/origin/):
git range-diff \
gitster/ab/config-based-hooks-base..nasamuffin/config-based-hooks-restart \
avar/es-avar/config-based-hooks-6..avar/avar-nasamuffin/config-based-hooks-restart-3There's a few miscellaneous fixes in there, e.g. it passes all commits with SANITIZE=leak now, and the "hook: allow running non-native hooks" commit is entirely gone except for the doc change, as it wasn't needed with the changes I've made in this v5 (we'd already allow running non-native hooks). Likewise all the fn() and fn_gently() function split wasn't needed anymore, as it's all "gentle" now. It has a trivial CI failure[6] (coccinelle nitpick), but other than that passes all tests.
1. https://lore.kernel.org/git/[email protected]/
2. https://lore.kernel.org/git/[email protected]/
3. https://github.com/nasamuffin/git/tree/config-based-hooks-restart
4. https://github.com/avar/git/tree/avar-nasamuffin/config-based-hooks-restart-3
5. https://github.com/avar/git/runs/3493829211
6. https://github.com/avar/git/runs/3493829211
Emily Shaffer (26):
hook.c: add a hook_exists() wrapper and use it in bugreport.c
hook: add 'run' subcommand
gc: use hook library for pre-auto-gc hook
rebase: convert pre-rebase to use hook.h
am: convert applypatch to use hook.h
hooks: convert 'post-checkout' hook to hook library
merge: convert post-merge to use hook.h
send-email: use 'git hook run' for 'sendemail-validate'
git-p4: use 'git hook' to run hooks
commit: convert {pre-commit,prepare-commit-msg} hook to hook.h
read-cache: convert post-index-change to use hook.h
receive-pack: convert push-to-checkout hook to hook.h
run-command: remove old run_hook_{le,ve}() hook API
run-command: allow stdin for run_processes_parallel
hook: support passing stdin to hooks
am: convert 'post-rewrite' hook to hook.h
run-command: add stdin callback for parallelization
hook: provide stdin by string_list or callback
hook: convert 'post-rewrite' hook in sequencer.c to hook.h
transport: convert pre-push hook to hook.h
reference-transaction: use hook.h to run hooks
run-command: allow capturing of collated output
hooks: allow callers to capture output
receive-pack: convert 'update' hook to hook.h
post-update: use hook.h library
receive-pack: convert receive hooks to hook.h
Ævar Arnfjörð Bjarmason (10):
Makefile: mark "check" target as .PHONY
Makefile: stop hardcoding {command,config}-list.h
Makefile: remove an out-of-date comment
hook.[ch]: move find_hook() from run-command.c to hook.c hook.c users: use "hook_exists()" instead of "find_hook()" hook-list.h: add a generated list of hooks, like config-list.h git hook run: add an --ignore-missing flag
hook tests: test for exact "pre-push" hook input hook tests: use a modern style for "pre-push" tests
hooks: fix a TOCTOU in "did we run a hook?" heuristic
.gitignore | 2 + Documentation/git-hook.txt | 51 +++++ Documentation/githooks.txt | 4 + Makefile | 26 ++- builtin.h | 1 + builtin/am.c | 29 +-- builtin/bugreport.c | 46 +---- builtin/checkout.c | 14 +- builtin/clone.c | 6 +- builtin/commit.c | 19 +- builtin/fetch.c | 1 + builtin/gc.c | 3 +- builtin/hook.c | 98 ++++++++++ builtin/merge.c | 21 +- builtin/rebase.c | 6 +- builtin/receive-pack.c | 285 +++++++++++++--------------- builtin/submodule--helper.c | 2 +- builtin/worktree.c | 29 ++- command-list.txt | 1 + commit.c | 16 +- commit.h | 3 +- compat/vcbuild/README | 2 +- config.mak.uname | 6 +- contrib/buildsystems/CMakeLists.txt | 7 + generate-hooklist.sh | 18 ++ git-p4.py | 72 +------ git-send-email.perl | 20 +- git.c | 1 + hook.c | 219 +++++++++++++++++++++ hook.h | 127 +++++++++++++ read-cache.c | 11 +- refs.c | 41 ++-- reset.c | 14 +- run-command.c | 157 +++++++-------- run-command.h | 55 +++--- sequencer.c | 86 ++++----- submodule.c | 1 + t/helper/test-run-command.c | 46 ++++- t/t0061-run-command.sh | 37 ++++ t/t1800-hook.sh | 151 +++++++++++++++ t/t5571-pre-push-hook.sh | 94 +++++---- t/t9001-send-email.sh | 4 +- transport.c | 57 ++---- 43 files changed, 1278 insertions(+), 611 deletions(-) create mode 100644 Documentation/git-hook.txt create mode 100644 builtin/hook.c create mode 100755 generate-hooklist.sh create mode 100644 hook.c create mode 100644 hook.h create mode 100755 t/t1800-hook.sh
Range-diff against v4:
1: 81fe1ed90d5 = 1: ac419613fdc Makefile: mark "check" target as .PHONY
2: 0f749530777 ! 2: a161b7f0a5c Makefile: stop hardcoding {command,config}-list.h @@ Commit message
this refactoring we'll only need to add the new file to the GENERATED_H variable, not EXCEPT_HDRS, the vcbuild/README etc.
- I have not tested the Windows-specific change in config.mak.uname
- being made here, but we use other variables from the Makefile in the
- same block, and the GENERATED_H is fully defined before we include
- config.mak.uname.
-
Hardcoding command-list.h there seems to have been a case of copy/paste programming in 976aaedca0 (msvc: add a Makefile target to pre-generate the Visual Studio solution, 2019-07-29). The
3: 644b31fe281 = 3: ffef1d3257e Makefile: remove an out-of-date comment 4: 89c4d44b0c3 ! 4: 545e16c6f04 hook.[ch]: move find_hook() to this new library
@@ Metadata
Author: Ævar Arnfjörð Bjarmason <[email protected]>
## Commit message ##
- hook.[ch]: move find_hook() to this new library
+ hook.[ch]: move find_hook() from run-command.c to hook.c
Move the find_hook() function from run-command.c to a new hook.c library. This change establishes a stub library that's pretty
@@ hook.h (new)
+#ifndef HOOK_H
+#define HOOK_H
+
-+/*
++/**
+ * Returns the path to the hook file, or NULL if the hook is missing
+ * or disabled. Note that this points to static storage that will be
-+ * overwritten by further calls to find_hook and run_hook_*.
++ * overwritten by further calls to find_hook().
+ */
+const char *find_hook(const char *name);
+
5: 3514e0c0251 ! 5: a9bc4519e9a hook.c: add a hook_exists() wrapper and use it in bugreport.c
@@ hook.h
*/
const char *find_hook(const char *name);
-+/*
++/**
+ * A boolean version of find_hook()
+ */
+int hook_exists(const char *hookname); 6: d5ef40f77dc ! 6: e99ec2e6f8f hook.c users: use "hook_exists()" insted of "find_hook()"
@@ Metadata
Author: Ævar Arnfjörð Bjarmason <[email protected]>
## Commit message ##
- hook.c users: use "hook_exists()" insted of "find_hook()"
+ hook.c users: use "hook_exists()" instead of "find_hook()"
Use the new hook_exists() function instead of find_hook() where the latter was called in boolean contexts. This make subsequent changes in
7: 4cfd72722c1 ! 7: 2ffb2332c8a hook-list.h: add a generated list of hooks, like config-list.h
@@ Commit message
hook-list.h: add a generated list of hooks, like config-list.h
Make githooks(5) the source of truth for what hooks git supports, and
- die hooks we don't know about in find_hook(). This ensures that the
- documentation and the C code's idea about existing hooks doesn't
- diverge.
+ punt out early on hooks we don't know about in find_hook(). This
+ ensures that the documentation and the C code's idea about existing
+ hooks doesn't diverge.
We still have Perl and Python code running its own hooks, but that'll be addressed by Emily Shaffer's upcoming "git hook run" command.
@@ Commit message
listing only knowing about 1/4 of the p4 hooks. It didn't know about the recent "reference-transaction" hook either.
+ We could make the find_hook() function die() or BUG() out if the new
+ known_hook() returned 0, but let's make it return NULL just as it does
+ when it can't find a hook of a known type. Making it die() is overly
+ anal, and unlikely to be what we need in catching stupid typos in the
+ name of some new hook hardcoded in git.git's sources. By making this
+ be tolerant of unknown hook names, changes in a later series to make
+ "git hook run" run arbitrary user-configured hook names will be easier
+ to implement.
+
I have not been able to directly test the CMake change being made here. Since 4c2c38e800 (ci: modification of main.yml to use cmake for vs-build job, 2020-06-26) some of the Windows CI has a hard dependency
@@ generate-hooklist.sh (new)
+ NULL,
+};
+EOF
-
- ## hook.c ##
-@@
- #include "cache.h"
- #include "hook.h"
- #include "run-command.h"
-+#include "hook-list.h"
-+
-+static int known_hook(const char *name)
-+{
-+ const char **p;
-+ size_t len = strlen(name);
-+ for (p = hook_name_list; *p; p++) {
-+ const char *hook = *p;
-+
-+ if (!strncmp(name, hook, len) && hook[len] == '\0')
-+ return 1;
-+ }
-+
-+ return 0;
-+}
-
- const char *find_hook(const char *name)
- {
- static struct strbuf path = STRBUF_INIT;
-
-+ if (!known_hook(name))
-+ die(_("the hook '%s' is not known to git, should be in hook-list.h via githooks(5)"),
-+ name);
-+
- strbuf_reset(&path);
- strbuf_git_path(&path, "hooks/%s", name);
- if (access(path.buf, X_OK) < 0) {
8: 7cb4a4cb69e ! 8: 72dd1010f5b hook: add 'run' subcommand
@@ builtin/hook.c (new)
+ /* Need to take into account core.hooksPath */
+ git_config(git_default_config, NULL);
+
++ /*
++ * We are not using a plain run_hooks() because we'd like to
++ * detect missing hooks. Let's find it ourselves and call
++ * run_hooks() instead.
++ */
+ hook_name = argv[0];
+ hook_path = find_hook(hook_name);
+ if (!hook_path) {
@@ git.c: static struct cmd_struct commands[] = {
## hook.c ##
@@
+ #include "cache.h"
#include "hook.h"
#include "run-command.h"
- #include "hook-list.h"
+#include "config.h"
- static int known_hook(const char *name)
+ const char *find_hook(const char *name)
{
- const char **p;
- size_t len = strlen(name);
-+ static int test_hooks_ok = -1;
-+
- for (p = hook_name_list; *p; p++) {
- const char *hook = *p;
-
-@@ hook.c: static int known_hook(const char *name)
- return 1;
- }
-
-+ if (test_hooks_ok == -1)
-+ test_hooks_ok = git_env_bool("GIT_TEST_FAKE_HOOKS", 0);
-+
-+ if (test_hooks_ok &&
-+ (!strcmp(name, "test-hook") ||
-+ !strcmp(name, "does-not-exist")))
-+ return 1;
-+
- return 0;
- }
-
@@ hook.c: int hook_exists(const char *name)
{
return !!find_hook(name);
@@ hook.h
@@
#ifndef HOOK_H
#define HOOK_H
-+#include "strbuf.h"
+#include "strvec.h"
-+#include "run-command.h"
-
- /*
- * Returns the path to the hook file, or NULL if the hook is missing
-@@ hook.h: const char *find_hook(const char *name);
- */
- int hook_exists(const char *hookname);
-
++
+struct hook {
+ /* The path to the hook */
+ const char *hook_path;
@@ hook.h: const char *find_hook(const char *name);
+ .args = STRVEC_INIT, \
+}
+
-+/*
-+ * Callback provided to feed_pipe_fn and consume_sideband_fn.
-+ */
+struct hook_cb_data {
+ /* rc reflects the cumulative failure state */
+ int rc;
@@ hook.h: const char *find_hook(const char *name);
+ struct hook *run_me;
+ struct run_hooks_opt *options;
+};
-+
+
+ /**
+ * Returns the path to the hook file, or NULL if the hook is missing
+@@ hook.h: const char *find_hook(const char *name);
+ */
+ int hook_exists(const char *hookname);
+
++/**
++ * Clear data from an initialized "struct run_hooks-opt".
++ */
+void run_hooks_opt_clear(struct run_hooks_opt *o);
+
+/**
@@ hook.h: const char *find_hook(const char *name);
## t/t1800-hook.sh (new) ##
@@
-+#!/bin/bash
++#!/bin/sh
+
+test_description='git-hook command'
+
@@ t/t1800-hook.sh (new)
+ grep "unknown option" err
+'
+
-+test_expect_success 'setup GIT_TEST_FAKE_HOOKS=true to permit "test-hook" and "does-not-exist" names"' '
-+ GIT_TEST_FAKE_HOOKS=true &&
-+ export GIT_TEST_FAKE_HOOKS
-+'
-+
+test_expect_success 'git hook run: nonexistent hook' '
+ cat >stderr.expect <<-\EOF &&
+ error: cannot find a hook named test-hook
9: 2b8500aa675 ! 9: 821cc9bf11e gc: use hook library for pre-auto-gc hook
@@ hook.c: int run_hooks(const char *hook_name, const char *hook_path,
+ }
+
+ ret = run_hooks(hook_name, hook_path, options);
++
+cleanup:
+ run_hooks_opt_clear(options);
++
+ return ret;
+}
@@ hook.h: void run_hooks_opt_clear(struct run_hooks_opt *o);
+ * with run_hooks().
+ *
+ * If "options" is provided calls run_hooks_opt_clear() on it for
-+ * you. If "options" is NULL a scratch one will be provided for you
-+ * before calling run_hooks().
++ * you. If "options" is NULL the default options from
++ * RUN_HOOKS_OPT_INIT will be used.
+ */
+int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options);
+
10: 3ee55d2c10f = 10: d71c90254ea rebase: convert pre-rebase to use hook.h
11: 050f20d14f0 = 11: ea3af2ccc4d am: convert applypatch to use hook.h
12: ac875d284da ! 12: fed0b52f88f hooks: convert 'post-checkout' hook to hook library
@@ hook.h: struct run_hooks_opt
/* Args to be passed to each hook */
struct strvec args;
+
-+ /* Resolve and run the "absolute_path(hook)" instead of
++ /*
++ * Resolve and run the "absolute_path(hook)" instead of
+ * "hook". Used for "git worktree" hooks
+ */
+ int absolute_path;
13: 69763bc2255 = 13: 53d8721a0e3 merge: convert post-merge to use hook.h
14: 2ca1ca1b8e4 ! 14: d60827a2856 git hook run: add an --ignore-missing flag
@@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
};
int ret;
@@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
- git_config(git_default_config, NULL);
-
+ /*
+ * We are not using a plain run_hooks() because we'd like to
+ * detect missing hooks. Let's find it ourselves and call
+- * run_hooks() instead.
++ * run_hooks() instead...
+ */
hook_name = argv[0];
+ if (ignore_missing)
++ /* ... act like a plain run_hooks() under --ignore-missing */
+ return run_hooks_oneshot(hook_name, &opt);
hook_path = find_hook(hook_name);
if (!hook_path) {
15: 5b66b04bec7 = 15: d4976a0821f send-email: use 'git hook run' for 'sendemail-validate'
16: 14a37a43db2 = 16: 99f3dcd1945 git-p4: use 'git hook' to run hooks
17: ad5d0e0e7de = 17: 509761454e6 commit: convert {pre-commit,prepare-commit-msg} hook to hook.h
18: 3d3a33e2674 = 18: e2c94d95427 read-cache: convert post-index-change to use hook.h
19: 893f8666301 ! 19: fa7d0d24ea2 receive-pack: convert push-to-checkout hook to hook.h @@ Commit message
Move the push-to-checkout hook away from run-command.h to and over to the new hook.h library.
+ This removes the last direct user of run_hook_le(), so we could remove
+ that function now, but let's leave that to a follow-up cleanup commit.
+
20: 070433deba5 = 20: 428bb5a6792 run-command: remove old run_hook_{le,ve}() hook API
21: 1028e0c1667 = 21: 994f6ad8602 run-command: allow stdin for run_processes_parallel
22: 639e59e9ed0 = 22: 3ccc654a664 hook: support passing stdin to hooks
23: 7d1925cca48 = 23: f548e3d15e7 am: convert 'post-rewrite' hook to hook.h
24: 0c24221b522 = 24: bb119fa7cc0 run-command: add stdin callback for parallelization
25: 05d1085f7eb ! 25: 2439f7752b8 hook: provide stdin by string_list or callback @@ hook.c: int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *optio
ret = 0;
## hook.h ##
-@@ hook.h: int hook_exists(const char *hookname); +@@
+ #ifndef HOOK_H
+ #define HOOK_H
++#include "strbuf.h"
+ #include "strvec.h"
++#include "run-command.h"
+
struct hook {
/* The path to the hook */
const char *hook_path;
@@ hook.h: struct run_hooks_opt
};
#define RUN_HOOKS_OPT_INIT { \
-@@ hook.h: struct run_hooks_opt
- .args = STRVEC_INIT, \
- }
+@@ hook.h: int run_hooks(const char *hookname, const char *hook_path,
+ */
+ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options);
-+/*
++/**
+ * To specify a 'struct string_list', set 'run_hooks_opt.feed_pipe_ctx' to the
-+ * string_list and set 'run_hooks_opt.feed_pipe' to 'pipe_from_string_list()'. ++ * string_list and set 'run_hooks_opt.feed_pipe' to pipe_from_string_list().
+ * This will pipe each string in the list to stdin, separated by newlines. (Do
+ * not inject your own newlines.)
+ */
+int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb); +
- /*
- * Callback provided to feed_pipe_fn and consume_sideband_fn.
- */
+ #endif
26: 4b7175af2e5 = 26: 48a380b3a91 hook: convert 'post-rewrite' hook in sequencer.c to hook.h 27: 3f24e056410 = 27: af6b9292aaa transport: convert pre-push hook to hook.h 28: ecf75f33233 ! 28: 957691f0b6d hook tests: test for exact "pre-push" hook input
@@ Commit message
Extend the tests added in ec55559f937 (push: Add support for pre-push hooks, 2013-01-13) to exhaustively test for the exact input we're
- expecting. This helps a parallel series that's refactoring how the
- hook is called, to e.g. make sure that we don't miss a trailing
- newline.
+ expecting. This ensures that we e.g. don't miss a trailing newline.
+
+ Appending to a file called "actual" is the established convention in
+ this test for hooks, see the rest of the tests added in
+ ec55559f937 (push: Add support for pre-push hooks, 2013-01-13). Let's
+ follow that convention here.
29: 2c961be94b4 ! 29: 88fe2621549 hook tests: use a modern style for "pre-push" tests
@@ t/t5571-pre-push-hook.sh: echo "$2" >>actual
-EOF
-
test_expect_success 'push with hook' '
-+ cat >expected <<-EOF &&
++ cat >expect <<-EOF &&
+ parent1
+ repo1
+ refs/heads/main $COMMIT2 refs/heads/foreign $COMMIT1
@@ t/t5571-pre-push-hook.sh: echo "$2" >>actual
+
git push parent1 main:foreign &&
- diff expected actual
-+ test_cmp expected actual
++ test_cmp expect actual
'
test_expect_success 'add a branch' '
30: 1ce456f9d9d = 30: 1d905e81779 reference-transaction: use hook.h to run hooks
31: 6e5f1f5bd3a = 31: fac56a9d8af run-command: allow capturing of collated output
32: 0b6e9c6d07a = 32: 7d185cdf9d1 hooks: allow callers to capture output
33: dcf63634338 = 33: c8150e1239f receive-pack: convert 'update' hook to hook.h
34: f352a485e59 = 34: a20ad847c14 post-update: use hook.h library
35: ceef2f3e804 = 35: 79c380be6ed receive-pack: convert receive hooks to hook.h
36: b71d7628b40 ! 36: fe056098534 hooks: fix a TOCTOU in "did we run a hook?" heuristic
@@ hook.h: struct hook_cb_data {
+ int *invoked_hook;
};
- void run_hooks_opt_clear(struct run_hooks_opt *o);
+ /**
## sequencer.c ##
@@ sequencer.c: static int run_prepare_commit_msg_hook(struct repository *r,Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 1/36 Makefile: mark "check" target as .PHONY
Fix a bug in 44c9e8594e (Fix up header file dependencies and add sparse checking rules, 2005-07-03), we never marked the phony "check" target as such.
Perhaps we should just remove it, since as of a combination of 912f9980d2 (Makefile: help people who run 'make check' by mistake, 2008-11-11) 0bcd9ae85d (sparse: Fix errors due to missing target-specific variables, 2011-04-21) we've been suggesting the user run "make sparse" directly.
But under that mode it still does something, as well as directing the user to run "make test" under non-sparse. So let's punt that and narrowly fix the PHONY bug.
--- Makefile | 1 + 1 file changed, 1 insertion(+)
Makefile +1 -0
@@ -2941,6 +2941,7 @@ hdr-check: $(HCO) style: git clang-format --style file --diff --extensions c,h +.PHONY: check check: config-list.h command-list.h @if sparse; \ then \
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 2/36 Makefile: stop hardcoding {command,config}-list.h
In reply to Ævar Arnfjörð BjarmasonChange various places that hardcode the names of these two files to refer to either $(GENERATED_H), or to a new generated-hdrs target. That target is consistent with the *-objs targets I recently added in 029bac01a8 (Makefile: add {program,xdiff,test,git,fuzz}-objs & objects targets, 2021-02-23).
A subsequent commit will add a new generated hook-list.h. By doing this refactoring we'll only need to add the new file to the GENERATED_H variable, not EXCEPT_HDRS, the vcbuild/README etc.
Hardcoding command-list.h there seems to have been a case of copy/paste programming in 976aaedca0 (msvc: add a Makefile target to pre-generate the Visual Studio solution, 2019-07-29). The config-list.h was added later in 709df95b78 (help: move list_config_help to builtin/help, 2020-04-16).
--- Makefile | 6 ++++-- compat/vcbuild/README | 2 +- config.mak.uname | 6 +++--- 3 files changed, 8 insertions(+), 6 deletions(-)
Makefile +4 -2
@@ -824,6 +824,8 @@ XDIFF_LIB = xdiff/lib.a GENERATED_H += command-list.h GENERATED_H += config-list.h +.PHONY: generated-hdrs +generated-hdrs: $(GENERATED_H) LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \ $(FIND) . \
@@ -2919,7 +2921,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE .PHONY: sparse $(SP_OBJ) sparse: $(SP_OBJ) -EXCEPT_HDRS := command-list.h config-list.h unicode-width.h compat/% xdiff/% +EXCEPT_HDRS := $(GENERATED_H) unicode-width.h compat/% xdiff/% ifndef GCRYPT_SHA256 EXCEPT_HDRS += sha256/gcrypt.h endif
@@ -2942,7 +2944,7 @@ style: git clang-format --style file --diff --extensions c,h .PHONY: check -check: config-list.h command-list.h +check: $(GENERATED_H) @if sparse; \ then \ echo >&2 "Use 'make sparse' instead"; \
compat/vcbuild/README +1 -1
@@ -92,7 +92,7 @@ The Steps of Build Git with VS2008 the git operations. 3. Inside Git's directory run the command: - make command-list.h config-list.h + make generated-hdrs to generate the header file needed to compile git. 4. Then either build Git with the GNU Make Makefile in the Git projects
config.mak.uname +3 -3
@@ -735,9 +735,9 @@ vcxproj: echo '</Project>') >git-remote-http/LinkOrCopyRemoteHttp.targets git add -f git/LinkOrCopyBuiltins.targets git-remote-http/LinkOrCopyRemoteHttp.targets - # Add command-list.h and config-list.h - $(MAKE) MSVC=1 SKIP_VCPKG=1 prefix=/mingw64 config-list.h command-list.h - git add -f config-list.h command-list.h + # Add generated headers + $(MAKE) MSVC=1 SKIP_VCPKG=1 prefix=/mingw64 $(GENERATED_H) + git add -f $(GENERATED_H) # Add scripts rm -f perl/perl.mak
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 3/36 Makefile: remove an out-of-date comment
In reply to Ævar Arnfjörð BjarmasonThis comment added in dfea575017 (Makefile: lazily compute header dependencies, 2010-01-26) has been out of date since 92b88eba9f (Makefile: use git ls-files to list header files, if possible, 2019-03-04), when we did exactly what it tells us not to do and added $(GENERATED_H) to $(OBJECTS) dependencies.
The rest of it was also somewhere between inaccurate and outdated, since as of b8ba629264 (Makefile: fold MISC_H into LIB_H, 2012-06-20) it's not followed by a list of header files, that got moved earlier in the file into LIB_H in 60d24dd255 (Makefile: fold XDIFF_H and VCSSVN_H into LIB_H, 2012-07-06).
Let's just remove it entirely, to the extent that we have anything useful to say here the comment on the "USE_COMPUTED_HEADER_DEPENDENCIES" variable a few lines above this change does the job for us.
--- Makefile | 7 ------- 1 file changed, 7 deletions(-)
Makefile +0 -7
@@ -2524,13 +2524,6 @@ ifneq ($(dep_files_present),) include $(dep_files_present) endif else -# Dependencies on header files, for platforms that do not support -# the gcc -MMD option. -# -# Dependencies on automatically generated headers such as command-list.h -# should _not_ be included here, since they are necessary even when -# building an object for the first time. - $(OBJECTS): $(LIB_H) $(GENERATED_H) endif
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 4/36 hook.[ch]: move find_hook() from run-command.c to hook.c
In reply to Ævar Arnfjörð BjarmasonMove the find_hook() function from run-command.c to a new hook.c library. This change establishes a stub library that's pretty pointless right now, but will see much wider use with Emily Shaffer's upcoming "configuration-based hooks" series.
Eventually all the hook related code will live in hook.[ch]. Let's start that process by moving the simple find_hook() function over as-is.
--- Makefile | 1 + builtin/am.c | 1 + builtin/bugreport.c | 2 +- builtin/commit.c | 1 + builtin/merge.c | 1 + builtin/receive-pack.c | 1 + builtin/worktree.c | 1 + hook.c | 37 +++++++++++++++++++++++++++++++++++++ hook.h | 11 +++++++++++ refs.c | 1 + run-command.c | 35 +---------------------------------- run-command.h | 7 ------- sequencer.c | 1 + transport.c | 1 + 14 files changed, 59 insertions(+), 42 deletions(-) create mode 100644 hook.c create mode 100644 hook.h
Makefile +1 -0
@@ -911,6 +911,7 @@ LIB_OBJS += hash-lookup.o LIB_OBJS += hashmap.o LIB_OBJS += help.o LIB_OBJS += hex.o +LIB_OBJS += hook.o LIB_OBJS += ident.o LIB_OBJS += json-writer.o LIB_OBJS += kwset.o
builtin/am.c +1 -0
@@ -11,6 +11,7 @@#include "parse-options.h" #include "dir.h" #include "run-command.h" +#include "hook.h" #include "quote.h" #include "tempfile.h" #include "lockfile.h"
builtin/bugreport.c +1 -1
@@ -3,7 +3,7 @@#include "strbuf.h" #include "help.h" #include "compat/compiler.h" -#include "run-command.h" +#include "hook.h" static void get_system_info (struct strbuf *sys_info)
builtin/commit.c +1 -0
@@ -19,6 +19,7 @@#include "revision.h" #include "wt-status.h" #include "run-command.h" +#include "hook.h" #include "refs.h" #include "log-tree.h" #include "strbuf.h"
builtin/merge.c +1 -0
@@ -13,6 +13,7 @@#include "builtin.h" #include "lockfile.h" #include "run-command.h" +#include "hook.h" #include "diff.h" #include "diff-merges.h" #include "refs.h"
builtin/receive-pack.c +1 -0
@@ -7,6 +7,7 @@#include "pkt-line.h" #include "sideband.h" #include "run-command.h" +#include "hook.h" #include "exec-cmd.h" #include "commit.h" #include "object.h"
builtin/worktree.c +1 -0
@@ -8,6 +8,7 @@#include "branch.h" #include "refs.h" #include "run-command.h" +#include "hook.h" #include "sigchain.h" #include "submodule.h" #include "utf8.h"
hook.c new file +37 -0
@@ -0,0 +1,37 @@ +#include "cache.h" +#include "hook.h" +#include "run-command.h" + +const char *find_hook (const char *name) +{ +static struct strbuf path =STRBUF_INIT ; + +strbuf_reset (&path); +strbuf_git_path (&path,"hooks/%s" , name); +if (access (path.buf,X_OK ) <0 ) { +int err = errno; + +#ifdef STRIP_EXTENSION +strbuf_addstr (&path,STRIP_EXTENSION ); +if (access (path.buf,X_OK ) >=0 ) +return path.buf; +if (errno ==EACCES ) + err = errno; +#endif + +if (err ==EACCES && advice_ignored_hook) { +static struct string_list advise_given =STRING_LIST_INIT_DUP ; + +if (!string_list_lookup (&advise_given, name)) { +string_list_insert (&advise_given, name); +advise (_ ("The '%s' hook was ignored because " +"it's not set as executable.\n" +"You can disable this warning with " +"`git config advice.ignoredHook false`." ), + path.buf); + } + } +return NULL ; + } +return path.buf; +}
hook.h new file +11 -0
@@ -0,0 +1,11 @@ +#ifndef HOOK_H +#define HOOK_H + +/** +* Returns the path to the hook file, or NULL if the hook is missing +* or disabled. Note that this points to static storage that will be +* overwritten by further calls to find_hook(). +*/ +const char *find_hook (const char *name); + +#endif
refs.c +1 -0
@@ -10,6 +10,7 @@#include "refs.h" #include "refs/refs-internal.h" #include "run-command.h" +#include "hook.h" #include "object-store.h" #include "object.h" #include "tag.h"
run-command.c +1 -34
@@ -8,6 +8,7 @@#include "string-list.h" #include "quote.h" #include "config.h" +#include "hook.h" void child_process_init (struct child_process *child) {
@@ -1319,40 +1320,6 @@ int async_with_fork(void)#endif } -const char *find_hook (const char *name) -{ -static struct strbuf path =STRBUF_INIT ; - -strbuf_reset (&path); -strbuf_git_path (&path,"hooks/%s" , name); -if (access (path.buf,X_OK ) <0 ) { -int err = errno; - -#ifdef STRIP_EXTENSION -strbuf_addstr (&path,STRIP_EXTENSION ); -if (access (path.buf,X_OK ) >=0 ) -return path.buf; -if (errno ==EACCES ) - err = errno; -#endif - -if (err ==EACCES && advice_ignored_hook) { -static struct string_list advise_given =STRING_LIST_INIT_DUP ; - -if (!string_list_lookup (&advise_given, name)) { -string_list_insert (&advise_given, name); -advise (_ ("The '%s' hook was ignored because " -"it's not set as executable.\n" -"You can disable this warning with " -"`git config advice.ignoredHook false`." ), - path.buf); - } - } -return NULL ; - } -return path.buf; -} -int run_hook_ve (const char *const *env,const char *name,va_list args) {struct child_process hook =CHILD_PROCESS_INIT ;
run-command.h +0 -7
@@ -204,13 +204,6 @@ int finish_command_in_signal(struct child_process *);*/ int run_command (struct child_process *); -/* -* Returns the path to the hook file, or NULL if the hook is missing -* or disabled. Note that this points to static storage that will be -* overwritten by further calls to find_hook and run_hook_*. -*/ -const char *find_hook (const char *name); -/** * Run a hook. * The first argument is a pathname to an index file, or NULL
sequencer.c +1 -0
@@ -8,6 +8,7 @@#include "sequencer.h" #include "tag.h" #include "run-command.h" +#include "hook.h" #include "exec-cmd.h" #include "utf8.h" #include "cache-tree.h"
transport.c +1 -0
@@ -2,6 +2,7 @@#include "config.h" #include "transport.h" #include "run-command.h" +#include "hook.h" #include "pkt-line.h" #include "fetch-pack.h" #include "remote.h"
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 5/36 hook.c: add a hook_exists() wrapper and use it in bugreport.c
In reply to Ævar Arnfjörð BjarmasonFrom: Emily Shaffer <[email protected]>
Add a boolean version of the find_hook() function for those callers who are only interested in checking whether the hook exists, not what the path to it is.
--- builtin/bugreport.c | 2 +- hook.c | 5 +++++ hook.h | 5 +++++ 3 files changed, 11 insertions(+), 1 deletion(-)
builtin/bugreport.c +1 -1
@@ -82,7 +82,7 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit) }for (i =0 ; i <ARRAY_SIZE (hook); i++) -if (find_hook (hook[i])) +if (hook_exists (hook[i]))strbuf_addf (hook_info,"%s\n" , hook[i]); }
hook.c +5 -0
@@ -35,3 +35,8 @@ const char *find_hook(const char *name) }return path.buf; } + +int hook_exists (const char *name) +{ +return !!find_hook (name); +}
hook.h +5 -0
@@ -8,4 +8,9 @@*/ const char *find_hook (const char *name); +/** +* A boolean version of find_hook() +*/ +int hook_exists (const char *hookname); +#endif
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 6/36 hook.c users: use "hook_exists()" instead of "find_hook()"
In reply to Ævar Arnfjörð BjarmasonUse the new hook_exists() function instead of find_hook() where the latter was called in boolean contexts. This make subsequent changes in a series where we further refactor the hook API clearer, as we won't conflate wanting to get the path of the hook with checking for its existence.
--- builtin/commit.c | 2 +- builtin/merge.c | 2 +- builtin/receive-pack.c | 2 +- sequencer.c | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-)
builtin/commit.c +1 -1
@@ -1052,7 +1052,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix,return 0 ; } -if (!no_verify &&find_hook ("pre-commit" )) { +if (!no_verify &&hook_exists ("pre-commit" )) {/* * Re-read the index as pre-commit hook could have updated it, * and write it out as a tree. We must do this before we invoke
builtin/merge.c +1 -1
@@ -850,7 +850,7 @@ static void prepare_to_commit(struct commit_list *remoteheads)* and write it out as a tree. We must do this before we invoke * the editor and after we invoke run_status above. */ -if (find_hook ("pre-merge-commit" )) +if (hook_exists ("pre-merge-commit" ))discard_cache ();read_cache_from (index_file);strbuf_addbuf (&msg, &merge_msg);
builtin/receive-pack.c +1 -1
@@ -1464,7 +1464,7 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *wstrvec_pushf (&env,"GIT_DIR=%s" ,absolute_path (git_dir)); -if (!find_hook (push_to_checkout_hook)) +if (!hook_exists (push_to_checkout_hook)) retval =push_to_deploy (sha1, &env, work_tree);else retval =push_to_checkout (sha1, &env, work_tree);
sequencer.c +1 -1
@@ -1446,7 +1446,7 @@ static int try_to_commit(struct repository *r, } } -if (find_hook ("prepare-commit-msg" )) { +if (hook_exists ("prepare-commit-msg" )) { res =run_prepare_commit_msg_hook (r, msg, hook_commit);if (res) goto out;
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 7/36 hook-list.h: add a generated list of hooks, like config-list.h
In reply to Ævar Arnfjörð BjarmasonMake githooks(5) the source of truth for what hooks git supports, and punt out early on hooks we don't know about in find_hook(). This ensures that the documentation and the C code's idea about existing hooks doesn't diverge.
We still have Perl and Python code running its own hooks, but that'll be addressed by Emily Shaffer's upcoming "git hook run" command.
This resolves a long-standing TODO item in bugreport.c of there being no centralized listing of hooks, and fixes a bug with the bugreport listing only knowing about 1/4 of the p4 hooks. It didn't know about the recent "reference-transaction" hook either.
We could make the find_hook() function die() or BUG() out if the new known_hook() returned 0, but let's make it return NULL just as it does when it can't find a hook of a known type. Making it die() is overly anal, and unlikely to be what we need in catching stupid typos in the name of some new hook hardcoded in git.git's sources. By making this be tolerant of unknown hook names, changes in a later series to make "git hook run" run arbitrary user-configured hook names will be easier to implement.
I have not been able to directly test the CMake change being made here. Since 4c2c38e800 (ci: modification of main.yml to use cmake for vs-build job, 2020-06-26) some of the Windows CI has a hard dependency on CMake, this change works there, and is to my eyes an obviously correct use of a pattern established in previous CMake changes, namely:
- 061c2240b1 (Introduce CMake support for configuring Git,
2020-06-12)
- 709df95b78 (help: move list_config_help to builtin/help,
2020-04-16) - 976aaedca0 (msvc: add a Makefile target to pre-generate the Visual Studio solution, 2019-07-29)
The LC_ALL=C is needed because at least in my locale the dash ("-") is ignored for the purposes of sorting, which results in a different order. I'm not aware of anything in git that has a hard dependency on the order, but e.g. the bugreport output would end up using whatever locale was in effect when git was compiled.
--- .gitignore | 1 + Makefile | 10 ++++++- builtin/bugreport.c | 44 ++++++----------------------- contrib/buildsystems/CMakeLists.txt | 7 +++++ generate-hooklist.sh | 18 ++++++++++++ 5 files changed, 43 insertions(+), 37 deletions(-) create mode 100755 generate-hooklist.sh
.gitignore +1 -0
@@ -190,6 +190,7 @@ /gitweb/static/gitweb.min.* /config-list.h /command-list.h +/hook-list.h *.tar.gz *.dsc *.deb
Makefile +9 -1
@@ -824,6 +824,8 @@ XDIFF_LIB = xdiff/lib.a GENERATED_H += command-list.h GENERATED_H += config-list.h +GENERATED_H += hook-list.h + .PHONY: generated-hdrs generated-hdrs: $(GENERATED_H)
@@ -2231,7 +2233,9 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS) help.sp help.s help.o: command-list.h -builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX +hook.sp hook.s hook.o: hook-list.h + +builtin/help.sp builtin/help.s builtin/help.o: config-list.h hook-list.h GIT-PREFIX builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \ '-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \ '-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2264,6 +2268,10 @@ command-list.h: $(wildcard Documentation/git*.txt) $(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \ command-list.txt >$@+ && mv $@+ $@ +hook-list.h: generate-hooklist.sh Documentation/githooks.txt + $(QUIET_GEN)$(SHELL_PATH) ./generate-hooklist.sh \ + >$@+ && mv $@+ $@ + SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\ $(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\ $(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
builtin/bugreport.c +8 -36
62 lines not on this page · raw message
contrib/buildsystems/CMakeLists.txt +7 -0
@@ -624,6 +624,13 @@ if(NOT EXISTS ${CMAKE_BINARY_DIR}/config-list.h) OUTPUT_FILE ${CMAKE_BINARY_DIR}/config-list.h) endif() +if(NOT EXISTS ${CMAKE_BINARY_DIR}/hook-list.h) + message("Generating hook-list.h") + execute_process(COMMAND ${SH_EXE} ${CMAKE_SOURCE_DIR}/generate-hooklist.sh + WORKING_DIRECTORY ${CMAKE_SOURCE_DIR} + OUTPUT_FILE ${CMAKE_BINARY_DIR}/hook-list.h) +endif() + include_directories(${CMAKE_BINARY_DIR}) #build
generate-hooklist.sh new file +18 -0
@@ -0,0 +1,18 @@ +#!/bin/sh +# +# Usage: ./generate-hooklist.sh >hook-list.h + +cat <<EOF +/* Automatically generated by generate-hooklist.sh */ + +static const char *hook_name_list[] = { +EOF + +sed -n -e '/^~~~~*$/ {x; s/^.*$/ "&",/; p;}; x' \ + <Documentation/githooks.txt | + LC_ALL=C sort + +cat <<EOF + NULL, +}; +EOF
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 8/36 hook: add 'run' subcommand
In reply to Ævar Arnfjörð BjarmasonFrom: Emily Shaffer <[email protected]>
In order to enable hooks to be run as an external process, by a standalone Git command, or by tools which wrap Git, provide an external means to run all configured hook commands for a given hook event.
Most of our hooks require more complex functionality than this, but let's start with the bare minimum required to support our simplest hooks.
In terms of implementation the usage_with_options() and "goto usage" pattern here mirrors that of builtin/{commit-graph,multi-pack-index}.c.
--- .gitignore | 1 + Documentation/git-hook.txt | 38 +++++++++++ Documentation/githooks.txt | 4 ++ Makefile | 1 + builtin.h | 1 + builtin/hook.c | 90 ++++++++++++++++++++++++++ command-list.txt | 1 + git.c | 1 + hook.c | 101 +++++++++++++++++++++++++++++ hook.h | 39 +++++++++++ t/t1800-hook.sh | 128 +++++++++++++++++++++++++++++++++++++ 11 files changed, 405 insertions(+) create mode 100644 Documentation/git-hook.txt create mode 100644 builtin/hook.c create mode 100755 t/t1800-hook.sh
.gitignore +1 -0
@@ -77,6 +77,7 @@ /git-grep /git-hash-object /git-help +/git-hook /git-http-backend /git-http-fetch /git-http-push
Documentation/git-hook.txt new file +38 -0
@@ -0,0 +1,38 @@ +git-hook(1) +=========== + +NAME +---- +git-hook - run git hooks + +SYNOPSIS +-------- +[verse] +'git hook' run <hook-name> [-- <hook-args>] + +DESCRIPTION +----------- + +This command is an interface to git hooks (see linkgit:githooks[5]). +Currently it only provides a convenience wrapper for running hooks for +use by git itself. In the future it might gain other functionality. + +SUBCOMMANDS +----------- + +run:: + Run the `<hook-name>` hook. See linkgit:githooks[5] for + the hook names we support. ++ +Any positional arguments to the hook should be passed after an +optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The +arguments (if any) differ by hook name, see linkgit:githooks[5] for +what those are. + +SEE ALSO +-------- +linkgit:githooks[5] + +GIT +--- +Part of the linkgit:git[1] suite
Documentation/githooks.txt +4 -0
@@ -698,6 +698,10 @@ and "0" meaning they were not. Only one parameter should be set to "1" when the hook runs. The hook running passing "1", "1" should not be possible. +SEE ALSO +-------- +linkgit:git-hook[1] + GIT --- Part of the linkgit:git[1] suite
Makefile +1 -0
@@ -1115,6 +1115,7 @@ BUILTIN_OBJS += builtin/get-tar-commit-id.o BUILTIN_OBJS += builtin/grep.o BUILTIN_OBJS += builtin/hash-object.o BUILTIN_OBJS += builtin/help.o +BUILTIN_OBJS += builtin/hook.o BUILTIN_OBJS += builtin/index-pack.o BUILTIN_OBJS += builtin/init-db.o BUILTIN_OBJS += builtin/interpret-trailers.o
builtin.h +1 -0
@@ -164,6 +164,7 @@ int cmd_get_tar_commit_id(int argc, const char **argv, const char *prefix);int cmd_grep (int argc,const char **argv,const char *prefix);int cmd_hash_object (int argc,const char **argv,const char *prefix);int cmd_help (int argc,const char **argv,const char *prefix); +int cmd_hook (int argc,const char **argv,const char *prefix);int cmd_index_pack (int argc,const char **argv,const char *prefix);int cmd_init_db (int argc,const char **argv,const char *prefix);int cmd_interpret_trailers (int argc,const char **argv,const char *prefix);
builtin/hook.c new file +90 -0
90 lines not on this page · raw message
command-list.txt +1 -0
@@ -103,6 +103,7 @@ git-grep mainporcelain info git-gui mainporcelain git-hash-object plumbingmanipulators git-help ancillaryinterrogators complete +git-hook mainporcelain git-http-backend synchingrepositories git-http-fetch synchelpers git-http-push synchelpers
git.c +1 -0
@@ -538,6 +538,7 @@ static struct cmd_struct commands[] = { { "grep", cmd_grep, RUN_SETUP_GENTLY }, { "hash-object", cmd_hash_object }, { "help", cmd_help }, + { "hook", cmd_hook, RUN_SETUP }, { "index-pack", cmd_index_pack, RUN_SETUP_GENTLY | NO_PARSEOPT }, { "init", cmd_init_db }, { "init-db", cmd_init_db },
hook.c +101 -0
110 lines not on this page · raw message
hook.h +39 -0
@@ -1,5 +1,33 @@#ifndef HOOK_H #define HOOK_H +#include "strvec.h" + +struct hook { +/* The path to the hook */ +const char *hook_path; +}; + +struct run_hooks_opt +{ +/* Environment vars to be set for each hook */ +struct strvec env; + +/* Args to be passed to each hook */ +struct strvec args; +}; + +#define RUN_HOOKS_OPT_INIT { \ + .env = STRVEC_INIT, \ + .args = STRVEC_INIT, \ +} + +struct hook_cb_data { +/* rc reflects the cumulative failure state */ +int rc; +const char *hook_name; +struct hook *run_me; +struct run_hooks_opt *options; +};/** * Returns the path to the hook file, or NULL if the hook is missing
@@ -13,4 +41,15 @@ const char *find_hook(const char *name);*/ int hook_exists (const char *hookname); +/** +* Clear data from an initialized "struct run_hooks-opt". +*/ +void run_hooks_opt_clear (struct run_hooks_opt *o); + +/** +* Takes an already resolved hook found via find_hook() and runs +* it. Does not call run_hooks_opt_clear() for you. +*/ +int run_hooks (const char *hookname,const char *hook_path, +struct run_hooks_opt *options);#endif
t/t1800-hook.sh new file +128 -0
128 lines not on this page · raw message
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 9/36 gc: use hook library for pre-auto-gc hook
In reply to Ævar Arnfjörð BjarmasonFrom: Emily Shaffer <[email protected]>
Move the pre-auto-gc hook away from run-command.h to and over to the new hook.h library.
To do this introduce a simple run_hooks_oneshot() wrapper, we'll be using it extensively for these simple cases of wanting to run a single hook under a given name, and having it free the memory we allocate for us.
--- builtin/gc.c | 3 ++- hook.c | 23 +++++++++++++++++++++++ hook.h | 13 +++++++++++++ 3 files changed, 38 insertions(+), 1 deletion(-)
builtin/gc.c +2 -1
@@ -32,6 +32,7 @@#include "remote.h" #include "object-store.h" #include "exec-cmd.h" +#include "hook.h" #define FAILED_RUN "failed to run %s"
@@ -394,7 +395,7 @@ static int need_to_gc(void)else return 0 ; -if (run_hook_le (NULL ,"pre-auto-gc" ,NULL )) +if (run_hooks_oneshot ("pre-auto-gc" ,NULL ))return 0 ;return 1 ; }
hook.c +23 -0
@@ -141,3 +141,26 @@ int run_hooks(const char *hook_name, const char *hook_path,return cb_data.rc; } + +int run_hooks_oneshot (const char *hook_name,struct run_hooks_opt *options) +{ +const char *hook_path; +int ret; +struct run_hooks_opt hook_opt_scratch =RUN_HOOKS_OPT_INIT ; + +if (!options) + options = &hook_opt_scratch; + + hook_path =find_hook (hook_name); +if (!hook_path) { + ret =0 ; + goto cleanup; + } + + ret =run_hooks (hook_name, hook_path, options); + +cleanup: +run_hooks_opt_clear (options); + +return ret; +}
hook.h +13 -0
@@ -49,7 +49,20 @@ void run_hooks_opt_clear(struct run_hooks_opt *o);/** * Takes an already resolved hook found via find_hook() and runs * it. Does not call run_hooks_opt_clear() for you. +* +* See run_hooks_oneshot() for the simpler one-shot API. */ int run_hooks (const char *hookname,const char *hook_path,struct run_hooks_opt *options); + +/** +* Calls find_hook() on your "hook_name" and runs the hooks (if any) +* with run_hooks(). +* +* If "options" is provided calls run_hooks_opt_clear() on it for +* you. If "options" is NULL the default options from +* RUN_HOOKS_OPT_INIT will be used. +*/ +int run_hooks_oneshot (const char *hook_name,struct run_hooks_opt *options); +#endif
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 10/36 rebase: convert pre-rebase to use hook.h
In reply to Ævar Arnfjörð BjarmasonFrom: Emily Shaffer <[email protected]>
Move the pre-rebase hook away from run-command.h to and over to the new hook.h library.
--- builtin/rebase.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
builtin/rebase.c +4 -2
@@ -28,6 +28,7 @@#include "sequencer.h" #include "rebase-interactive.h" #include "reset.h" +#include "hook.h" #define DEFAULT_REFLOG_ACTION "rebase"
@@ -1313,6 +1314,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)char *squash_onto_name =NULL ;int reschedule_failed_exec =-1 ;int allow_preemptive_ff =1 ; +struct run_hooks_opt hook_opt =RUN_HOOKS_OPT_INIT ;struct option builtin_rebase_options[] = { OPT_STRING(0, "onto", &options.onto_name, N_("revision"),
@@ -2022,9 +2024,9 @@ int cmd_rebase(int argc, const char **argv, const char *prefix) }/* If a hook exists, give it a chance to interrupt*/ +strvec_pushl (&hook_opt.args, options.upstream_arg, argc ? argv[0 ] :NULL ,NULL );if (!ok_to_skip_pre_rebase && -run_hook_le (NULL ,"pre-rebase" , options.upstream_arg, - argc ? argv[0 ] :NULL ,NULL )) +run_hooks_oneshot ("pre-rebase" , &hook_opt))die (_ ("The pre-rebase hook refused to rebase." ));if (options.flags &REBASE_DIFFSTAT ) {
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 11/36 am: convert applypatch to use hook.h
In reply to Ævar Arnfjörð BjarmasonFrom: Emily Shaffer <[email protected]>
Teach pre-applypatch, post-applypatch, and applypatch-msg to use the hook.h library instead of the run-command.h library.
--- builtin/am.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
builtin/am.c +5 -3
@@ -446,9 +446,11 @@ static void am_destroy(const struct am_state *state)static int run_applypatch_msg_hook (struct am_state *state) {int ret; +struct run_hooks_opt opt =RUN_HOOKS_OPT_INIT ;assert (state->msg); - ret =run_hook_le (NULL ,"applypatch-msg" ,am_path (state,"final-commit" ),NULL ); +strvec_push (&opt.args,am_path (state,"final-commit" )); + ret =run_hooks_oneshot ("applypatch-msg" , &opt);if (!ret) {FREE_AND_NULL (state->msg);
@@ -1609,7 +1611,7 @@ static void do_commit(const struct am_state *state)const char *reflog_msg, *author, *committer =NULL ;struct strbuf sb =STRBUF_INIT ; -if (run_hook_le (NULL ,"pre-applypatch" ,NULL )) +if (run_hooks_oneshot ("pre-applypatch" ,NULL ))exit (1 );if (write_cache_as_tree (&tree,0 ,NULL ))
@@ -1661,7 +1663,7 @@ static void do_commit(const struct am_state *state)fclose (fp); } -run_hook_le (NULL ,"post-applypatch" ,NULL ); +run_hooks_oneshot ("post-applypatch" ,NULL );strbuf_release (&sb); }
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 12/36 hooks: convert 'post-checkout' hook to hook library
In reply to Ævar Arnfjörð BjarmasonFrom: Emily Shaffer <[email protected]>
Move the running of the 'post-checkout' hook away from run-command.h to the new hook.h library. For "worktree" this requires a change to it to run the hooks from a given directory.
We could strictly speaking skip the "absolute_path" flag and just check if "dir" is specified, but let's split them up for clarity, as well as for any future user who'd like to set "dir" but not implicitly change the argument to an absolute path.
--- builtin/checkout.c | 14 +++++++++----- builtin/clone.c | 6 ++++-- builtin/worktree.c | 28 ++++++++++++---------------- hook.c | 9 +++++++++ hook.h | 9 +++++++++ read-cache.c | 1 + reset.c | 14 ++++++++++---- 7 files changed, 54 insertions(+), 27 deletions(-)
builtin/checkout.c +9 -5
@@ -9,6 +9,7 @@#include "config.h" #include "diff.h" #include "dir.h" +#include "hook.h" #include "ll-merge.h" #include "lockfile.h" #include "merge-recursive.h"
@@ -106,13 +107,16 @@ struct branch_info {static int post_checkout_hook (struct commit *old_commit,struct commit *new_commit,int changed) { -return run_hook_le (NULL ,"post-checkout" , -oid_to_hex (old_commit ? &old_commit->object.oid :null_oid ()), -oid_to_hex (new_commit ? &new_commit->object.oid :null_oid ()), - changed ?"1" :"0" ,NULL ); +struct run_hooks_opt opt =RUN_HOOKS_OPT_INIT ; +/* "new_commit" can be NULL when checking out from the index before a commit exists. */ - +strvec_pushl (&opt.args, +oid_to_hex (old_commit ? &old_commit->object.oid :null_oid ()), +oid_to_hex (new_commit ? &new_commit->object.oid :null_oid ()), + changed ?"1" :"0" , +NULL ); +return run_hooks_oneshot ("post-checkout" , &opt); } static int update_some(const struct object_id *oid, struct strbuf *base,
builtin/clone.c +4 -2
@@ -32,6 +32,7 @@#include "connected.h" #include "packfile.h" #include "list-objects-filter-options.h" +#include "hook.h" /* * Overall FIXMEs:
@@ -775,6 +776,7 @@ static int checkout(int submodule_progress)struct tree *tree;struct tree_desc t;int err =0 ; +struct run_hooks_opt hook_opt =RUN_HOOKS_OPT_INIT ;if (option_no_checkout)return 0 ;
@@ -820,8 +822,8 @@ static int checkout(int submodule_progress)if (write_locked_index (&the_index, &lock_file,COMMIT_LOCK ))die (_ ("unable to write new index file" )); - err |=run_hook_le (NULL ,"post-checkout" ,oid_to_hex (null_oid ()), -oid_to_hex (&oid),"1" ,NULL ); +strvec_pushl (&hook_opt.args,oid_to_hex (null_oid ()),oid_to_hex (&oid),"1" ,NULL ); + err |=run_hooks_oneshot ("post-checkout" , &hook_opt);if (!err && (option_recurse_submodules.nr >0 )) {struct strvec args =STRVEC_INIT ;
builtin/worktree.c +12 -16
@@ -382,22 +382,18 @@ static int add_worktree(const char *path, const char *refname,* is_junk is cleared, but do return appropriate code when hook fails. */ if (!ret && opts->checkout) { -const char *hook =find_hook ("post-checkout" ); -if (hook) { -const char *env[] = {"GIT_DIR" ,"GIT_WORK_TREE" ,NULL }; - cp.git_cmd =0 ; - cp.no_stdin =1 ; - cp.stdout_to_stderr =1 ; - cp.dir = path; - cp.env = env; - cp.argv =NULL ; - cp.trace2_hook_name ="post-checkout" ; -strvec_pushl (&cp.args,absolute_path (hook), -oid_to_hex (null_oid ()), -oid_to_hex (&commit->object.oid), -"1" ,NULL ); - ret =run_command (&cp); - } +struct run_hooks_opt opt =RUN_HOOKS_OPT_INIT ; + +strvec_pushl (&opt.env,"GIT_DIR" ,"GIT_WORK_TREE" ,NULL ); +strvec_pushl (&opt.args, +oid_to_hex (null_oid ()), +oid_to_hex (&commit->object.oid), +"1" , +NULL ); + opt.dir = path; + opt.absolute_path =1 ; + + ret =run_hooks_oneshot ("post-checkout" , &opt); }strvec_clear (&child_env);
hook.c +9 -0
@@ -63,6 +63,7 @@ static int pick_next_hook(struct child_process *cp, cp->env = hook_cb->options->env.v; cp->stdout_to_stderr =1 ; cp->trace2_hook_name = hook_cb->hook_name; + cp->dir = hook_cb->options->dir;/* add command */ strvec_push (&cp->args, run_me->hook_path);
@@ -116,6 +117,7 @@ static int notify_hook_finished(int result,int run_hooks (const char *hook_name,const char *hook_path,struct run_hooks_opt *options) { +struct strbuf abs_path =STRBUF_INIT ;struct hook my_hook = { .hook_path = hook_path, };
@@ -129,6 +131,10 @@ int run_hooks(const char *hook_name, const char *hook_path,if (!options)BUG ("a struct run_hooks_opt must be provided to run_hooks" ); +if (options->absolute_path) { +strbuf_add_absolute_path (&abs_path, hook_path); + my_hook.hook_path = abs_path.buf; + } cb_data.run_me = &my_hook; run_processes_parallel_tr2(jobs,
@@ -139,6 +145,9 @@ int run_hooks(const char *hook_name, const char *hook_path,"hook" , hook_name); +if (options->absolute_path) +strbuf_release (&abs_path); +return cb_data.rc; }
hook.h +9 -0
@@ -14,6 +14,15 @@ struct run_hooks_opt/* Args to be passed to each hook */ struct strvec args; + +/* +* Resolve and run the "absolute_path(hook)" instead of +* "hook". Used for "git worktree" hooks +*/ +int absolute_path; + +/* Path to initial working directory for subprocess */ +const char *dir; };#define RUN_HOOKS_OPT_INIT { \
read-cache.c +1 -0
@@ -28,6 +28,7 @@#include "sparse-index.h" #include "csum-file.h" #include "promisor-remote.h" +#include "hook.h" /* Mask for the name length in ce_flags in the on-disk index */
reset.c +10 -4
@@ -7,6 +7,7 @@#include "tree-walk.h" #include "tree.h" #include "unpack-trees.h" +#include "hook.h" int reset_head(struct repository *r, struct object_id *oid, const char *action, const char *switch_to_branch, unsigned flags,
@@ -125,10 +126,15 @@ int reset_head(struct repository *r, struct object_id *oid, const char *action, ret =create_symref ("HEAD" , switch_to_branch, reflog_head); } -if (run_hook) -run_hook_le (NULL ,"post-checkout" , -oid_to_hex (orig ? orig :null_oid ()), -oid_to_hex (oid),"1" ,NULL ); +if (run_hook) { +struct run_hooks_opt opt =RUN_HOOKS_OPT_INIT ; +strvec_pushl (&opt.args, +oid_to_hex (orig ? orig :null_oid ()), +oid_to_hex (oid), +"1" , +NULL ); +run_hooks_oneshot ("post-checkout" , &opt); + } leave_reset_head:strbuf_release (&msg);
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 13/36 merge: convert post-merge to use hook.h
In reply to Ævar Arnfjörð BjarmasonFrom: Emily Shaffer <[email protected]>
Teach post-merge to use the hook.h library instead of the run-command.h library to run hooks.
--- builtin/merge.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
builtin/merge.c +3 -1
@@ -448,6 +448,7 @@ static void finish(struct commit *head_commit, const struct object_id *new_head, const char *msg) { struct strbuf reflog_message = STRBUF_INIT; +struct run_hooks_opt opt =RUN_HOOKS_OPT_INIT ;const struct object_id *head = &head_commit->object.oid;if (!msg)
@@ -489,7 +490,8 @@ static void finish(struct commit *head_commit, }/* Run a post-merge hook */ -run_hook_le (NULL ,"post-merge" , squash ?"1" :"0" ,NULL ); +strvec_push (&opt.args, squash ?"1" :"0" ); +run_hooks_oneshot ("post-merge" , &opt);apply_autostash (git_path_merge_autostash (the_repository));strbuf_release (&reflog_message);
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 14/36 git hook run: add an --ignore-missing flag
In reply to Ævar Arnfjörð BjarmasonFor certain one-shot hooks we'd like to optimistically run them, and not complain if they don't exist. This will be used by send-email in a subsequent commit.
--- Documentation/git-hook.txt | 10 +++++++++- builtin/hook.c | 10 ++++++++-- t/t1800-hook.sh | 5 +++++ 3 files changed, 22 insertions(+), 3 deletions(-)
Documentation/git-hook.txt +9 -1
@@ -8,7 +8,7 @@ git-hook - run git hooks SYNOPSIS -------- [verse] -'git hook' run <hook-name> [-- <hook-args>] +'git hook' run [--ignore-missing] <hook-name> [-- <hook-args>] DESCRIPTION -----------
@@ -29,6 +29,14 @@ optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The arguments (if any) differ by hook name, see linkgit:githooks[5] for what those are. +OPTIONS +------- + +--ignore-missing:: + Ignore any missing hook by quietly returning zero. Used for + tools that want to do a blind one-shot run of a hook that may + or may not be present. + SEE ALSO -------- linkgit:githooks[5]
builtin/hook.c +8 -2
@@ -7,7 +7,7 @@#include "strvec.h" #define BUILTIN_HOOK_RUN_USAGE \ - N_("git hook run <hook-name> [-- <hook-args>]") + N_("git hook run [--ignore-missing] <hook-name> [-- <hook-args>]")static const char *const builtin_hook_usage[] = {BUILTIN_HOOK_RUN_USAGE ,
@@ -23,9 +23,12 @@ static int run(int argc, const char **argv, const char *prefix) {int i;struct run_hooks_opt opt =RUN_HOOKS_OPT_INIT ; +int ignore_missing =0 ;const char *hook_name;const char *hook_path;struct option run_options[] = { +OPT_BOOL (0 ,"ignore-missing" , &ignore_missing, +N_ ("exit quietly with a zero exit code if the requested hook cannot be found" )),OPT_END (), };int ret;
@@ -55,9 +58,12 @@ static int run(int argc, const char **argv, const char *prefix)/* * We are not using a plain run_hooks() because we'd like to * detect missing hooks. Let's find it ourselves and call -* run_hooks() instead. +* run_hooks() instead... */ hook_name = argv[0 ]; +if (ignore_missing) +/* ... act like a plain run_hooks() under --ignore-missing */ +return run_hooks_oneshot (hook_name, &opt); hook_path =find_hook (hook_name);if (!hook_path) {error ("cannot find a hook named %s" , hook_name);
t/t1800-hook.sh +5 -0
@@ -20,6 +20,11 @@ test_expect_success 'git hook run: nonexistent hook' ' test_cmp stderr.expect stderr.actual ' +test_expect_success 'git hook run: nonexistent hook with --ignore-missing' ' + git hook run --ignore-missing does-not-exist 2>stderr.actual && + test_must_be_empty stderr.actual +' + test_expect_success 'git hook run: basic' ' write_script .git/hooks/test-hook <<-EOF && echo Test hook
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 15/36 send-email: use 'git hook run' for 'sendemail-validate'
In reply to Ævar Arnfjörð BjarmasonFrom: Emily Shaffer <[email protected]>
Change the "sendmail-validate" hook to be run via the "git hook run" wrapper instead of via a direct invocation.
This is the smallest possibly change to get "send-email" using "git hook run". We still check the hook itself with "-x", and set a "GIT_DIR" variable, both of which are asserted by our tests. We'll need to get rid of this special behavior if we start running N hooks, but for now let's be as close to bug-for-bug compatible as possible.
--- git-send-email.perl | 20 ++++++++++++-------- t/t9001-send-email.sh | 4 ++-- 2 files changed, 14 insertions(+), 10 deletions(-)
git-send-email.perl +12 -8
@@ -195,13 +195,13 @@ sub format_2822_time { my $editor; sub system_or_msg { - my ($args, $msg) = @_; + my ($args, $msg, $cmd_name) = @_; system(@$args); my $signalled = $? & 127; my $exit_code = $? >> 8; return unless $signalled or $exit_code; - my @sprintf_args = ($args->[0], $exit_code); + my @sprintf_args = ($cmd_name ? $cmd_name : $args->[0], $exit_code); if (defined $msg) { # Quiet the 'redundant' warning category, except we # need to support down to Perl 5.8, so we can't do a
@@ -2031,10 +2031,10 @@ sub validate_patch { my ($fn, $xfer_encoding) = @_; if ($repo) { + my $hook_name = 'sendemail-validate'; my $hooks_path = $repo->command_oneline('rev-parse', '--git-path', 'hooks'); require File::Spec; - my $validate_hook = File::Spec->catfile($hooks_path, - 'sendemail-validate'); + my $validate_hook = File::Spec->catfile($hooks_path, $hook_name); my $hook_error; if (-x $validate_hook) { require Cwd;
@@ -2044,13 +2044,17 @@ sub validate_patch { chdir($repo->wc_path() or $repo->repo_path()) or die("chdir: $!"); local $ENV{"GIT_DIR"} = $repo->repo_path(); - $hook_error = system_or_msg([$validate_hook, $target]); + my @validate_hook = ("git", "hook", "run", "--ignore-missing", $hook_name, "--", $target); + $hook_error = system_or_msg(\@validate_hook, undef, + "git hook run $hook_name -- <patch>"); chdir($cwd_save) or die("chdir: $!"); } if ($hook_error) { - die sprintf(__("fatal: %s: rejected by sendemail-validate hook\n" . - "%s\n" . - "warning: no patches were sent\n"), $fn, $hook_error); + $hook_error = sprintf(__("fatal: %s: rejected by %s hook\n" . + $hook_error . "\n" . + "warning: no patches were sent\n"), + $fn, $hook_name); + die $hook_error; } }
t/t9001-send-email.sh +2 -2
@@ -539,7 +539,7 @@ test_expect_success $PREREQ "--validate respects relative core.hooksPath path" ' test_path_is_file my-hooks.ran && cat >expect <<-EOF && fatal: longline.patch: rejected by sendemail-validate hook - fatal: command '"'"'my-hooks/sendemail-validate'"'"' died with exit code 1 + fatal: command '"'"'git hook run sendemail-validate -- <patch>'"'"' died with exit code 1 warning: no patches were sent EOF test_cmp expect actual
@@ -558,7 +558,7 @@ test_expect_success $PREREQ "--validate respects absolute core.hooksPath path" ' test_path_is_file my-hooks.ran && cat >expect <<-EOF && fatal: longline.patch: rejected by sendemail-validate hook - fatal: command '"'"'$hooks_path/sendemail-validate'"'"' died with exit code 1 + fatal: command '"'"'git hook run sendemail-validate -- <patch>'"'"' died with exit code 1 warning: no patches were sent EOF test_cmp expect actual
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 16/36 git-p4: use 'git hook' to run hooks
In reply to Ævar Arnfjörð BjarmasonFrom: Emily Shaffer <[email protected]>
Instead of duplicating the behavior of run-command.h:run_hook_le() in Python, we can directly call 'git hook run'. We emulate the existence check with the --ignore-missing flag.
As this is the last hook execution in git.git to not go through "git hook run" or the hook.[ch] library we can now be absolutely sure that our assertion in hook.c that only hooks known by the generated (from githooks(5)) hook-list.h are permitted.
--- git-p4.py | 72 ++++++------------------------------------------------- 1 file changed, 7 insertions(+), 65 deletions(-)
git-p4.py +7 -65
@@ -207,71 +207,13 @@ def decode_path(path): return path def run_git_hook(cmd, param=[]): - """Execute a hook if the hook exists.""" - if verbose: - sys.stderr.write("Looking for hook: %s\n" % cmd) - sys.stderr.flush() - - hooks_path = gitConfig("core.hooksPath") - if len(hooks_path) <= 0: - hooks_path = os.path.join(os.environ["GIT_DIR"], "hooks") - - if not isinstance(param, list): - param=[param] - - # resolve hook file name, OS depdenent - hook_file = os.path.join(hooks_path, cmd) - if platform.system() == 'Windows': - if not os.path.isfile(hook_file): - # look for the file with an extension - files = glob.glob(hook_file + ".*") - if not files: - return True - files.sort() - hook_file = files.pop() - while hook_file.upper().endswith(".SAMPLE"): - # The file is a sample hook. We don't want it - if len(files) > 0: - hook_file = files.pop() - else: - return True - - if not os.path.isfile(hook_file) or not os.access(hook_file, os.X_OK): - return True - - return run_hook_command(hook_file, param) == 0 - -def run_hook_command(cmd, param): - """Executes a git hook command - cmd = the command line file to be executed. This can be - a file that is run by OS association. - - param = a list of parameters to pass to the cmd command - - On windows, the extension is checked to see if it should - be run with the Git for Windows Bash shell. If there - is no file extension, the file is deemed a bash shell - and will be handed off to sh.exe. Otherwise, Windows - will be called with the shell to handle the file assocation. - - For non Windows operating systems, the file is called - as an executable. - """ - cli = [cmd] + param - use_shell = False - if platform.system() == 'Windows': - (root,ext) = os.path.splitext(cmd) - if ext == "": - exe_path = os.environ.get("EXEPATH") - if exe_path is None: - exe_path = "" - else: - exe_path = os.path.join(exe_path, "bin") - cli = [os.path.join(exe_path, "SH.EXE")] + cli - else: - use_shell = True - return subprocess.call(cli, shell=use_shell) - + """args are specified with -a <arg> -a <arg> -a <arg>""" + args = ['git', 'hook', 'run', '--ignore-missing', cmd] + if param: + args.append("--") + for p in param: + args.append(p) + return subprocess.call(args) == 0 def write_pipe(c, stdin): if verbose:
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 17/36 commit: convert {pre-commit,prepare-commit-msg} hook to hook.h
In reply to Ævar Arnfjörð BjarmasonFrom: Emily Shaffer <[email protected]>
Move these hooks hook away from run-command.h to and over to the new hook.h library.
--- commit.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-)
commit.c +8 -7
@@ -21,6 +21,7 @@#include "commit-reach.h" #include "run-command.h" #include "shallow.h" +#include "hook.h" static struct commit_extra_header *read_commit_extra_header_lines (const char *buf,size_t len,const char **);
@@ -1698,22 +1699,22 @@ size_t ignore_non_trailer(const char *buf, size_t len)int run_commit_hook (int editor_is_used,const char *index_file,const char *name, ...) { -struct strvec hook_env =STRVEC_INIT ; +struct run_hooks_opt opt =RUN_HOOKS_OPT_INIT ;va_list args; -int ret; +const char *arg; -strvec_pushf (&hook_env,"GIT_INDEX_FILE=%s" , index_file); +strvec_pushf (&opt.env,"GIT_INDEX_FILE=%s" , index_file);/* * Let the hook know that no editor will be launched. */ if (!editor_is_used) -strvec_push (&hook_env,"GIT_EDITOR=:" ); +strvec_push (&opt.env,"GIT_EDITOR=:" );va_start (args, name); - ret =run_hook_ve (hook_env.v, name, args); + while ((arg = va_arg(args, const char *))) +strvec_push (&opt.args, arg);va_end (args); -strvec_clear (&hook_env); -return ret; +return run_hooks_oneshot (name, &opt); }
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 18/36 read-cache: convert post-index-change to use hook.h
In reply to Ævar Arnfjörð BjarmasonFrom: Emily Shaffer <[email protected]>
Move the post-index-change hook away from run-command.h to and over to the new hook.h library.
This removes the last direct user of run_hook_ve(), so we can make the function static now. It'll be removed entirely soon.
--- read-cache.c | 10 +++++++--- run-command.c | 2 +- run-command.h | 1 - 3 files changed, 8 insertions(+), 5 deletions(-)
read-cache.c +7 -3
@@ -3069,6 +3069,7 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l {int ret;int was_full = !istate->sparse_index; +struct run_hooks_opt hook_opt =RUN_HOOKS_OPT_INIT ; ret =convert_to_sparse (istate);
@@ -3097,9 +3098,12 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *lelse ret =close_lock_file_gently (lock); -run_hook_le (NULL ,"post-index-change" , - istate->updated_workdir ?"1" :"0" , - istate->updated_skipworktree ?"1" :"0" ,NULL ); +strvec_pushl (&hook_opt.args, + istate->updated_workdir ?"1" :"0" , + istate->updated_skipworktree ?"1" :"0" , +NULL ); +run_hooks_oneshot ("post-index-change" , &hook_opt); + istate->updated_workdir =0 ; istate->updated_skipworktree =0 ;
run-command.c +1 -1
@@ -1320,7 +1320,7 @@ int async_with_fork(void)#endif } -int run_hook_ve (const char *const *env,const char *name,va_list args) +static int run_hook_ve (const char *const *env,const char *name,va_list args) {struct child_process hook =CHILD_PROCESS_INIT ;const char *p;
run-command.h +0 -1
@@ -219,7 +219,6 @@ int run_command(struct child_process *);*/ LAST_ARG_MUST_BE_NULL int run_hook_le(const char *const *env, const char *name, ...); -int run_hook_ve (const char *const *env,const char *name,va_list args);/* * Trigger an auto-gc
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 19/36 receive-pack: convert push-to-checkout hook to hook.h
In reply to Ævar Arnfjörð BjarmasonFrom: Emily Shaffer <[email protected]>
Move the push-to-checkout hook away from run-command.h to and over to the new hook.h library.
This removes the last direct user of run_hook_le(), so we could remove that function now, but let's leave that to a follow-up cleanup commit.
--- builtin/receive-pack.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
builtin/receive-pack.c +5 -2
@@ -1435,9 +1435,12 @@ static const char *push_to_checkout(unsigned char *hash,struct strvec *env, const char *work_tree) { + struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT; +strvec_pushf (env,"GIT_WORK_TREE=%s" ,absolute_path (work_tree)); -if (run_hook_le (env->v, push_to_checkout_hook, -hash_to_hex (hash),NULL )) +strvec_pushv (&opt.env, env->v); +strvec_push (&opt.args,hash_to_hex (hash)); +if (run_hooks_oneshot (push_to_checkout_hook, &opt))return "push-to-checkout hook declined" ;else return NULL ;
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 20/36 run-command: remove old run_hook_{le,ve}() hook API
In reply to Ævar Arnfjörð BjarmasonFrom: Emily Shaffer <[email protected]>
The new hook.h library has replaced all run-command.h hook-related functionality. So let's delete this dead code.
--- run-command.c | 32 -------------------------------- run-command.h | 16 ---------------- 2 files changed, 48 deletions(-)
run-command.c +0 -32
@@ -1320,38 +1320,6 @@ int async_with_fork(void)#endif } -static int run_hook_ve (const char *const *env,const char *name,va_list args) -{ -struct child_process hook =CHILD_PROCESS_INIT ; -const char *p; - - p =find_hook (name); -if (!p) -return 0 ; - -strvec_push (&hook.args, p); - while ((p = va_arg(args, const char *))) -strvec_push (&hook.args, p); - hook.env = env; - hook.no_stdin =1 ; - hook.stdout_to_stderr =1 ; - hook.trace2_hook_name = name; - -return run_command (&hook); -} - -int run_hook_le (const char *const *env,const char *name, ...) -{ -va_list args; -int ret; - -va_start (args, name); - ret =run_hook_ve (env, name, args); -va_end (args); - -return ret; -} -struct io_pump {/* initialized by caller */ int fd;
run-command.h +0 -16
@@ -204,22 +204,6 @@ int finish_command_in_signal(struct child_process *);*/ int run_command (struct child_process *); -/** -* Run a hook. -* The first argument is a pathname to an index file, or NULL -* if the hook uses the default index file or no index is needed. -* The second argument is the name of the hook. -* The further arguments correspond to the hook arguments. -* The last argument has to be NULL to terminate the arguments list. -* If the hook does not exist or is not executable, the return -* value will be zero. -* If it is executable, the hook will be executed and the exit -* status of the hook is returned. -* On execution, .stdout_to_stderr and .no_stdin will be set. -*/ -LAST_ARG_MUST_BE_NULL -int run_hook_le(const char *const *env, const char *name, ...); -/* * Trigger an auto-gc */
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 21/36 run-command: allow stdin for run_processes_parallel
In reply to Ævar Arnfjörð BjarmasonFrom: Emily Shaffer <[email protected]>
While it makes sense not to inherit stdin from the parent process to avoid deadlocking, it's not necessary to completely ban stdin to children. An informed user should be able to configure stdin safely. By setting some_child.process.no_stdin=1 before calling get_next_task() we provide a reasonable default behavior but enable users to set up stdin streaming for themselves during the callback.
some_child.process.stdout_to_stderr, however, remains unmodifiable by get_next_task() - the rest of the run_processes_parallel() API depends on child output in stderr.
--- run-command.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
run-command.c +8 -1
@@ -1627,6 +1627,14 @@ static int pp_start_one(struct parallel_processes *pp)if (i == pp->max_processes)BUG ("bookkeeping is hard" ); +/* +* By default, do not inherit stdin from the parent process - otherwise, +* all children would share stdin! Users may overwrite this to provide +* something to the child's stdin by having their 'get_next_task' +* callback assign 0 to .no_stdin and an appropriate integer to .in. +*/ + pp->children[i].process.no_stdin =1 ; + code = pp->get_next_task(&pp->children[i].process, &pp->children[i].err, pp->data,
@@ -1638,7 +1646,6 @@ static int pp_start_one(struct parallel_processes *pp) } pp->children[i].process.err =-1 ; pp->children[i].process.stdout_to_stderr =1 ; - pp->children[i].process.no_stdin =1 ;if (start_command (&pp->children[i].process)) { code = pp->start_failure(&pp->children[i].err,
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 22/36 hook: support passing stdin to hooks
In reply to Ævar Arnfjörð BjarmasonFrom: Emily Shaffer <[email protected]>
Some hooks (such as post-rewrite) need to take input via stdin. Previously, callers provided stdin to hooks by setting run-command.h:child_process.in, which takes a FD. Callers would open the file in question themselves before calling run-command(). However, since we will now need to seek to the front of the file and read it again for every hook which runs, hook.h:run_command() takes a path and handles FD management itself. Since this file is opened for read only, it should not prevent later parallel execution support.
On the frontend, this is supported by asking for a file path, rather than by reading stdin. Reading directly from stdin would involve caching the entire stdin (to memory or to disk) and reading it back from the beginning to each hook. We'd want to support cases like insufficient memory or storage for the file. While this may prove useful later, for now the path of least resistance is to just ask the user to make this interim file themselves.
--- Documentation/git-hook.txt | 7 ++++++- builtin/hook.c | 4 +++- hook.c | 8 +++++++- hook.h | 3 +++ t/t1800-hook.sh | 18 ++++++++++++++++++ 5 files changed, 37 insertions(+), 3 deletions(-)
Documentation/git-hook.txt +6 -1
@@ -8,7 +8,7 @@ git-hook - run git hooks SYNOPSIS -------- [verse] -'git hook' run [--ignore-missing] <hook-name> [-- <hook-args>] +'git hook' run [--to-stdin=<path>] [--ignore-missing] <hook-name> [-- <hook-args>] DESCRIPTION -----------
@@ -32,6 +32,11 @@ what those are. OPTIONS ------- +--to-stdin:: + For "run"; Specify a file which will be streamed into the + hook's stdin. The hook will receive the entire file from + beginning to EOF. + --ignore-missing:: Ignore any missing hook by quietly returning zero. Used for tools that want to do a blind one-shot run of a hook that may
builtin/hook.c +3 -1
@@ -7,7 +7,7 @@#include "strvec.h" #define BUILTIN_HOOK_RUN_USAGE \ - N_("git hook run [--ignore-missing] <hook-name> [-- <hook-args>]") + N_("git hook run [--ignore-missing] [--to-stdin=<path>] <hook-name> [-- <hook-args>]")static const char *const builtin_hook_usage[] = {BUILTIN_HOOK_RUN_USAGE ,
@@ -29,6 +29,8 @@ static int run(int argc, const char **argv, const char *prefix)struct option run_options[] = {OPT_BOOL (0 ,"ignore-missing" , &ignore_missing,N_ ("exit quietly with a zero exit code if the requested hook cannot be found" )), +OPT_STRING (0 ,"to-stdin" , &opt.path_to_stdin,N_ ("path" ), +N_ ("file to read into hooks' stdin" )),OPT_END (), };int ret;
hook.c +7 -1
@@ -59,7 +59,13 @@ static int pick_next_hook(struct child_process *cp,if (!run_me)return 0 ; - cp->no_stdin =1 ; +/* reopen the file for stdin; run_command closes it. */ +if (hook_cb->options->path_to_stdin) { + cp->no_stdin =0 ; + cp->in =xopen (hook_cb->options->path_to_stdin,O_RDONLY ); + }else { + cp->no_stdin =1 ; + } cp->env = hook_cb->options->env.v; cp->stdout_to_stderr =1 ; cp->trace2_hook_name = hook_cb->hook_name;
hook.h +3 -0
@@ -23,6 +23,9 @@ struct run_hooks_opt/* Path to initial working directory for subprocess */ const char *dir; + +/* Path to file which should be piped to stdin for each hook */ +const char *path_to_stdin; };#define RUN_HOOKS_OPT_INIT { \
t/t1800-hook.sh +18 -0
@@ -130,4 +130,22 @@ test_expect_success 'git -c core.hooksPath=<PATH> hook run' ' test_cmp expect actual ' +test_expect_success 'stdin to hooks' ' + write_script .git/hooks/test-hook <<-\EOF && + echo BEGIN stdin + cat + echo END stdin + EOF + + cat >expect <<-EOF && + BEGIN stdin + hello + END stdin + EOF + + echo hello >input && + git hook run --to-stdin=input test-hook 2>actual && + test_cmp expect actual +' + test_done
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 23/36 am: convert 'post-rewrite' hook to hook.h
In reply to Ævar Arnfjörð BjarmasonFrom: Emily Shaffer <[email protected]>
--- builtin/am.c | 20 ++++---------------- 1 file changed, 4 insertions(+), 16 deletions(-)
builtin/am.c +4 -16
@@ -467,24 +467,12 @@ static int run_applypatch_msg_hook(struct am_state *state)*/ static int run_post_rewrite_hook (const struct am_state *state) { -struct child_process cp =CHILD_PROCESS_INIT ; -const char *hook =find_hook ("post-rewrite" ); -int ret; - -if (!hook) -return 0 ; - -strvec_push (&cp.args, hook); -strvec_push (&cp.args,"rebase" ); - - cp.in =xopen (am_path (state,"rewritten" ),O_RDONLY ); - cp.stdout_to_stderr =1 ; - cp.trace2_hook_name ="post-rewrite" ; +struct run_hooks_opt opt =RUN_HOOKS_OPT_INIT ; - ret =run_command (&cp); +strvec_push (&opt.args,"rebase" ); + opt.path_to_stdin =am_path (state,"rewritten" ); -close (cp.in); -return ret; +return run_hooks_oneshot ("post-rewrite" , &opt); }/**
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 24/36 run-command: add stdin callback for parallelization
In reply to Ævar Arnfjörð BjarmasonFrom: Emily Shaffer <[email protected]>
If a user of the run_processes_parallel() API wants to pipe a large amount of information to stdin of each parallel command, that information could exceed the buffer of the pipe allocated for that process's stdin. Generally this is solved by repeatedly writing to child_process.in between calls to start_command() and finish_command(); run_processes_parallel() did not provide users an opportunity to access child_process at that time.
Because the data might be extremely large (for example, a list of all refs received during a push from a client) simply taking a string_list or strbuf is not as scalable as using a callback; the rest of the run_processes_parallel() API also uses callbacks, so making this feature match the rest of the API reduces mental load on the user.
--- builtin/fetch.c | 1 + builtin/submodule--helper.c | 2 +- hook.c | 1 + run-command.c | 54 +++++++++++++++++++++++++++++++++++-- run-command.h | 17 +++++++++++- submodule.c | 1 + t/helper/test-run-command.c | 31 ++++++++++++++++++--- t/t0061-run-command.sh | 30 +++++++++++++++++++++ 8 files changed, 129 insertions(+), 8 deletions(-)
builtin/fetch.c +1 -0
@@ -1819,6 +1819,7 @@ static int fetch_multiple(struct string_list *list, int max_children) result =run_processes_parallel_tr2 (max_children, &fetch_next_remote, &fetch_failed_to_start, +NULL , &fetch_finished, &state,"fetch" ,"parallel/fetch" );
builtin/submodule--helper.c +1 -1
@@ -2311,7 +2311,7 @@ static int update_submodules(struct submodule_update_clone *suc)int i;run_processes_parallel_tr2 (suc->max_jobs, update_clone_get_next_task, - update_clone_start_failure, + update_clone_start_failure,NULL , update_clone_task_finished, suc,"submodule" ,"parallel/update" );
hook.c +1 -0
@@ -146,6 +146,7 @@ int run_hooks(const char *hook_name, const char *hook_path, run_processes_parallel_tr2(jobs, pick_next_hook, notify_start_failure, + NULL, notify_hook_finished, &cb_data, "hook",
run-command.c +52 -2
125 lines not on this page · raw message
run-command.h +16 -1
@@ -422,6 +422,20 @@ typedef int (*start_failure_fn)(struct strbuf *out,void *pp_cb, void *pp_task_cb); +/** +* This callback is called repeatedly on every child process who requests +* start_command() to create a pipe by setting child_process.in < 0. +* +* pp_cb is the callback cookie as passed into run_processes_parallel, and +* pp_task_cb is the callback cookie as passed into get_next_task_fn. +* The contents of 'send' will be read into the pipe and passed to the pipe. +* +* Return nonzero to close the pipe. +*/ +typedef int (*feed_pipe_fn )(struct strbuf *pipe, +void *pp_cb, +void *pp_task_cb); +/** * This callback is called on every child process that finished processing. *
@@ -456,10 +470,11 @@ typedef int (*task_finished_fn)(int result,int run_processes_parallel (int n,get_next_task_fn ,start_failure_fn , +feed_pipe_fn ,task_finished_fn ,void *pp_cb);int run_processes_parallel_tr2 (int n,get_next_task_fn ,start_failure_fn , -task_finished_fn ,void *pp_cb, +feed_pipe_fn ,task_finished_fn ,void *pp_cb,const char *tr2_category,const char *tr2_label);/**
submodule.c +1 -0
@@ -1632,6 +1632,7 @@ int fetch_populated_submodules(struct repository *r,run_processes_parallel_tr2 (max_parallel_jobs, get_next_submodule, fetch_start_failure, +NULL , fetch_finish, &spf,"submodule" ,"parallel/fetch" );
t/helper/test-run-command.c +27 -4
@@ -32,8 +32,13 @@ static int parallel_next(struct child_process *cp,return 0 ;strvec_pushv (&cp->args, d->argv); + cp->in = d->in; + cp->no_stdin = d->no_stdin;strbuf_addstr (err,"preloaded output of a child\n" ); number_callbacks++; + + *task_cb =xmalloc (sizeof (int )); + *(int *)(*task_cb) =2 ;return 1 ; }
@@ -55,6 +60,17 @@ static int task_finished(int result,return 1 ; } +static int test_stdin (struct strbuf *pipe,void *cb,void *task_cb) +{ +int *lines_remaining = task_cb; + +if (*lines_remaining) +strbuf_addf (pipe,"sample stdin %d\n" , --(*lines_remaining)); + +return !(*lines_remaining); +} + +struct testsuite {struct string_list tests, failed;int next;
@@ -185,7 +201,7 @@ static int testsuite(int argc, const char **argv) suite.tests.nr, max_jobs); ret =run_processes_parallel (max_jobs, next_test, test_failed, - test_finished, &suite); + test_stdin, test_finished, &suite);if (suite.failed.nr >0 ) { ret =1 ;
@@ -413,15 +429,22 @@ int cmd__run_command(int argc, const char **argv)if (!strcmp (argv[1 ],"run-command-parallel" ))exit (run_processes_parallel (jobs, parallel_next, -NULL ,NULL , &proc)); +NULL ,NULL ,NULL , &proc));if (!strcmp (argv[1 ],"run-command-abort" ))exit (run_processes_parallel (jobs, parallel_next, -NULL , task_finished, &proc)); +NULL ,NULL , task_finished, &proc));if (!strcmp (argv[1 ],"run-command-no-jobs" ))exit (run_processes_parallel (jobs, no_job, -NULL , task_finished, &proc)); +NULL ,NULL , task_finished, &proc)); + +if (!strcmp (argv[1 ],"run-command-stdin" )) { + proc.in =-1 ; + proc.no_stdin =0 ; +exit (run_processes_parallel (jobs, parallel_next,NULL , + test_stdin,NULL , &proc)); + }fprintf (stderr,"check usage\n" );return 1 ;
t/t0061-run-command.sh +30 -0
@@ -143,6 +143,36 @@ test_expect_success 'run_command runs in parallel with more tasks than jobs avai test_cmp expect actual ' +cat >expect <<-EOF +preloaded output of a child +listening for stdin: +sample stdin 1 +sample stdin 0 +preloaded output of a child +listening for stdin: +sample stdin 1 +sample stdin 0 +preloaded output of a child +listening for stdin: +sample stdin 1 +sample stdin 0 +preloaded output of a child +listening for stdin: +sample stdin 1 +sample stdin 0 +EOF + +test_expect_success 'run_command listens to stdin' ' + write_script stdin-script <<-\EOF && + echo "listening for stdin:" + while read line; do + echo "$line" + done + EOF + test-tool run-command run-command-stdin 2 ./stdin-script 2>actual && + test_cmp expect actual +' + cat >expect <<-EOF preloaded output of a child asking for a quick stop
Signature · 1 line
2.33.0.816.g1ba32acadee
ab/config-based-hooks-N status (was Re: [PATCH v5 24/36] run-command: add stdin callback for parallelization)
On Thu, Sep 02 2021, Ævar Arnfjörð Bjarmason wrote:
Emily, there's a...:
Quoted 7 lines
t/helper/test-run-command.c +0
-0@@ -32,8 +32,13 @@ static int parallel_next(struct child_process *cp,return 0 ;
t/helper/test-run-command.c static int parallel_next(struct child_process *cp, · line 43
Quoted 10 lines
t/helper/test-run-command.c +5
-0strvec_pushv (&cp->args, d->argv); + cp->in = d->in; + cp->no_stdin = d->no_stdin;strbuf_addstr (err,"preloaded output of a child\n" ); number_callbacks++; + + *task_cb =xmalloc (sizeof (int )); + *(int *)(*task_cb) =2 ;return 1 ; }
Probably trivial to solve failure here in t0061-run-command.sh if you compile with SANITIZE=leak. This failed in combination with my[1] (but for anyone reading along, this patch has been ejected from "seen" a while ago).
More generally: The equivalent of 01-07/36 of this series is being merged into "next". As described in a plan to submit this topic incrementally I was hoping to do 08-20/36 next, i.e. up to "run-command:
remove old run_hook_{le,ve}() hook API". See [2] for that plan.
You've been inactive on-list recently, it would be nice to time this so that by the time it gets to 21-36/36 (which I was planning to split in two per [2]) that you'd have time to review/help with outstanding issues etc, for eventually re-submitting your "config based hooks" on top once this all lands.
1. https://lore.kernel.org/git/[email protected]/
2. https://lore.kernel.org/git/[email protected]/
Re: ab/config-based-hooks-N status (was Re: [PATCH v5 24/36] run-command: add stdin callback for parallelization)
Quoted 32 lines · Ævar Arnfjörð Bjarmason
On Thu, Sep 02 2021, Ævar Arnfjörð Bjarmason wrote:
Emily, there's a...:
Quoted 7 lines
t/helper/test-run-command.c +0
-0@@ -32,8 +32,13 @@ static int parallel_next(struct child_process *cp,return 0 ;t/helper/test-run-command.c static int parallel_next(struct child_process *cp, · line 43
Quoted 10 lines
t/helper/test-run-command.c +5
-0strvec_pushv (&cp->args, d->argv); + cp->in = d->in; + cp->no_stdin = d->no_stdin;strbuf_addstr (err,"preloaded output of a child\n" ); number_callbacks++; + + *task_cb =xmalloc (sizeof (int )); + *(int *)(*task_cb) =2 ;return 1 ; }Probably trivial to solve failure here in
t0061-run-command.shif you compile with SANITIZE=leak. This failed in combination with my[1] (but for anyone reading along, this patch has been ejected from "seen" a while ago).More generally: The equivalent of 01-07/36 of this series is being merged into "next". As described in a plan to submit this topic incrementally I was hoping to do 08-20/36 next, i.e. up to "run-command:
remove old run_hook_{le,ve}() hook API". See [2] for that plan.
Quoted 8 lines
You've been inactive on-list recently, it would be nice to time this so that by the time it gets to 21-36/36 (which I was planning to split in two per [2]) that you'd have time to review/help with outstanding issues etc, for eventually re-submitting your "config based hooks" on top once this all lands.
1. https://lore.kernel.org/git/[email protected]/
2. https://lore.kernel.org/git/[email protected]/
Since I had a reason to look at this again, this fixes it. I've squashed it into my "base" branch, but it won't be in the next batch I submit (I have the cut-off point before this commit):
t/helper/test-run-command.c +13 -3
@@ -61,11 +61,21 @@ static void test_consume_sideband(struct strbuf *output, void *cb)fclose (sideband); } +static int task_free (int result, +struct strbuf *err, +void *pp_cb, +void *pp_task_cb) +{ +free (pp_task_cb); +return 0 ; +} +static int task_finished (int result,struct strbuf *err,void *pp_cb,void *pp_task_cb) { +task_free (0 ,NULL ,NULL , pp_task_cb);strbuf_addstr (err,"asking for a quick stop\n" );return 1 ; }
@@ -438,7 +448,7 @@ int cmd__run_command(int argc, const char **argv)if (!strcmp (argv[1 ],"run-command-parallel" ))exit (run_processes_parallel (jobs, parallel_next, -NULL ,NULL ,NULL ,NULL , &proc)); +NULL ,NULL ,NULL , task_free, &proc));if (!strcmp (argv[1 ],"run-command-abort" )) exit(run_processes_parallel(jobs, parallel_next,
@@ -452,12 +462,12 @@ int cmd__run_command(int argc, const char **argv) proc.in =-1 ; proc.no_stdin =0 ;exit (run_processes_parallel (jobs, parallel_next,NULL , - test_stdin,NULL ,NULL , &proc)); + test_stdin,NULL , task_free, &proc)); }if (!strcmp (argv[1 ],"run-command-sideband" ))exit (run_processes_parallel (jobs, parallel_next,NULL ,NULL , - test_consume_sideband,NULL , + test_consume_sideband, task_free, &proc));fprintf (stderr,"check usage\n" );
PATCH v5 25/36 hook: provide stdin by string_list or callback
In reply to Ævar Arnfjörð BjarmasonFrom: Emily Shaffer <[email protected]>
In cases where a hook requires only a small amount of information via stdin, it should be simple for users to provide a string_list alone. But in more complicated cases where the stdin is too large to hold in memory, let's instead provide a callback the users can populate line after line.
--- hook.c | 33 ++++++++++++++++++++++++++++++++- hook.h | 29 +++++++++++++++++++++++++++++ 2 files changed, 61 insertions(+), 1 deletion(-)
hook.c +32 -1
@@ -48,6 +48,29 @@ void run_hooks_opt_clear(struct run_hooks_opt *o)strvec_clear (&o->args); } +int pipe_from_string_list (struct strbuf *pipe,void *pp_cb,void *pp_task_cb) +{ +int *item_idx; +struct hook *ctx = pp_task_cb; +struct hook_cb_data *hook_cb = pp_cb; +struct string_list *to_pipe = hook_cb->options->feed_pipe_ctx; + +/* Bootstrap the state manager if necessary. */ +if (!ctx->feed_pipe_cb_data) { + ctx->feed_pipe_cb_data =xmalloc (sizeof (unsigned int )); + *(int *)ctx->feed_pipe_cb_data =0 ; + } + + item_idx = ctx->feed_pipe_cb_data; + +if (*item_idx < to_pipe->nr) { +strbuf_addf (pipe,"%s\n" , to_pipe->items[*item_idx].string); + (*item_idx)++; +return 0 ; + } +return 1 ; +} + static int pick_next_hook(struct child_process *cp, struct strbuf *out, void *pp_cb,
@@ -63,6 +86,10 @@ static int pick_next_hook(struct child_process *cp,if (hook_cb->options->path_to_stdin) { cp->no_stdin =0 ; cp->in =xopen (hook_cb->options->path_to_stdin,O_RDONLY ); + }else if (hook_cb->options->feed_pipe) { +/* ask for start_command() to make a pipe for us */ + cp->in =-1 ; + cp->no_stdin =0 ; }else { cp->no_stdin =1 ; }
@@ -146,7 +173,7 @@ int run_hooks(const char *hook_name, const char *hook_path, run_processes_parallel_tr2(jobs, pick_next_hook, notify_start_failure, - NULL, + options->feed_pipe, notify_hook_finished, &cb_data, "hook",
@@ -154,6 +181,7 @@ int run_hooks(const char *hook_name, const char *hook_path,if (options->absolute_path)strbuf_release (&abs_path); +free (my_hook.feed_pipe_cb_data);return cb_data.rc; }
@@ -167,6 +195,9 @@ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)if (!options) options = &hook_opt_scratch; +if (options->path_to_stdin && options->feed_pipe) +BUG ("choose only one method to populate stdin" ); + hook_path =find_hook (hook_name);if (!hook_path) { ret =0 ;
hook.h +29 -0
@@ -1,10 +1,18 @@#ifndef HOOK_H #define HOOK_H +#include "strbuf.h" #include "strvec.h" +#include "run-command.h" struct hook {/* The path to the hook */ const char *hook_path; + +/* +* Use this to keep state for your feed_pipe_fn if you are using +* run_hooks_opt.feed_pipe. Otherwise, do not touch it. +*/ +void *feed_pipe_cb_data; };struct run_hooks_opt
@@ -26,6 +34,19 @@ struct run_hooks_opt/* Path to file which should be piped to stdin for each hook */ const char *path_to_stdin; + +/* +* Callback and state pointer to ask for more content to pipe to stdin. +* Will be called repeatedly, for each hook. See +* hook.c:pipe_from_stdin() for an example. Keep per-hook state in +* hook.feed_pipe_cb_data (per process). Keep initialization context in +* feed_pipe_ctx (shared by all processes). +* +* See 'pipe_from_string_list()' for info about how to specify a +* string_list as the stdin input instead of writing your own handler. +*/ +feed_pipe_fn feed_pipe; +void *feed_pipe_ctx; };#define RUN_HOOKS_OPT_INIT { \
@@ -77,4 +98,12 @@ int run_hooks(const char *hookname, const char *hook_path,*/ int run_hooks_oneshot (const char *hook_name,struct run_hooks_opt *options); +/** +* To specify a 'struct string_list', set 'run_hooks_opt.feed_pipe_ctx' to the +* string_list and set 'run_hooks_opt.feed_pipe' to pipe_from_string_list(). +* This will pipe each string in the list to stdin, separated by newlines. (Do +* not inject your own newlines.) +*/ +int pipe_from_string_list (struct strbuf *pipe,void *pp_cb,void *pp_task_cb); +#endif
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 26/36 hook: convert 'post-rewrite' hook in sequencer.c to hook.h
In reply to Ævar Arnfjörð BjarmasonFrom: Emily Shaffer <[email protected]>
By using 'hook.h' for 'post-rewrite', we simplify hook invocations by not needing to put together our own 'struct child_process'.
The signal handling that's being removed by this commit now takes place in run-command.h:run_processes_parallel(), so it is OK to remove them here.
--- sequencer.c | 79 ++++++++++++++++++++++------------------------------- 1 file changed, 32 insertions(+), 47 deletions(-)
sequencer.c +32 -47
@@ -35,6 +35,7 @@#include "commit-reach.h" #include "rebase-interactive.h" #include "reset.h" +#include "string-list.h" #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
@@ -1147,33 +1148,27 @@ int update_head_with_reflog(const struct commit *old_head,static int run_rewrite_hook (const struct object_id *oldoid,const struct object_id *newoid) { -struct child_process proc =CHILD_PROCESS_INIT ; -const char *argv[3 ]; +struct run_hooks_opt opt =RUN_HOOKS_OPT_INIT ; +struct strbuf tmp =STRBUF_INIT ; +struct string_list to_stdin =STRING_LIST_INIT_DUP ;int code; -struct strbuf sb =STRBUF_INIT ; - argv[0 ] =find_hook ("post-rewrite" ); -if (!argv[0 ]) -return 0 ; +strvec_push (&opt.args,"amend" ); - argv[1 ] ="amend" ; - argv[2 ] =NULL ; - - proc.argv = argv; - proc.in =-1 ; - proc.stdout_to_stderr =1 ; - proc.trace2_hook_name ="post-rewrite" ; - - code =start_command (&proc); -if (code) -return code; -strbuf_addf (&sb,"%s %s\n" ,oid_to_hex (oldoid),oid_to_hex (newoid)); -sigchain_push (SIGPIPE ,SIG_IGN ); -write_in_full (proc.in, sb.buf, sb.len); -close (proc.in); -strbuf_release (&sb); -sigchain_pop (SIGPIPE ); -return finish_command (&proc); +strbuf_addf (&tmp, +"%s %s" , +oid_to_hex (oldoid), +oid_to_hex (newoid)); +string_list_append (&to_stdin, tmp.buf); + + opt.feed_pipe = pipe_from_string_list; + opt.feed_pipe_ctx = &to_stdin; + + code =run_hooks_oneshot ("post-rewrite" , &opt); + +strbuf_release (&tmp); +string_list_clear (&to_stdin,0 ); +return code; } void commit_post_rewrite(struct repository *r,
@@ -4526,30 +4521,20 @@ static int pick_commits(struct repository *r,flush_rewritten_pending ();if (!stat (rebase_path_rewritten_list (), &st) && st.st_size >0 ) { -struct child_process child =CHILD_PROCESS_INIT ; -const char *post_rewrite_hook = -find_hook ("post-rewrite" ); - - child.in =open (rebase_path_rewritten_list (),O_RDONLY ); - child.git_cmd =1 ; -strvec_push (&child.args,"notes" ); -strvec_push (&child.args,"copy" ); -strvec_push (&child.args,"--for-rewrite=rebase" ); +struct child_process notes_cp =CHILD_PROCESS_INIT ; +struct run_hooks_opt hook_opt =RUN_HOOKS_OPT_INIT ; + + notes_cp.in =open (rebase_path_rewritten_list (),O_RDONLY ); + notes_cp.git_cmd =1 ; +strvec_push (¬es_cp.args,"notes" ); +strvec_push (¬es_cp.args,"copy" ); +strvec_push (¬es_cp.args,"--for-rewrite=rebase" );/* we don't care if this copying failed */ -run_command (&child); - -if (post_rewrite_hook) { -struct child_process hook =CHILD_PROCESS_INIT ; - - hook.in =open (rebase_path_rewritten_list (), -O_RDONLY ); - hook.stdout_to_stderr =1 ; - hook.trace2_hook_name ="post-rewrite" ; -strvec_push (&hook.args, post_rewrite_hook); -strvec_push (&hook.args,"rebase" ); -/* we don't care if this hook failed */ -run_command (&hook); - } +run_command (¬es_cp); + + hook_opt.path_to_stdin =rebase_path_rewritten_list (); +strvec_push (&hook_opt.args,"rebase" ); +run_hooks_oneshot ("post-rewrite" , &hook_opt); }apply_autostash (rebase_path_autostash ());
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 27/36 transport: convert pre-push hook to hook.h
In reply to Ævar Arnfjörð BjarmasonFrom: Emily Shaffer <[email protected]>
Move the pre-push hook away from run-command.h to and over to the new hook.h library.
--- transport.c | 56 ++++++++++++++--------------------------------------- 1 file changed, 14 insertions(+), 42 deletions(-)
transport.c +14 -42
@@ -1203,63 +1203,35 @@ static void die_with_unpushed_submodules(struct string_list *needs_pushing)static int run_pre_push_hook (struct transport *transport,struct ref *remote_refs) { -int ret =0 , x; +int ret =0 ; +struct run_hooks_opt opt =RUN_HOOKS_OPT_INIT ;struct ref *r; -struct child_process proc =CHILD_PROCESS_INIT ; -struct strbuf buf; -const char *argv[4 ]; - -if (!(argv[0 ] =find_hook ("pre-push" ))) -return 0 ; - - argv[1 ] = transport->remote->name; - argv[2 ] = transport->url; - argv[3 ] =NULL ; - - proc.argv = argv; - proc.in =-1 ; - proc.trace2_hook_name ="pre-push" ; - -if (start_command (&proc)) { -finish_command (&proc); -return -1 ; - } +struct string_list to_stdin =STRING_LIST_INIT_NODUP ; -sigchain_push (SIGPIPE ,SIG_IGN ); - -strbuf_init (&buf,256 ); +strvec_push (&opt.args, transport->remote->name); +strvec_push (&opt.args, transport->url);for (r = remote_refs; r; r = r->next) { +struct strbuf buf =STRBUF_INIT ; +if (!r->peer_ref)continue ;if (r->status ==REF_STATUS_REJECT_NONFASTFORWARD )continue ;if (r->status ==REF_STATUS_REJECT_STALE )continue ;if (r->status ==REF_STATUS_REJECT_REMOTE_UPDATED )continue ;if (r->status ==REF_STATUS_UPTODATE )continue ; -strbuf_reset (&buf); -strbuf_addf ( &buf,"%s %s %s %s\n" , +strbuf_addf (&buf,"%s %s %s %s" , r->peer_ref->name,oid_to_hex (&r->new_oid), r->name,oid_to_hex (&r->old_oid)); - -if (write_in_full (proc.in, buf.buf, buf.len) <0 ) { -/* We do not mind if a hook does not read all refs. */ -if (errno !=EPIPE ) - ret =-1 ; -break ; - } +string_list_append (&to_stdin,strbuf_detach (&buf,NULL )); } -strbuf_release (&buf); - - x =close (proc.in); -if (!ret) - ret = x; - -sigchain_pop (SIGPIPE ); + opt.feed_pipe = pipe_from_string_list; + opt.feed_pipe_ctx = &to_stdin; - x =finish_command (&proc); -if (!ret) - ret = x; + ret =run_hooks_oneshot ("pre-push" , &opt); + to_stdin.strdup_strings =1 ; +string_list_clear (&to_stdin,0 );return ret; }
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 28/36 hook tests: test for exact "pre-push" hook input
In reply to Ævar Arnfjörð BjarmasonExtend the tests added in ec55559f937 (push: Add support for pre-push hooks, 2013-01-13) to exhaustively test for the exact input we're expecting. This ensures that we e.g. don't miss a trailing newline.
Appending to a file called "actual" is the established convention in this test for hooks, see the rest of the tests added in ec55559f937 (push: Add support for pre-push hooks, 2013-01-13). Let's follow that convention here.
--- t/t5571-pre-push-hook.sh | 23 ++++++++++++++++++----- 1 file changed, 18 insertions(+), 5 deletions(-)
t/t5571-pre-push-hook.sh +18 -5
@@ -11,7 +11,7 @@ HOOKDIR="$(git rev-parse --git-dir)/hooks" HOOK="$HOOKDIR/pre-push" mkdir -p "$HOOKDIR" write_script "$HOOK" <<EOF -cat >/dev/null +cat >actual exit 0 EOF
@@ -20,10 +20,16 @@ test_expect_success 'setup' ' git init --bare repo1 && git remote add parent1 repo1 && test_commit one && - git push parent1 HEAD:foreign + cat >expect <<-EOF && + HEAD $(git rev-parse HEAD) refs/heads/foreign $(test_oid zero) + EOF + + test_when_finished "rm actual" && + git push parent1 HEAD:foreign && + test_cmp expect actual ' write_script "$HOOK" <<EOF -cat >/dev/null +cat >actual exit 1 EOF
@@ -32,11 +38,18 @@ export COMMIT1 test_expect_success 'push with failing hook' ' test_commit two && - test_must_fail git push parent1 HEAD + cat >expect <<-EOF && + HEAD $(git rev-parse HEAD) refs/heads/main $(test_oid zero) + EOF + + test_when_finished "rm actual" && + test_must_fail git push parent1 HEAD && + test_cmp expect actual ' test_expect_success '--no-verify bypasses hook' ' - git push --no-verify parent1 HEAD + git push --no-verify parent1 HEAD && + test_path_is_missing actual ' COMMIT2="$(git rev-parse HEAD)"
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 29/36 hook tests: use a modern style for "pre-push" tests
In reply to Ævar Arnfjörð BjarmasonIndent the here-docs and use "test_cmp" instead of "diff" in tests added in ec55559f937 (push: Add support for pre-push hooks, 2013-01-13). Let's also use the more typical "expect" instead of "expected" to be consistent with the rest of the test file.
--- t/t5571-pre-push-hook.sh | 71 ++++++++++++++++++++-------------------- 1 file changed, 35 insertions(+), 36 deletions(-)
t/t5571-pre-push-hook.sh +35 -36
@@ -61,15 +61,15 @@ echo "$2" >>actual cat >>actual EOF -cat >expected <<EOF -parent1 -repo1 -refs/heads/main $COMMIT2 refs/heads/foreign $COMMIT1 -EOF - test_expect_success 'push with hook' ' + cat >expect <<-EOF && + parent1 + repo1 + refs/heads/main $COMMIT2 refs/heads/foreign $COMMIT1 + EOF + git push parent1 main:foreign && - diff expected actual + test_cmp expect actual ' test_expect_success 'add a branch' '
@@ -80,49 +80,48 @@ test_expect_success 'add a branch' ' COMMIT3="$(git rev-parse HEAD)" export COMMIT3 -cat >expected <<EOF -parent1 -repo1 -refs/heads/other $COMMIT3 refs/heads/foreign $COMMIT2 -EOF - test_expect_success 'push to default' ' + cat >expect <<-EOF && + parent1 + repo1 + refs/heads/other $COMMIT3 refs/heads/foreign $COMMIT2 + EOF git push && - diff expected actual + test_cmp expect actual ' -cat >expected <<EOF -parent1 -repo1 -refs/tags/one $COMMIT1 refs/tags/tag1 $ZERO_OID -HEAD~ $COMMIT2 refs/heads/prev $ZERO_OID -EOF - test_expect_success 'push non-branches' ' + cat >expect <<-EOF && + parent1 + repo1 + refs/tags/one $COMMIT1 refs/tags/tag1 $ZERO_OID + HEAD~ $COMMIT2 refs/heads/prev $ZERO_OID + EOF + git push parent1 one:tag1 HEAD~:refs/heads/prev && - diff expected actual + test_cmp expect actual ' -cat >expected <<EOF -parent1 -repo1 -(delete) $ZERO_OID refs/heads/prev $COMMIT2 -EOF - test_expect_success 'push delete' ' + cat >expect <<-EOF && + parent1 + repo1 + (delete) $ZERO_OID refs/heads/prev $COMMIT2 + EOF + git push parent1 :prev && - diff expected actual + test_cmp expect actual ' -cat >expected <<EOF -repo1 -repo1 -HEAD $COMMIT3 refs/heads/other $ZERO_OID -EOF - test_expect_success 'push to URL' ' + cat >expect <<-EOF && + repo1 + repo1 + HEAD $COMMIT3 refs/heads/other $ZERO_OID + EOF + git push repo1 HEAD && - diff expected actual + test_cmp expect actual ' test_expect_success 'set up many-ref tests' '
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 30/36 reference-transaction: use hook.h to run hooks
In reply to Ævar Arnfjörð BjarmasonFrom: Emily Shaffer <[email protected]>
--- refs.c | 40 +++++++++++++--------------------------- 1 file changed, 13 insertions(+), 27 deletions(-)
refs.c +13 -27
@@ -2062,47 +2062,33 @@ int ref_update_reject_duplicates(struct string_list *refnames,static int run_transaction_hook (struct ref_transaction *transaction,const char *state) { -struct child_process proc =CHILD_PROCESS_INIT ; -struct strbuf buf =STRBUF_INIT ; -const char *hook; +struct run_hooks_opt opt =RUN_HOOKS_OPT_INIT ; +struct string_list to_stdin =STRING_LIST_INIT_NODUP ;int ret =0 , i; - hook =find_hook ("reference-transaction" ); -if (!hook) -return ret; - -strvec_pushl (&proc.args, hook, state,NULL ); - proc.in =-1 ; - proc.stdout_to_stderr =1 ; - proc.trace2_hook_name ="reference-transaction" ; - - ret =start_command (&proc); -if (ret) +if (!hook_exists ("reference-transaction" ))return ret; -sigchain_push (SIGPIPE ,SIG_IGN ); +strvec_push (&opt.args, state);for (i =0 ; i < transaction->nr; i++) {struct ref_update *update = transaction->updates[i]; +struct strbuf buf =STRBUF_INIT ; -strbuf_reset (&buf); -strbuf_addf (&buf,"%s %s %s\n" , +strbuf_addf (&buf,"%s %s %s" ,oid_to_hex (&update->old_oid),oid_to_hex (&update->new_oid), update->refname); - -if (write_in_full (proc.in, buf.buf, buf.len) <0 ) { -if (errno !=EPIPE ) - ret =-1 ; -break ; - } +string_list_append (&to_stdin,strbuf_detach (&buf,NULL )); } -close (proc.in); -sigchain_pop (SIGPIPE ); -strbuf_release (&buf); + opt.feed_pipe = pipe_from_string_list; + opt.feed_pipe_ctx = &to_stdin; + + ret =run_hooks_oneshot ("reference-transaction" , &opt); + to_stdin.strdup_strings =1 ; +string_list_clear (&to_stdin,0 ); - ret |=finish_command (&proc);return ret; }
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 31/36 run-command: allow capturing of collated output
In reply to Ævar Arnfjörð BjarmasonFrom: Emily Shaffer <[email protected]>
Some callers, for example server-side hooks which wish to relay hook output to clients across a transport, want to capture what would normally print to stderr and do something else with it. Allow that via a callback.
By calling the callback regardless of whether there's output available, we allow clients to send e.g. a keepalive if necessary.
Because we expose a strbuf, not a fd or FILE*, there's no need to create a temporary pipe or similar - we can just skip the print to stderr and instead hand it to the caller.
--- builtin/fetch.c | 2 +- builtin/submodule--helper.c | 2 +- hook.c | 1 + run-command.c | 33 +++++++++++++++++++++++++-------- run-command.h | 18 +++++++++++++++++- submodule.c | 2 +- t/helper/test-run-command.c | 25 ++++++++++++++++++++----- t/t0061-run-command.sh | 7 +++++++ 8 files changed, 73 insertions(+), 17 deletions(-)
builtin/fetch.c +1 -1
@@ -1819,7 +1819,7 @@ static int fetch_multiple(struct string_list *list, int max_children) result =run_processes_parallel_tr2 (max_children, &fetch_next_remote, &fetch_failed_to_start, -NULL , +NULL ,NULL , &fetch_finished, &state,"fetch" ,"parallel/fetch" );
builtin/submodule--helper.c +1 -1
@@ -2311,7 +2311,7 @@ static int update_submodules(struct submodule_update_clone *suc)int i;run_processes_parallel_tr2 (suc->max_jobs, update_clone_get_next_task, - update_clone_start_failure,NULL , + update_clone_start_failure,NULL ,NULL , update_clone_task_finished, suc,"submodule" ,"parallel/update" );
hook.c +1 -0
@@ -174,6 +174,7 @@ int run_hooks(const char *hook_name, const char *hook_path, pick_next_hook, notify_start_failure, options->feed_pipe, +NULL , notify_hook_finished, &cb_data, "hook",
run-command.c +25 -8
@@ -1493,6 +1493,7 @@ struct parallel_processes {get_next_task_fn get_next_task;start_failure_fn start_failure;feed_pipe_fn feed_pipe; +consume_sideband_fn consume_sideband;task_finished_fn task_finished;struct {
@@ -1558,6 +1559,7 @@ static void pp_init(struct parallel_processes *pp,get_next_task_fn get_next_task, start_failure_fn start_failure, feed_pipe_fn feed_pipe, + consume_sideband_fn consume_sideband, task_finished_fn task_finished, void *data) {
@@ -1578,6 +1580,7 @@ static void pp_init(struct parallel_processes *pp, pp->start_failure = start_failure ? start_failure : default_start_failure; pp->feed_pipe = feed_pipe ? feed_pipe : default_feed_pipe; pp->task_finished = task_finished ? task_finished : default_task_finished; + pp->consume_sideband = consume_sideband; pp->nr_processes =0 ; pp->output_owner =0 ;
@@ -1614,7 +1617,10 @@ static void pp_cleanup(struct parallel_processes *pp)* When get_next_task added messages to the buffer in its last * iteration, the buffered output is non empty. */ -strbuf_write (&pp->buffered_output, stderr); +if (pp->consume_sideband) + pp->consume_sideband (&pp->buffered_output, pp->data); +else +strbuf_write (&pp->buffered_output, stderr);strbuf_release (&pp->buffered_output);sigchain_pop_common ();
@@ -1735,9 +1741,13 @@ static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout)static void pp_output (struct parallel_processes *pp) {int i = pp->output_owner; +if (pp->children[i].state ==GIT_CP_WORKING && pp->children[i].err.len) { -strbuf_write (&pp->children[i].err, stderr); +if (pp->consume_sideband) + pp->consume_sideband (&pp->children[i].err, pp->data); +else +strbuf_write (&pp->children[i].err, stderr);strbuf_reset (&pp->children[i].err); } }
@@ -1776,11 +1786,15 @@ static int pp_collect_finished(struct parallel_processes *pp)strbuf_addbuf (&pp->buffered_output, &pp->children[i].err);strbuf_reset (&pp->children[i].err); }else { -strbuf_write (&pp->children[i].err, stderr); +/* Output errors, then all other finished child processes */ +if (pp->consume_sideband) { + pp->consume_sideband (&pp->children[i].err, pp->data); + pp->consume_sideband (&pp->buffered_output, pp->data); + }else { +strbuf_write (&pp->children[i].err, stderr); +strbuf_write (&pp->buffered_output, stderr); + }strbuf_reset (&pp->children[i].err); - -/* Output all other finished child processes */ -strbuf_write (&pp->buffered_output, stderr);strbuf_reset (&pp->buffered_output);/*
@@ -1804,6 +1818,7 @@ int run_processes_parallel(int n,get_next_task_fn get_next_task, start_failure_fn start_failure, feed_pipe_fn feed_pipe, + consume_sideband_fn consume_sideband, task_finished_fn task_finished, void *pp_cb) {
@@ -1814,7 +1829,7 @@ int run_processes_parallel(int n,sigchain_push (SIGPIPE ,SIG_IGN ); -pp_init (&pp, n, get_next_task, start_failure, feed_pipe, task_finished, pp_cb); +pp_init (&pp, n, get_next_task, start_failure, feed_pipe, consume_sideband, task_finished, pp_cb);while (1 ) { for (i = 0; i < spawn_cap && !pp.shutdown &&
@@ -1852,6 +1867,7 @@ int run_processes_parallel(int n,int run_processes_parallel_tr2 (int n,get_next_task_fn get_next_task,start_failure_fn start_failure,feed_pipe_fn feed_pipe, +consume_sideband_fn consume_sideband,task_finished_fn task_finished,void *pp_cb,const char *tr2_category,const char *tr2_label) {
@@ -1861,7 +1877,8 @@ int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task, ((n < 1) ? online_cpus() : n)); result =run_processes_parallel (n, get_next_task, start_failure, - feed_pipe, task_finished, pp_cb); + feed_pipe, consume_sideband, + task_finished, pp_cb);trace2_region_leave (tr2_category, tr2_label,NULL );
run-command.h +17 -1
@@ -436,6 +436,20 @@ typedef int (*feed_pipe_fn)(struct strbuf *pipe,void *pp_cb, void *pp_task_cb); +/** +* If this callback is provided, instead of collating process output to stderr, +* they will be collated into a new pipe. consume_sideband_fn will be called +* repeatedly. When output is available on that pipe, it will be contained in +* 'output'. But it will be called with an empty 'output' too, to allow for +* keepalives or similar operations if necessary. +* +* pp_cb is the callback cookie as passed into run_processes_parallel. +* +* Since this callback is provided with the collated output, no task cookie is +* provided. +*/ +typedef void (*consume_sideband_fn )(struct strbuf *output,void *pp_cb); +/** * This callback is called on every child process that finished processing. *
@@ -471,10 +485,12 @@ int run_processes_parallel(int n, get_next_task_fn, start_failure_fn, feed_pipe_fn, + consume_sideband_fn, task_finished_fn, void *pp_cb);int run_processes_parallel_tr2 (int n,get_next_task_fn ,start_failure_fn , -feed_pipe_fn ,task_finished_fn ,void *pp_cb, +feed_pipe_fn ,consume_sideband_fn , +task_finished_fn ,void *pp_cb,const char *tr2_category,const char *tr2_label);/**
submodule.c +1 -1
@@ -1632,7 +1632,7 @@ int fetch_populated_submodules(struct repository *r,run_processes_parallel_tr2 (max_parallel_jobs, get_next_submodule, fetch_start_failure, -NULL , +NULL ,NULL , fetch_finish, &spf,"submodule" ,"parallel/fetch" );
t/helper/test-run-command.c +20 -5
@@ -51,6 +51,16 @@ static int no_job(struct child_process *cp,return 0 ; } +static void test_consume_sideband (struct strbuf *output,void *cb) +{ +FILE *sideband; + + sideband =fopen ("./sideband" ,"a" ); + +strbuf_write (output, sideband); +fclose (sideband); +} + static int task_finished(int result, struct strbuf *err, void *pp_cb,
@@ -201,7 +211,7 @@ static int testsuite(int argc, const char **argv) suite.tests.nr, max_jobs); ret =run_processes_parallel (max_jobs, next_test, test_failed, - test_stdin, test_finished, &suite); + test_stdin,NULL , test_finished, &suite);if (suite.failed.nr >0 ) { ret =1 ;
@@ -429,23 +439,28 @@ int cmd__run_command(int argc, const char **argv)if (!strcmp (argv[1 ],"run-command-parallel" ))exit (run_processes_parallel (jobs, parallel_next, -NULL ,NULL ,NULL , &proc)); +NULL ,NULL ,NULL ,NULL , &proc));if (!strcmp (argv[1 ],"run-command-abort" ))exit (run_processes_parallel (jobs, parallel_next, -NULL ,NULL , task_finished, &proc)); +NULL ,NULL ,NULL , task_finished, &proc));if (!strcmp (argv[1 ],"run-command-no-jobs" ))exit (run_processes_parallel (jobs, no_job, -NULL ,NULL , task_finished, &proc)); +NULL ,NULL ,NULL , task_finished, &proc));if (!strcmp (argv[1 ],"run-command-stdin" )) { proc.in =-1 ; proc.no_stdin =0 ;exit (run_processes_parallel (jobs, parallel_next,NULL , - test_stdin,NULL , &proc)); + test_stdin,NULL ,NULL , &proc)); } +if (!strcmp (argv[1 ],"run-command-sideband" )) +exit (run_processes_parallel (jobs, parallel_next,NULL ,NULL , + test_consume_sideband,NULL , + &proc)); +fprintf (stderr,"check usage\n" );return 1 ; }
t/t0061-run-command.sh +7 -0
@@ -143,6 +143,13 @@ test_expect_success 'run_command runs in parallel with more tasks than jobs avai test_cmp expect actual ' +test_expect_success 'run_command can divert output' ' + test_when_finished rm sideband && + test-tool run-command run-command-sideband 3 sh -c "printf \"%s\n%s\n\" Hello World" 2>actual && + test_must_be_empty actual && + test_cmp expect sideband +' + cat >expect <<-EOF preloaded output of a child listening for stdin:
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 32/36 hooks: allow callers to capture output
In reply to Ævar Arnfjörð BjarmasonFrom: Emily Shaffer <[email protected]>
Some server-side hooks will require capturing output to send over sideband instead of printing directly to stderr. Expose that capability.
--- hook.c | 3 ++- hook.h | 8 ++++++++ 2 files changed, 10 insertions(+), 1 deletion(-)
hook.c +2 -1
@@ -174,12 +174,13 @@ int run_hooks(const char *hook_name, const char *hook_path, pick_next_hook, notify_start_failure, options->feed_pipe, -NULL , + options->consume_sideband, notify_hook_finished, &cb_data,"hook" , hook_name); +if (options->absolute_path)strbuf_release (&abs_path);free (my_hook.feed_pipe_cb_data);
hook.h +8 -0
@@ -47,6 +47,14 @@ struct run_hooks_opt*/ feed_pipe_fn feed_pipe;void *feed_pipe_ctx; + +/* +* Populate this to capture output and prevent it from being printed to +* stderr. This will be passed directly through to +* run_command:run_parallel_processes(). See t/helper/test-run-command.c +* for an example. +*/ +consume_sideband_fn consume_sideband; };#define RUN_HOOKS_OPT_INIT { \
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 33/36 receive-pack: convert 'update' hook to hook.h
In reply to Ævar Arnfjörð BjarmasonFrom: Emily Shaffer <[email protected]>
This makes use of the new sideband API in hook.h added in the preceding commit.
--- builtin/receive-pack.c | 62 ++++++++++++++++++++++++++++-------------- 1 file changed, 41 insertions(+), 21 deletions(-)
builtin/receive-pack.c +41 -21
@@ -937,33 +937,53 @@ static int run_receive_hook(struct command *commands,return status; } -static int run_update_hook (struct command *cmd) +static void hook_output_to_sideband (struct strbuf *output,void *cb_data) { -const char *argv[5 ]; -struct child_process proc =CHILD_PROCESS_INIT ; -int code; +int keepalive_active =0 ; - argv[0 ] =find_hook ("update" ); -if (!argv[0 ]) -return 0 ; +if (keepalive_in_sec <=0 ) + use_keepalive =KEEPALIVE_NEVER ; +if (use_keepalive ==KEEPALIVE_ALWAYS ) + keepalive_active =1 ; - argv[1 ] = cmd->ref_name; - argv[2 ] =oid_to_hex (&cmd->old_oid); - argv[3 ] =oid_to_hex (&cmd->new_oid); - argv[4 ] =NULL ; +/* send a keepalive if there is no data to write */ +if (keepalive_active && !output->len) { +static const char buf[] ="0005\1" ; +write_or_die (1 , buf,sizeof (buf) -1 ); +return ; + } - proc.no_stdin =1 ; - proc.stdout_to_stderr =1 ; - proc.err = use_sideband ?-1 :0 ; - proc.argv = argv; - proc.trace2_hook_name ="update" ; +if (use_keepalive ==KEEPALIVE_AFTER_NUL && !keepalive_active) { +const char *first_null =memchr (output->buf,'\0' , output->len); +if (first_null) { +/* The null bit is excluded. */ +size_t before_null = first_null - output->buf; +size_t after_null = output->len - (before_null +1 ); + keepalive_active =1 ; +send_sideband (1 ,2 , output->buf, before_null, use_sideband); +send_sideband (1 ,2 , first_null +1 , after_null, use_sideband); + +return ; + } + } + +send_sideband (1 ,2 , output->buf, output->len, use_sideband); +} + +static int run_update_hook (struct command *cmd) +{ +struct run_hooks_opt opt =RUN_HOOKS_OPT_INIT ; + +strvec_pushl (&opt.args, + cmd->ref_name, +oid_to_hex (&cmd->old_oid), +oid_to_hex (&cmd->new_oid), +NULL ); - code =start_command (&proc); -if (code) -return code;if (use_sideband) -copy_to_sideband (proc.err,-1 ,NULL ); -return finish_command (&proc); + opt.consume_sideband = hook_output_to_sideband; + +return run_hooks_oneshot ("update" , &opt); } static struct command *find_command_by_refname(struct command *list,
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 34/36 post-update: use hook.h library
In reply to Ævar Arnfjörð BjarmasonFrom: Emily Shaffer <[email protected]>
--- builtin/receive-pack.c | 25 ++++++------------------- 1 file changed, 6 insertions(+), 19 deletions(-)
builtin/receive-pack.c +6 -19
@@ -1650,33 +1650,20 @@ static const char *update(struct command *cmd, struct shallow_info *si)static void run_update_post_hook (struct command *commands) {struct command *cmd; -struct child_process proc =CHILD_PROCESS_INIT ; -const char *hook; - - hook =find_hook ("post-update" ); -if (!hook) -return ; +struct run_hooks_opt opt =RUN_HOOKS_OPT_INIT ;for (cmd = commands; cmd; cmd = cmd->next) {if (cmd->error_string || cmd->did_not_exist)continue ; -if (!proc.args.nr) -strvec_push (&proc.args, hook); -strvec_push (&proc.args, cmd->ref_name); +strvec_push (&opt.args, cmd->ref_name); } -if (!proc.args.nr) +if (!opt.args.nr)return ; - proc.no_stdin =1 ; - proc.stdout_to_stderr =1 ; - proc.err = use_sideband ?-1 :0 ; - proc.trace2_hook_name ="post-update" ; +if (use_sideband) + opt.consume_sideband = hook_output_to_sideband; -if (!start_command (&proc)) { -if (use_sideband) -copy_to_sideband (proc.err,-1 ,NULL ); -finish_command (&proc); - } +run_hooks_oneshot ("post-update" , &opt); } static void check_aliased_update_internal(struct command *cmd,
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 35/36 receive-pack: convert receive hooks to hook.h
In reply to Ævar Arnfjörð BjarmasonFrom: Emily Shaffer <[email protected]>
--- builtin/receive-pack.c | 190 ++++++++++++++++++----------------------- 1 file changed, 83 insertions(+), 107 deletions(-)
builtin/receive-pack.c +83 -107
@@ -747,7 +747,7 @@ static int check_cert_push_options(const struct string_list *push_options)return retval; } -static void prepare_push_cert_sha1 (struct child_process *proc) +static void prepare_push_cert_sha1 (struct run_hooks_opt *opt) {static int already_done;
@@ -771,110 +771,42 @@ static void prepare_push_cert_sha1(struct child_process *proc) nonce_status =check_nonce (push_cert.buf, bogs); }if (!is_null_oid (&push_cert_oid)) { -strvec_pushf (&proc->env_array,"GIT_PUSH_CERT=%s" , +strvec_pushf (&opt->env,"GIT_PUSH_CERT=%s" ,oid_to_hex (&push_cert_oid)); -strvec_pushf (&proc->env_array,"GIT_PUSH_CERT_SIGNER=%s" , +strvec_pushf (&opt->env,"GIT_PUSH_CERT_SIGNER=%s" , sigcheck.signer ? sigcheck.signer :"" ); -strvec_pushf (&proc->env_array,"GIT_PUSH_CERT_KEY=%s" , +strvec_pushf (&opt->env,"GIT_PUSH_CERT_KEY=%s" , sigcheck.key ? sigcheck.key :"" ); -strvec_pushf (&proc->env_array,"GIT_PUSH_CERT_STATUS=%c" , +strvec_pushf (&opt->env,"GIT_PUSH_CERT_STATUS=%c" , sigcheck.result);if (push_cert_nonce) { -strvec_pushf (&proc->env_array, +strvec_pushf (&opt->env,"GIT_PUSH_CERT_NONCE=%s" , push_cert_nonce); -strvec_pushf (&proc->env_array, +strvec_pushf (&opt->env,"GIT_PUSH_CERT_NONCE_STATUS=%s" , nonce_status);if (nonce_status ==NONCE_SLOP ) -strvec_pushf (&proc->env_array, +strvec_pushf (&opt->env,"GIT_PUSH_CERT_NONCE_SLOP=%ld" , nonce_stamp_slop); } } } +struct receive_hook_feed_context { +struct command *cmd; +int skip_broken; +}; +struct receive_hook_feed_state {struct command *cmd;struct ref_push_report *report;int skip_broken;struct strbuf buf; -const struct string_list *push_options; }; -typedef int (*feed_fn )(void *,const char **,size_t *); -static int run_and_feed_hook (const char *hook_name,feed_fn feed, -struct receive_hook_feed_state *feed_state) -{ -struct child_process proc =CHILD_PROCESS_INIT ; -struct async muxer; -const char *argv[2 ]; -int code; - - argv[0 ] =find_hook (hook_name); -if (!argv[0 ]) -return 0 ; - - argv[1 ] =NULL ; - - proc.argv = argv; - proc.in =-1 ; - proc.stdout_to_stderr =1 ; - proc.trace2_hook_name = hook_name; - -if (feed_state->push_options) { -int i; -for (i =0 ; i < feed_state->push_options->nr; i++) -strvec_pushf (&proc.env_array, -"GIT_PUSH_OPTION_%d=%s" , i, - feed_state->push_options->items[i].string); -strvec_pushf (&proc.env_array,"GIT_PUSH_OPTION_COUNT=%d" , - feed_state->push_options->nr); - }else -strvec_pushf (&proc.env_array,"GIT_PUSH_OPTION_COUNT" ); - -if (tmp_objdir) -strvec_pushv (&proc.env_array,tmp_objdir_env (tmp_objdir)); - -if (use_sideband) { -memset (&muxer,0 ,sizeof (muxer)); - muxer.proc = copy_to_sideband; - muxer.in =-1 ; - code =start_async (&muxer); -if (code) -return code; - proc.err = muxer.in; - } - -prepare_push_cert_sha1 (&proc); - - code =start_command (&proc); -if (code) { -if (use_sideband) -finish_async (&muxer); -return code; - } - -sigchain_push (SIGPIPE ,SIG_IGN ); - -while (1 ) { -const char *buf; -size_t n; -if (feed (feed_state, &buf, &n)) -break ; -if (write_in_full (proc.in, buf, n) <0 ) -break ; - } -close (proc.in); -if (use_sideband) -finish_async (&muxer); - -sigchain_pop (SIGPIPE ); - -return finish_command (&proc); -} - -static int feed_receive_hook (void *state_,const char **bufp,size_t *sizep) +static int feed_receive_hook (void *state_) {struct receive_hook_feed_state *state = state_;struct command *cmd = state->cmd;
@@ -883,9 +815,7 @@ static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep) state->skip_broken && (cmd->error_string || cmd->did_not_exist)) cmd = cmd->next;if (!cmd) -return -1 ;/* EOF */ -if (!bufp) -return 0 ;/* OK, can feed something. */ +return 1 ;/* EOF - close the pipe*/ strbuf_reset (&state->buf);if (!state->report) state->report = cmd->report;
@@ -909,32 +839,36 @@ static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep) cmd->ref_name); state->cmd = cmd->next; } -if (bufp) { - *bufp = state->buf.buf; - *sizep = state->buf.len; - }return 0 ; } -static int run_receive_hook (struct command *commands, -const char *hook_name, -int skip_broken, -const struct string_list *push_options) +static int feed_receive_hook_cb (struct strbuf *pipe,void *pp_cb,void *pp_task_cb) { -struct receive_hook_feed_state state; -int status; - -strbuf_init (&state.buf,0 ); - state.cmd = commands; - state.skip_broken = skip_broken; - state.report =NULL ; -if (feed_receive_hook (&state,NULL ,NULL )) -return 0 ; - state.cmd = commands; - state.push_options = push_options; - status =run_and_feed_hook (hook_name, feed_receive_hook, &state); -strbuf_release (&state.buf); -return status; +struct hook *hook = pp_task_cb; +struct receive_hook_feed_state *feed_state = hook->feed_pipe_cb_data; +int rc; + +/* first-time setup */ +if (!feed_state) { +struct hook_cb_data *hook_cb = pp_cb; +struct run_hooks_opt *opt = hook_cb->options; +struct receive_hook_feed_context *ctx = opt->feed_pipe_ctx; +if (!ctx) +BUG ("run_hooks_opt.feed_pipe_ctx required for receive hook" ); + + feed_state =xmalloc (sizeof (struct receive_hook_feed_state )); +strbuf_init (&feed_state->buf,0 ); + feed_state->cmd = ctx->cmd; + feed_state->skip_broken = ctx->skip_broken; + feed_state->report =NULL ; + + hook->feed_pipe_cb_data = feed_state; + } + + rc =feed_receive_hook (feed_state); +if (!rc) +strbuf_addbuf (pipe, &feed_state->buf); +return rc; }static void hook_output_to_sideband (struct strbuf *output,void *cb_data)
@@ -970,6 +904,48 @@ static void hook_output_to_sideband(struct strbuf *output, void *cb_data)send_sideband (1 ,2 , output->buf, output->len, use_sideband); } +static int run_receive_hook (struct command *commands, +const char *hook_name, +int skip_broken, +const struct string_list *push_options) +{ +struct run_hooks_opt opt =RUN_HOOKS_OPT_INIT ; +struct receive_hook_feed_context ctx; +struct command *iter = commands; + +/* if there are no valid commands, don't invoke the hook at all. */ +while (iter && skip_broken && (iter->error_string || iter->did_not_exist)) + iter = iter->next; +if (!iter) +return 0 ; + +if (push_options) { +int i; +for (i =0 ; i < push_options->nr; i++) +strvec_pushf (&opt.env,"GIT_PUSH_OPTION_%d=%s" , i, + push_options->items[i].string); +strvec_pushf (&opt.env,"GIT_PUSH_OPTION_COUNT=%d" , push_options->nr); + }else +strvec_push (&opt.env,"GIT_PUSH_OPTION_COUNT" ); + +if (tmp_objdir) +strvec_pushv (&opt.env,tmp_objdir_env (tmp_objdir)); + +prepare_push_cert_sha1 (&opt); + +/* set up sideband printer */ +if (use_sideband) + opt.consume_sideband = hook_output_to_sideband; + +/* set up stdin callback */ + ctx.cmd = commands; + ctx.skip_broken = skip_broken; + opt.feed_pipe = feed_receive_hook_cb; + opt.feed_pipe_ctx = &ctx; + +return run_hooks_oneshot (hook_name, &opt); +} +static int run_update_hook (struct command *cmd) {struct run_hooks_opt opt =RUN_HOOKS_OPT_INIT ;
Signature · 1 line
2.33.0.816.g1ba32acadee
PATCH v5 36/36 hooks: fix a TOCTOU in "did we run a hook?" heuristic
In reply to Ævar Arnfjörð BjarmasonFix a Time-of-check to time-of-use (TOCTOU) race in code added in 680ee550d72 (commit: skip discarding the index if there is no pre-commit hook, 2017-08-14).
We can fix the race passing around information about whether or not we ran the hook in question, instead of running hook_exists() after the fact to check if the hook in question exists. This problem has been noted on-list when 680ee550d72 was discussed[1], but had not been fixed.
In addition to fixing this for the pre-commit hook as suggested there I'm also fixing this for the pre-merge-commit hook. See 6098817fd7f (git-merge: honor pre-merge-commit hook, 2019-08-07) for the introduction of its previous behavior.
Let's also change this for the push-to-checkout hook. Now instead of checking if the hook exists and either doing a push to checkout or a push to deploy we'll always attempt a push to checkout. If the hook doesn't exist we'll fall back on push to deploy. The same behavior as before, without the TOCTOU race. See 0855331941b (receive-pack:
support push-to-checkout hook, 2014-12-01) for the introduction of the previous behavior.
This leaves uses of hook_exists() in two places that matter. The "reference-transaction" check in refs.c, see 67541597670 (refs:
implement reference transaction hook, 2020-06-19), and the prepare-commit-msg hook, see 66618a50f9c (sequencer: run 'prepare-commit-msg' hook, 2018-01-24).
In both of those cases we're saving ourselves CPU time by not preparing data for the hook that we'll then do nothing with if we don't have the hook. So using this "invoked_hook" pattern doesn't make sense in those cases.
More importantly, in those cases the worst we'll do is miss that we "should" run the hook because a new hook appeared, whereas in the pre-commit and pre-merge-commit cases we'll skip an important discard_cache() on the bases of our faulty guess.
I do think none of these races really matter in practice. It would be some one-off issue as a hook was added or removed. I did think it was stupid that we didn't pass a "did this run?" flag instead of doing this guessing at a distance though, so now we're not guessing anymore.
1. https://lore.kernel.org/git/[email protected]/
--- builtin/commit.c | 18 +++++++++++------- builtin/merge.c | 16 ++++++++++------ builtin/receive-pack.c | 8 +++++--- commit.c | 1 + commit.h | 3 ++- hook.c | 4 ++++ hook.h | 10 ++++++++++ sequencer.c | 4 ++-- 8 files changed, 45 insertions(+), 19 deletions(-)
builtin/commit.c +11 -7
@@ -725,11 +725,13 @@ static int prepare_to_commit(const char *index_file, const char *prefix,int clean_message_contents = (cleanup_mode !=COMMIT_MSG_CLEANUP_NONE );int old_display_comment_prefix;int merge_contains_scissors =0 ; +int invoked_hook =0 ;/* This checks and barfs if author is badly specified */ determine_author_info (author_ident); -if (!no_verify &&run_commit_hook (use_editor, index_file,"pre-commit" ,NULL )) +if (!no_verify &&run_commit_hook (use_editor, index_file, &invoked_hook, +"pre-commit" ,NULL ))return 0 ;if (squash_message) {
@@ -1052,10 +1054,10 @@ static int prepare_to_commit(const char *index_file, const char *prefix,return 0 ; } -if (!no_verify &&hook_exists ("pre-commit" )) { +if (!no_verify && invoked_hook) {/* -* Re-read the index as pre-commit hook could have updated it, -* and write it out as a tree. We must do this before we invoke +* Re-read the index as the pre-commit-commit hook was invoked +* and could have updated it. We must do this before we invoke * the editor and after we invoke run_status above. */ discard_cache ();
@@ -1067,7 +1069,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix,return 0 ; } -if (run_commit_hook (use_editor, index_file,"prepare-commit-msg" , +if (run_commit_hook (use_editor, index_file,NULL ,"prepare-commit-msg" ,git_path_commit_editmsg (), hook_arg1, hook_arg2,NULL ))return 0 ;
@@ -1084,7 +1086,8 @@ static int prepare_to_commit(const char *index_file, const char *prefix, }if (!no_verify && -run_commit_hook (use_editor, index_file,"commit-msg" ,git_path_commit_editmsg (),NULL )) { +run_commit_hook (use_editor, index_file,NULL ,"commit-msg" , +git_path_commit_editmsg (),NULL )) {return 0 ; }
@@ -1843,7 +1846,8 @@ int cmd_commit(int argc, const char **argv, const char *prefix)repo_rerere (the_repository,0 );run_auto_maintenance (quiet); -run_commit_hook (use_editor,get_index_file (),"post-commit" ,NULL ); +run_commit_hook (use_editor,get_index_file (),NULL ,"post-commit" , +NULL );if (amend && !no_post_rewrite) {commit_post_rewrite (the_repository, current_head, &oid); }
builtin/merge.c +10 -6
@@ -844,15 +844,18 @@ static void prepare_to_commit(struct commit_list *remoteheads) {struct strbuf msg =STRBUF_INIT ;const char *index_file =get_index_file (); +int invoked_hook =0 ; -if (!no_verify &&run_commit_hook (0 < option_edit, index_file,"pre-merge-commit" ,NULL )) +if (!no_verify &&run_commit_hook (0 < option_edit, index_file, + &invoked_hook,"pre-merge-commit" , +NULL ))abort_commit (remoteheads,NULL );/* -* Re-read the index as pre-merge-commit hook could have updated it, -* and write it out as a tree. We must do this before we invoke +* Re-read the index as the pre-merge-commit hook was invoked +* and could have updated it. We must do this before we invoke * the editor and after we invoke run_status above. */ -if (hook_exists ("pre-merge-commit" )) +if (invoked_hook)discard_cache ();read_cache_from (index_file);strbuf_addbuf (&msg, &merge_msg);
@@ -875,7 +878,8 @@ static void prepare_to_commit(struct commit_list *remoteheads)append_signoff (&msg,ignore_non_trailer (msg.buf, msg.len),0 );write_merge_heads (remoteheads);write_file_buf (git_path_merge_msg (the_repository), msg.buf, msg.len); -if (run_commit_hook (0 < option_edit,get_index_file (),"prepare-commit-msg" , +if (run_commit_hook (0 < option_edit,get_index_file (),NULL , +"prepare-commit-msg" ,git_path_merge_msg (the_repository),"merge" ,NULL ))abort_commit (remoteheads,NULL );if (0 < option_edit) {
@@ -884,7 +888,7 @@ static void prepare_to_commit(struct commit_list *remoteheads) }if (!no_verify &&run_commit_hook (0 < option_edit,get_index_file (), -"commit-msg" , +NULL ,"commit-msg" ,git_path_merge_msg (the_repository),NULL ))abort_commit (remoteheads,NULL );
builtin/receive-pack.c +5 -3
@@ -1428,10 +1428,12 @@ static const char *push_to_deploy(unsigned char *sha1,static const char *push_to_checkout_hook ="push-to-checkout" ;static const char *push_to_checkout (unsigned char *hash, +int *invoked_hook,struct strvec *env,const char *work_tree) {struct run_hooks_opt opt =RUN_HOOKS_OPT_INIT ; + opt.invoked_hook = invoked_hook;strvec_pushf (env,"GIT_WORK_TREE=%s" ,absolute_path (work_tree));strvec_pushv (&opt.env, env->v);
@@ -1446,6 +1448,7 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *w {const char *retval, *work_tree, *git_dir =NULL ;struct strvec env =STRVEC_INIT ; +int invoked_hook =0 ;if (worktree && worktree->path) work_tree = worktree->path;
@@ -1463,10 +1466,9 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *wstrvec_pushf (&env,"GIT_DIR=%s" ,absolute_path (git_dir)); -if (!hook_exists (push_to_checkout_hook)) + retval =push_to_checkout (sha1, &invoked_hook, &env, work_tree); +if (!invoked_hook) retval =push_to_deploy (sha1, &env, work_tree); -else - retval =push_to_checkout (sha1, &env, work_tree);strvec_clear (&env);return retval;
commit.c +1 -0
@@ -1697,6 +1697,7 @@ size_t ignore_non_trailer(const char *buf, size_t len) }int run_commit_hook (int editor_is_used,const char *index_file, +int *invoked_hook,const char *name, ...) {struct run_hooks_opt opt =RUN_HOOKS_OPT_INIT ;
commit.h +2 -1
@@ -363,7 +363,8 @@ int compare_commits_by_commit_date(const void *a_, const void *b_, void *unused)int compare_commits_by_gen_then_commit_date (const void *a_,const void *b_,void *unused);LAST_ARG_MUST_BE_NULL -int run_commit_hook(int editor_is_used, const char *index_file, const char *name, ...); +int run_commit_hook(int editor_is_used, const char *index_file, +int *invoked_hook,const char *name, ...);/* Sign a commit or tag buffer, storing the result in a header. */ int sign_with_header (struct strbuf *buf,const char *keyid);
hook.c +4 -0
@@ -144,6 +144,9 @@ static int notify_hook_finished(int result, hook_cb->rc |= result; +if (hook_cb->invoked_hook) + *hook_cb->invoked_hook =1 ; +return 0 ; }
@@ -158,6 +161,7 @@ int run_hooks(const char *hook_name, const char *hook_path, .rc = 0, .hook_name = hook_name, .options = options, + .invoked_hook = options->invoked_hook, };int jobs =1 ;
hook.h +10 -0
@@ -55,6 +55,15 @@ struct run_hooks_opt* for an example. */ consume_sideband_fn consume_sideband; + +/* +* A pointer which if provided will be set to 1 or 0 depending +* on if a hook was invoked (i.e. existed), regardless of +* whether or not that was successful. Used for avoiding +* TOCTOU races in code that would otherwise call hook_exist() +* after a "maybe hook run" to see if a hook was invoked. +*/ +int *invoked_hook; };#define RUN_HOOKS_OPT_INIT { \
@@ -68,6 +77,7 @@ struct hook_cb_data {const char *hook_name;struct hook *run_me;struct run_hooks_opt *options; +int *invoked_hook; };/**
sequencer.c +2 -2
@@ -1203,7 +1203,7 @@ static int run_prepare_commit_msg_hook(struct repository *r, }else { arg1 ="message" ; } -if (run_commit_hook (0 , r->index_file,"prepare-commit-msg" , name, +if (run_commit_hook (0 , r->index_file,NULL ,"prepare-commit-msg" , name, arg1, arg2,NULL )) ret =error (_ ("'prepare-commit-msg' hook failed" ));
@@ -1533,7 +1533,7 @@ static int try_to_commit(struct repository *r, goto out; } -run_commit_hook (0 , r->index_file,"post-commit" ,NULL ); +run_commit_hook (0 , r->index_file,NULL ,"post-commit" ,NULL );if (flags &AMEND_MSG )commit_post_rewrite (r, current_head, oid);
Signature · 1 line
2.33.0.816.g1ba32acadee
RE: [PATCH v2 00/30] Minimal restart of "config-based-hooks"
In reply to Ævar Arnfjörð BjarmasonQuoted 4 lines · Ævar Arnfjörð Bjarmason
I proposed splitting Emily's "hook config" topic[1] into at least a topic that retains all current behavior of the codebase, and merely refactors existing behavior to new APIs, and then doing behavior changes later.
This is good but can you add an introduction that is more than "Emily's hook config topic"? What is this patch series doing? (assume that Emily's patches don't exist)
Signature · 1 line
Felipe Contreras