Skip to contentSkip to conversation

propose config-based hooks

PATCH Superseded Emily Shaffer 17 participants 649 messages May 21, 2020 – Oct 12, 2021 Page 7 of 7

Subsystems (2)
lore
Share & tools
lore
#608 lore raw

Re: [PATCH v4 00/36] Run hooks via "git run hook" & hook library

In reply to Emily Shaffer

On Thu, Aug 19 2021, Emily Shaffer wrote:

I'm submitting a re-roll of this soon, just some notes.

Quoted 88 lines · Ævar Arnfjörð Bjarmason
Quoted 7 lines

This is a v4 re-roll of the "Base for "config-based-hooks" topic.
* Almost all the callers were just "one-shot" callers, I introduced a new run_hooks_oneshot() function for those, which gets rid of the verbosity around memory management, see e.g. the "builtin/gc.c" in the range-diff below. That run_hooks_oneshot() can also take a NULL set of options.

I discussed this elswhere, but just to make doubly sure that comment doesn't get lost, here's a link to my mail:
https://lore.kernel.org/git/YR2jLdYQA5CVzX5h%40google.com

Quoted 8 lines

Range-diff against v3:

 1:  27c94247f87 =  1:  81fe1ed90d5 Makefile: mark "check" target as .PHONY
 2:  6e164edb0b0 !  2:  0f749530777 Makefile: stop hardcoding {command,config}-list.h
 3:  ddae86802e2 !  3:  644b31fe281 Makefile: remove an out-of-date comment
 4:  58c37e4f06e =  4:  89c4d44b0c3 hook.[ch]: move find_hook() to this new library
 5:  0cf7e078ef4 =  5:  3514e0c0251 hook.c: add a hook_exists() wrapper and use it in bugreport.c
 -:  ----------- >  6:  d5ef40f77dc hook.c users: use "hook_exists()" insted of "find_hook()"
 6:  f343fc7ae66 !  7:  4cfd72722c1 hook-list.h: add a generated list of hooks, like config-list.h

Since these 6 were broken out and then brought back into this series, I'll review them individually downthread.

Quoted 8 lines

7: cf4b06bfdf8 ! 8: 7cb4a4cb69e hook: add 'run' subcommand

    @@ builtin/hook.c (new)
     +#include "strbuf.h"
     +#include "strvec.h"
     +
    ++#define BUILTIN_HOOK_RUN_USAGE \
    ++	N_("git hook run <hook-name> [-- <hook-args>]")
    ++

Nice, now we avoid string duplication here...

Quoted 2 lines
     +static const char * const builtin_hook_usage[] = {
    -+	N_("git hook <command> [...]"),

...and there is no point including this vague thing that has a more specific description right after. Ok.

Quoted 4 lines
    -+	N_("git hook run <hook-name> [-- <hook-args>]"),
    ++	BUILTIN_HOOK_RUN_USAGE,
     +	NULL
     +};

[...]

Quoted 19 lines
    @@ builtin/hook.c (new)
     +{
     +	int i;
     +	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	int rc = 0;
     +	const char *hook_name;
     +	const char *hook_path;
    -+
     +	struct option run_options[] = {
     +		OPT_END(),
     +	};
    ++	int ret;
     +
     +	argc = parse_options(argc, argv, prefix, run_options,
     +			     builtin_hook_run_usage,
    -+			     PARSE_OPT_KEEP_UNKNOWN | PARSE_OPT_KEEP_DASHDASH);
    -+
    -+	if (argc > 1) {
    -+		if (strcmp(argv[1], "--") &&
Quoted 8 lines
    -+		    strcmp(argv[1], "--end-of-options"))
    -+			/* Having a -- for "run" is mandatory */
    -+			usage_with_options(builtin_hook_usage, run_options);
    -+		/* Add our arguments, start after -- */
    -+		for (i = 2 ; i < argc; i++)
    -+			strvec_push(&opt.args, argv[i]);
    -+	}
    ++			     PARSE_OPT_KEEP_DASHDASH);

Nice - this code is being broken up and moved later.

Quoted 10 lines
     +
    -+	/* Need to take into account core.hooksPath */
    -+	git_config(git_default_config, NULL);
    ++	if (!argc)
    ++		goto usage;
     +
     +	/*
    -+	 * We are not using run_hooks() because we'd like to detect
    -+	 * missing hooks. Let's find it ourselves and call
    -+	 * run_found_hooks() instead.
Quoted 3 lines

This comment disappears entirely. I am not too terribly upset about it, since this behavior changes in my series anyway, but it seems like it might have been an accident?

I removed it intentionally, I thought it was clear enough from the control flow, but added it back in. Thanks.

Quoted 266 lines
Quoted 43 lines
    ++	 * Having a -- for "run" when providing <hook-args> is
    ++	 * mandatory.
     +	 */
    ++	if (argc > 1 && strcmp(argv[1], "--") &&
    ++	    strcmp(argv[1], "--end-of-options"))
    ++		goto usage;
    ++
    ++	/* Add our arguments, start after -- */
    ++	for (i = 2 ; i < argc; i++)
    ++		strvec_push(&opt.args, argv[i]);
    ++
    ++	/* Need to take into account core.hooksPath */
    ++	git_config(git_default_config, NULL);
    ++
     +	hook_name = argv[0];
     +	hook_path = find_hook(hook_name);
     +	if (!hook_path) {
     +		error("cannot find a hook named %s", hook_name);
     +		return 1;
     +	}
    -+	rc = run_found_hooks(hook_name, hook_path, &opt);
     +
    ++	ret = run_hooks(hook_name, hook_path, &opt);
     +	run_hooks_opt_clear(&opt);
    -+
    -+	return rc;
    ++	return ret;
    ++usage:
    ++	usage_with_options(builtin_hook_run_usage, run_options);
     +}
     +
     +int cmd_hook(int argc, const char **argv, const char *prefix)
    @@ builtin/hook.c (new)
     +	struct option builtin_hook_options[] = {
     +		OPT_END(),
     +	};
    ++
     +	argc = parse_options(argc, argv, NULL, builtin_hook_options,
     +			     builtin_hook_usage, PARSE_OPT_STOP_AT_NON_OPTION);
     +	if (!argc)
    -+		usage_with_options(builtin_hook_usage, builtin_hook_options);
    ++		goto usage;
     +
Quoted 8 lines
     +	if (!strcmp(argv[0], "run"))
     +		return run(argc, argv, prefix);
    -+	else
    -+		usage_with_options(builtin_hook_usage, builtin_hook_options);
    ++
    ++usage:
    ++	usage_with_options(builtin_hook_usage, builtin_hook_options);
     +}

The goto pattern seems readable enough elsewhere, though.

Quoted 8 lines
      ## command-list.txt ##
    @@ hook.c: int hook_exists(const char *name)
     +	struct hook_cb_data *hook_cb = pp_cb;
     +	struct hook *run_me = hook_cb->run_me;
     +
    ++	if (!run_me)
    ++		return 0;
    ++

Now we protect ourselves from the repeated calls to pick_next_hook(). Good.

Quoted 6 lines
     +	cp->no_stdin = 1;
     +	cp->env = hook_cb->options->env.v;
     +	cp->stdout_to_stderr = 1;
    @@ hook.c: int hook_exists(const char *name)
     +	/* Provide context for errors if necessary */
     +	*pp_task_cb = run_me;
Quoted 8 lines
     +
    ++	/*
    ++	 * This pick_next_hook() will be called again, we're only
    ++	 * running one hook, so indicate that no more work will be
    ++	 * done.
    ++	 */
    ++	hook_cb->run_me = NULL;
    ++

And clearly explaining the caller's behavior here. Sure.

Quoted 9 lines
     +	return 1;
     +}
     +
    @@ hook.c: int hook_exists(const char *name)
     +
     +	hook_cb->rc |= result;
     +
    -+	return 1;
    ++	return 0;

And finally, report "everything is fine" on task finished. Thanks.

Quoted 6 lines

+}
+
-+int run_found_hooks(const char *hook_name, const char *hook_path,
-+ struct run_hooks_opt *options)
++int run_hooks(const char *hook_name, const char *hook_path,
++ struct run_hooks_opt *options)

Ok. With the rename, we only have "verbose way to call it" and "speedy way to call it". No more "run_found_hooks()". Thanks, I think I like this better - I found "run_found_hooks()" to be a little ambiguous.

Quoted 16 lines
    @@ hook.c: int hook_exists(const char *name)
     +				   hook_name);
     +
     +	return cb_data.rc;
    -+}
    -+
    -+int run_hooks(const char *hook_name, struct run_hooks_opt *options)
    -+{
    -+	const char *hook_path;
    -+	int ret;
    -+	if (!options)
    -+		BUG("a struct run_hooks_opt must be provided to run_hooks");
    -+
    -+	hook_path = find_hook(hook_name);
    -+
    -+	/*
Quoted 8 lines
    -+	 * If you need to act on a missing hook, use run_found_hooks()
    -+	 * instead
    -+	 */
    -+	if (!hook_path)
    -+		return 0;
    -+
    -+	ret = run_found_hooks(hook_name, hook_path, options);
    -+	return ret;

Ok - so we wait for another patch to introduce run_hooks_oneshot. Fine.

Quoted 7 lines
     +}
     
      ## hook.h ##
    @@ hook.h: const char *find_hook(const char *name);
     +
     +	/* Args to be passed to each hook */
     +	struct strvec args;
Quoted 8 lines
    -+
    -+	/*
    -+	 * Number of threads to parallelize across, currently a stub,
    -+	 * we use the parallel API for future-proofing, but we always
    -+	 * have one hook of a given name, so this is always an
    -+	 * implicit 1 for now.
    -+	 */
    -+	int jobs;

Thanks. I think this makes more sense than the weird "futureproofing" state it was in before.

Quoted 4 lines
      ## t/t1800-hook.sh (new) ##
    @@ t/t1800-hook.sh (new)
     +
     +test_expect_success 'git hook usage' '
Quoted 8 lines
     +	test_expect_code 129 git hook &&
    -+	test_expect_code 129 git hook -h &&
    -+	test_expect_code 129 git hook run -h
    ++	test_expect_code 129 git hook run &&
    ++	test_expect_code 129 git hook run -h &&
    ++	test_expect_code 129 git hook run --unknown 2>err &&
    ++	grep "unknown option" err
     +'

Nice. I was able to use this test myself when I added 'git hook list', too.

The changes to this patch look good to me.

Quoted 5 lines

8: 7209f73f281 ! 9: 2b8500aa675 gc: use hook library for pre-auto-gc hook
@@ Metadata
## Commit message ##
gc: use hook library for pre-auto-gc hook

Quoted 10 lines

- Using the hook.h library instead of the run-command.h library to run
- pre-auto-gc means that those hooks can be set up in config files, as
- well as in the hookdir. pre-auto-gc is called only from builtin/gc.c.
+ Move the pre-auto-gc hook away from run-command.h to and over to the
+ new hook.h library.
+
+ To do this introduce a simple run_hooks_oneshot() wrapper, we'll be
+ using it extensively for these simple cases of wanting to run a single
+ hook under a given name, and having it free the memory we allocate for
+ us.

Cool, so we have an excuse for introducing the oneshot wrapper right away. Thanks.

Quoted 14 lines
    -@@ builtin/gc.c: static void add_repack_incremental_option(void)
    - 
    - static int need_to_gc(void)
    - {
    -+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
    -+
    - 	/*
    - 	 * Setting gc.auto to 0 or negative can disable the
    - 	 * automatic gc.
     @@ builtin/gc.c: static int need_to_gc(void)
      	else
      		return 0;
      
     -	if (run_hook_le(NULL, "pre-auto-gc", NULL))
Quoted 8 lines
    -+	if (run_hooks("pre-auto-gc", &hook_opt)) {
    -+		run_hooks_opt_clear(&hook_opt);
    ++	if (run_hooks_oneshot("pre-auto-gc", NULL))
      		return 0;
    -+	}
    -+	run_hooks_opt_clear(&hook_opt);
      	return 1;
      }

Nice - the callsite looks much tidier now.

Quoted 7 lines
    +
    + ## hook.c ##
    +@@ hook.c: int run_hooks(const char *hook_name, const char *hook_path,
      
    + 	return cb_data.rc;
    + }
    ++
Quoted 8 lines
    ++int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
    ++{
    ++	const char *hook_path;
    ++	int ret;
    ++	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT;
    ++
    ++	if (!options)
    ++		options = &hook_opt_scratch;

Ah, it still takes an options (optionally, ha ha) because run_hooks() would require the caller to provide the hook list/hook path. Ok. Seems fine, and I like that the '_oneshot' naming change makes it less surprising that run_hooks_opt_clear() will get called for you.

Quoted 23 lines
    ++
    ++	hook_path = find_hook(hook_name);
    ++	if (!hook_path) {
    ++		ret = 0;
    ++		goto cleanup;
    ++	}
    ++
    ++	ret = run_hooks(hook_name, hook_path, options);
    ++cleanup:
    ++	run_hooks_opt_clear(options);
    ++	return ret;
    ++}
    +
    + ## hook.h ##
    +@@ hook.h: void run_hooks_opt_clear(struct run_hooks_opt *o);
    + /**
    +  * Takes an already resolved hook found via find_hook() and runs
    +  * it. Does not call run_hooks_opt_clear() for you.
    ++ *
    ++ * See run_hooks_oneshot() for the simpler one-shot API.
    +  */
    + int run_hooks(const char *hookname, const char *hook_path,
    + 	      struct run_hooks_opt *options);
Quoted 8 lines
    ++
    ++/**
    ++ * Calls find_hook() on your "hook_name" and runs the hooks (if any)
    ++ * with run_hooks().
    ++ *
    ++ * If "options" is provided calls run_hooks_opt_clear() on it for
    ++ * you. If "options" is NULL a scratch one will be provided for you
    ++ * before calling run_hooks().

"A scratch one will be provided for you" doesn't sound quite right - it's not exposed to the caller at all, but the comment sounds like you're handing the caller this scratch struct. Maybe it's better to say "the default options from RUN_HOOKS_OPT_INIT will be used"?

Quoted 4 lines

As a bonus, if you directly reference the initter macro, then it will automatically become clear to folks reading the documentation what the expected parallelism is on their hook, as the parallelism (_SERIES or _PARALLEL) is included on the name of the macro in my change later.

*Nod*, did that.

Quoted 66 lines
Quoted 13 lines
    ++ */
    ++int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options);
    ++
    + #endif

9: e9a1e7cf61e ! 10: 3ee55d2c10f rebase: teach pre-rebase to use hook.h
@@ Metadata
Author: Emily Shaffer <[email protected]>

## Commit message ##
- rebase: teach pre-rebase to use hook.h
+ rebase: convert pre-rebase to use hook.h

Quoted 9 lines

Move the pre-rebase hook away from run-command.h to and over to the new hook.h library.
@@ builtin/rebase.c: int cmd_rebase(int argc, const char **argv, const char *prefix
if (!ok_to_skip_pre_rebase &&
- run_hook_le(NULL, "pre-rebase", options.upstream_arg,
- argc ? argv[0] : NULL, NULL))
-+ run_hooks("pre-rebase", &hook_opt)) {
-+ run_hooks_opt_clear(&hook_opt);
++ run_hooks_oneshot("pre-rebase", &hook_opt))

Ok, it just uses the oneshot call instead. Looks good.

Quoted 3 lines
10:  1d087269303 ! 11:  050f20d14f0 am: convert applypatch hooks to use config
11:  32eec5dc2f0 ! 12:  ac875d284da hooks: convert 'post-checkout' hook to hook library
12:  e9fa3f67593 ! 13:  69763bc2255 merge: use config-based hooks for post-merge hook

The above all just use the oneshot instead, and have some minor tweaks to commit messages to lose references to the config. Looks good.

Quoted 11 lines

13: 12347d901bb ! 14: 2ca1ca1b8e4 git hook run: add an --ignore-missing flag

    @@ Documentation/git-hook.txt: optional `--` (or `--end-of-options`, see linkgit:gi
      linkgit:githooks[5]
     
      ## builtin/hook.c ##
    +@@
    + #include "strvec.h"
    + 
    + #define BUILTIN_HOOK_RUN_USAGE \
    +-	N_("git hook run <hook-name> [-- <hook-args>]")
    ++	N_("git hook run [--ignore-missing] <hook-name> [-- <hook-args>]")

Ok. We update the usage in the macro here instead. Nice.

Quoted 12 lines
     @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
    - 	/*
    - 	 * We are not using run_hooks() because we'd like to detect
    - 	 * missing hooks. Let's find it ourselves and call
    --	 * run_found_hooks() instead.
    -+	 * run_found_hooks() instead...
    - 	 */
    + 	git_config(git_default_config, NULL);
    + 
      	hook_name = argv[0];
    ++	if (ignore_missing)
    ++		return run_hooks_oneshot(hook_name, &opt);
Quoted 8 lines
      	hook_path = find_hook(hook_name);
      	if (!hook_path) {
    -+		/* ... act like run_hooks() under --ignore-missing */
    -+		if (ignore_missing)
    -+			return 0;
      		error("cannot find a hook named %s", hook_name);
    - 		return 1;
    - 	}
Quoted 6 lines

Hm. I actually think this was clearer before. I *think* that is because you and I have different opinions on the clarity of multiple returns :)

But I think it is a little confusing to say "ok, we will call an entirely different entry point in case of this one thing which we will have to check anyways".

Tried to reword this per the above..

Quoted 12 lines
Quoted 11 lines

14: 71d209b4077 ! 15: 5b66b04bec7 send-email: use 'git hook run' for 'sendemail-validate'

    @@ git-send-email.perl: sub validate_patch {
      	if ($repo) {
     +		my $hook_name = 'sendemail-validate';
      		my $hooks_path = $repo->command_oneline('rev-parse', '--git-path', 'hooks');
    --		my $validate_hook = catfile($hooks_path,
    + 		require File::Spec;
    +-		my $validate_hook = File::Spec->catfile($hooks_path,
     -					    'sendemail-validate');
    -+		my $validate_hook = catfile($hooks_path, $hook_name);
    ++		my $validate_hook = File::Spec->catfile($hooks_path, $hook_name);
Quoted 32 lines

Ok, we are adapting to some other change around File::Spec that happened to git-send-email.perl elsewhere. The change to this patch looks good to me.

Quoted 28 lines
      		my $hook_error;
      		if (-x $validate_hook) {
    - 			my $target = abs_path($fn);
    + 			require Cwd;
     @@ git-send-email.perl: sub validate_patch {
      			chdir($repo->wc_path() or $repo->repo_path())
      				or die("chdir: $!");
15:  246a82b55b2 = 16:  14a37a43db2 git-p4: use 'git hook' to run hooks
16:  e3f8482d803 ! 17:  ad5d0e0e7de commit: use hook.h to execute hooks

@@ Metadata
Author: Emily Shaffer <[email protected]>

## Commit message ##
- commit: use hook.h to execute hooks
+ commit: convert {pre-commit,prepare-commit-msg} hook to hook.h

- Teach run_commit_hook() to call hook.h instead of run-command.h. This
- covers 'pre-commit', 'commit-msg', and
- 'prepare-commit-msg'.
-
- Additionally, ask the hook library - not run-command - whether any
- hooks will be run, as it's possible hooks may exist in the config but
- not the hookdir.
-
- Because all but 'post-commit' hooks are expected to make some state
- change, force all but 'post-commit' hook to run in series. 'post-commit'
- "is meant primarily for notification, and cannot affect the outcome of
- git commit," so it is fine to run in parallel.

I tried to address all of the above.

Quoted 3 lines

I am a little bummed that the patch reordering effectively removes the justification for parallel/serial execution for each hook from 'git blame'. Oh well.

But it'll be there if you add it to your commit to migrate them to the parallel macros, and that'll be easier to find with "blame" than having it tangled up in the previous "do all the hook things" changes.

So a note to reword/rebase that into your series...

Quoted 26 lines
Quoted 7 lines

+ Move these hooks hook away from run-command.h to and over to the new
+ hook.h library.

## commit.c ##

Otherwise the main change here is dealing with other improvements earlier in the series, like 'hook_exists()' and 'run_hooks_oneshot()'. Looks fine.

Quoted 8 lines

17: 6ed61071c5e ! 18: 3d3a33e2674 read-cache: convert post-index-change hook to use config
@@ Metadata
Author: Emily Shaffer <[email protected]>

## Commit message ##
- read-cache: convert post-index-change hook to use config
+ read-cache: convert post-index-change to use hook.h

Quoted 5 lines

- By using hook.h instead of run-command.h to run, post-index-change hooks
- can now be specified in the config in addition to the hookdir.
- post-index-change is not run anywhere besides in read-cache.c.
+ Move the post-index-change hook away from run-command.h to and over to
+ the new hook.h library.

Quoted 3 lines

I do not think it is necessary to drop the mention about 'post-index-change' being run anywhere else, but that's just a nit, not worth fixing :)

I removed this in rewording/removing now-false mentions of "this does config".

I didn't add the post-index-change mentiond back in, I figured to do it consistently I'd need to start with pre-auto-gc saying it's only handled in builtin/gc.c & all the way down the line...

Quoted 21 lines
Quoted 11 lines
     
         This removes the last direct user of run_hook_ve(), so we can make the
         function static now. It'll be removed entirely soon.
    @@ read-cache.c: static int do_write_locked_index(struct index_state *istate, struc
     +		     istate->updated_workdir ? "1" : "0",
     +		     istate->updated_skipworktree ? "1" : "0",
     +		     NULL);
    -+	run_hooks("post-index-change", &hook_opt);
    -+	run_hooks_opt_clear(&hook_opt);
    ++	run_hooks_oneshot("post-index-change", &hook_opt);
     +

Mechanical change to using run_hooks_oneshot(). OK.

Quoted 7 lines
      	istate->updated_workdir = 0;
      	istate->updated_skipworktree = 0;

18: e4ef3f4548a ! 19: 893f8666301 receive-pack: convert push-to-checkout hook to hook.h
@@ Metadata
## Commit message ##
receive-pack: convert push-to-checkout hook to hook.h

Quoted 8 lines
Quoted 5 lines
    -    By using hook.h instead of run-command.h to invoke push-to-checkout,
    -    hooks can now be specified in the config as well as in the hookdir.
    -    push-to-checkout is not called anywhere but in builtin/receive-pack.c.
    +    Move the push-to-checkout hook away from run-command.h to and over to
    +    the new hook.h library.

You made a comment in the commit before about the final instance of run_hook_ve; do you want to make a similar one here about run_hook_le?

Willd.o

Quoted 295 lines

The range-diff here is mechanical so looks fine to me.

Quoted 9 lines
19:  e3dda367ec9 = 20:  070433deba5 run-command: remove old run_hook_{le,ve}() hook API
20:  477d75bf579 = 21:  1028e0c1667 run-command: allow stdin for run_processes_parallel
21:  b7c0ee9719a ! 22:  639e59e9ed0 hook: support passing stdin to hooks

@@ Documentation/git-hook.txt: what those are.

## builtin/hook.c ##
@@
+ #include "strvec.h"

Quoted 16 lines

- static const char * const builtin_hook_usage[] = {
- N_("git hook <command> [...]"),
-- N_("git hook run <hook-name> [-- <hook-args>]"),
-+ N_("git hook run [<args>] <hook-name> [-- <hook-args>]"),
- NULL
- };
-
- static const char * const builtin_hook_run_usage[] = {
- N_("git hook run <hook-name> [-- <hook-args>]"),
-+ N_("git hook run [--to-stdin=<path>] <hook-name> [-- <hook-args>]"),
- NULL
- };
+ #define BUILTIN_HOOK_RUN_USAGE \
+- N_("git hook run [--ignore-missing] <hook-name> [-- <hook-args>]")
++ N_("git hook run [--ignore-missing] [--to-stdin=<path>] <hook-name> [-- <hook-args>]")

Just coping with the macro instead of the duplicated strings. Ok.

The range-diff looks fine to me.

Quoted 26 lines

22: 4035069a98c ! 23: 7d1925cca48 am: convert 'post-rewrite' hook to hook.h

    @@ builtin/am.c: static int run_applypatch_msg_hook(struct am_state *state)
      {
     -	struct child_process cp = CHILD_PROCESS_INIT;
     -	const char *hook = find_hook("post-rewrite");
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    - 	int ret;
    - 
    +-	int ret;
    +-
     -	if (!hook)
     -		return 0;
     -
    @@ builtin/am.c: static int run_applypatch_msg_hook(struct am_state *state)
     -	cp.in = xopen(am_path(state, "rewritten"), O_RDONLY);
     -	cp.stdout_to_stderr = 1;
     -	cp.trace2_hook_name = "post-rewrite";
    -+	strvec_push(&opt.args, "rebase");
    -+	opt.path_to_stdin = am_path(state, "rewritten");
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
      
     -	ret = run_command(&cp);
    -+	ret = run_hooks("post-rewrite", &opt);
    ++	strvec_push(&opt.args, "rebase");
    ++	opt.path_to_stdin = am_path(state, "rewritten");
Quoted 8 lines
     -	close(cp.in);
    -+	run_hooks_opt_clear(&opt);
    - 	return ret;
    +-	return ret;
    ++	return run_hooks_oneshot("post-rewrite", &opt);
      }
      
    + /**

Man, the range-diff did a terrible job of summarizing this :)

The patch itself looks great, though, with the switch to using run_hooks_oneshot. Thanks.

Quoted 14 lines

24: da46c859c1c ! 25: 05d1085f7eb hook: provide stdin by string_list or callback
@@ hook.c: static int pick_next_hook(struct child_process *cp,
} else {
cp->no_stdin = 1;
}
-@@ hook.c: int run_found_hooks(const char *hook_name, const char *hook_path,
- run_processes_parallel_tr2(options->jobs,
+@@ hook.c: int run_hooks(const char *hook_name, const char *hook_path,
+ run_processes_parallel_tr2(jobs,
pick_next_hook, notify_start_failure,
- NULL,
@@ hook.c: int run_found_hooks(const char *hook_name, const char *hook_path,
notify_hook_finished,

Quoted 8 lines

&cb_data,
"hook",
-@@ hook.c: int run_hooks(const char *hook_name, struct run_hooks_opt *options) +@@ hook.c: int run_hooks(const char *hook_name, const char *hook_path, +
+ if (options->absolute_path)
+ strbuf_release(&abs_path);
++ free(my_hook.feed_pipe_cb_data);

Nice catch. Thanks.

Quoted 33 lines
25:  7343be28ef4 ! 26:  4b7175af2e5 hook: convert 'post-rewrite' hook in sequencer.c to hook.h
26:  85bf13a0835 ! 27:  3f24e056410 transport: convert pre-push hook to use config
    @@ Metadata
     Author: Emily Shaffer <[email protected]>
     
      ## Commit message ##
    -    transport: convert pre-push hook to use config
    +    transport: convert pre-push hook to hook.h
     
    -    By using the hook.h:run_hooks API, pre-push hooks can be specified in
    -    the config as well as in the hookdir.
    +    Move the pre-push hook away from run-command.h to and over to the new
    +    hook.h library.
     
         Signed-off-by: Emily Shaffer <[email protected]>
         Signed-off-by: Ævar Arnfjörð Bjarmason <[email protected]>
    @@ transport.c: static void die_with_unpushed_submodules(struct string_list *needs_
     -	int ret = 0, x;
     +	int ret = 0;
     +	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct strbuf tmp = STRBUF_INIT;
      	struct ref *r;
     -	struct child_process proc = CHILD_PROCESS_INIT;
     -	struct strbuf buf;
    @@ transport.c: static void die_with_unpushed_submodules(struct string_list *needs_
     -		finish_command(&proc);
     -		return -1;
     -	}
    --
    --	sigchain_push(SIGPIPE, SIG_IGN);
    -+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
    ++	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
Quoted 8 lines
    +-	sigchain_push(SIGPIPE, SIG_IGN);
    +-
     -	strbuf_init(&buf, 256);
     +	strvec_push(&opt.args, transport->remote->name);
     +	strvec_push(&opt.args, transport->url);
      
      	for (r = remote_refs; r; r = r->next) {
    ++		struct strbuf buf = STRBUF_INIT;

Ah. It is not being freed because its lifetime is being managed by to_stdin instead. Ok, cool!

Quoted 26 lines
    ++
      		if (!r->peer_ref) continue;
    -@@ transport.c: static int run_pre_push_hook(struct transport *transport,
    + 		if (r->status == REF_STATUS_REJECT_NONFASTFORWARD) continue;
    + 		if (r->status == REF_STATUS_REJECT_STALE) continue;
      		if (r->status == REF_STATUS_REJECT_REMOTE_UPDATED) continue;
      		if (r->status == REF_STATUS_UPTODATE) continue;
      
     -		strbuf_reset(&buf);
     -		strbuf_addf( &buf, "%s %s %s %s\n",
    -+		strbuf_reset(&tmp);
    -+		strbuf_addf(&tmp, "%s %s %s %s",
    ++		strbuf_addf(&buf, "%s %s %s %s",
      			 r->peer_ref->name, oid_to_hex(&r->new_oid),
      			 r->name, oid_to_hex(&r->old_oid));
     -
    @@ transport.c: static int run_pre_push_hook(struct transport *transport,
     -				ret = -1;
     -			break;
     -		}
    -+		string_list_append(&to_stdin, tmp.buf);
    ++		string_list_append(&to_stdin, strbuf_detach(&buf, NULL));
      	}
      
     -	strbuf_release(&buf);
    @@ transport.c: static int run_pre_push_hook(struct transport *transport,
Quoted 8 lines
     -	x = finish_command(&proc);
     -	if (!ret)
     -		ret = x;
    -+	ret = run_hooks("pre-push", &opt);
    -+	run_hooks_opt_clear(&opt);
    -+	strbuf_release(&tmp);
    ++	ret = run_hooks_oneshot("pre-push", &opt);
    ++	to_stdin.strdup_strings = 1;

And this is a typical change to the oneshot function. Cool.

Quoted 5 lines
     +	string_list_clear(&to_stdin, 0);
      
      	return ret;
 -:  ----------- > 28:  ecf75f33233 hook tests: test for exact "pre-push" hook input
 -:  ----------- > 29:  2c961be94b4 hook tests: use a modern style for "pre-push" tests

I'll take a look at these separately.

Quoted 33 lines

27: 331014bad17 ! 30: 1ce456f9d9d reference-transaction: use hook.h to run hooks

    @@ refs.c: int ref_update_reject_duplicates(struct string_list *refnames,
      				const char *state)
      {
     -	struct child_process proc = CHILD_PROCESS_INIT;
    - 	struct strbuf buf = STRBUF_INIT;
    +-	struct strbuf buf = STRBUF_INIT;
     -	const char *hook;
     +	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
    ++	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
      	int ret = 0, i;
    -+	char o[GIT_MAX_HEXSZ + 1], n[GIT_MAX_HEXSZ + 1];
      
     -	hook = find_hook("reference-transaction");
     -	if (!hook)
    -+	if (!hook_exists("reference-transaction"))
    - 		return ret;
    - 
    +-		return ret;
    +-
     -	strvec_pushl(&proc.args, hook, state, NULL);
     -	proc.in = -1;
     -	proc.stdout_to_stderr = 1;
    @@ refs.c: int ref_update_reject_duplicates(struct string_list *refnames,
     -
     -	ret = start_command(&proc);
     -	if (ret)
    --		return ret;
    --
    ++	if (!hook_exists("reference-transaction"))
    + 		return ret;
    + 
Quoted 8 lines
     -	sigchain_push(SIGPIPE, SIG_IGN);
     +	strvec_push(&opt.args, state);
      
      	for (i = 0; i < transaction->nr; i++) {
      		struct ref_update *update = transaction->updates[i];
    -+		oid_to_hex_r(o, &update->old_oid);
    -+		oid_to_hex_r(n, &update->new_oid);
    ++		struct strbuf buf = STRBUF_INIT;

Ah, this is doing the same thing as with 'pre-push'. Cool.

Quoted 25 lines
      
    - 		strbuf_reset(&buf);
    +-		strbuf_reset(&buf);
     -		strbuf_addf(&buf, "%s %s %s\n",
    --			    oid_to_hex(&update->old_oid),
    --			    oid_to_hex(&update->new_oid),
    --			    update->refname);
    ++		strbuf_addf(&buf, "%s %s %s",
    + 			    oid_to_hex(&update->old_oid),
    + 			    oid_to_hex(&update->new_oid),
    + 			    update->refname);
     -
     -		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
     -			if (errno != EPIPE)
     -				ret = -1;
     -			break;
     -		}
    -+		strbuf_addf(&buf, "%s %s %s", o, n, update->refname);
    -+		string_list_append(&to_stdin, buf.buf);
    ++		string_list_append(&to_stdin, strbuf_detach(&buf, NULL));
      	}
      
     -	close(proc.in);
     -	sigchain_pop(SIGPIPE);
    +-	strbuf_release(&buf);
Quoted 8 lines
     +	opt.feed_pipe = pipe_from_string_list;
     +	opt.feed_pipe_ctx = &to_stdin;
     +
    -+	ret = run_hooks("reference-transaction", &opt);
    -+	run_hooks_opt_clear(&opt);
    - 	strbuf_release(&buf);
    ++	ret = run_hooks_oneshot("reference-transaction", &opt);
    ++	to_stdin.strdup_strings = 1;

And turning on strdup_strings here means we will free them when we call string_list_clear. Ok.

Quoted 12 lines
     +	string_list_clear(&to_stdin, 0);
      
     -	ret |= finish_command(&proc);
32:  db70b59b3bd ! 35:  ceef2f3e804 receive-pack: convert receive hooks to hook.h
    @@ builtin/receive-pack.c: static void hook_output_to_sideband(struct strbuf *outpu
     +{
     +	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
     +	struct receive_hook_feed_context ctx;
    -+	int rc;
     +	struct command *iter = commands;
     +
     +	/* if there are no valid commands, don't invoke the hook at all. */
Quoted 8 lines
    @@ builtin/receive-pack.c: static void hook_output_to_sideband(struct strbuf *outpu
     +	if (!iter)
     +		return 0;
     +
    -+	/* pre-receive hooks should run in series as the hook updates refs */
    -+	if (!strcmp(hook_name, "pre-receive"))
    -+		opt.jobs = 1;
    -+

Hm, interesting. I'll note this as a case to watch for when I add the .jobs option back in.

Thanks. I'll take a look at the newly added/copied over patches next.

- Emily

#609 lore raw

v5 cover Run hooks via "git run hook" & hook library

In reply to Ævar Arnfjörð Bjarmason

This topic refactors our hook execution to use a new hook.[ch] API and rew "git hook run" tool in the case of the Perl & Python scripts that execute hooks. It should contain no functional changes in how hooks are run.

It's mainly authored by Emily Schaffer & is followed-up by her series to make hooks configurable via normal "git config": [1]

See [2] for the v4 of this. A range-diff follows, but some of the main changes:

* The part where we die() if we're asked about a hook not listed in githooks(5) is gone. This makes things in Emily's follow-on topic much easier.

* Various whitespace/small refactoring changes designed to make Emily's follow-on topic smaller.

* Changed "/*" to "/**" comments in hook.h as appropriate (API comments).

* I also re-arranged some functions in the header file to make those API comments easier to follow. I.e. we start with structs, and then various functions grouped by their respective functionality.

* Various small commit message / comment rewording etc., some in response to Emily's feedback, others that I found myself.

[The rest here is a summary of how this topic interacts with a re-roll of Emily's topic coming after this, feel free to skip it if only reading this topic]:

I have my own version of Emily's series rebased on top of this with some significant changes / fixes, which I mainly rebased/hacked up to validate that all the changes here made sense.

That version is an extensive edit not of her v3 (which I believe maps to her own 30ffe98601e) but the cf1f8e34a34 tip I found at the tip of her "config-based-hooks-restart" a couple of days ago)[3], which already incorporated some of my own feedback. That version is at my avar-nasamuffin/config-based-hooks-restart-3[4].

Emily: You should be able to run something like this to get a sensible range-diff of it v.s. what you have now (probably with s/nasamuffin/origin/):

    git range-diff \
    gitster/ab/config-based-hooks-base..nasamuffin/config-based-hooks-restart \
    avar/es-avar/config-based-hooks-6..avar/avar-nasamuffin/config-based-hooks-restart-3

There's a few miscellaneous fixes in there, e.g. it passes all commits with SANITIZE=leak now, and the "hook: allow running non-native hooks" commit is entirely gone except for the doc change, as it wasn't needed with the changes I've made in this v5 (we'd already allow running non-native hooks). Likewise all the fn() and fn_gently() function split wasn't needed anymore, as it's all "gentle" now. It has a trivial CI failure[6] (coccinelle nitpick), but other than that passes all tests.

1. https://lore.kernel.org/git/[email protected]/
2. https://lore.kernel.org/git/[email protected]/
3. https://github.com/nasamuffin/git/tree/config-based-hooks-restart
4. https://github.com/avar/git/tree/avar-nasamuffin/config-based-hooks-restart-3
5. https://github.com/avar/git/runs/3493829211
6. https://github.com/avar/git/runs/3493829211

Emily Shaffer (26):
hook.c: add a hook_exists() wrapper and use it in bugreport.c
hook: add 'run' subcommand
gc: use hook library for pre-auto-gc hook
rebase: convert pre-rebase to use hook.h
am: convert applypatch to use hook.h
hooks: convert 'post-checkout' hook to hook library
merge: convert post-merge to use hook.h
send-email: use 'git hook run' for 'sendemail-validate'
git-p4: use 'git hook' to run hooks
commit: convert {pre-commit,prepare-commit-msg} hook to hook.h
read-cache: convert post-index-change to use hook.h
receive-pack: convert push-to-checkout hook to hook.h
run-command: remove old run_hook_{le,ve}() hook API
run-command: allow stdin for run_processes_parallel
hook: support passing stdin to hooks
am: convert 'post-rewrite' hook to hook.h
run-command: add stdin callback for parallelization
hook: provide stdin by string_list or callback
hook: convert 'post-rewrite' hook in sequencer.c to hook.h
transport: convert pre-push hook to hook.h
reference-transaction: use hook.h to run hooks
run-command: allow capturing of collated output
hooks: allow callers to capture output
receive-pack: convert 'update' hook to hook.h
post-update: use hook.h library
receive-pack: convert receive hooks to hook.h

Ævar Arnfjörð Bjarmason (10):
Makefile: mark "check" target as .PHONY
Makefile: stop hardcoding {command,config}-list.h
Makefile: remove an out-of-date comment
hook.[ch]: move find_hook() from run-command.c to hook.c hook.c users: use "hook_exists()" instead of "find_hook()" hook-list.h: add a generated list of hooks, like config-list.h git hook run: add an --ignore-missing flag
hook tests: test for exact "pre-push" hook input hook tests: use a modern style for "pre-push" tests
hooks: fix a TOCTOU in "did we run a hook?" heuristic

 .gitignore                          |   2 +
 Documentation/git-hook.txt          |  51 +++++
 Documentation/githooks.txt          |   4 +
 Makefile                            |  26 ++-
 builtin.h                           |   1 +
 builtin/am.c                        |  29 +--
 builtin/bugreport.c                 |  46 +----
 builtin/checkout.c                  |  14 +-
 builtin/clone.c                     |   6 +-
 builtin/commit.c                    |  19 +-
 builtin/fetch.c                     |   1 +
 builtin/gc.c                        |   3 +-
 builtin/hook.c                      |  98 ++++++++++
 builtin/merge.c                     |  21 +-
 builtin/rebase.c                    |   6 +-
 builtin/receive-pack.c              | 285 +++++++++++++---------------
 builtin/submodule--helper.c         |   2 +-
 builtin/worktree.c                  |  29 ++-
 command-list.txt                    |   1 +
 commit.c                            |  16 +-
 commit.h                            |   3 +-
 compat/vcbuild/README               |   2 +-
 config.mak.uname                    |   6 +-
 contrib/buildsystems/CMakeLists.txt |   7 +
 generate-hooklist.sh                |  18 ++
 git-p4.py                           |  72 +------
 git-send-email.perl                 |  20 +-
 git.c                               |   1 +
 hook.c                              | 219 +++++++++++++++++++++
 hook.h                              | 127 +++++++++++++
 read-cache.c                        |  11 +-
 refs.c                              |  41 ++--
 reset.c                             |  14 +-
 run-command.c                       | 157 +++++++--------
 run-command.h                       |  55 +++---
 sequencer.c                         |  86 ++++-----
 submodule.c                         |   1 +
 t/helper/test-run-command.c         |  46 ++++-
 t/t0061-run-command.sh              |  37 ++++
 t/t1800-hook.sh                     | 151 +++++++++++++++
 t/t5571-pre-push-hook.sh            |  94 +++++----
 t/t9001-send-email.sh               |   4 +-
 transport.c                         |  57 ++----
 43 files changed, 1278 insertions(+), 611 deletions(-)
 create mode 100644 Documentation/git-hook.txt
 create mode 100644 builtin/hook.c
 create mode 100755 generate-hooklist.sh
 create mode 100644 hook.c
 create mode 100644 hook.h
 create mode 100755 t/t1800-hook.sh

Range-diff against v4:

 1:  81fe1ed90d5 =  1:  ac419613fdc Makefile: mark "check" target as .PHONY
 2:  0f749530777 !  2:  a161b7f0a5c Makefile: stop hardcoding {command,config}-list.h

@@ Commit message
this refactoring we'll only need to add the new file to the GENERATED_H variable, not EXCEPT_HDRS, the vcbuild/README etc.

- I have not tested the Windows-specific change in config.mak.uname
- being made here, but we use other variables from the Makefile in the
- same block, and the GENERATED_H is fully defined before we include
- config.mak.uname.
-
Hardcoding command-list.h there seems to have been a case of copy/paste programming in 976aaedca0 (msvc: add a Makefile target to pre-generate the Visual Studio solution, 2019-07-29). The

 3:  644b31fe281 =  3:  ffef1d3257e Makefile: remove an out-of-date comment
 4:  89c4d44b0c3 !  4:  545e16c6f04 hook.[ch]: move find_hook() to this new library

@@ Metadata
Author: Ævar Arnfjörð Bjarmason <[email protected]>

## Commit message ##
- hook.[ch]: move find_hook() to this new library
+ hook.[ch]: move find_hook() from run-command.c to hook.c

Move the find_hook() function from run-command.c to a new hook.c library. This change establishes a stub library that's pretty
@@ hook.h (new)
+#ifndef HOOK_H
+#define HOOK_H
+
-+/*
++/**
+ * Returns the path to the hook file, or NULL if the hook is missing
+ * or disabled. Note that this points to static storage that will be
-+ * overwritten by further calls to find_hook and run_hook_*.
++ * overwritten by further calls to find_hook().
+ */
+const char *find_hook(const char *name);
+
5: 3514e0c0251 ! 5: a9bc4519e9a hook.c: add a hook_exists() wrapper and use it in bugreport.c

    @@ hook.h
       */
      const char *find_hook(const char *name);
      
    -+/*
    ++/**
     + * A boolean version of find_hook()
     + */
     +int hook_exists(const char *hookname);

6: d5ef40f77dc ! 6: e99ec2e6f8f hook.c users: use "hook_exists()" insted of "find_hook()"
@@ Metadata
Author: Ævar Arnfjörð Bjarmason <[email protected]>

## Commit message ##
- hook.c users: use "hook_exists()" insted of "find_hook()"
+ hook.c users: use "hook_exists()" instead of "find_hook()"

Use the new hook_exists() function instead of find_hook() where the latter was called in boolean contexts. This make subsequent changes in
7: 4cfd72722c1 ! 7: 2ffb2332c8a hook-list.h: add a generated list of hooks, like config-list.h
@@ Commit message
hook-list.h: add a generated list of hooks, like config-list.h

Make githooks(5) the source of truth for what hooks git supports, and
- die hooks we don't know about in find_hook(). This ensures that the
- documentation and the C code's idea about existing hooks doesn't
- diverge.
+ punt out early on hooks we don't know about in find_hook(). This
+ ensures that the documentation and the C code's idea about existing
+ hooks doesn't diverge.

We still have Perl and Python code running its own hooks, but that'll be addressed by Emily Shaffer's upcoming "git hook run" command.
@@ Commit message
listing only knowing about 1/4 of the p4 hooks. It didn't know about the recent "reference-transaction" hook either.

+ We could make the find_hook() function die() or BUG() out if the new
+ known_hook() returned 0, but let's make it return NULL just as it does
+ when it can't find a hook of a known type. Making it die() is overly
+ anal, and unlikely to be what we need in catching stupid typos in the
+ name of some new hook hardcoded in git.git's sources. By making this
+ be tolerant of unknown hook names, changes in a later series to make
+ "git hook run" run arbitrary user-configured hook names will be easier
+ to implement.
+
I have not been able to directly test the CMake change being made here. Since 4c2c38e800 (ci: modification of main.yml to use cmake for vs-build job, 2020-06-26) some of the Windows CI has a hard dependency
@@ generate-hooklist.sh (new)
+ NULL,
+};
+EOF
-
- ## hook.c ##
-@@
- #include "cache.h"
- #include "hook.h"
- #include "run-command.h"
-+#include "hook-list.h"
-+
-+static int known_hook(const char *name)
-+{
-+ const char **p;
-+ size_t len = strlen(name);
-+ for (p = hook_name_list; *p; p++) {
-+ const char *hook = *p;
-+
-+ if (!strncmp(name, hook, len) && hook[len] == '\0')
-+ return 1;
-+ }
-+
-+ return 0;
-+}
-
- const char *find_hook(const char *name)
- {
- static struct strbuf path = STRBUF_INIT;
-
-+ if (!known_hook(name))
-+ die(_("the hook '%s' is not known to git, should be in hook-list.h via githooks(5)"),
-+ name);
-+
- strbuf_reset(&path);
- strbuf_git_path(&path, "hooks/%s", name);
- if (access(path.buf, X_OK) < 0) {
8: 7cb4a4cb69e ! 8: 72dd1010f5b hook: add 'run' subcommand

    @@ builtin/hook.c (new)
     +	/* Need to take into account core.hooksPath */
     +	git_config(git_default_config, NULL);
     +
    ++	/*
    ++	 * We are not using a plain run_hooks() because we'd like to
    ++	 * detect missing hooks. Let's find it ourselves and call
    ++	 * run_hooks() instead.
    ++	 */
     +	hook_name = argv[0];
     +	hook_path = find_hook(hook_name);
     +	if (!hook_path) {
    @@ git.c: static struct cmd_struct commands[] = {
     
      ## hook.c ##
     @@
    + #include "cache.h"
      #include "hook.h"
      #include "run-command.h"
    - #include "hook-list.h"
     +#include "config.h"
      
    - static int known_hook(const char *name)
    + const char *find_hook(const char *name)
      {
    - 	const char **p;
    - 	size_t len = strlen(name);
    -+	static int test_hooks_ok = -1;
    -+
    - 	for (p = hook_name_list; *p; p++) {
    - 		const char *hook = *p;
    - 
    -@@ hook.c: static int known_hook(const char *name)
    - 			return 1;
    - 	}
    - 
    -+	if (test_hooks_ok == -1)
    -+		test_hooks_ok = git_env_bool("GIT_TEST_FAKE_HOOKS", 0);
    -+
    -+	if (test_hooks_ok &&
    -+	    (!strcmp(name, "test-hook") ||
    -+	     !strcmp(name, "does-not-exist")))
    -+		return 1;
    -+
    - 	return 0;
    - }
    - 
     @@ hook.c: int hook_exists(const char *name)
      {
      	return !!find_hook(name);
    @@ hook.h
     @@
      #ifndef HOOK_H
      #define HOOK_H
    -+#include "strbuf.h"
     +#include "strvec.h"
    -+#include "run-command.h"
    - 
    - /*
    -  * Returns the path to the hook file, or NULL if the hook is missing
    -@@ hook.h: const char *find_hook(const char *name);
    -  */
    - int hook_exists(const char *hookname);
    - 
    ++
     +struct hook {
     +	/* The path to the hook */
     +	const char *hook_path;
    @@ hook.h: const char *find_hook(const char *name);
     +	.args = STRVEC_INIT, \
     +}
     +
    -+/*
    -+ * Callback provided to feed_pipe_fn and consume_sideband_fn.
    -+ */
     +struct hook_cb_data {
     +	/* rc reflects the cumulative failure state */
     +	int rc;
    @@ hook.h: const char *find_hook(const char *name);
     +	struct hook *run_me;
     +	struct run_hooks_opt *options;
     +};
    -+
    + 
    + /**
    +  * Returns the path to the hook file, or NULL if the hook is missing
    +@@ hook.h: const char *find_hook(const char *name);
    +  */
    + int hook_exists(const char *hookname);
    + 
    ++/**
    ++ * Clear data from an initialized "struct run_hooks-opt".
    ++ */
     +void run_hooks_opt_clear(struct run_hooks_opt *o);
     +
     +/**
    @@ hook.h: const char *find_hook(const char *name);
     
      ## t/t1800-hook.sh (new) ##
     @@
    -+#!/bin/bash
    ++#!/bin/sh
     +
     +test_description='git-hook command'
     +
    @@ t/t1800-hook.sh (new)
     +	grep "unknown option" err
     +'
     +
    -+test_expect_success 'setup GIT_TEST_FAKE_HOOKS=true to permit "test-hook" and "does-not-exist" names"' '
    -+	GIT_TEST_FAKE_HOOKS=true &&
    -+	export GIT_TEST_FAKE_HOOKS
    -+'
    -+
     +test_expect_success 'git hook run: nonexistent hook' '
     +	cat >stderr.expect <<-\EOF &&
     +	error: cannot find a hook named test-hook
 9:  2b8500aa675 !  9:  821cc9bf11e gc: use hook library for pre-auto-gc hook
    @@ hook.c: int run_hooks(const char *hook_name, const char *hook_path,
     +	}
     +
     +	ret = run_hooks(hook_name, hook_path, options);
    ++
     +cleanup:
     +	run_hooks_opt_clear(options);
    ++
     +	return ret;
     +}
     
    @@ hook.h: void run_hooks_opt_clear(struct run_hooks_opt *o);
     + * with run_hooks().
     + *
     + * If "options" is provided calls run_hooks_opt_clear() on it for
    -+ * you. If "options" is NULL a scratch one will be provided for you
    -+ * before calling run_hooks().
    ++ * you. If "options" is NULL the default options from
    ++ * RUN_HOOKS_OPT_INIT will be used.
     + */
     +int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options);
     +
10:  3ee55d2c10f = 10:  d71c90254ea rebase: convert pre-rebase to use hook.h
11:  050f20d14f0 = 11:  ea3af2ccc4d am: convert applypatch to use hook.h
12:  ac875d284da ! 12:  fed0b52f88f hooks: convert 'post-checkout' hook to hook library
    @@ hook.h: struct run_hooks_opt
      	/* Args to be passed to each hook */
      	struct strvec args;
     +
    -+	/* Resolve and run the "absolute_path(hook)" instead of
    ++	/*
    ++	 * Resolve and run the "absolute_path(hook)" instead of
     +	 * "hook". Used for "git worktree" hooks
     +	 */
     +	int absolute_path;
13:  69763bc2255 = 13:  53d8721a0e3 merge: convert post-merge to use hook.h
14:  2ca1ca1b8e4 ! 14:  d60827a2856 git hook run: add an --ignore-missing flag
    @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
      	};
      	int ret;
     @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
    - 	git_config(git_default_config, NULL);
    - 
    + 	/*
    + 	 * We are not using a plain run_hooks() because we'd like to
    + 	 * detect missing hooks. Let's find it ourselves and call
    +-	 * run_hooks() instead.
    ++	 * run_hooks() instead...
    + 	 */
      	hook_name = argv[0];
     +	if (ignore_missing)
    ++		/* ... act like a plain run_hooks() under --ignore-missing */
     +		return run_hooks_oneshot(hook_name, &opt);
      	hook_path = find_hook(hook_name);
      	if (!hook_path) {
15:  5b66b04bec7 = 15:  d4976a0821f send-email: use 'git hook run' for 'sendemail-validate'
16:  14a37a43db2 = 16:  99f3dcd1945 git-p4: use 'git hook' to run hooks
17:  ad5d0e0e7de = 17:  509761454e6 commit: convert {pre-commit,prepare-commit-msg} hook to hook.h
18:  3d3a33e2674 = 18:  e2c94d95427 read-cache: convert post-index-change to use hook.h
19:  893f8666301 ! 19:  fa7d0d24ea2 receive-pack: convert push-to-checkout hook to hook.h

@@ Commit message
Move the push-to-checkout hook away from run-command.h to and over to the new hook.h library.

+ This removes the last direct user of run_hook_le(), so we could remove
+ that function now, but let's leave that to a follow-up cleanup commit.
+

20:  070433deba5 = 20:  428bb5a6792 run-command: remove old run_hook_{le,ve}() hook API
21:  1028e0c1667 = 21:  994f6ad8602 run-command: allow stdin for run_processes_parallel
22:  639e59e9ed0 = 22:  3ccc654a664 hook: support passing stdin to hooks
23:  7d1925cca48 = 23:  f548e3d15e7 am: convert 'post-rewrite' hook to hook.h
24:  0c24221b522 = 24:  bb119fa7cc0 run-command: add stdin callback for parallelization
25:  05d1085f7eb ! 25:  2439f7752b8 hook: provide stdin by string_list or callback

@@ hook.c: int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *optio
ret = 0;

## hook.h ##
-@@ hook.h: int hook_exists(const char *hookname); +@@
+ #ifndef HOOK_H
+ #define HOOK_H
++#include "strbuf.h"
+ #include "strvec.h"
++#include "run-command.h"
+
struct hook {
/* The path to the hook */
const char *hook_path;
@@ hook.h: struct run_hooks_opt
};

#define RUN_HOOKS_OPT_INIT { \
-@@ hook.h: struct run_hooks_opt
- .args = STRVEC_INIT, \
- }
+@@ hook.h: int run_hooks(const char *hookname, const char *hook_path,
+ */
+ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options);

-+/*
++/**
+ * To specify a 'struct string_list', set 'run_hooks_opt.feed_pipe_ctx' to the
-+ * string_list and set 'run_hooks_opt.feed_pipe' to 'pipe_from_string_list()'. ++ * string_list and set 'run_hooks_opt.feed_pipe' to pipe_from_string_list().
+ * This will pipe each string in the list to stdin, separated by newlines. (Do
+ * not inject your own newlines.)
+ */
+int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb); +
- /*
- * Callback provided to feed_pipe_fn and consume_sideband_fn.
- */
+ #endif

26:  4b7175af2e5 = 26:  48a380b3a91 hook: convert 'post-rewrite' hook in sequencer.c to hook.h
27:  3f24e056410 = 27:  af6b9292aaa transport: convert pre-push hook to hook.h
28:  ecf75f33233 ! 28:  957691f0b6d hook tests: test for exact "pre-push" hook input

@@ Commit message

Extend the tests added in ec55559f937 (push: Add support for pre-push hooks, 2013-01-13) to exhaustively test for the exact input we're
- expecting. This helps a parallel series that's refactoring how the
- hook is called, to e.g. make sure that we don't miss a trailing
- newline.
+ expecting. This ensures that we e.g. don't miss a trailing newline.
+
+ Appending to a file called "actual" is the established convention in
+ this test for hooks, see the rest of the tests added in
+ ec55559f937 (push: Add support for pre-push hooks, 2013-01-13). Let's
+ follow that convention here.

29: 2c961be94b4 ! 29: 88fe2621549 hook tests: use a modern style for "pre-push" tests

    @@ t/t5571-pre-push-hook.sh: echo "$2" >>actual
     -EOF
     -
      test_expect_success 'push with hook' '
    -+	cat >expected <<-EOF &&
    ++	cat >expect <<-EOF &&
     +	parent1
     +	repo1
     +	refs/heads/main $COMMIT2 refs/heads/foreign $COMMIT1
    @@ t/t5571-pre-push-hook.sh: echo "$2" >>actual
     +
      	git push parent1 main:foreign &&
     -	diff expected actual
    -+	test_cmp expected actual
    ++	test_cmp expect actual
      '
      
      test_expect_success 'add a branch' '
30:  1ce456f9d9d = 30:  1d905e81779 reference-transaction: use hook.h to run hooks
31:  6e5f1f5bd3a = 31:  fac56a9d8af run-command: allow capturing of collated output
32:  0b6e9c6d07a = 32:  7d185cdf9d1 hooks: allow callers to capture output
33:  dcf63634338 = 33:  c8150e1239f receive-pack: convert 'update' hook to hook.h
34:  f352a485e59 = 34:  a20ad847c14 post-update: use hook.h library
35:  ceef2f3e804 = 35:  79c380be6ed receive-pack: convert receive hooks to hook.h
36:  b71d7628b40 ! 36:  fe056098534 hooks: fix a TOCTOU in "did we run a hook?" heuristic
    @@ hook.h: struct hook_cb_data {
     +	int *invoked_hook;
      };
      
    - void run_hooks_opt_clear(struct run_hooks_opt *o);
    + /**
     
      ## sequencer.c ##
     @@ sequencer.c: static int run_prepare_commit_msg_hook(struct repository *r,
Signature · 1 line
2.33.0.816.g1ba32acadee
#610 lore raw

PATCH v5 1/36 Makefile: mark "check" target as .PHONY

Fix a bug in 44c9e8594e (Fix up header file dependencies and add sparse checking rules, 2005-07-03), we never marked the phony "check" target as such.

Perhaps we should just remove it, since as of a combination of 912f9980d2 (Makefile: help people who run 'make check' by mistake, 2008-11-11) 0bcd9ae85d (sparse: Fix errors due to missing target-specific variables, 2011-04-21) we've been suggesting the user run "make sparse" directly.

But under that mode it still does something, as well as directing the user to run "make test" under non-sparse. So let's punt that and narrowly fix the PHONY bug.

---
 Makefile | 1 +
 1 file changed, 1 insertion(+)
Makefile +1 -0
index d1feab008fc..5ec23e6b59a 100644
@@ -2941,6 +2941,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
+.PHONY: check
 check: config-list.h command-list.h
 	@if sparse; \
 	then \
Signature · 1 line
2.33.0.816.g1ba32acadee
#611 lore raw

PATCH v5 2/36 Makefile: stop hardcoding {command,config}-list.h

In reply to Ævar Arnfjörð Bjarmason

Change various places that hardcode the names of these two files to refer to either $(GENERATED_H), or to a new generated-hdrs target. That target is consistent with the *-objs targets I recently added in 029bac01a8 (Makefile: add {program,xdiff,test,git,fuzz}-objs & objects targets, 2021-02-23).

A subsequent commit will add a new generated hook-list.h. By doing this refactoring we'll only need to add the new file to the GENERATED_H variable, not EXCEPT_HDRS, the vcbuild/README etc.

Hardcoding command-list.h there seems to have been a case of copy/paste programming in 976aaedca0 (msvc: add a Makefile target to pre-generate the Visual Studio solution, 2019-07-29). The config-list.h was added later in 709df95b78 (help: move list_config_help to builtin/help, 2020-04-16).

---
 Makefile              | 6 ++++--
 compat/vcbuild/README | 2 +-
 config.mak.uname      | 6 +++---
 3 files changed, 8 insertions(+), 6 deletions(-)
Makefile +4 -2
index 5ec23e6b59a..4f1f13b5f0a 100644
@@ -824,6 +824,8 @@ XDIFF_LIB = xdiff/lib.a
 
 GENERATED_H += command-list.h
 GENERATED_H += config-list.h
+.PHONY: generated-hdrs
+generated-hdrs: $(GENERATED_H)
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -2919,7 +2921,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-EXCEPT_HDRS := command-list.h config-list.h unicode-width.h compat/% xdiff/%
+EXCEPT_HDRS := $(GENERATED_H) unicode-width.h compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
 endif
@@ -2942,7 +2944,7 @@ style:
 	git clang-format --style file --diff --extensions c,h
 
 .PHONY: check
-check: config-list.h command-list.h
+check: $(GENERATED_H)
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
compat/vcbuild/README +1 -1
index 51fb083dbbe..29ec1d0f104 100644
@@ -92,7 +92,7 @@ The Steps of Build Git with VS2008
    the git operations.
 
 3. Inside Git's directory run the command:
-       make command-list.h config-list.h
+       make generated-hdrs
    to generate the header file needed to compile git.
 
 4. Then either build Git with the GNU Make Makefile in the Git projects
config.mak.uname +3 -3
index 76516aaa9a5..8aac06eb094 100644
@@ -735,9 +735,9 @@ vcxproj:
 	 echo '</Project>') >git-remote-http/LinkOrCopyRemoteHttp.targets
 	git add -f git/LinkOrCopyBuiltins.targets git-remote-http/LinkOrCopyRemoteHttp.targets
 
-	# Add command-list.h and config-list.h
-	$(MAKE) MSVC=1 SKIP_VCPKG=1 prefix=/mingw64 config-list.h command-list.h
-	git add -f config-list.h command-list.h
+	# Add generated headers
+	$(MAKE) MSVC=1 SKIP_VCPKG=1 prefix=/mingw64 $(GENERATED_H)
+	git add -f $(GENERATED_H)
 
 	# Add scripts
 	rm -f perl/perl.mak
Signature · 1 line
2.33.0.816.g1ba32acadee
#612 lore raw

PATCH v5 3/36 Makefile: remove an out-of-date comment

In reply to Ævar Arnfjörð Bjarmason

This comment added in dfea575017 (Makefile: lazily compute header dependencies, 2010-01-26) has been out of date since 92b88eba9f (Makefile: use git ls-files to list header files, if possible, 2019-03-04), when we did exactly what it tells us not to do and added $(GENERATED_H) to $(OBJECTS) dependencies.

The rest of it was also somewhere between inaccurate and outdated, since as of b8ba629264 (Makefile: fold MISC_H into LIB_H, 2012-06-20) it's not followed by a list of header files, that got moved earlier in the file into LIB_H in 60d24dd255 (Makefile: fold XDIFF_H and VCSSVN_H into LIB_H, 2012-07-06).

Let's just remove it entirely, to the extent that we have anything useful to say here the comment on the "USE_COMPUTED_HEADER_DEPENDENCIES" variable a few lines above this change does the job for us.

---
 Makefile | 7 -------
 1 file changed, 7 deletions(-)
Makefile +0 -7
index 4f1f13b5f0a..53c67f4a726 100644
@@ -2524,13 +2524,6 @@ ifneq ($(dep_files_present),)
 include $(dep_files_present)
 endif
 else
-# Dependencies on header files, for platforms that do not support
-# the gcc -MMD option.
-#
-# Dependencies on automatically generated headers such as command-list.h
-# should _not_ be included here, since they are necessary even when
-# building an object for the first time.
-
 $(OBJECTS): $(LIB_H) $(GENERATED_H)
 endif
 
Signature · 1 line
2.33.0.816.g1ba32acadee
#613 lore raw

PATCH v5 4/36 hook.[ch]: move find_hook() from run-command.c to hook.c

In reply to Ævar Arnfjörð Bjarmason

Move the find_hook() function from run-command.c to a new hook.c library. This change establishes a stub library that's pretty pointless right now, but will see much wider use with Emily Shaffer's upcoming "configuration-based hooks" series.

Eventually all the hook related code will live in hook.[ch]. Let's start that process by moving the simple find_hook() function over as-is.

---
 Makefile               |  1 +
 builtin/am.c           |  1 +
 builtin/bugreport.c    |  2 +-
 builtin/commit.c       |  1 +
 builtin/merge.c        |  1 +
 builtin/receive-pack.c |  1 +
 builtin/worktree.c     |  1 +
 hook.c                 | 37 +++++++++++++++++++++++++++++++++++++
 hook.h                 | 11 +++++++++++
 refs.c                 |  1 +
 run-command.c          | 35 +----------------------------------
 run-command.h          |  7 -------
 sequencer.c            |  1 +
 transport.c            |  1 +
 14 files changed, 59 insertions(+), 42 deletions(-)
 create mode 100644 hook.c
 create mode 100644 hook.h
Makefile +1 -0
index 53c67f4a726..ae256672780 100644
@@ -911,6 +911,7 @@ LIB_OBJS += hash-lookup.o
 LIB_OBJS += hashmap.o
 LIB_OBJS += help.o
 LIB_OBJS += hex.o
+LIB_OBJS += hook.o
 LIB_OBJS += ident.o
 LIB_OBJS += json-writer.o
 LIB_OBJS += kwset.o
builtin/am.c +1 -0
index 0c2ad96b70e..c603f3cebdf 100644
@@ -11,6 +11,7 @@
 #include "parse-options.h"
 #include "dir.h"
 #include "run-command.h"
+#include "hook.h"
 #include "quote.h"
 #include "tempfile.h"
 #include "lockfile.h"
builtin/bugreport.c +1 -1
index 9915a5841de..596f079a7f9 100644
@@ -3,7 +3,7 @@
 #include "strbuf.h"
 #include "help.h"
 #include "compat/compiler.h"
-#include "run-command.h"
+#include "hook.h"
 
 
 static void get_system_info(struct strbuf *sys_info)
builtin/commit.c +1 -0
index 243c626307c..26c8fad6366 100644
@@ -19,6 +19,7 @@
 #include "revision.h"
 #include "wt-status.h"
 #include "run-command.h"
+#include "hook.h"
 #include "refs.h"
 #include "log-tree.h"
 #include "strbuf.h"
builtin/merge.c +1 -0
index febb0c99c93..2fc78f54e43 100644
@@ -13,6 +13,7 @@
 #include "builtin.h"
 #include "lockfile.h"
 #include "run-command.h"
+#include "hook.h"
 #include "diff.h"
 #include "diff-merges.h"
 #include "refs.h"
builtin/receive-pack.c +1 -0
index 2d1f97e1ca7..97aebdc15bd 100644
@@ -7,6 +7,7 @@
 #include "pkt-line.h"
 #include "sideband.h"
 #include "run-command.h"
+#include "hook.h"
 #include "exec-cmd.h"
 #include "commit.h"
 #include "object.h"
builtin/worktree.c +1 -0
index 0d0a80da61f..d22ece93e1a 100644
@@ -8,6 +8,7 @@
 #include "branch.h"
 #include "refs.h"
 #include "run-command.h"
+#include "hook.h"
 #include "sigchain.h"
 #include "submodule.h"
 #include "utf8.h"
hook.c new file +37 -0
new file mode 100644
index 00000000000..c4dbef1d0ef
@@ -0,0 +1,37 @@
+#include "cache.h"
+#include "hook.h"
+#include "run-command.h"
+
+const char *find_hook(const char *name)
+{
+	static struct strbuf path = STRBUF_INIT;
+
+	strbuf_reset(&path);
+	strbuf_git_path(&path, "hooks/%s", name);
+	if (access(path.buf, X_OK) < 0) {
+		int err = errno;
+
+#ifdef STRIP_EXTENSION
+		strbuf_addstr(&path, STRIP_EXTENSION);
+		if (access(path.buf, X_OK) >= 0)
+			return path.buf;
+		if (errno == EACCES)
+			err = errno;
+#endif
+
+		if (err == EACCES && advice_ignored_hook) {
+			static struct string_list advise_given = STRING_LIST_INIT_DUP;
+
+			if (!string_list_lookup(&advise_given, name)) {
+				string_list_insert(&advise_given, name);
+				advise(_("The '%s' hook was ignored because "
+					 "it's not set as executable.\n"
+					 "You can disable this warning with "
+					 "`git config advice.ignoredHook false`."),
+				       path.buf);
+			}
+		}
+		return NULL;
+	}
+	return path.buf;
+}
hook.h new file +11 -0
new file mode 100644
index 00000000000..56a7b3c9164
@@ -0,0 +1,11 @@
+#ifndef HOOK_H
+#define HOOK_H
+
+/**
+ * Returns the path to the hook file, or NULL if the hook is missing
+ * or disabled. Note that this points to static storage that will be
+ * overwritten by further calls to find_hook().
+ */
+const char *find_hook(const char *name);
+
+#endif
refs.c +1 -0
index 8b9f7c3a80a..6211692eaae 100644
@@ -10,6 +10,7 @@
 #include "refs.h"
 #include "refs/refs-internal.h"
 #include "run-command.h"
+#include "hook.h"
 #include "object-store.h"
 #include "object.h"
 #include "tag.h"
run-command.c +1 -34
index f72e72cce73..352f5be1646 100644
@@ -8,6 +8,7 @@
 #include "string-list.h"
 #include "quote.h"
 #include "config.h"
+#include "hook.h"
 
 void child_process_init(struct child_process *child)
 {
@@ -1319,40 +1320,6 @@ int async_with_fork(void)
 #endif
 }
 
-const char *find_hook(const char *name)
-{
-	static struct strbuf path = STRBUF_INIT;
-
-	strbuf_reset(&path);
-	strbuf_git_path(&path, "hooks/%s", name);
-	if (access(path.buf, X_OK) < 0) {
-		int err = errno;
-
-#ifdef STRIP_EXTENSION
-		strbuf_addstr(&path, STRIP_EXTENSION);
-		if (access(path.buf, X_OK) >= 0)
-			return path.buf;
-		if (errno == EACCES)
-			err = errno;
-#endif
-
-		if (err == EACCES && advice_ignored_hook) {
-			static struct string_list advise_given = STRING_LIST_INIT_DUP;
-
-			if (!string_list_lookup(&advise_given, name)) {
-				string_list_insert(&advise_given, name);
-				advise(_("The '%s' hook was ignored because "
-					 "it's not set as executable.\n"
-					 "You can disable this warning with "
-					 "`git config advice.ignoredHook false`."),
-				       path.buf);
-			}
-		}
-		return NULL;
-	}
-	return path.buf;
-}
-
 int run_hook_ve(const char *const *env, const char *name, va_list args)
 {
 	struct child_process hook = CHILD_PROCESS_INIT;
run-command.h +0 -7
index af1296769f9..f76b740f927 100644
@@ -204,13 +204,6 @@ int finish_command_in_signal(struct child_process *);
  */
 int run_command(struct child_process *);
 
-/*
- * Returns the path to the hook file, or NULL if the hook is missing
- * or disabled. Note that this points to static storage that will be
- * overwritten by further calls to find_hook and run_hook_*.
- */
-const char *find_hook(const char *name);
-
 /**
  * Run a hook.
  * The first argument is a pathname to an index file, or NULL
sequencer.c +1 -0
index 1ceb4c0d7fd..3214f02f393 100644
@@ -8,6 +8,7 @@
 #include "sequencer.h"
 #include "tag.h"
 #include "run-command.h"
+#include "hook.h"
 #include "exec-cmd.h"
 #include "utf8.h"
 #include "cache-tree.h"
transport.c +1 -0
index 17e9629710a..77e196f75f5 100644
@@ -2,6 +2,7 @@
 #include "config.h"
 #include "transport.h"
 #include "run-command.h"
+#include "hook.h"
 #include "pkt-line.h"
 #include "fetch-pack.h"
 #include "remote.h"
Signature · 1 line
2.33.0.816.g1ba32acadee
#614 lore raw

PATCH v5 5/36 hook.c: add a hook_exists() wrapper and use it in bugreport.c

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Add a boolean version of the find_hook() function for those callers who are only interested in checking whether the hook exists, not what the path to it is.

---
 builtin/bugreport.c | 2 +-
 hook.c              | 5 +++++
 hook.h              | 5 +++++
 3 files changed, 11 insertions(+), 1 deletion(-)
builtin/bugreport.c +1 -1
index 596f079a7f9..941c8d5e270 100644
@@ -82,7 +82,7 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 	}
 
 	for (i = 0; i < ARRAY_SIZE(hook); i++)
-		if (find_hook(hook[i]))
+		if (hook_exists(hook[i]))
 			strbuf_addf(hook_info, "%s\n", hook[i]);
 }
 
hook.c +5 -0
index c4dbef1d0ef..97cd799a320 100644
@@ -35,3 +35,8 @@ const char *find_hook(const char *name)
 	}
 	return path.buf;
 }
+
+int hook_exists(const char *name)
+{
+	return !!find_hook(name);
+}
hook.h +5 -0
index 56a7b3c9164..368754b1201 100644
@@ -8,4 +8,9 @@
  */
 const char *find_hook(const char *name);
 
+/**
+ * A boolean version of find_hook()
+ */
+int hook_exists(const char *hookname);
+
 #endif
Signature · 1 line
2.33.0.816.g1ba32acadee
#615 lore raw

PATCH v5 6/36 hook.c users: use "hook_exists()" instead of "find_hook()"

In reply to Ævar Arnfjörð Bjarmason

Use the new hook_exists() function instead of find_hook() where the latter was called in boolean contexts. This make subsequent changes in a series where we further refactor the hook API clearer, as we won't conflate wanting to get the path of the hook with checking for its existence.

---
 builtin/commit.c       | 2 +-
 builtin/merge.c        | 2 +-
 builtin/receive-pack.c | 2 +-
 sequencer.c            | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)
builtin/commit.c +1 -1
index 26c8fad6366..f6ca9d04c78 100644
@@ -1052,7 +1052,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (!no_verify && find_hook("pre-commit")) {
+	if (!no_verify && hook_exists("pre-commit")) {
 		/*
 		 * Re-read the index as pre-commit hook could have updated it,
 		 * and write it out as a tree.  We must do this before we invoke
builtin/merge.c +1 -1
index 2fc78f54e43..d75a438298b 100644
@@ -850,7 +850,7 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 	 * and write it out as a tree.  We must do this before we invoke
 	 * the editor and after we invoke run_status above.
 	 */
-	if (find_hook("pre-merge-commit"))
+	if (hook_exists("pre-merge-commit"))
 		discard_cache();
 	read_cache_from(index_file);
 	strbuf_addbuf(&msg, &merge_msg);
builtin/receive-pack.c +1 -1
index 97aebdc15bd..91fa799b66e 100644
@@ -1464,7 +1464,7 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *w
 
 	strvec_pushf(&env, "GIT_DIR=%s", absolute_path(git_dir));
 
-	if (!find_hook(push_to_checkout_hook))
+	if (!hook_exists(push_to_checkout_hook))
 		retval = push_to_deploy(sha1, &env, work_tree);
 	else
 		retval = push_to_checkout(sha1, &env, work_tree);
sequencer.c +1 -1
index 3214f02f393..8999edf8d7b 100644
@@ -1446,7 +1446,7 @@ static int try_to_commit(struct repository *r,
 		}
 	}
 
-	if (find_hook("prepare-commit-msg")) {
+	if (hook_exists("prepare-commit-msg")) {
 		res = run_prepare_commit_msg_hook(r, msg, hook_commit);
 		if (res)
 			goto out;
Signature · 1 line
2.33.0.816.g1ba32acadee
#616 lore raw

PATCH v5 7/36 hook-list.h: add a generated list of hooks, like config-list.h

In reply to Ævar Arnfjörð Bjarmason

Make githooks(5) the source of truth for what hooks git supports, and punt out early on hooks we don't know about in find_hook(). This ensures that the documentation and the C code's idea about existing hooks doesn't diverge.

We still have Perl and Python code running its own hooks, but that'll be addressed by Emily Shaffer's upcoming "git hook run" command.

This resolves a long-standing TODO item in bugreport.c of there being no centralized listing of hooks, and fixes a bug with the bugreport listing only knowing about 1/4 of the p4 hooks. It didn't know about the recent "reference-transaction" hook either.

We could make the find_hook() function die() or BUG() out if the new known_hook() returned 0, but let's make it return NULL just as it does when it can't find a hook of a known type. Making it die() is overly anal, and unlikely to be what we need in catching stupid typos in the name of some new hook hardcoded in git.git's sources. By making this be tolerant of unknown hook names, changes in a later series to make "git hook run" run arbitrary user-configured hook names will be easier to implement.

I have not been able to directly test the CMake change being made here. Since 4c2c38e800 (ci: modification of main.yml to use cmake for vs-build job, 2020-06-26) some of the Windows CI has a hard dependency on CMake, this change works there, and is to my eyes an obviously correct use of a pattern established in previous CMake changes, namely:

- 061c2240b1 (Introduce CMake support for configuring Git,

    2020-06-12)
 - 709df95b78 (help: move list_config_help to builtin/help,
    2020-04-16)

- 976aaedca0 (msvc: add a Makefile target to pre-generate the Visual Studio solution, 2019-07-29)

The LC_ALL=C is needed because at least in my locale the dash ("-") is ignored for the purposes of sorting, which results in a different order. I'm not aware of anything in git that has a hard dependency on the order, but e.g. the bugreport output would end up using whatever locale was in effect when git was compiled.

---
 .gitignore                          |  1 +
 Makefile                            | 10 ++++++-
 builtin/bugreport.c                 | 44 ++++++-----------------------
 contrib/buildsystems/CMakeLists.txt |  7 +++++
 generate-hooklist.sh                | 18 ++++++++++++
 5 files changed, 43 insertions(+), 37 deletions(-)
 create mode 100755 generate-hooklist.sh
.gitignore +1 -0
index 311841f9bed..6be9de41ae8 100644
@@ -190,6 +190,7 @@
 /gitweb/static/gitweb.min.*
 /config-list.h
 /command-list.h
+/hook-list.h
 *.tar.gz
 *.dsc
 *.deb
Makefile +9 -1
index ae256672780..ed06b4e9634 100644
@@ -824,6 +824,8 @@ XDIFF_LIB = xdiff/lib.a
 
 GENERATED_H += command-list.h
 GENERATED_H += config-list.h
+GENERATED_H += hook-list.h
+
 .PHONY: generated-hdrs
 generated-hdrs: $(GENERATED_H)
 
@@ -2231,7 +2233,9 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
+hook.sp hook.s hook.o: hook-list.h
+
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h hook-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2264,6 +2268,10 @@ command-list.h: $(wildcard Documentation/git*.txt)
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+hook-list.h: generate-hooklist.sh Documentation/githooks.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-hooklist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
builtin/bugreport.c +8 -36

62 lines not on this page · raw message

contrib/buildsystems/CMakeLists.txt +7 -0
index 171b4124afe..fd1399c440f 100644
@@ -624,6 +624,13 @@ if(NOT EXISTS ${CMAKE_BINARY_DIR}/config-list.h)
 			OUTPUT_FILE ${CMAKE_BINARY_DIR}/config-list.h)
 endif()
 
+if(NOT EXISTS ${CMAKE_BINARY_DIR}/hook-list.h)
+	message("Generating hook-list.h")
+	execute_process(COMMAND ${SH_EXE} ${CMAKE_SOURCE_DIR}/generate-hooklist.sh
+			WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
+			OUTPUT_FILE ${CMAKE_BINARY_DIR}/hook-list.h)
+endif()
+
 include_directories(${CMAKE_BINARY_DIR})
 
 #build
generate-hooklist.sh new file +18 -0
new file mode 100755
index 00000000000..6d4e56d1a31
@@ -0,0 +1,18 @@
+#!/bin/sh
+#
+# Usage: ./generate-hooklist.sh >hook-list.h
+
+cat <<EOF
+/* Automatically generated by generate-hooklist.sh */
+
+static const char *hook_name_list[] = {
+EOF
+
+sed -n -e '/^~~~~*$/ {x; s/^.*$/	"&",/; p;}; x' \
+	<Documentation/githooks.txt |
+	LC_ALL=C sort
+
+cat <<EOF
+	NULL,
+};
+EOF
Signature · 1 line
2.33.0.816.g1ba32acadee
#617 lore raw

PATCH v5 8/36 hook: add 'run' subcommand

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

In order to enable hooks to be run as an external process, by a standalone Git command, or by tools which wrap Git, provide an external means to run all configured hook commands for a given hook event.

Most of our hooks require more complex functionality than this, but let's start with the bare minimum required to support our simplest hooks.

In terms of implementation the usage_with_options() and "goto usage" pattern here mirrors that of builtin/{commit-graph,multi-pack-index}.c.

---
 .gitignore                 |   1 +
 Documentation/git-hook.txt |  38 +++++++++++
 Documentation/githooks.txt |   4 ++
 Makefile                   |   1 +
 builtin.h                  |   1 +
 builtin/hook.c             |  90 ++++++++++++++++++++++++++
 command-list.txt           |   1 +
 git.c                      |   1 +
 hook.c                     | 101 +++++++++++++++++++++++++++++
 hook.h                     |  39 +++++++++++
 t/t1800-hook.sh            | 128 +++++++++++++++++++++++++++++++++++++
 11 files changed, 405 insertions(+)
 create mode 100644 Documentation/git-hook.txt
 create mode 100644 builtin/hook.c
 create mode 100755 t/t1800-hook.sh
.gitignore +1 -0
index 6be9de41ae8..66189ca3cdc 100644
@@ -77,6 +77,7 @@
 /git-grep
 /git-hash-object
 /git-help
+/git-hook
 /git-http-backend
 /git-http-fetch
 /git-http-push
Documentation/git-hook.txt new file +38 -0
new file mode 100644
index 00000000000..660d6a992a0
@@ -0,0 +1,38 @@
+git-hook(1)
+===========
+
+NAME
+----
+git-hook - run git hooks
+
+SYNOPSIS
+--------
+[verse]
+'git hook' run <hook-name> [-- <hook-args>]
+
+DESCRIPTION
+-----------
+
+This command is an interface to git hooks (see linkgit:githooks[5]).
+Currently it only provides a convenience wrapper for running hooks for
+use by git itself. In the future it might gain other functionality.
+
+SUBCOMMANDS
+-----------
+
+run::
+	Run the `<hook-name>` hook. See linkgit:githooks[5] for
+	the hook names we support.
++
+Any positional arguments to the hook should be passed after an
+optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The
+arguments (if any) differ by hook name, see linkgit:githooks[5] for
+what those are.
+
+SEE ALSO
+--------
+linkgit:githooks[5]
+
+GIT
+---
+Part of the linkgit:git[1] suite
Documentation/githooks.txt +4 -0
index b51959ff941..a16e62bc8c8 100644
@@ -698,6 +698,10 @@ and "0" meaning they were not.
 Only one parameter should be set to "1" when the hook runs.  The hook
 running passing "1", "1" should not be possible.
 
+SEE ALSO
+--------
+linkgit:git-hook[1]
+
 GIT
 ---
 Part of the linkgit:git[1] suite
Makefile +1 -0
index ed06b4e9634..bbd416511a3 100644
@@ -1115,6 +1115,7 @@ BUILTIN_OBJS += builtin/get-tar-commit-id.o
 BUILTIN_OBJS += builtin/grep.o
 BUILTIN_OBJS += builtin/hash-object.o
 BUILTIN_OBJS += builtin/help.o
+BUILTIN_OBJS += builtin/hook.o
 BUILTIN_OBJS += builtin/index-pack.o
 BUILTIN_OBJS += builtin/init-db.o
 BUILTIN_OBJS += builtin/interpret-trailers.o
builtin.h +1 -0
index 16ecd5586f0..91740c15149 100644
@@ -164,6 +164,7 @@ int cmd_get_tar_commit_id(int argc, const char **argv, const char *prefix);
 int cmd_grep(int argc, const char **argv, const char *prefix);
 int cmd_hash_object(int argc, const char **argv, const char *prefix);
 int cmd_help(int argc, const char **argv, const char *prefix);
+int cmd_hook(int argc, const char **argv, const char *prefix);
 int cmd_index_pack(int argc, const char **argv, const char *prefix);
 int cmd_init_db(int argc, const char **argv, const char *prefix);
 int cmd_interpret_trailers(int argc, const char **argv, const char *prefix);
builtin/hook.c new file +90 -0

90 lines not on this page · raw message

command-list.txt +1 -0
index a289f09ed6f..9ccd8e5aebe 100644
@@ -103,6 +103,7 @@ git-grep                                mainporcelain           info
 git-gui                                 mainporcelain
 git-hash-object                         plumbingmanipulators
 git-help                                ancillaryinterrogators          complete
+git-hook                                mainporcelain
 git-http-backend                        synchingrepositories
 git-http-fetch                          synchelpers
 git-http-push                           synchelpers
git.c +1 -0
index 18bed9a9964..540909c391f 100644
@@ -538,6 +538,7 @@ static struct cmd_struct commands[] = {
 	{ "grep", cmd_grep, RUN_SETUP_GENTLY },
 	{ "hash-object", cmd_hash_object },
 	{ "help", cmd_help },
+	{ "hook", cmd_hook, RUN_SETUP },
 	{ "index-pack", cmd_index_pack, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "init", cmd_init_db },
 	{ "init-db", cmd_init_db },
hook.c +101 -0

110 lines not on this page · raw message

hook.h +39 -0
index 368754b1201..d5bc1dc74c9 100644
@@ -1,5 +1,33 @@
 #ifndef HOOK_H
 #define HOOK_H
+#include "strvec.h"
+
+struct hook {
+	/* The path to the hook */
+	const char *hook_path;
+};
+
+struct run_hooks_opt
+{
+	/* Environment vars to be set for each hook */
+	struct strvec env;
+
+	/* Args to be passed to each hook */
+	struct strvec args;
+};
+
+#define RUN_HOOKS_OPT_INIT { \
+	.env = STRVEC_INIT, \
+	.args = STRVEC_INIT, \
+}
+
+struct hook_cb_data {
+	/* rc reflects the cumulative failure state */
+	int rc;
+	const char *hook_name;
+	struct hook *run_me;
+	struct run_hooks_opt *options;
+};
 
 /**
  * Returns the path to the hook file, or NULL if the hook is missing
@@ -13,4 +41,15 @@ const char *find_hook(const char *name);
  */
 int hook_exists(const char *hookname);
 
+/**
+ * Clear data from an initialized "struct run_hooks-opt".
+ */
+void run_hooks_opt_clear(struct run_hooks_opt *o);
+
+/**
+ * Takes an already resolved hook found via find_hook() and runs
+ * it. Does not call run_hooks_opt_clear() for you.
+ */
+int run_hooks(const char *hookname, const char *hook_path,
+	      struct run_hooks_opt *options);
 #endif
t/t1800-hook.sh new file +128 -0

128 lines not on this page · raw message

Signature · 1 line
2.33.0.816.g1ba32acadee
#618 lore raw

PATCH v5 9/36 gc: use hook library for pre-auto-gc hook

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Move the pre-auto-gc hook away from run-command.h to and over to the new hook.h library.

To do this introduce a simple run_hooks_oneshot() wrapper, we'll be using it extensively for these simple cases of wanting to run a single hook under a given name, and having it free the memory we allocate for us.

---
 builtin/gc.c |  3 ++-
 hook.c       | 23 +++++++++++++++++++++++
 hook.h       | 13 +++++++++++++
 3 files changed, 38 insertions(+), 1 deletion(-)
builtin/gc.c +2 -1
index 6ce5ca45126..c91a0786836 100644
@@ -32,6 +32,7 @@
 #include "remote.h"
 #include "object-store.h"
 #include "exec-cmd.h"
+#include "hook.h"
 
 #define FAILED_RUN "failed to run %s"
 
@@ -394,7 +395,7 @@ static int need_to_gc(void)
 	else
 		return 0;
 
-	if (run_hook_le(NULL, "pre-auto-gc", NULL))
+	if (run_hooks_oneshot("pre-auto-gc", NULL))
 		return 0;
 	return 1;
 }
hook.c +23 -0
index dd67801c962..1776895289e 100644
@@ -141,3 +141,26 @@ int run_hooks(const char *hook_name, const char *hook_path,
 
 	return cb_data.rc;
 }
+
+int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
+{
+	const char *hook_path;
+	int ret;
+	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT;
+
+	if (!options)
+		options = &hook_opt_scratch;
+
+	hook_path = find_hook(hook_name);
+	if (!hook_path) {
+		ret = 0;
+		goto cleanup;
+	}
+
+	ret = run_hooks(hook_name, hook_path, options);
+
+cleanup:
+	run_hooks_opt_clear(options);
+
+	return ret;
+}
hook.h +13 -0
index d5bc1dc74c9..b7a106a6e2b 100644
@@ -49,7 +49,20 @@ void run_hooks_opt_clear(struct run_hooks_opt *o);
 /**
  * Takes an already resolved hook found via find_hook() and runs
  * it. Does not call run_hooks_opt_clear() for you.
+ *
+ * See run_hooks_oneshot() for the simpler one-shot API.
  */
 int run_hooks(const char *hookname, const char *hook_path,
 	      struct run_hooks_opt *options);
+
+/**
+ * Calls find_hook() on your "hook_name" and runs the hooks (if any)
+ * with run_hooks().
+ *
+ * If "options" is provided calls run_hooks_opt_clear() on it for
+ * you. If "options" is NULL the default options from
+ * RUN_HOOKS_OPT_INIT will be used.
+ */
+int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options);
+
 #endif
Signature · 1 line
2.33.0.816.g1ba32acadee
#619 lore raw

PATCH v5 10/36 rebase: convert pre-rebase to use hook.h

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Move the pre-rebase hook away from run-command.h to and over to the new hook.h library.

---
 builtin/rebase.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
builtin/rebase.c +4 -2
index c284a7ace19..ee68a1df492 100644
@@ -28,6 +28,7 @@
 #include "sequencer.h"
 #include "rebase-interactive.h"
 #include "reset.h"
+#include "hook.h"
 
 #define DEFAULT_REFLOG_ACTION "rebase"
 
@@ -1313,6 +1314,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	char *squash_onto_name = NULL;
 	int reschedule_failed_exec = -1;
 	int allow_preemptive_ff = 1;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
 	struct option builtin_rebase_options[] = {
 		OPT_STRING(0, "onto", &options.onto_name,
 			   N_("revision"),
@@ -2022,9 +2024,9 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	}
 
 	/* If a hook exists, give it a chance to interrupt*/
+	strvec_pushl(&hook_opt.args, options.upstream_arg, argc ? argv[0] : NULL, NULL);
 	if (!ok_to_skip_pre_rebase &&
-	    run_hook_le(NULL, "pre-rebase", options.upstream_arg,
-			argc ? argv[0] : NULL, NULL))
+	    run_hooks_oneshot("pre-rebase", &hook_opt))
 		die(_("The pre-rebase hook refused to rebase."));
 
 	if (options.flags & REBASE_DIFFSTAT) {
Signature · 1 line
2.33.0.816.g1ba32acadee
#620 lore raw

PATCH v5 11/36 am: convert applypatch to use hook.h

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Teach pre-applypatch, post-applypatch, and applypatch-msg to use the hook.h library instead of the run-command.h library.

---
 builtin/am.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
builtin/am.c +5 -3
index c603f3cebdf..e444b18b64a 100644
@@ -446,9 +446,11 @@ static void am_destroy(const struct am_state *state)
 static int run_applypatch_msg_hook(struct am_state *state)
 {
 	int ret;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 
 	assert(state->msg);
-	ret = run_hook_le(NULL, "applypatch-msg", am_path(state, "final-commit"), NULL);
+	strvec_push(&opt.args, am_path(state, "final-commit"));
+	ret = run_hooks_oneshot("applypatch-msg", &opt);
 
 	if (!ret) {
 		FREE_AND_NULL(state->msg);
@@ -1609,7 +1611,7 @@ static void do_commit(const struct am_state *state)
 	const char *reflog_msg, *author, *committer = NULL;
 	struct strbuf sb = STRBUF_INIT;
 
-	if (run_hook_le(NULL, "pre-applypatch", NULL))
+	if (run_hooks_oneshot("pre-applypatch", NULL))
 		exit(1);
 
 	if (write_cache_as_tree(&tree, 0, NULL))
@@ -1661,7 +1663,7 @@ static void do_commit(const struct am_state *state)
 		fclose(fp);
 	}
 
-	run_hook_le(NULL, "post-applypatch", NULL);
+	run_hooks_oneshot("post-applypatch", NULL);
 
 	strbuf_release(&sb);
 }
Signature · 1 line
2.33.0.816.g1ba32acadee
#621 lore raw

PATCH v5 12/36 hooks: convert 'post-checkout' hook to hook library

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Move the running of the 'post-checkout' hook away from run-command.h to the new hook.h library. For "worktree" this requires a change to it to run the hooks from a given directory.

We could strictly speaking skip the "absolute_path" flag and just check if "dir" is specified, but let's split them up for clarity, as well as for any future user who'd like to set "dir" but not implicitly change the argument to an absolute path.

---
 builtin/checkout.c | 14 +++++++++-----
 builtin/clone.c    |  6 ++++--
 builtin/worktree.c | 28 ++++++++++++----------------
 hook.c             |  9 +++++++++
 hook.h             |  9 +++++++++
 read-cache.c       |  1 +
 reset.c            | 14 ++++++++++----
 7 files changed, 54 insertions(+), 27 deletions(-)
builtin/checkout.c +9 -5
index b5d477919a7..863b02a7d7c 100644
@@ -9,6 +9,7 @@
 #include "config.h"
 #include "diff.h"
 #include "dir.h"
+#include "hook.h"
 #include "ll-merge.h"
 #include "lockfile.h"
 #include "merge-recursive.h"
@@ -106,13 +107,16 @@ struct branch_info {
 static int post_checkout_hook(struct commit *old_commit, struct commit *new_commit,
 			      int changed)
 {
-	return run_hook_le(NULL, "post-checkout",
-			   oid_to_hex(old_commit ? &old_commit->object.oid : null_oid()),
-			   oid_to_hex(new_commit ? &new_commit->object.oid : null_oid()),
-			   changed ? "1" : "0", NULL);
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+
 	/* "new_commit" can be NULL when checking out from the index before
 	   a commit exists. */
-
+	strvec_pushl(&opt.args,
+		     oid_to_hex(old_commit ? &old_commit->object.oid : null_oid()),
+		     oid_to_hex(new_commit ? &new_commit->object.oid : null_oid()),
+		     changed ? "1" : "0",
+		     NULL);
+	return run_hooks_oneshot("post-checkout", &opt);
 }
 
 static int update_some(const struct object_id *oid, struct strbuf *base,
builtin/clone.c +4 -2
index 66fe66679c8..27fc05ee511 100644
@@ -32,6 +32,7 @@
 #include "connected.h"
 #include "packfile.h"
 #include "list-objects-filter-options.h"
+#include "hook.h"
 
 /*
  * Overall FIXMEs:
@@ -775,6 +776,7 @@ static int checkout(int submodule_progress)
 	struct tree *tree;
 	struct tree_desc t;
 	int err = 0;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
 
 	if (option_no_checkout)
 		return 0;
@@ -820,8 +822,8 @@ static int checkout(int submodule_progress)
 	if (write_locked_index(&the_index, &lock_file, COMMIT_LOCK))
 		die(_("unable to write new index file"));
 
-	err |= run_hook_le(NULL, "post-checkout", oid_to_hex(null_oid()),
-			   oid_to_hex(&oid), "1", NULL);
+	strvec_pushl(&hook_opt.args, oid_to_hex(null_oid()), oid_to_hex(&oid), "1", NULL);
+	err |= run_hooks_oneshot("post-checkout", &hook_opt);
 
 	if (!err && (option_recurse_submodules.nr > 0)) {
 		struct strvec args = STRVEC_INIT;
builtin/worktree.c +12 -16
index d22ece93e1a..330867c19bf 100644
@@ -382,22 +382,18 @@ static int add_worktree(const char *path, const char *refname,
 	 * is_junk is cleared, but do return appropriate code when hook fails.
 	 */
 	if (!ret && opts->checkout) {
-		const char *hook = find_hook("post-checkout");
-		if (hook) {
-			const char *env[] = { "GIT_DIR", "GIT_WORK_TREE", NULL };
-			cp.git_cmd = 0;
-			cp.no_stdin = 1;
-			cp.stdout_to_stderr = 1;
-			cp.dir = path;
-			cp.env = env;
-			cp.argv = NULL;
-			cp.trace2_hook_name = "post-checkout";
-			strvec_pushl(&cp.args, absolute_path(hook),
-				     oid_to_hex(null_oid()),
-				     oid_to_hex(&commit->object.oid),
-				     "1", NULL);
-			ret = run_command(&cp);
-		}
+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+
+		strvec_pushl(&opt.env, "GIT_DIR", "GIT_WORK_TREE", NULL);
+		strvec_pushl(&opt.args,
+			     oid_to_hex(null_oid()),
+			     oid_to_hex(&commit->object.oid),
+			     "1",
+			     NULL);
+		opt.dir = path;
+		opt.absolute_path = 1;
+
+		ret = run_hooks_oneshot("post-checkout", &opt);
 	}
 
 	strvec_clear(&child_env);
hook.c +9 -0
index 1776895289e..8a923e52c1b 100644
@@ -63,6 +63,7 @@ static int pick_next_hook(struct child_process *cp,
 	cp->env = hook_cb->options->env.v;
 	cp->stdout_to_stderr = 1;
 	cp->trace2_hook_name = hook_cb->hook_name;
+	cp->dir = hook_cb->options->dir;
 
 	/* add command */
 	strvec_push(&cp->args, run_me->hook_path);
@@ -116,6 +117,7 @@ static int notify_hook_finished(int result,
 int run_hooks(const char *hook_name, const char *hook_path,
 	      struct run_hooks_opt *options)
 {
+	struct strbuf abs_path = STRBUF_INIT;
 	struct hook my_hook = {
 		.hook_path = hook_path,
 	};
@@ -129,6 +131,10 @@ int run_hooks(const char *hook_name, const char *hook_path,
 	if (!options)
 		BUG("a struct run_hooks_opt must be provided to run_hooks");
 
+	if (options->absolute_path) {
+		strbuf_add_absolute_path(&abs_path, hook_path);
+		my_hook.hook_path = abs_path.buf;
+	}
 	cb_data.run_me = &my_hook;
 
 	run_processes_parallel_tr2(jobs,
@@ -139,6 +145,9 @@ int run_hooks(const char *hook_name, const char *hook_path,
 				   "hook",
 				   hook_name);
 
+	if (options->absolute_path)
+		strbuf_release(&abs_path);
+
 	return cb_data.rc;
 }
 
hook.h +9 -0
index b7a106a6e2b..3edd937e198 100644
@@ -14,6 +14,15 @@ struct run_hooks_opt
 
 	/* Args to be passed to each hook */
 	struct strvec args;
+
+	/*
+	 * Resolve and run the "absolute_path(hook)" instead of
+	 * "hook". Used for "git worktree" hooks
+	 */
+	int absolute_path;
+
+	/* Path to initial working directory for subprocess */
+	const char *dir;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
read-cache.c +1 -0
index 9048ef9e905..7a9ffea0885 100644
@@ -28,6 +28,7 @@
 #include "sparse-index.h"
 #include "csum-file.h"
 #include "promisor-remote.h"
+#include "hook.h"
 
 /* Mask for the name length in ce_flags in the on-disk index */
 
reset.c +10 -4
index 79310ae071b..1237ced8a58 100644
@@ -7,6 +7,7 @@
 #include "tree-walk.h"
 #include "tree.h"
 #include "unpack-trees.h"
+#include "hook.h"
 
 int reset_head(struct repository *r, struct object_id *oid, const char *action,
 	       const char *switch_to_branch, unsigned flags,
@@ -125,10 +126,15 @@ int reset_head(struct repository *r, struct object_id *oid, const char *action,
 			ret = create_symref("HEAD", switch_to_branch,
 					    reflog_head);
 	}
-	if (run_hook)
-		run_hook_le(NULL, "post-checkout",
-			    oid_to_hex(orig ? orig : null_oid()),
-			    oid_to_hex(oid), "1", NULL);
+	if (run_hook) {
+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+		strvec_pushl(&opt.args,
+			     oid_to_hex(orig ? orig : null_oid()),
+			     oid_to_hex(oid),
+			     "1",
+			     NULL);
+		run_hooks_oneshot("post-checkout", &opt);
+	}
 
 leave_reset_head:
 	strbuf_release(&msg);
Signature · 1 line
2.33.0.816.g1ba32acadee
#622 lore raw

PATCH v5 13/36 merge: convert post-merge to use hook.h

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Teach post-merge to use the hook.h library instead of the run-command.h library to run hooks.

---
 builtin/merge.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
builtin/merge.c +3 -1
index d75a438298b..ca9b3ba4827 100644
@@ -448,6 +448,7 @@ static void finish(struct commit *head_commit,
 		   const struct object_id *new_head, const char *msg)
 {
 	struct strbuf reflog_message = STRBUF_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	const struct object_id *head = &head_commit->object.oid;
 
 	if (!msg)
@@ -489,7 +490,8 @@ static void finish(struct commit *head_commit,
 	}
 
 	/* Run a post-merge hook */
-	run_hook_le(NULL, "post-merge", squash ? "1" : "0", NULL);
+	strvec_push(&opt.args, squash ? "1" : "0");
+	run_hooks_oneshot("post-merge", &opt);
 
 	apply_autostash(git_path_merge_autostash(the_repository));
 	strbuf_release(&reflog_message);
Signature · 1 line
2.33.0.816.g1ba32acadee
#623 lore raw

PATCH v5 14/36 git hook run: add an --ignore-missing flag

In reply to Ævar Arnfjörð Bjarmason

For certain one-shot hooks we'd like to optimistically run them, and not complain if they don't exist. This will be used by send-email in a subsequent commit.

---
 Documentation/git-hook.txt | 10 +++++++++-
 builtin/hook.c             | 10 ++++++++--
 t/t1800-hook.sh            |  5 +++++
 3 files changed, 22 insertions(+), 3 deletions(-)
Documentation/git-hook.txt +9 -1
index 660d6a992a0..097fb9de63b 100644
@@ -8,7 +8,7 @@ git-hook - run git hooks
 SYNOPSIS
 --------
 [verse]
-'git hook' run <hook-name> [-- <hook-args>]
+'git hook' run [--ignore-missing] <hook-name> [-- <hook-args>]
 
 DESCRIPTION
 -----------
@@ -29,6 +29,14 @@ optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The
 arguments (if any) differ by hook name, see linkgit:githooks[5] for
 what those are.
 
+OPTIONS
+-------
+
+--ignore-missing::
+	Ignore any missing hook by quietly returning zero. Used for
+	tools that want to do a blind one-shot run of a hook that may
+	or may not be present.
+
 SEE ALSO
 --------
 linkgit:githooks[5]
builtin/hook.c +8 -2
index 012a2973b38..76d49e672f4 100644
@@ -7,7 +7,7 @@
 #include "strvec.h"
 
 #define BUILTIN_HOOK_RUN_USAGE \
-	N_("git hook run <hook-name> [-- <hook-args>]")
+	N_("git hook run [--ignore-missing] <hook-name> [-- <hook-args>]")
 
 static const char * const builtin_hook_usage[] = {
 	BUILTIN_HOOK_RUN_USAGE,
@@ -23,9 +23,12 @@ static int run(int argc, const char **argv, const char *prefix)
 {
 	int i;
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	int ignore_missing = 0;
 	const char *hook_name;
 	const char *hook_path;
 	struct option run_options[] = {
+		OPT_BOOL(0, "ignore-missing", &ignore_missing,
+			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
 		OPT_END(),
 	};
 	int ret;
@@ -55,9 +58,12 @@ static int run(int argc, const char **argv, const char *prefix)
 	/*
 	 * We are not using a plain run_hooks() because we'd like to
 	 * detect missing hooks. Let's find it ourselves and call
-	 * run_hooks() instead.
+	 * run_hooks() instead...
 	 */
 	hook_name = argv[0];
+	if (ignore_missing)
+		/* ... act like a plain run_hooks() under --ignore-missing */
+		return run_hooks_oneshot(hook_name, &opt);
 	hook_path = find_hook(hook_name);
 	if (!hook_path) {
 		error("cannot find a hook named %s", hook_name);
t/t1800-hook.sh +5 -0
index d0a3796f28e..feb95b8fc8d 100755
@@ -20,6 +20,11 @@ test_expect_success 'git hook run: nonexistent hook' '
 	test_cmp stderr.expect stderr.actual
 '
 
+test_expect_success 'git hook run: nonexistent hook with --ignore-missing' '
+	git hook run --ignore-missing does-not-exist 2>stderr.actual &&
+	test_must_be_empty stderr.actual
+'
+
 test_expect_success 'git hook run: basic' '
 	write_script .git/hooks/test-hook <<-EOF &&
 	echo Test hook
Signature · 1 line
2.33.0.816.g1ba32acadee
#624 lore raw

PATCH v5 15/36 send-email: use 'git hook run' for 'sendemail-validate'

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Change the "sendmail-validate" hook to be run via the "git hook run" wrapper instead of via a direct invocation.

This is the smallest possibly change to get "send-email" using "git hook run". We still check the hook itself with "-x", and set a "GIT_DIR" variable, both of which are asserted by our tests. We'll need to get rid of this special behavior if we start running N hooks, but for now let's be as close to bug-for-bug compatible as possible.

---
 git-send-email.perl   | 20 ++++++++++++--------
 t/t9001-send-email.sh |  4 ++--
 2 files changed, 14 insertions(+), 10 deletions(-)
git-send-email.perl +12 -8
index e65d969d0bb..126850d974b 100755
@@ -195,13 +195,13 @@ sub format_2822_time {
 my $editor;
 
 sub system_or_msg {
-	my ($args, $msg) = @_;
+	my ($args, $msg, $cmd_name) = @_;
 	system(@$args);
 	my $signalled = $? & 127;
 	my $exit_code = $? >> 8;
 	return unless $signalled or $exit_code;
 
-	my @sprintf_args = ($args->[0], $exit_code);
+	my @sprintf_args = ($cmd_name ? $cmd_name : $args->[0], $exit_code);
 	if (defined $msg) {
 		# Quiet the 'redundant' warning category, except we
 		# need to support down to Perl 5.8, so we can't do a
@@ -2031,10 +2031,10 @@ sub validate_patch {
 	my ($fn, $xfer_encoding) = @_;
 
 	if ($repo) {
+		my $hook_name = 'sendemail-validate';
 		my $hooks_path = $repo->command_oneline('rev-parse', '--git-path', 'hooks');
 		require File::Spec;
-		my $validate_hook = File::Spec->catfile($hooks_path,
-					    'sendemail-validate');
+		my $validate_hook = File::Spec->catfile($hooks_path, $hook_name);
 		my $hook_error;
 		if (-x $validate_hook) {
 			require Cwd;
@@ -2044,13 +2044,17 @@ sub validate_patch {
 			chdir($repo->wc_path() or $repo->repo_path())
 				or die("chdir: $!");
 			local $ENV{"GIT_DIR"} = $repo->repo_path();
-			$hook_error = system_or_msg([$validate_hook, $target]);
+			my @validate_hook = ("git", "hook", "run", "--ignore-missing", $hook_name, "--", $target);
+			$hook_error = system_or_msg(\@validate_hook, undef,
+						       "git hook run $hook_name -- <patch>");
 			chdir($cwd_save) or die("chdir: $!");
 		}
 		if ($hook_error) {
-			die sprintf(__("fatal: %s: rejected by sendemail-validate hook\n" .
-				       "%s\n" .
-				       "warning: no patches were sent\n"), $fn, $hook_error);
+			$hook_error = sprintf(__("fatal: %s: rejected by %s hook\n" .
+						 $hook_error . "\n" .
+						 "warning: no patches were sent\n"),
+					      $fn, $hook_name);
+			die $hook_error;
 		}
 	}
 
t/t9001-send-email.sh +2 -2
index 57fc10e7f82..9ec7d75f0ff 100755
@@ -539,7 +539,7 @@ test_expect_success $PREREQ "--validate respects relative core.hooksPath path" '
 	test_path_is_file my-hooks.ran &&
 	cat >expect <<-EOF &&
 	fatal: longline.patch: rejected by sendemail-validate hook
-	fatal: command '"'"'my-hooks/sendemail-validate'"'"' died with exit code 1
+	fatal: command '"'"'git hook run sendemail-validate -- <patch>'"'"' died with exit code 1
 	warning: no patches were sent
 	EOF
 	test_cmp expect actual
@@ -558,7 +558,7 @@ test_expect_success $PREREQ "--validate respects absolute core.hooksPath path" '
 	test_path_is_file my-hooks.ran &&
 	cat >expect <<-EOF &&
 	fatal: longline.patch: rejected by sendemail-validate hook
-	fatal: command '"'"'$hooks_path/sendemail-validate'"'"' died with exit code 1
+	fatal: command '"'"'git hook run sendemail-validate -- <patch>'"'"' died with exit code 1
 	warning: no patches were sent
 	EOF
 	test_cmp expect actual
Signature · 1 line
2.33.0.816.g1ba32acadee
#625 lore raw

PATCH v5 16/36 git-p4: use 'git hook' to run hooks

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Instead of duplicating the behavior of run-command.h:run_hook_le() in Python, we can directly call 'git hook run'. We emulate the existence check with the --ignore-missing flag.

As this is the last hook execution in git.git to not go through "git hook run" or the hook.[ch] library we can now be absolutely sure that our assertion in hook.c that only hooks known by the generated (from githooks(5)) hook-list.h are permitted.

---
 git-p4.py | 72 ++++++-------------------------------------------------
 1 file changed, 7 insertions(+), 65 deletions(-)
git-p4.py +7 -65
index 2b4500226aa..1f24cbf0bca 100755
@@ -207,71 +207,13 @@ def decode_path(path):
         return path
 
 def run_git_hook(cmd, param=[]):
-    """Execute a hook if the hook exists."""
-    if verbose:
-        sys.stderr.write("Looking for hook: %s\n" % cmd)
-        sys.stderr.flush()
-
-    hooks_path = gitConfig("core.hooksPath")
-    if len(hooks_path) <= 0:
-        hooks_path = os.path.join(os.environ["GIT_DIR"], "hooks")
-
-    if not isinstance(param, list):
-        param=[param]
-
-    # resolve hook file name, OS depdenent
-    hook_file = os.path.join(hooks_path, cmd)
-    if platform.system() == 'Windows':
-        if not os.path.isfile(hook_file):
-            # look for the file with an extension
-            files = glob.glob(hook_file + ".*")
-            if not files:
-                return True
-            files.sort()
-            hook_file = files.pop()
-            while hook_file.upper().endswith(".SAMPLE"):
-                # The file is a sample hook. We don't want it
-                if len(files) > 0:
-                    hook_file = files.pop()
-                else:
-                    return True
-
-    if not os.path.isfile(hook_file) or not os.access(hook_file, os.X_OK):
-        return True
-
-    return run_hook_command(hook_file, param) == 0
-
-def run_hook_command(cmd, param):
-    """Executes a git hook command
-       cmd = the command line file to be executed. This can be
-       a file that is run by OS association.
-
-       param = a list of parameters to pass to the cmd command
-
-       On windows, the extension is checked to see if it should
-       be run with the Git for Windows Bash shell.  If there
-       is no file extension, the file is deemed a bash shell
-       and will be handed off to sh.exe. Otherwise, Windows
-       will be called with the shell to handle the file assocation.
-
-       For non Windows operating systems, the file is called
-       as an executable.
-    """
-    cli = [cmd] + param
-    use_shell = False
-    if platform.system() == 'Windows':
-        (root,ext) = os.path.splitext(cmd)
-        if ext == "":
-            exe_path = os.environ.get("EXEPATH")
-            if exe_path is None:
-                exe_path = ""
-            else:
-                exe_path = os.path.join(exe_path, "bin")
-            cli = [os.path.join(exe_path, "SH.EXE")] + cli
-        else:
-            use_shell = True
-    return subprocess.call(cli, shell=use_shell)
-
+    """args are specified with -a <arg> -a <arg> -a <arg>"""
+    args = ['git', 'hook', 'run', '--ignore-missing', cmd]
+    if param:
+        args.append("--")
+        for p in param:
+            args.append(p)
+    return subprocess.call(args) == 0
 
 def write_pipe(c, stdin):
     if verbose:
Signature · 1 line
2.33.0.816.g1ba32acadee
#626 lore raw

PATCH v5 17/36 commit: convert {pre-commit,prepare-commit-msg} hook to hook.h

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Move these hooks hook away from run-command.h to and over to the new hook.h library.

---
 commit.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)
commit.c +8 -7
index 143f472c0f2..63d7943a86d 100644
@@ -21,6 +21,7 @@
 #include "commit-reach.h"
 #include "run-command.h"
 #include "shallow.h"
+#include "hook.h"
 
 static struct commit_extra_header *read_commit_extra_header_lines(const char *buf, size_t len, const char **);
 
@@ -1698,22 +1699,22 @@ size_t ignore_non_trailer(const char *buf, size_t len)
 int run_commit_hook(int editor_is_used, const char *index_file,
 		    const char *name, ...)
 {
-	struct strvec hook_env = STRVEC_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	va_list args;
-	int ret;
+	const char *arg;
 
-	strvec_pushf(&hook_env, "GIT_INDEX_FILE=%s", index_file);
+	strvec_pushf(&opt.env, "GIT_INDEX_FILE=%s", index_file);
 
 	/*
 	 * Let the hook know that no editor will be launched.
 	 */
 	if (!editor_is_used)
-		strvec_push(&hook_env, "GIT_EDITOR=:");
+		strvec_push(&opt.env, "GIT_EDITOR=:");
 
 	va_start(args, name);
-	ret = run_hook_ve(hook_env.v, name, args);
+	while ((arg = va_arg(args, const char *)))
+		strvec_push(&opt.args, arg);
 	va_end(args);
-	strvec_clear(&hook_env);
 
-	return ret;
+	return run_hooks_oneshot(name, &opt);
 }
Signature · 1 line
2.33.0.816.g1ba32acadee
#627 lore raw

PATCH v5 18/36 read-cache: convert post-index-change to use hook.h

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Move the post-index-change hook away from run-command.h to and over to the new hook.h library.

This removes the last direct user of run_hook_ve(), so we can make the function static now. It'll be removed entirely soon.

---
 read-cache.c  | 10 +++++++---
 run-command.c |  2 +-
 run-command.h |  1 -
 3 files changed, 8 insertions(+), 5 deletions(-)
read-cache.c +7 -3
index 7a9ffea0885..875f6c1dea5 100644
@@ -3069,6 +3069,7 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 {
 	int ret;
 	int was_full = !istate->sparse_index;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
 
 	ret = convert_to_sparse(istate);
 
@@ -3097,9 +3098,12 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 	else
 		ret = close_lock_file_gently(lock);
 
-	run_hook_le(NULL, "post-index-change",
-			istate->updated_workdir ? "1" : "0",
-			istate->updated_skipworktree ? "1" : "0", NULL);
+	strvec_pushl(&hook_opt.args,
+		     istate->updated_workdir ? "1" : "0",
+		     istate->updated_skipworktree ? "1" : "0",
+		     NULL);
+	run_hooks_oneshot("post-index-change", &hook_opt);
+
 	istate->updated_workdir = 0;
 	istate->updated_skipworktree = 0;
 
run-command.c +1 -1
index 352f5be1646..b4341ba1c7b 100644
@@ -1320,7 +1320,7 @@ int async_with_fork(void)
 #endif
 }
 
-int run_hook_ve(const char *const *env, const char *name, va_list args)
+static int run_hook_ve(const char *const *env, const char *name, va_list args)
 {
 	struct child_process hook = CHILD_PROCESS_INIT;
 	const char *p;
run-command.h +0 -1
index f76b740f927..7a867d41217 100644
@@ -219,7 +219,6 @@ int run_command(struct child_process *);
  */
 LAST_ARG_MUST_BE_NULL
 int run_hook_le(const char *const *env, const char *name, ...);
-int run_hook_ve(const char *const *env, const char *name, va_list args);
 
 /*
  * Trigger an auto-gc
Signature · 1 line
2.33.0.816.g1ba32acadee
#628 lore raw

PATCH v5 19/36 receive-pack: convert push-to-checkout hook to hook.h

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Move the push-to-checkout hook away from run-command.h to and over to the new hook.h library.

This removes the last direct user of run_hook_le(), so we could remove that function now, but let's leave that to a follow-up cleanup commit.

---
 builtin/receive-pack.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
builtin/receive-pack.c +5 -2
index 91fa799b66e..a7d03bbc7d3 100644
@@ -1435,9 +1435,12 @@ static const char *push_to_checkout(unsigned char *hash,
 				    struct strvec *env,
 				    const char *work_tree)
 {
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+
 	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
-	if (run_hook_le(env->v, push_to_checkout_hook,
-			hash_to_hex(hash), NULL))
+	strvec_pushv(&opt.env, env->v);
+	strvec_push(&opt.args, hash_to_hex(hash));
+	if (run_hooks_oneshot(push_to_checkout_hook, &opt))
 		return "push-to-checkout hook declined";
 	else
 		return NULL;
Signature · 1 line
2.33.0.816.g1ba32acadee
#629 lore raw

PATCH v5 20/36 run-command: remove old run_hook_{le,ve}() hook API

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

The new hook.h library has replaced all run-command.h hook-related functionality. So let's delete this dead code.

---
 run-command.c | 32 --------------------------------
 run-command.h | 16 ----------------
 2 files changed, 48 deletions(-)
run-command.c +0 -32
index b4341ba1c7b..1399243de8a 100644
@@ -1320,38 +1320,6 @@ int async_with_fork(void)
 #endif
 }
 
-static int run_hook_ve(const char *const *env, const char *name, va_list args)
-{
-	struct child_process hook = CHILD_PROCESS_INIT;
-	const char *p;
-
-	p = find_hook(name);
-	if (!p)
-		return 0;
-
-	strvec_push(&hook.args, p);
-	while ((p = va_arg(args, const char *)))
-		strvec_push(&hook.args, p);
-	hook.env = env;
-	hook.no_stdin = 1;
-	hook.stdout_to_stderr = 1;
-	hook.trace2_hook_name = name;
-
-	return run_command(&hook);
-}
-
-int run_hook_le(const char *const *env, const char *name, ...)
-{
-	va_list args;
-	int ret;
-
-	va_start(args, name);
-	ret = run_hook_ve(env, name, args);
-	va_end(args);
-
-	return ret;
-}
-
 struct io_pump {
 	/* initialized by caller */
 	int fd;
run-command.h +0 -16
index 7a867d41217..cfb6887e4ae 100644
@@ -204,22 +204,6 @@ int finish_command_in_signal(struct child_process *);
  */
 int run_command(struct child_process *);
 
-/**
- * Run a hook.
- * The first argument is a pathname to an index file, or NULL
- * if the hook uses the default index file or no index is needed.
- * The second argument is the name of the hook.
- * The further arguments correspond to the hook arguments.
- * The last argument has to be NULL to terminate the arguments list.
- * If the hook does not exist or is not executable, the return
- * value will be zero.
- * If it is executable, the hook will be executed and the exit
- * status of the hook is returned.
- * On execution, .stdout_to_stderr and .no_stdin will be set.
- */
-LAST_ARG_MUST_BE_NULL
-int run_hook_le(const char *const *env, const char *name, ...);
-
 /*
  * Trigger an auto-gc
  */
Signature · 1 line
2.33.0.816.g1ba32acadee
#630 lore raw

PATCH v5 21/36 run-command: allow stdin for run_processes_parallel

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

While it makes sense not to inherit stdin from the parent process to avoid deadlocking, it's not necessary to completely ban stdin to children. An informed user should be able to configure stdin safely. By setting some_child.process.no_stdin=1 before calling get_next_task() we provide a reasonable default behavior but enable users to set up stdin streaming for themselves during the callback.

some_child.process.stdout_to_stderr, however, remains unmodifiable by get_next_task() - the rest of the run_processes_parallel() API depends on child output in stderr.

---
 run-command.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
run-command.c +8 -1
index 1399243de8a..482ee2d76c6 100644
@@ -1627,6 +1627,14 @@ static int pp_start_one(struct parallel_processes *pp)
 	if (i == pp->max_processes)
 		BUG("bookkeeping is hard");
 
+	/*
+	 * By default, do not inherit stdin from the parent process - otherwise,
+	 * all children would share stdin! Users may overwrite this to provide
+	 * something to the child's stdin by having their 'get_next_task'
+	 * callback assign 0 to .no_stdin and an appropriate integer to .in.
+	 */
+	pp->children[i].process.no_stdin = 1;
+
 	code = pp->get_next_task(&pp->children[i].process,
 				 &pp->children[i].err,
 				 pp->data,
@@ -1638,7 +1646,6 @@ static int pp_start_one(struct parallel_processes *pp)
 	}
 	pp->children[i].process.err = -1;
 	pp->children[i].process.stdout_to_stderr = 1;
-	pp->children[i].process.no_stdin = 1;
 
 	if (start_command(&pp->children[i].process)) {
 		code = pp->start_failure(&pp->children[i].err,
Signature · 1 line
2.33.0.816.g1ba32acadee
#631 lore raw

PATCH v5 22/36 hook: support passing stdin to hooks

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Some hooks (such as post-rewrite) need to take input via stdin. Previously, callers provided stdin to hooks by setting run-command.h:child_process.in, which takes a FD. Callers would open the file in question themselves before calling run-command(). However, since we will now need to seek to the front of the file and read it again for every hook which runs, hook.h:run_command() takes a path and handles FD management itself. Since this file is opened for read only, it should not prevent later parallel execution support.

On the frontend, this is supported by asking for a file path, rather than by reading stdin. Reading directly from stdin would involve caching the entire stdin (to memory or to disk) and reading it back from the beginning to each hook. We'd want to support cases like insufficient memory or storage for the file. While this may prove useful later, for now the path of least resistance is to just ask the user to make this interim file themselves.

---
 Documentation/git-hook.txt |  7 ++++++-
 builtin/hook.c             |  4 +++-
 hook.c                     |  8 +++++++-
 hook.h                     |  3 +++
 t/t1800-hook.sh            | 18 ++++++++++++++++++
 5 files changed, 37 insertions(+), 3 deletions(-)
Documentation/git-hook.txt +6 -1
index 097fb9de63b..fa68c1f3912 100644
@@ -8,7 +8,7 @@ git-hook - run git hooks
 SYNOPSIS
 --------
 [verse]
-'git hook' run [--ignore-missing] <hook-name> [-- <hook-args>]
+'git hook' run [--to-stdin=<path>] [--ignore-missing] <hook-name> [-- <hook-args>]
 
 DESCRIPTION
 -----------
@@ -32,6 +32,11 @@ what those are.
 OPTIONS
 -------
 
+--to-stdin::
+	For "run"; Specify a file which will be streamed into the
+	hook's stdin. The hook will receive the entire file from
+	beginning to EOF.
+
 --ignore-missing::
 	Ignore any missing hook by quietly returning zero. Used for
 	tools that want to do a blind one-shot run of a hook that may
builtin/hook.c +3 -1
index 76d49e672f4..fae69068201 100644
@@ -7,7 +7,7 @@
 #include "strvec.h"
 
 #define BUILTIN_HOOK_RUN_USAGE \
-	N_("git hook run [--ignore-missing] <hook-name> [-- <hook-args>]")
+	N_("git hook run [--ignore-missing] [--to-stdin=<path>] <hook-name> [-- <hook-args>]")
 
 static const char * const builtin_hook_usage[] = {
 	BUILTIN_HOOK_RUN_USAGE,
@@ -29,6 +29,8 @@ static int run(int argc, const char **argv, const char *prefix)
 	struct option run_options[] = {
 		OPT_BOOL(0, "ignore-missing", &ignore_missing,
 			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
+		OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"),
+			   N_("file to read into hooks' stdin")),
 		OPT_END(),
 	};
 	int ret;
hook.c +7 -1
index 8a923e52c1b..d156b0dc800 100644
@@ -59,7 +59,13 @@ static int pick_next_hook(struct child_process *cp,
 	if (!run_me)
 		return 0;
 
-	cp->no_stdin = 1;
+	/* reopen the file for stdin; run_command closes it. */
+	if (hook_cb->options->path_to_stdin) {
+		cp->no_stdin = 0;
+		cp->in = xopen(hook_cb->options->path_to_stdin, O_RDONLY);
+	} else {
+		cp->no_stdin = 1;
+	}
 	cp->env = hook_cb->options->env.v;
 	cp->stdout_to_stderr = 1;
 	cp->trace2_hook_name = hook_cb->hook_name;
hook.h +3 -0
index 3edd937e198..253725e197b 100644
@@ -23,6 +23,9 @@ struct run_hooks_opt
 
 	/* Path to initial working directory for subprocess */
 	const char *dir;
+
+	/* Path to file which should be piped to stdin for each hook */
+	const char *path_to_stdin;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
t/t1800-hook.sh +18 -0
index feb95b8fc8d..6431b19e392 100755
@@ -130,4 +130,22 @@ test_expect_success 'git -c core.hooksPath=<PATH> hook run' '
 	test_cmp expect actual
 '
 
+test_expect_success 'stdin to hooks' '
+	write_script .git/hooks/test-hook <<-\EOF &&
+	echo BEGIN stdin
+	cat
+	echo END stdin
+	EOF
+
+	cat >expect <<-EOF &&
+	BEGIN stdin
+	hello
+	END stdin
+	EOF
+
+	echo hello >input &&
+	git hook run --to-stdin=input test-hook 2>actual &&
+	test_cmp expect actual
+'
+
 test_done
Signature · 1 line
2.33.0.816.g1ba32acadee
#632 lore raw

PATCH v5 23/36 am: convert 'post-rewrite' hook to hook.h

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

---
 builtin/am.c | 20 ++++----------------
 1 file changed, 4 insertions(+), 16 deletions(-)
builtin/am.c +4 -16
index e444b18b64a..9e3d4d9ab44 100644
@@ -467,24 +467,12 @@ static int run_applypatch_msg_hook(struct am_state *state)
  */
 static int run_post_rewrite_hook(const struct am_state *state)
 {
-	struct child_process cp = CHILD_PROCESS_INIT;
-	const char *hook = find_hook("post-rewrite");
-	int ret;
-
-	if (!hook)
-		return 0;
-
-	strvec_push(&cp.args, hook);
-	strvec_push(&cp.args, "rebase");
-
-	cp.in = xopen(am_path(state, "rewritten"), O_RDONLY);
-	cp.stdout_to_stderr = 1;
-	cp.trace2_hook_name = "post-rewrite";
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 
-	ret = run_command(&cp);
+	strvec_push(&opt.args, "rebase");
+	opt.path_to_stdin = am_path(state, "rewritten");
 
-	close(cp.in);
-	return ret;
+	return run_hooks_oneshot("post-rewrite", &opt);
 }
 
 /**
Signature · 1 line
2.33.0.816.g1ba32acadee
#633 lore raw

PATCH v5 24/36 run-command: add stdin callback for parallelization

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

If a user of the run_processes_parallel() API wants to pipe a large amount of information to stdin of each parallel command, that information could exceed the buffer of the pipe allocated for that process's stdin. Generally this is solved by repeatedly writing to child_process.in between calls to start_command() and finish_command(); run_processes_parallel() did not provide users an opportunity to access child_process at that time.

Because the data might be extremely large (for example, a list of all refs received during a push from a client) simply taking a string_list or strbuf is not as scalable as using a callback; the rest of the run_processes_parallel() API also uses callbacks, so making this feature match the rest of the API reduces mental load on the user.

---
 builtin/fetch.c             |  1 +
 builtin/submodule--helper.c |  2 +-
 hook.c                      |  1 +
 run-command.c               | 54 +++++++++++++++++++++++++++++++++++--
 run-command.h               | 17 +++++++++++-
 submodule.c                 |  1 +
 t/helper/test-run-command.c | 31 ++++++++++++++++++---
 t/t0061-run-command.sh      | 30 +++++++++++++++++++++
 8 files changed, 129 insertions(+), 8 deletions(-)
builtin/fetch.c +1 -0
index e064687dbdc..b18ada08842 100644
@@ -1819,6 +1819,7 @@ static int fetch_multiple(struct string_list *list, int max_children)
 		result = run_processes_parallel_tr2(max_children,
 						    &fetch_next_remote,
 						    &fetch_failed_to_start,
+						    NULL,
 						    &fetch_finished,
 						    &state,
 						    "fetch", "parallel/fetch");
builtin/submodule--helper.c +1 -1
index ef2776a9e45..0b73739c160 100644
@@ -2311,7 +2311,7 @@ static int update_submodules(struct submodule_update_clone *suc)
 	int i;
 
 	run_processes_parallel_tr2(suc->max_jobs, update_clone_get_next_task,
-				   update_clone_start_failure,
+				   update_clone_start_failure, NULL,
 				   update_clone_task_finished, suc, "submodule",
 				   "parallel/update");
 
hook.c +1 -0
index d156b0dc800..45983d08aed 100644
@@ -146,6 +146,7 @@ int run_hooks(const char *hook_name, const char *hook_path,
 	run_processes_parallel_tr2(jobs,
 				   pick_next_hook,
 				   notify_start_failure,
+				   NULL,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
run-command.c +52 -2

125 lines not on this page · raw message

run-command.h +16 -1
index cfb6887e4ae..80d394664ae 100644
@@ -422,6 +422,20 @@ typedef int (*start_failure_fn)(struct strbuf *out,
 				void *pp_cb,
 				void *pp_task_cb);
 
+/**
+ * This callback is called repeatedly on every child process who requests
+ * start_command() to create a pipe by setting child_process.in < 0.
+ *
+ * pp_cb is the callback cookie as passed into run_processes_parallel, and
+ * pp_task_cb is the callback cookie as passed into get_next_task_fn.
+ * The contents of 'send' will be read into the pipe and passed to the pipe.
+ *
+ * Return nonzero to close the pipe.
+ */
+typedef int (*feed_pipe_fn)(struct strbuf *pipe,
+			    void *pp_cb,
+			    void *pp_task_cb);
+
 /**
  * This callback is called on every child process that finished processing.
  *
@@ -456,10 +470,11 @@ typedef int (*task_finished_fn)(int result,
 int run_processes_parallel(int n,
 			   get_next_task_fn,
 			   start_failure_fn,
+			   feed_pipe_fn,
 			   task_finished_fn,
 			   void *pp_cb);
 int run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn,
-			       task_finished_fn, void *pp_cb,
+			       feed_pipe_fn, task_finished_fn, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label);
 
 /**
submodule.c +1 -0
index 8e611fe1dbf..db1700a502d 100644
@@ -1632,6 +1632,7 @@ int fetch_populated_submodules(struct repository *r,
 	run_processes_parallel_tr2(max_parallel_jobs,
 				   get_next_submodule,
 				   fetch_start_failure,
+				   NULL,
 				   fetch_finish,
 				   &spf,
 				   "submodule", "parallel/fetch");
t/helper/test-run-command.c +27 -4
index 7ae03dc7123..9348184d303 100644
@@ -32,8 +32,13 @@ static int parallel_next(struct child_process *cp,
 		return 0;
 
 	strvec_pushv(&cp->args, d->argv);
+	cp->in = d->in;
+	cp->no_stdin = d->no_stdin;
 	strbuf_addstr(err, "preloaded output of a child\n");
 	number_callbacks++;
+
+	*task_cb = xmalloc(sizeof(int));
+	*(int*)(*task_cb) = 2;
 	return 1;
 }
 
@@ -55,6 +60,17 @@ static int task_finished(int result,
 	return 1;
 }
 
+static int test_stdin(struct strbuf *pipe, void *cb, void *task_cb)
+{
+	int *lines_remaining = task_cb;
+
+	if (*lines_remaining)
+		strbuf_addf(pipe, "sample stdin %d\n", --(*lines_remaining));
+
+	return !(*lines_remaining);
+}
+
+
 struct testsuite {
 	struct string_list tests, failed;
 	int next;
@@ -185,7 +201,7 @@ static int testsuite(int argc, const char **argv)
 		suite.tests.nr, max_jobs);
 
 	ret = run_processes_parallel(max_jobs, next_test, test_failed,
-				     test_finished, &suite);
+				     test_stdin, test_finished, &suite);
 
 	if (suite.failed.nr > 0) {
 		ret = 1;
@@ -413,15 +429,22 @@ int cmd__run_command(int argc, const char **argv)
 
 	if (!strcmp(argv[1], "run-command-parallel"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, &proc));
+					    NULL, NULL, NULL, &proc));
 
 	if (!strcmp(argv[1], "run-command-abort"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, task_finished, &proc));
+					    NULL, NULL, task_finished, &proc));
 
 	if (!strcmp(argv[1], "run-command-no-jobs"))
 		exit(run_processes_parallel(jobs, no_job,
-					    NULL, task_finished, &proc));
+					    NULL, NULL, task_finished, &proc));
+
+	if (!strcmp(argv[1], "run-command-stdin")) {
+		proc.in = -1;
+		proc.no_stdin = 0;
+		exit (run_processes_parallel(jobs, parallel_next, NULL,
+					     test_stdin, NULL, &proc));
+	}
 
 	fprintf(stderr, "check usage\n");
 	return 1;
t/t0061-run-command.sh +30 -0
index 7d599675e35..87759482ad1 100755
@@ -143,6 +143,36 @@ test_expect_success 'run_command runs in parallel with more tasks than jobs avai
 	test_cmp expect actual
 '
 
+cat >expect <<-EOF
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+EOF
+
+test_expect_success 'run_command listens to stdin' '
+	write_script stdin-script <<-\EOF &&
+	echo "listening for stdin:"
+	while read line; do
+		echo "$line"
+	done
+	EOF
+	test-tool run-command run-command-stdin 2 ./stdin-script 2>actual &&
+	test_cmp expect actual
+'
+
 cat >expect <<-EOF
 preloaded output of a child
 asking for a quick stop
Signature · 1 line
2.33.0.816.g1ba32acadee
#634 lore raw

ab/config-based-hooks-N status (was Re: [PATCH v5 24/36] run-command: add stdin callback for parallelization)

On Thu, Sep 02 2021, Ævar Arnfjörð Bjarmason wrote:

Emily, there's a...:

Quoted 7 lines
t/helper/test-run-command.c +0 -0
index 7ae03dc7123..9348184d303 100644
@@ -32,8 +32,13 @@ static int parallel_next(struct child_process *cp,
 		return 0;
 

t/helper/test-run-command.c static int parallel_next(struct child_process *cp, · line 43

Quoted 10 lines
t/helper/test-run-command.c +5 -0
 	strvec_pushv(&cp->args, d->argv);
+	cp->in = d->in;
+	cp->no_stdin = d->no_stdin;
 	strbuf_addstr(err, "preloaded output of a child\n");
 	number_callbacks++;
+
+	*task_cb = xmalloc(sizeof(int));
+	*(int*)(*task_cb) = 2;
 	return 1;
 }

Probably trivial to solve failure here in t0061-run-command.sh if you compile with SANITIZE=leak. This failed in combination with my[1] (but for anyone reading along, this patch has been ejected from "seen" a while ago).

More generally: The equivalent of 01-07/36 of this series is being merged into "next". As described in a plan to submit this topic incrementally I was hoping to do 08-20/36 next, i.e. up to "run-command:
remove old run_hook_{le,ve}() hook API". See [2] for that plan.

You've been inactive on-list recently, it would be nice to time this so that by the time it gets to 21-36/36 (which I was planning to split in two per [2]) that you'd have time to review/help with outstanding issues etc, for eventually re-submitting your "config based hooks" on top once this all lands.

1. https://lore.kernel.org/git/[email protected]/
2. https://lore.kernel.org/git/[email protected]/

#635 lore raw

Re: ab/config-based-hooks-N status (was Re: [PATCH v5 24/36] run-command: add stdin callback for parallelization)

Quoted 32 lines · Ævar Arnfjörð Bjarmason

On Thu, Sep 02 2021, Ævar Arnfjörð Bjarmason wrote:

Emily, there's a...:

Quoted 7 lines
t/helper/test-run-command.c +0 -0
index 7ae03dc7123..9348184d303 100644
@@ -32,8 +32,13 @@ static int parallel_next(struct child_process *cp,
 		return 0;
 

t/helper/test-run-command.c static int parallel_next(struct child_process *cp, · line 43

Quoted 10 lines
t/helper/test-run-command.c +5 -0
 	strvec_pushv(&cp->args, d->argv);
+	cp->in = d->in;
+	cp->no_stdin = d->no_stdin;
 	strbuf_addstr(err, "preloaded output of a child\n");
 	number_callbacks++;
+
+	*task_cb = xmalloc(sizeof(int));
+	*(int*)(*task_cb) = 2;
 	return 1;
 }

Probably trivial to solve failure here in t0061-run-command.sh if you compile with SANITIZE=leak. This failed in combination with my[1] (but for anyone reading along, this patch has been ejected from "seen" a while ago).

More generally: The equivalent of 01-07/36 of this series is being merged into "next". As described in a plan to submit this topic incrementally I was hoping to do 08-20/36 next, i.e. up to "run-command:
remove old run_hook_{le,ve}() hook API". See [2] for that plan.

Quoted 8 lines

You've been inactive on-list recently, it would be nice to time this so that by the time it gets to 21-36/36 (which I was planning to split in two per [2]) that you'd have time to review/help with outstanding issues etc, for eventually re-submitting your "config based hooks" on top once this all lands.

1. https://lore.kernel.org/git/[email protected]/
2. https://lore.kernel.org/git/[email protected]/

Since I had a reason to look at this again, this fixes it. I've squashed it into my "base" branch, but it won't be in the next batch I submit (I have the cut-off point before this commit):

t/helper/test-run-command.c +13 -3
index fa25bcbbc0d..e9b4214d163 100644
@@ -61,11 +61,21 @@ static void test_consume_sideband(struct strbuf *output, void *cb)
 	fclose(sideband);
 }
 
+static int task_free(int result,
+		     struct strbuf *err,
+		     void *pp_cb,
+		     void *pp_task_cb)
+{
+	free(pp_task_cb);
+	return 0;
+}
+
 static int task_finished(int result,
 			 struct strbuf *err,
 			 void *pp_cb,
 			 void *pp_task_cb)
 {
+	task_free(0, NULL, NULL, pp_task_cb);
 	strbuf_addstr(err, "asking for a quick stop\n");
 	return 1;
 }
@@ -438,7 +448,7 @@ int cmd__run_command(int argc, const char **argv)
 
 	if (!strcmp(argv[1], "run-command-parallel"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, NULL, NULL, &proc));
+					    NULL, NULL, NULL, task_free, &proc));
 
 	if (!strcmp(argv[1], "run-command-abort"))
 		exit(run_processes_parallel(jobs, parallel_next,
@@ -452,12 +462,12 @@ int cmd__run_command(int argc, const char **argv)
 		proc.in = -1;
 		proc.no_stdin = 0;
 		exit (run_processes_parallel(jobs, parallel_next, NULL,
-					     test_stdin, NULL, NULL, &proc));
+					     test_stdin, NULL, task_free, &proc));
 	}
 
 	if (!strcmp(argv[1], "run-command-sideband"))
 		exit(run_processes_parallel(jobs, parallel_next, NULL, NULL,
-					    test_consume_sideband, NULL,
+					    test_consume_sideband, task_free,
 					    &proc));
 
 	fprintf(stderr, "check usage\n");
#636 lore raw

PATCH v5 25/36 hook: provide stdin by string_list or callback

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

In cases where a hook requires only a small amount of information via stdin, it should be simple for users to provide a string_list alone. But in more complicated cases where the stdin is too large to hold in memory, let's instead provide a callback the users can populate line after line.

---
 hook.c | 33 ++++++++++++++++++++++++++++++++-
 hook.h | 29 +++++++++++++++++++++++++++++
 2 files changed, 61 insertions(+), 1 deletion(-)
hook.c +32 -1
index 45983d08aed..7a6ef527443 100644
@@ -48,6 +48,29 @@ void run_hooks_opt_clear(struct run_hooks_opt *o)
 	strvec_clear(&o->args);
 }
 
+int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb)
+{
+	int *item_idx;
+	struct hook *ctx = pp_task_cb;
+	struct hook_cb_data *hook_cb = pp_cb;
+	struct string_list *to_pipe = hook_cb->options->feed_pipe_ctx;
+
+	/* Bootstrap the state manager if necessary. */
+	if (!ctx->feed_pipe_cb_data) {
+		ctx->feed_pipe_cb_data = xmalloc(sizeof(unsigned int));
+		*(int*)ctx->feed_pipe_cb_data = 0;
+	}
+
+	item_idx = ctx->feed_pipe_cb_data;
+
+	if (*item_idx < to_pipe->nr) {
+		strbuf_addf(pipe, "%s\n", to_pipe->items[*item_idx].string);
+		(*item_idx)++;
+		return 0;
+	}
+	return 1;
+}
+
 static int pick_next_hook(struct child_process *cp,
 			  struct strbuf *out,
 			  void *pp_cb,
@@ -63,6 +86,10 @@ static int pick_next_hook(struct child_process *cp,
 	if (hook_cb->options->path_to_stdin) {
 		cp->no_stdin = 0;
 		cp->in = xopen(hook_cb->options->path_to_stdin, O_RDONLY);
+	} else if (hook_cb->options->feed_pipe) {
+		/* ask for start_command() to make a pipe for us */
+		cp->in = -1;
+		cp->no_stdin = 0;
 	} else {
 		cp->no_stdin = 1;
 	}
@@ -146,7 +173,7 @@ int run_hooks(const char *hook_name, const char *hook_path,
 	run_processes_parallel_tr2(jobs,
 				   pick_next_hook,
 				   notify_start_failure,
-				   NULL,
+				   options->feed_pipe,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
@@ -154,6 +181,7 @@ int run_hooks(const char *hook_name, const char *hook_path,
 
 	if (options->absolute_path)
 		strbuf_release(&abs_path);
+	free(my_hook.feed_pipe_cb_data);
 
 	return cb_data.rc;
 }
@@ -167,6 +195,9 @@ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 	if (!options)
 		options = &hook_opt_scratch;
 
+	if (options->path_to_stdin && options->feed_pipe)
+		BUG("choose only one method to populate stdin");
+
 	hook_path = find_hook(hook_name);
 	if (!hook_path) {
 		ret = 0;
hook.h +29 -0
index 253725e197b..4e26b8d658d 100644
@@ -1,10 +1,18 @@
 #ifndef HOOK_H
 #define HOOK_H
+#include "strbuf.h"
 #include "strvec.h"
+#include "run-command.h"
 
 struct hook {
 	/* The path to the hook */
 	const char *hook_path;
+
+	/*
+	 * Use this to keep state for your feed_pipe_fn if you are using
+	 * run_hooks_opt.feed_pipe. Otherwise, do not touch it.
+	 */
+	void *feed_pipe_cb_data;
 };
 
 struct run_hooks_opt
@@ -26,6 +34,19 @@ struct run_hooks_opt
 
 	/* Path to file which should be piped to stdin for each hook */
 	const char *path_to_stdin;
+
+	/*
+	 * Callback and state pointer to ask for more content to pipe to stdin.
+	 * Will be called repeatedly, for each hook. See
+	 * hook.c:pipe_from_stdin() for an example. Keep per-hook state in
+	 * hook.feed_pipe_cb_data (per process). Keep initialization context in
+	 * feed_pipe_ctx (shared by all processes).
+	 *
+	 * See 'pipe_from_string_list()' for info about how to specify a
+	 * string_list as the stdin input instead of writing your own handler.
+	 */
+	feed_pipe_fn feed_pipe;
+	void *feed_pipe_ctx;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
@@ -77,4 +98,12 @@ int run_hooks(const char *hookname, const char *hook_path,
  */
 int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options);
 
+/**
+ * To specify a 'struct string_list', set 'run_hooks_opt.feed_pipe_ctx' to the
+ * string_list and set 'run_hooks_opt.feed_pipe' to pipe_from_string_list().
+ * This will pipe each string in the list to stdin, separated by newlines.  (Do
+ * not inject your own newlines.)
+ */
+int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb);
+
 #endif
Signature · 1 line
2.33.0.816.g1ba32acadee
#637 lore raw

PATCH v5 26/36 hook: convert 'post-rewrite' hook in sequencer.c to hook.h

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

By using 'hook.h' for 'post-rewrite', we simplify hook invocations by not needing to put together our own 'struct child_process'.

The signal handling that's being removed by this commit now takes place in run-command.h:run_processes_parallel(), so it is OK to remove them here.

---
 sequencer.c | 79 ++++++++++++++++++++++-------------------------------
 1 file changed, 32 insertions(+), 47 deletions(-)
sequencer.c +32 -47
index 8999edf8d7b..549b583b277 100644
@@ -35,6 +35,7 @@
 #include "commit-reach.h"
 #include "rebase-interactive.h"
 #include "reset.h"
+#include "string-list.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -1147,33 +1148,27 @@ int update_head_with_reflog(const struct commit *old_head,
 static int run_rewrite_hook(const struct object_id *oldoid,
 			    const struct object_id *newoid)
 {
-	struct child_process proc = CHILD_PROCESS_INIT;
-	const char *argv[3];
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct strbuf tmp = STRBUF_INIT;
+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 	int code;
-	struct strbuf sb = STRBUF_INIT;
 
-	argv[0] = find_hook("post-rewrite");
-	if (!argv[0])
-		return 0;
+	strvec_push(&opt.args, "amend");
 
-	argv[1] = "amend";
-	argv[2] = NULL;
-
-	proc.argv = argv;
-	proc.in = -1;
-	proc.stdout_to_stderr = 1;
-	proc.trace2_hook_name = "post-rewrite";
-
-	code = start_command(&proc);
-	if (code)
-		return code;
-	strbuf_addf(&sb, "%s %s\n", oid_to_hex(oldoid), oid_to_hex(newoid));
-	sigchain_push(SIGPIPE, SIG_IGN);
-	write_in_full(proc.in, sb.buf, sb.len);
-	close(proc.in);
-	strbuf_release(&sb);
-	sigchain_pop(SIGPIPE);
-	return finish_command(&proc);
+	strbuf_addf(&tmp,
+		    "%s %s",
+		    oid_to_hex(oldoid),
+		    oid_to_hex(newoid));
+	string_list_append(&to_stdin, tmp.buf);
+
+	opt.feed_pipe = pipe_from_string_list;
+	opt.feed_pipe_ctx = &to_stdin;
+
+	code = run_hooks_oneshot("post-rewrite", &opt);
+
+	strbuf_release(&tmp);
+	string_list_clear(&to_stdin, 0);
+	return code;
 }
 
 void commit_post_rewrite(struct repository *r,
@@ -4526,30 +4521,20 @@ static int pick_commits(struct repository *r,
 		flush_rewritten_pending();
 		if (!stat(rebase_path_rewritten_list(), &st) &&
 				st.st_size > 0) {
-			struct child_process child = CHILD_PROCESS_INIT;
-			const char *post_rewrite_hook =
-				find_hook("post-rewrite");
-
-			child.in = open(rebase_path_rewritten_list(), O_RDONLY);
-			child.git_cmd = 1;
-			strvec_push(&child.args, "notes");
-			strvec_push(&child.args, "copy");
-			strvec_push(&child.args, "--for-rewrite=rebase");
+			struct child_process notes_cp = CHILD_PROCESS_INIT;
+			struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+
+			notes_cp.in = open(rebase_path_rewritten_list(), O_RDONLY);
+			notes_cp.git_cmd = 1;
+			strvec_push(&notes_cp.args, "notes");
+			strvec_push(&notes_cp.args, "copy");
+			strvec_push(&notes_cp.args, "--for-rewrite=rebase");
 			/* we don't care if this copying failed */
-			run_command(&child);
-
-			if (post_rewrite_hook) {
-				struct child_process hook = CHILD_PROCESS_INIT;
-
-				hook.in = open(rebase_path_rewritten_list(),
-					O_RDONLY);
-				hook.stdout_to_stderr = 1;
-				hook.trace2_hook_name = "post-rewrite";
-				strvec_push(&hook.args, post_rewrite_hook);
-				strvec_push(&hook.args, "rebase");
-				/* we don't care if this hook failed */
-				run_command(&hook);
-			}
+			run_command(&notes_cp);
+
+			hook_opt.path_to_stdin = rebase_path_rewritten_list();
+			strvec_push(&hook_opt.args, "rebase");
+			run_hooks_oneshot("post-rewrite", &hook_opt);
 		}
 		apply_autostash(rebase_path_autostash());
 
Signature · 1 line
2.33.0.816.g1ba32acadee
#638 lore raw

PATCH v5 27/36 transport: convert pre-push hook to hook.h

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Move the pre-push hook away from run-command.h to and over to the new hook.h library.

---
 transport.c | 56 ++++++++++++++---------------------------------------
 1 file changed, 14 insertions(+), 42 deletions(-)
transport.c +14 -42
index 77e196f75f5..4ca8fc0391d 100644
@@ -1203,63 +1203,35 @@ static void die_with_unpushed_submodules(struct string_list *needs_pushing)
 static int run_pre_push_hook(struct transport *transport,
 			     struct ref *remote_refs)
 {
-	int ret = 0, x;
+	int ret = 0;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	struct ref *r;
-	struct child_process proc = CHILD_PROCESS_INIT;
-	struct strbuf buf;
-	const char *argv[4];
-
-	if (!(argv[0] = find_hook("pre-push")))
-		return 0;
-
-	argv[1] = transport->remote->name;
-	argv[2] = transport->url;
-	argv[3] = NULL;
-
-	proc.argv = argv;
-	proc.in = -1;
-	proc.trace2_hook_name = "pre-push";
-
-	if (start_command(&proc)) {
-		finish_command(&proc);
-		return -1;
-	}
+	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
 
-	sigchain_push(SIGPIPE, SIG_IGN);
-
-	strbuf_init(&buf, 256);
+	strvec_push(&opt.args, transport->remote->name);
+	strvec_push(&opt.args, transport->url);
 
 	for (r = remote_refs; r; r = r->next) {
+		struct strbuf buf = STRBUF_INIT;
+
 		if (!r->peer_ref) continue;
 		if (r->status == REF_STATUS_REJECT_NONFASTFORWARD) continue;
 		if (r->status == REF_STATUS_REJECT_STALE) continue;
 		if (r->status == REF_STATUS_REJECT_REMOTE_UPDATED) continue;
 		if (r->status == REF_STATUS_UPTODATE) continue;
 
-		strbuf_reset(&buf);
-		strbuf_addf( &buf, "%s %s %s %s\n",
+		strbuf_addf(&buf, "%s %s %s %s",
 			 r->peer_ref->name, oid_to_hex(&r->new_oid),
 			 r->name, oid_to_hex(&r->old_oid));
-
-		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
-			/* We do not mind if a hook does not read all refs. */
-			if (errno != EPIPE)
-				ret = -1;
-			break;
-		}
+		string_list_append(&to_stdin, strbuf_detach(&buf, NULL));
 	}
 
-	strbuf_release(&buf);
-
-	x = close(proc.in);
-	if (!ret)
-		ret = x;
-
-	sigchain_pop(SIGPIPE);
+	opt.feed_pipe = pipe_from_string_list;
+	opt.feed_pipe_ctx = &to_stdin;
 
-	x = finish_command(&proc);
-	if (!ret)
-		ret = x;
+	ret = run_hooks_oneshot("pre-push", &opt);
+	to_stdin.strdup_strings = 1;
+	string_list_clear(&to_stdin, 0);
 
 	return ret;
 }
Signature · 1 line
2.33.0.816.g1ba32acadee
#639 lore raw

PATCH v5 28/36 hook tests: test for exact "pre-push" hook input

In reply to Ævar Arnfjörð Bjarmason

Extend the tests added in ec55559f937 (push: Add support for pre-push hooks, 2013-01-13) to exhaustively test for the exact input we're expecting. This ensures that we e.g. don't miss a trailing newline.

Appending to a file called "actual" is the established convention in this test for hooks, see the rest of the tests added in ec55559f937 (push: Add support for pre-push hooks, 2013-01-13). Let's follow that convention here.

---
 t/t5571-pre-push-hook.sh | 23 ++++++++++++++++++-----
 1 file changed, 18 insertions(+), 5 deletions(-)
t/t5571-pre-push-hook.sh +18 -5
index ad8d5804f7b..d2857a6fbc0 100755
@@ -11,7 +11,7 @@ HOOKDIR="$(git rev-parse --git-dir)/hooks"
 HOOK="$HOOKDIR/pre-push"
 mkdir -p "$HOOKDIR"
 write_script "$HOOK" <<EOF
-cat >/dev/null
+cat >actual
 exit 0
 EOF
 
@@ -20,10 +20,16 @@ test_expect_success 'setup' '
 	git init --bare repo1 &&
 	git remote add parent1 repo1 &&
 	test_commit one &&
-	git push parent1 HEAD:foreign
+	cat >expect <<-EOF &&
+	HEAD $(git rev-parse HEAD) refs/heads/foreign $(test_oid zero)
+	EOF
+
+	test_when_finished "rm actual" &&
+	git push parent1 HEAD:foreign &&
+	test_cmp expect actual
 '
 write_script "$HOOK" <<EOF
-cat >/dev/null
+cat >actual
 exit 1
 EOF
 
@@ -32,11 +38,18 @@ export COMMIT1
 
 test_expect_success 'push with failing hook' '
 	test_commit two &&
-	test_must_fail git push parent1 HEAD
+	cat >expect <<-EOF &&
+	HEAD $(git rev-parse HEAD) refs/heads/main $(test_oid zero)
+	EOF
+
+	test_when_finished "rm actual" &&
+	test_must_fail git push parent1 HEAD &&
+	test_cmp expect actual
 '
 
 test_expect_success '--no-verify bypasses hook' '
-	git push --no-verify parent1 HEAD
+	git push --no-verify parent1 HEAD &&
+	test_path_is_missing actual
 '
 
 COMMIT2="$(git rev-parse HEAD)"
Signature · 1 line
2.33.0.816.g1ba32acadee
#640 lore raw

PATCH v5 29/36 hook tests: use a modern style for "pre-push" tests

In reply to Ævar Arnfjörð Bjarmason

Indent the here-docs and use "test_cmp" instead of "diff" in tests added in ec55559f937 (push: Add support for pre-push hooks, 2013-01-13). Let's also use the more typical "expect" instead of "expected" to be consistent with the rest of the test file.

---
 t/t5571-pre-push-hook.sh | 71 ++++++++++++++++++++--------------------
 1 file changed, 35 insertions(+), 36 deletions(-)
t/t5571-pre-push-hook.sh +35 -36
index d2857a6fbc0..cc1a3e23668 100755
@@ -61,15 +61,15 @@ echo "$2" >>actual
 cat >>actual
 EOF
 
-cat >expected <<EOF
-parent1
-repo1
-refs/heads/main $COMMIT2 refs/heads/foreign $COMMIT1
-EOF
-
 test_expect_success 'push with hook' '
+	cat >expect <<-EOF &&
+	parent1
+	repo1
+	refs/heads/main $COMMIT2 refs/heads/foreign $COMMIT1
+	EOF
+
 	git push parent1 main:foreign &&
-	diff expected actual
+	test_cmp expect actual
 '
 
 test_expect_success 'add a branch' '
@@ -80,49 +80,48 @@ test_expect_success 'add a branch' '
 COMMIT3="$(git rev-parse HEAD)"
 export COMMIT3
 
-cat >expected <<EOF
-parent1
-repo1
-refs/heads/other $COMMIT3 refs/heads/foreign $COMMIT2
-EOF
-
 test_expect_success 'push to default' '
+	cat >expect <<-EOF &&
+	parent1
+	repo1
+	refs/heads/other $COMMIT3 refs/heads/foreign $COMMIT2
+	EOF
 	git push &&
-	diff expected actual
+	test_cmp expect actual
 '
 
-cat >expected <<EOF
-parent1
-repo1
-refs/tags/one $COMMIT1 refs/tags/tag1 $ZERO_OID
-HEAD~ $COMMIT2 refs/heads/prev $ZERO_OID
-EOF
-
 test_expect_success 'push non-branches' '
+	cat >expect <<-EOF &&
+	parent1
+	repo1
+	refs/tags/one $COMMIT1 refs/tags/tag1 $ZERO_OID
+	HEAD~ $COMMIT2 refs/heads/prev $ZERO_OID
+	EOF
+
 	git push parent1 one:tag1 HEAD~:refs/heads/prev &&
-	diff expected actual
+	test_cmp expect actual
 '
 
-cat >expected <<EOF
-parent1
-repo1
-(delete) $ZERO_OID refs/heads/prev $COMMIT2
-EOF
-
 test_expect_success 'push delete' '
+	cat >expect <<-EOF &&
+	parent1
+	repo1
+	(delete) $ZERO_OID refs/heads/prev $COMMIT2
+	EOF
+
 	git push parent1 :prev &&
-	diff expected actual
+	test_cmp expect actual
 '
 
-cat >expected <<EOF
-repo1
-repo1
-HEAD $COMMIT3 refs/heads/other $ZERO_OID
-EOF
-
 test_expect_success 'push to URL' '
+	cat >expect <<-EOF &&
+	repo1
+	repo1
+	HEAD $COMMIT3 refs/heads/other $ZERO_OID
+	EOF
+
 	git push repo1 HEAD &&
-	diff expected actual
+	test_cmp expect actual
 '
 
 test_expect_success 'set up many-ref tests' '
Signature · 1 line
2.33.0.816.g1ba32acadee
#641 lore raw

PATCH v5 30/36 reference-transaction: use hook.h to run hooks

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

---
 refs.c | 40 +++++++++++++---------------------------
 1 file changed, 13 insertions(+), 27 deletions(-)
refs.c +13 -27
index 6211692eaae..73d4a939267 100644
@@ -2062,47 +2062,33 @@ int ref_update_reject_duplicates(struct string_list *refnames,
 static int run_transaction_hook(struct ref_transaction *transaction,
 				const char *state)
 {
-	struct child_process proc = CHILD_PROCESS_INIT;
-	struct strbuf buf = STRBUF_INIT;
-	const char *hook;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
 	int ret = 0, i;
 
-	hook = find_hook("reference-transaction");
-	if (!hook)
-		return ret;
-
-	strvec_pushl(&proc.args, hook, state, NULL);
-	proc.in = -1;
-	proc.stdout_to_stderr = 1;
-	proc.trace2_hook_name = "reference-transaction";
-
-	ret = start_command(&proc);
-	if (ret)
+	if (!hook_exists("reference-transaction"))
 		return ret;
 
-	sigchain_push(SIGPIPE, SIG_IGN);
+	strvec_push(&opt.args, state);
 
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
+		struct strbuf buf = STRBUF_INIT;
 
-		strbuf_reset(&buf);
-		strbuf_addf(&buf, "%s %s %s\n",
+		strbuf_addf(&buf, "%s %s %s",
 			    oid_to_hex(&update->old_oid),
 			    oid_to_hex(&update->new_oid),
 			    update->refname);
-
-		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
-			if (errno != EPIPE)
-				ret = -1;
-			break;
-		}
+		string_list_append(&to_stdin, strbuf_detach(&buf, NULL));
 	}
 
-	close(proc.in);
-	sigchain_pop(SIGPIPE);
-	strbuf_release(&buf);
+	opt.feed_pipe = pipe_from_string_list;
+	opt.feed_pipe_ctx = &to_stdin;
+
+	ret = run_hooks_oneshot("reference-transaction", &opt);
+	to_stdin.strdup_strings = 1;
+	string_list_clear(&to_stdin, 0);
 
-	ret |= finish_command(&proc);
 	return ret;
 }
 
Signature · 1 line
2.33.0.816.g1ba32acadee
#642 lore raw

PATCH v5 31/36 run-command: allow capturing of collated output

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Some callers, for example server-side hooks which wish to relay hook output to clients across a transport, want to capture what would normally print to stderr and do something else with it. Allow that via a callback.

By calling the callback regardless of whether there's output available, we allow clients to send e.g. a keepalive if necessary.

Because we expose a strbuf, not a fd or FILE*, there's no need to create a temporary pipe or similar - we can just skip the print to stderr and instead hand it to the caller.

---
 builtin/fetch.c             |  2 +-
 builtin/submodule--helper.c |  2 +-
 hook.c                      |  1 +
 run-command.c               | 33 +++++++++++++++++++++++++--------
 run-command.h               | 18 +++++++++++++++++-
 submodule.c                 |  2 +-
 t/helper/test-run-command.c | 25 ++++++++++++++++++++-----
 t/t0061-run-command.sh      |  7 +++++++
 8 files changed, 73 insertions(+), 17 deletions(-)
builtin/fetch.c +1 -1
index b18ada08842..27b63ee5200 100644
@@ -1819,7 +1819,7 @@ static int fetch_multiple(struct string_list *list, int max_children)
 		result = run_processes_parallel_tr2(max_children,
 						    &fetch_next_remote,
 						    &fetch_failed_to_start,
-						    NULL,
+						    NULL, NULL,
 						    &fetch_finished,
 						    &state,
 						    "fetch", "parallel/fetch");
builtin/submodule--helper.c +1 -1
index 0b73739c160..fbceab08b0d 100644
@@ -2311,7 +2311,7 @@ static int update_submodules(struct submodule_update_clone *suc)
 	int i;
 
 	run_processes_parallel_tr2(suc->max_jobs, update_clone_get_next_task,
-				   update_clone_start_failure, NULL,
+				   update_clone_start_failure, NULL, NULL,
 				   update_clone_task_finished, suc, "submodule",
 				   "parallel/update");
 
hook.c +1 -0
index 7a6ef527443..169287ad152 100644
@@ -174,6 +174,7 @@ int run_hooks(const char *hook_name, const char *hook_path,
 				   pick_next_hook,
 				   notify_start_failure,
 				   options->feed_pipe,
+				   NULL,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
run-command.c +25 -8
index f1616858d18..aacc336f951 100644
@@ -1493,6 +1493,7 @@ struct parallel_processes {
 	get_next_task_fn get_next_task;
 	start_failure_fn start_failure;
 	feed_pipe_fn feed_pipe;
+	consume_sideband_fn consume_sideband;
 	task_finished_fn task_finished;
 
 	struct {
@@ -1558,6 +1559,7 @@ static void pp_init(struct parallel_processes *pp,
 		    get_next_task_fn get_next_task,
 		    start_failure_fn start_failure,
 		    feed_pipe_fn feed_pipe,
+		    consume_sideband_fn consume_sideband,
 		    task_finished_fn task_finished,
 		    void *data)
 {
@@ -1578,6 +1580,7 @@ static void pp_init(struct parallel_processes *pp,
 	pp->start_failure = start_failure ? start_failure : default_start_failure;
 	pp->feed_pipe = feed_pipe ? feed_pipe : default_feed_pipe;
 	pp->task_finished = task_finished ? task_finished : default_task_finished;
+	pp->consume_sideband = consume_sideband;
 
 	pp->nr_processes = 0;
 	pp->output_owner = 0;
@@ -1614,7 +1617,10 @@ static void pp_cleanup(struct parallel_processes *pp)
 	 * When get_next_task added messages to the buffer in its last
 	 * iteration, the buffered output is non empty.
 	 */
-	strbuf_write(&pp->buffered_output, stderr);
+	if (pp->consume_sideband)
+		pp->consume_sideband(&pp->buffered_output, pp->data);
+	else
+		strbuf_write(&pp->buffered_output, stderr);
 	strbuf_release(&pp->buffered_output);
 
 	sigchain_pop_common();
@@ -1735,9 +1741,13 @@ static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout)
 static void pp_output(struct parallel_processes *pp)
 {
 	int i = pp->output_owner;
+
 	if (pp->children[i].state == GIT_CP_WORKING &&
 	    pp->children[i].err.len) {
-		strbuf_write(&pp->children[i].err, stderr);
+		if (pp->consume_sideband)
+			pp->consume_sideband(&pp->children[i].err, pp->data);
+		else
+			strbuf_write(&pp->children[i].err, stderr);
 		strbuf_reset(&pp->children[i].err);
 	}
 }
@@ -1776,11 +1786,15 @@ static int pp_collect_finished(struct parallel_processes *pp)
 			strbuf_addbuf(&pp->buffered_output, &pp->children[i].err);
 			strbuf_reset(&pp->children[i].err);
 		} else {
-			strbuf_write(&pp->children[i].err, stderr);
+			/* Output errors, then all other finished child processes */
+			if (pp->consume_sideband) {
+				pp->consume_sideband(&pp->children[i].err, pp->data);
+				pp->consume_sideband(&pp->buffered_output, pp->data);
+			} else {
+				strbuf_write(&pp->children[i].err, stderr);
+				strbuf_write(&pp->buffered_output, stderr);
+			}
 			strbuf_reset(&pp->children[i].err);
-
-			/* Output all other finished child processes */
-			strbuf_write(&pp->buffered_output, stderr);
 			strbuf_reset(&pp->buffered_output);
 
 			/*
@@ -1804,6 +1818,7 @@ int run_processes_parallel(int n,
 			   get_next_task_fn get_next_task,
 			   start_failure_fn start_failure,
 			   feed_pipe_fn feed_pipe,
+			   consume_sideband_fn consume_sideband,
 			   task_finished_fn task_finished,
 			   void *pp_cb)
 {
@@ -1814,7 +1829,7 @@ int run_processes_parallel(int n,
 
 	sigchain_push(SIGPIPE, SIG_IGN);
 
-	pp_init(&pp, n, get_next_task, start_failure, feed_pipe, task_finished, pp_cb);
+	pp_init(&pp, n, get_next_task, start_failure, feed_pipe, consume_sideband, task_finished, pp_cb);
 	while (1) {
 		for (i = 0;
 		    i < spawn_cap && !pp.shutdown &&
@@ -1852,6 +1867,7 @@ int run_processes_parallel(int n,
 int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 			       start_failure_fn start_failure,
 			       feed_pipe_fn feed_pipe,
+			       consume_sideband_fn consume_sideband,
 			       task_finished_fn task_finished, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label)
 {
@@ -1861,7 +1877,8 @@ int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 				   ((n < 1) ? online_cpus() : n));
 
 	result = run_processes_parallel(n, get_next_task, start_failure,
-					feed_pipe, task_finished, pp_cb);
+					feed_pipe, consume_sideband,
+					task_finished, pp_cb);
 
 	trace2_region_leave(tr2_category, tr2_label, NULL);
 
run-command.h +17 -1
index 80d394664ae..e321d23bbd2 100644
@@ -436,6 +436,20 @@ typedef int (*feed_pipe_fn)(struct strbuf *pipe,
 			    void *pp_cb,
 			    void *pp_task_cb);
 
+/**
+ * If this callback is provided, instead of collating process output to stderr,
+ * they will be collated into a new pipe. consume_sideband_fn will be called
+ * repeatedly. When output is available on that pipe, it will be contained in
+ * 'output'. But it will be called with an empty 'output' too, to allow for
+ * keepalives or similar operations if necessary.
+ *
+ * pp_cb is the callback cookie as passed into run_processes_parallel.
+ *
+ * Since this callback is provided with the collated output, no task cookie is
+ * provided.
+ */
+typedef void (*consume_sideband_fn)(struct strbuf *output, void *pp_cb);
+
 /**
  * This callback is called on every child process that finished processing.
  *
@@ -471,10 +485,12 @@ int run_processes_parallel(int n,
 			   get_next_task_fn,
 			   start_failure_fn,
 			   feed_pipe_fn,
+			   consume_sideband_fn,
 			   task_finished_fn,
 			   void *pp_cb);
 int run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn,
-			       feed_pipe_fn, task_finished_fn, void *pp_cb,
+			       feed_pipe_fn, consume_sideband_fn,
+			       task_finished_fn, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label);
 
 /**
submodule.c +1 -1
index db1700a502d..32364d8bd56 100644
@@ -1632,7 +1632,7 @@ int fetch_populated_submodules(struct repository *r,
 	run_processes_parallel_tr2(max_parallel_jobs,
 				   get_next_submodule,
 				   fetch_start_failure,
-				   NULL,
+				   NULL, NULL,
 				   fetch_finish,
 				   &spf,
 				   "submodule", "parallel/fetch");
t/helper/test-run-command.c +20 -5
index 9348184d303..d53db6d11c4 100644
@@ -51,6 +51,16 @@ static int no_job(struct child_process *cp,
 	return 0;
 }
 
+static void test_consume_sideband(struct strbuf *output, void *cb)
+{
+	FILE *sideband;
+
+	sideband = fopen("./sideband", "a");
+
+	strbuf_write(output, sideband);
+	fclose(sideband);
+}
+
 static int task_finished(int result,
 			 struct strbuf *err,
 			 void *pp_cb,
@@ -201,7 +211,7 @@ static int testsuite(int argc, const char **argv)
 		suite.tests.nr, max_jobs);
 
 	ret = run_processes_parallel(max_jobs, next_test, test_failed,
-				     test_stdin, test_finished, &suite);
+				     test_stdin, NULL, test_finished, &suite);
 
 	if (suite.failed.nr > 0) {
 		ret = 1;
@@ -429,23 +439,28 @@ int cmd__run_command(int argc, const char **argv)
 
 	if (!strcmp(argv[1], "run-command-parallel"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, NULL, &proc));
+					    NULL, NULL, NULL, NULL, &proc));
 
 	if (!strcmp(argv[1], "run-command-abort"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, task_finished, &proc));
+					    NULL, NULL, NULL, task_finished, &proc));
 
 	if (!strcmp(argv[1], "run-command-no-jobs"))
 		exit(run_processes_parallel(jobs, no_job,
-					    NULL, NULL, task_finished, &proc));
+					    NULL, NULL, NULL, task_finished, &proc));
 
 	if (!strcmp(argv[1], "run-command-stdin")) {
 		proc.in = -1;
 		proc.no_stdin = 0;
 		exit (run_processes_parallel(jobs, parallel_next, NULL,
-					     test_stdin, NULL, &proc));
+					     test_stdin, NULL, NULL, &proc));
 	}
 
+	if (!strcmp(argv[1], "run-command-sideband"))
+		exit(run_processes_parallel(jobs, parallel_next, NULL, NULL,
+					    test_consume_sideband, NULL,
+					    &proc));
+
 	fprintf(stderr, "check usage\n");
 	return 1;
 }
t/t0061-run-command.sh +7 -0
index 87759482ad1..e99f6c7f445 100755
@@ -143,6 +143,13 @@ test_expect_success 'run_command runs in parallel with more tasks than jobs avai
 	test_cmp expect actual
 '
 
+test_expect_success 'run_command can divert output' '
+	test_when_finished rm sideband &&
+	test-tool run-command run-command-sideband 3 sh -c "printf \"%s\n%s\n\" Hello World" 2>actual &&
+	test_must_be_empty actual &&
+	test_cmp expect sideband
+'
+
 cat >expect <<-EOF
 preloaded output of a child
 listening for stdin:
Signature · 1 line
2.33.0.816.g1ba32acadee
#643 lore raw

PATCH v5 32/36 hooks: allow callers to capture output

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Some server-side hooks will require capturing output to send over sideband instead of printing directly to stderr. Expose that capability.

---
 hook.c | 3 ++-
 hook.h | 8 ++++++++
 2 files changed, 10 insertions(+), 1 deletion(-)
hook.c +2 -1
index 169287ad152..7f93fe40dba 100644
@@ -174,12 +174,13 @@ int run_hooks(const char *hook_name, const char *hook_path,
 				   pick_next_hook,
 				   notify_start_failure,
 				   options->feed_pipe,
-				   NULL,
+				   options->consume_sideband,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
 				   hook_name);
 
+
 	if (options->absolute_path)
 		strbuf_release(&abs_path);
 	free(my_hook.feed_pipe_cb_data);
hook.h +8 -0
index 4e26b8d658d..a85e3e6981b 100644
@@ -47,6 +47,14 @@ struct run_hooks_opt
 	 */
 	feed_pipe_fn feed_pipe;
 	void *feed_pipe_ctx;
+
+	/*
+	 * Populate this to capture output and prevent it from being printed to
+	 * stderr. This will be passed directly through to
+	 * run_command:run_parallel_processes(). See t/helper/test-run-command.c
+	 * for an example.
+	 */
+	consume_sideband_fn consume_sideband;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
Signature · 1 line
2.33.0.816.g1ba32acadee
#644 lore raw

PATCH v5 33/36 receive-pack: convert 'update' hook to hook.h

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

This makes use of the new sideband API in hook.h added in the preceding commit.

---
 builtin/receive-pack.c | 62 ++++++++++++++++++++++++++++--------------
 1 file changed, 41 insertions(+), 21 deletions(-)
builtin/receive-pack.c +41 -21
index a7d03bbc7d3..31ce4ece4e7 100644
@@ -937,33 +937,53 @@ static int run_receive_hook(struct command *commands,
 	return status;
 }
 
-static int run_update_hook(struct command *cmd)
+static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
 {
-	const char *argv[5];
-	struct child_process proc = CHILD_PROCESS_INIT;
-	int code;
+	int keepalive_active = 0;
 
-	argv[0] = find_hook("update");
-	if (!argv[0])
-		return 0;
+	if (keepalive_in_sec <= 0)
+		use_keepalive = KEEPALIVE_NEVER;
+	if (use_keepalive == KEEPALIVE_ALWAYS)
+		keepalive_active = 1;
 
-	argv[1] = cmd->ref_name;
-	argv[2] = oid_to_hex(&cmd->old_oid);
-	argv[3] = oid_to_hex(&cmd->new_oid);
-	argv[4] = NULL;
+	/* send a keepalive if there is no data to write */
+	if (keepalive_active && !output->len) {
+		static const char buf[] = "0005\1";
+		write_or_die(1, buf, sizeof(buf) - 1);
+		return;
+	}
 
-	proc.no_stdin = 1;
-	proc.stdout_to_stderr = 1;
-	proc.err = use_sideband ? -1 : 0;
-	proc.argv = argv;
-	proc.trace2_hook_name = "update";
+	if (use_keepalive == KEEPALIVE_AFTER_NUL && !keepalive_active) {
+		const char *first_null = memchr(output->buf, '\0', output->len);
+		if (first_null) {
+			/* The null bit is excluded. */
+			size_t before_null = first_null - output->buf;
+			size_t after_null = output->len - (before_null + 1);
+			keepalive_active = 1;
+			send_sideband(1, 2, output->buf, before_null, use_sideband);
+			send_sideband(1, 2, first_null + 1, after_null, use_sideband);
+
+			return;
+		}
+	}
+
+	send_sideband(1, 2, output->buf, output->len, use_sideband);
+}
+
+static int run_update_hook(struct command *cmd)
+{
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+
+	strvec_pushl(&opt.args,
+		     cmd->ref_name,
+		     oid_to_hex(&cmd->old_oid),
+		     oid_to_hex(&cmd->new_oid),
+		     NULL);
 
-	code = start_command(&proc);
-	if (code)
-		return code;
 	if (use_sideband)
-		copy_to_sideband(proc.err, -1, NULL);
-	return finish_command(&proc);
+		opt.consume_sideband = hook_output_to_sideband;
+
+	return run_hooks_oneshot("update", &opt);
 }
 
 static struct command *find_command_by_refname(struct command *list,
Signature · 1 line
2.33.0.816.g1ba32acadee
#645 lore raw

PATCH v5 34/36 post-update: use hook.h library

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

---
 builtin/receive-pack.c | 25 ++++++-------------------
 1 file changed, 6 insertions(+), 19 deletions(-)
builtin/receive-pack.c +6 -19
index 31ce4ece4e7..26e302aab85 100644
@@ -1650,33 +1650,20 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 static void run_update_post_hook(struct command *commands)
 {
 	struct command *cmd;
-	struct child_process proc = CHILD_PROCESS_INIT;
-	const char *hook;
-
-	hook = find_hook("post-update");
-	if (!hook)
-		return;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 
 	for (cmd = commands; cmd; cmd = cmd->next) {
 		if (cmd->error_string || cmd->did_not_exist)
 			continue;
-		if (!proc.args.nr)
-			strvec_push(&proc.args, hook);
-		strvec_push(&proc.args, cmd->ref_name);
+		strvec_push(&opt.args, cmd->ref_name);
 	}
-	if (!proc.args.nr)
+	if (!opt.args.nr)
 		return;
 
-	proc.no_stdin = 1;
-	proc.stdout_to_stderr = 1;
-	proc.err = use_sideband ? -1 : 0;
-	proc.trace2_hook_name = "post-update";
+	if (use_sideband)
+		opt.consume_sideband = hook_output_to_sideband;
 
-	if (!start_command(&proc)) {
-		if (use_sideband)
-			copy_to_sideband(proc.err, -1, NULL);
-		finish_command(&proc);
-	}
+	run_hooks_oneshot("post-update", &opt);
 }
 
 static void check_aliased_update_internal(struct command *cmd,
Signature · 1 line
2.33.0.816.g1ba32acadee
#646 lore raw

PATCH v5 35/36 receive-pack: convert receive hooks to hook.h

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

---
 builtin/receive-pack.c | 190 ++++++++++++++++++-----------------------
 1 file changed, 83 insertions(+), 107 deletions(-)
builtin/receive-pack.c +83 -107
index 26e302aab85..c3984680d7f 100644
@@ -747,7 +747,7 @@ static int check_cert_push_options(const struct string_list *push_options)
 	return retval;
 }
 
-static void prepare_push_cert_sha1(struct child_process *proc)
+static void prepare_push_cert_sha1(struct run_hooks_opt *opt)
 {
 	static int already_done;
 
@@ -771,110 +771,42 @@ static void prepare_push_cert_sha1(struct child_process *proc)
 		nonce_status = check_nonce(push_cert.buf, bogs);
 	}
 	if (!is_null_oid(&push_cert_oid)) {
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT=%s",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT=%s",
 			     oid_to_hex(&push_cert_oid));
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_SIGNER=%s",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT_SIGNER=%s",
 			     sigcheck.signer ? sigcheck.signer : "");
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_KEY=%s",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT_KEY=%s",
 			     sigcheck.key ? sigcheck.key : "");
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_STATUS=%c",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT_STATUS=%c",
 			     sigcheck.result);
 		if (push_cert_nonce) {
-			strvec_pushf(&proc->env_array,
+			strvec_pushf(&opt->env,
 				     "GIT_PUSH_CERT_NONCE=%s",
 				     push_cert_nonce);
-			strvec_pushf(&proc->env_array,
+			strvec_pushf(&opt->env,
 				     "GIT_PUSH_CERT_NONCE_STATUS=%s",
 				     nonce_status);
 			if (nonce_status == NONCE_SLOP)
-				strvec_pushf(&proc->env_array,
+				strvec_pushf(&opt->env,
 					     "GIT_PUSH_CERT_NONCE_SLOP=%ld",
 					     nonce_stamp_slop);
 		}
 	}
 }
 
+struct receive_hook_feed_context {
+	struct command *cmd;
+	int skip_broken;
+};
+
 struct receive_hook_feed_state {
 	struct command *cmd;
 	struct ref_push_report *report;
 	int skip_broken;
 	struct strbuf buf;
-	const struct string_list *push_options;
 };
 
-typedef int (*feed_fn)(void *, const char **, size_t *);
-static int run_and_feed_hook(const char *hook_name, feed_fn feed,
-			     struct receive_hook_feed_state *feed_state)
-{
-	struct child_process proc = CHILD_PROCESS_INIT;
-	struct async muxer;
-	const char *argv[2];
-	int code;
-
-	argv[0] = find_hook(hook_name);
-	if (!argv[0])
-		return 0;
-
-	argv[1] = NULL;
-
-	proc.argv = argv;
-	proc.in = -1;
-	proc.stdout_to_stderr = 1;
-	proc.trace2_hook_name = hook_name;
-
-	if (feed_state->push_options) {
-		int i;
-		for (i = 0; i < feed_state->push_options->nr; i++)
-			strvec_pushf(&proc.env_array,
-				     "GIT_PUSH_OPTION_%d=%s", i,
-				     feed_state->push_options->items[i].string);
-		strvec_pushf(&proc.env_array, "GIT_PUSH_OPTION_COUNT=%d",
-			     feed_state->push_options->nr);
-	} else
-		strvec_pushf(&proc.env_array, "GIT_PUSH_OPTION_COUNT");
-
-	if (tmp_objdir)
-		strvec_pushv(&proc.env_array, tmp_objdir_env(tmp_objdir));
-
-	if (use_sideband) {
-		memset(&muxer, 0, sizeof(muxer));
-		muxer.proc = copy_to_sideband;
-		muxer.in = -1;
-		code = start_async(&muxer);
-		if (code)
-			return code;
-		proc.err = muxer.in;
-	}
-
-	prepare_push_cert_sha1(&proc);
-
-	code = start_command(&proc);
-	if (code) {
-		if (use_sideband)
-			finish_async(&muxer);
-		return code;
-	}
-
-	sigchain_push(SIGPIPE, SIG_IGN);
-
-	while (1) {
-		const char *buf;
-		size_t n;
-		if (feed(feed_state, &buf, &n))
-			break;
-		if (write_in_full(proc.in, buf, n) < 0)
-			break;
-	}
-	close(proc.in);
-	if (use_sideband)
-		finish_async(&muxer);
-
-	sigchain_pop(SIGPIPE);
-
-	return finish_command(&proc);
-}
-
-static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep)
+static int feed_receive_hook(void *state_)
 {
 	struct receive_hook_feed_state *state = state_;
 	struct command *cmd = state->cmd;
@@ -883,9 +815,7 @@ static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep)
 	       state->skip_broken && (cmd->error_string || cmd->did_not_exist))
 		cmd = cmd->next;
 	if (!cmd)
-		return -1; /* EOF */
-	if (!bufp)
-		return 0; /* OK, can feed something. */
+		return 1; /* EOF - close the pipe*/
 	strbuf_reset(&state->buf);
 	if (!state->report)
 		state->report = cmd->report;
@@ -909,32 +839,36 @@ static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep)
 			    cmd->ref_name);
 		state->cmd = cmd->next;
 	}
-	if (bufp) {
-		*bufp = state->buf.buf;
-		*sizep = state->buf.len;
-	}
 	return 0;
 }
 
-static int run_receive_hook(struct command *commands,
-			    const char *hook_name,
-			    int skip_broken,
-			    const struct string_list *push_options)
+static int feed_receive_hook_cb(struct strbuf *pipe, void *pp_cb, void *pp_task_cb)
 {
-	struct receive_hook_feed_state state;
-	int status;
-
-	strbuf_init(&state.buf, 0);
-	state.cmd = commands;
-	state.skip_broken = skip_broken;
-	state.report = NULL;
-	if (feed_receive_hook(&state, NULL, NULL))
-		return 0;
-	state.cmd = commands;
-	state.push_options = push_options;
-	status = run_and_feed_hook(hook_name, feed_receive_hook, &state);
-	strbuf_release(&state.buf);
-	return status;
+	struct hook *hook = pp_task_cb;
+	struct receive_hook_feed_state *feed_state = hook->feed_pipe_cb_data;
+	int rc;
+
+	/* first-time setup */
+	if (!feed_state) {
+		struct hook_cb_data *hook_cb = pp_cb;
+		struct run_hooks_opt *opt = hook_cb->options;
+		struct receive_hook_feed_context *ctx = opt->feed_pipe_ctx;
+		if (!ctx)
+			BUG("run_hooks_opt.feed_pipe_ctx required for receive hook");
+
+		feed_state = xmalloc(sizeof(struct receive_hook_feed_state));
+		strbuf_init(&feed_state->buf, 0);
+		feed_state->cmd = ctx->cmd;
+		feed_state->skip_broken = ctx->skip_broken;
+		feed_state->report = NULL;
+
+		hook->feed_pipe_cb_data = feed_state;
+	}
+
+	rc = feed_receive_hook(feed_state);
+	if (!rc)
+		strbuf_addbuf(pipe, &feed_state->buf);
+	return rc;
 }
 
 static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
@@ -970,6 +904,48 @@ static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
 	send_sideband(1, 2, output->buf, output->len, use_sideband);
 }
 
+static int run_receive_hook(struct command *commands,
+			    const char *hook_name,
+			    int skip_broken,
+			    const struct string_list *push_options)
+{
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct receive_hook_feed_context ctx;
+	struct command *iter = commands;
+
+	/* if there are no valid commands, don't invoke the hook at all. */
+	while (iter && skip_broken && (iter->error_string || iter->did_not_exist))
+		iter = iter->next;
+	if (!iter)
+		return 0;
+
+	if (push_options) {
+		int i;
+		for (i = 0; i < push_options->nr; i++)
+			strvec_pushf(&opt.env, "GIT_PUSH_OPTION_%d=%s", i,
+				     push_options->items[i].string);
+		strvec_pushf(&opt.env, "GIT_PUSH_OPTION_COUNT=%d", push_options->nr);
+	} else
+		strvec_push(&opt.env, "GIT_PUSH_OPTION_COUNT");
+
+	if (tmp_objdir)
+		strvec_pushv(&opt.env, tmp_objdir_env(tmp_objdir));
+
+	prepare_push_cert_sha1(&opt);
+
+	/* set up sideband printer */
+	if (use_sideband)
+		opt.consume_sideband = hook_output_to_sideband;
+
+	/* set up stdin callback */
+	ctx.cmd = commands;
+	ctx.skip_broken = skip_broken;
+	opt.feed_pipe = feed_receive_hook_cb;
+	opt.feed_pipe_ctx = &ctx;
+
+	return run_hooks_oneshot(hook_name, &opt);
+}
+
 static int run_update_hook(struct command *cmd)
 {
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
Signature · 1 line
2.33.0.816.g1ba32acadee
#647 lore raw

PATCH v5 36/36 hooks: fix a TOCTOU in "did we run a hook?" heuristic

In reply to Ævar Arnfjörð Bjarmason

Fix a Time-of-check to time-of-use (TOCTOU) race in code added in 680ee550d72 (commit: skip discarding the index if there is no pre-commit hook, 2017-08-14).

We can fix the race passing around information about whether or not we ran the hook in question, instead of running hook_exists() after the fact to check if the hook in question exists. This problem has been noted on-list when 680ee550d72 was discussed[1], but had not been fixed.

In addition to fixing this for the pre-commit hook as suggested there I'm also fixing this for the pre-merge-commit hook. See 6098817fd7f (git-merge: honor pre-merge-commit hook, 2019-08-07) for the introduction of its previous behavior.

Let's also change this for the push-to-checkout hook. Now instead of checking if the hook exists and either doing a push to checkout or a push to deploy we'll always attempt a push to checkout. If the hook doesn't exist we'll fall back on push to deploy. The same behavior as before, without the TOCTOU race. See 0855331941b (receive-pack:
support push-to-checkout hook, 2014-12-01) for the introduction of the previous behavior.

This leaves uses of hook_exists() in two places that matter. The "reference-transaction" check in refs.c, see 67541597670 (refs:
implement reference transaction hook, 2020-06-19), and the prepare-commit-msg hook, see 66618a50f9c (sequencer: run 'prepare-commit-msg' hook, 2018-01-24).

In both of those cases we're saving ourselves CPU time by not preparing data for the hook that we'll then do nothing with if we don't have the hook. So using this "invoked_hook" pattern doesn't make sense in those cases.

More importantly, in those cases the worst we'll do is miss that we "should" run the hook because a new hook appeared, whereas in the pre-commit and pre-merge-commit cases we'll skip an important discard_cache() on the bases of our faulty guess.

I do think none of these races really matter in practice. It would be some one-off issue as a hook was added or removed. I did think it was stupid that we didn't pass a "did this run?" flag instead of doing this guessing at a distance though, so now we're not guessing anymore.

1. https://lore.kernel.org/git/[email protected]/

---
 builtin/commit.c       | 18 +++++++++++-------
 builtin/merge.c        | 16 ++++++++++------
 builtin/receive-pack.c |  8 +++++---
 commit.c               |  1 +
 commit.h               |  3 ++-
 hook.c                 |  4 ++++
 hook.h                 | 10 ++++++++++
 sequencer.c            |  4 ++--
 8 files changed, 45 insertions(+), 19 deletions(-)
builtin/commit.c +11 -7
index f6ca9d04c78..986146e3205 100644
@@ -725,11 +725,13 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 	int clean_message_contents = (cleanup_mode != COMMIT_MSG_CLEANUP_NONE);
 	int old_display_comment_prefix;
 	int merge_contains_scissors = 0;
+	int invoked_hook = 0;
 
 	/* This checks and barfs if author is badly specified */
 	determine_author_info(author_ident);
 
-	if (!no_verify && run_commit_hook(use_editor, index_file, "pre-commit", NULL))
+	if (!no_verify && run_commit_hook(use_editor, index_file, &invoked_hook,
+					  "pre-commit", NULL))
 		return 0;
 
 	if (squash_message) {
@@ -1052,10 +1054,10 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (!no_verify && hook_exists("pre-commit")) {
+	if (!no_verify && invoked_hook) {
 		/*
-		 * Re-read the index as pre-commit hook could have updated it,
-		 * and write it out as a tree.  We must do this before we invoke
+		 * Re-read the index as the pre-commit-commit hook was invoked
+		 * and could have updated it. We must do this before we invoke
 		 * the editor and after we invoke run_status above.
 		 */
 		discard_cache();
@@ -1067,7 +1069,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (run_commit_hook(use_editor, index_file, "prepare-commit-msg",
+	if (run_commit_hook(use_editor, index_file, NULL, "prepare-commit-msg",
 			    git_path_commit_editmsg(), hook_arg1, hook_arg2, NULL))
 		return 0;
 
@@ -1084,7 +1086,8 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 	}
 
 	if (!no_verify &&
-	    run_commit_hook(use_editor, index_file, "commit-msg", git_path_commit_editmsg(), NULL)) {
+	    run_commit_hook(use_editor, index_file, NULL, "commit-msg",
+			    git_path_commit_editmsg(), NULL)) {
 		return 0;
 	}
 
@@ -1843,7 +1846,8 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 
 	repo_rerere(the_repository, 0);
 	run_auto_maintenance(quiet);
-	run_commit_hook(use_editor, get_index_file(), "post-commit", NULL);
+	run_commit_hook(use_editor, get_index_file(), NULL, "post-commit",
+			NULL);
 	if (amend && !no_post_rewrite) {
 		commit_post_rewrite(the_repository, current_head, &oid);
 	}
builtin/merge.c +10 -6
index ca9b3ba4827..f215f264cc8 100644
@@ -844,15 +844,18 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 {
 	struct strbuf msg = STRBUF_INIT;
 	const char *index_file = get_index_file();
+	int invoked_hook = 0;
 
-	if (!no_verify && run_commit_hook(0 < option_edit, index_file, "pre-merge-commit", NULL))
+	if (!no_verify && run_commit_hook(0 < option_edit, index_file,
+					  &invoked_hook, "pre-merge-commit",
+					  NULL))
 		abort_commit(remoteheads, NULL);
 	/*
-	 * Re-read the index as pre-merge-commit hook could have updated it,
-	 * and write it out as a tree.  We must do this before we invoke
+	 * Re-read the index as the pre-merge-commit hook was invoked
+	 * and could have updated it. We must do this before we invoke
 	 * the editor and after we invoke run_status above.
 	 */
-	if (hook_exists("pre-merge-commit"))
+	if (invoked_hook)
 		discard_cache();
 	read_cache_from(index_file);
 	strbuf_addbuf(&msg, &merge_msg);
@@ -875,7 +878,8 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 		append_signoff(&msg, ignore_non_trailer(msg.buf, msg.len), 0);
 	write_merge_heads(remoteheads);
 	write_file_buf(git_path_merge_msg(the_repository), msg.buf, msg.len);
-	if (run_commit_hook(0 < option_edit, get_index_file(), "prepare-commit-msg",
+	if (run_commit_hook(0 < option_edit, get_index_file(), NULL,
+			    "prepare-commit-msg",
 			    git_path_merge_msg(the_repository), "merge", NULL))
 		abort_commit(remoteheads, NULL);
 	if (0 < option_edit) {
@@ -884,7 +888,7 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 	}
 
 	if (!no_verify && run_commit_hook(0 < option_edit, get_index_file(),
-					  "commit-msg",
+					  NULL, "commit-msg",
 					  git_path_merge_msg(the_repository), NULL))
 		abort_commit(remoteheads, NULL);
 
builtin/receive-pack.c +5 -3
index c3984680d7f..ebec6f3bb10 100644
@@ -1428,10 +1428,12 @@ static const char *push_to_deploy(unsigned char *sha1,
 static const char *push_to_checkout_hook = "push-to-checkout";
 
 static const char *push_to_checkout(unsigned char *hash,
+				    int *invoked_hook,
 				    struct strvec *env,
 				    const char *work_tree)
 {
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	opt.invoked_hook = invoked_hook;
 
 	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
 	strvec_pushv(&opt.env, env->v);
@@ -1446,6 +1448,7 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *w
 {
 	const char *retval, *work_tree, *git_dir = NULL;
 	struct strvec env = STRVEC_INIT;
+	int invoked_hook = 0;
 
 	if (worktree && worktree->path)
 		work_tree = worktree->path;
@@ -1463,10 +1466,9 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *w
 
 	strvec_pushf(&env, "GIT_DIR=%s", absolute_path(git_dir));
 
-	if (!hook_exists(push_to_checkout_hook))
+	retval = push_to_checkout(sha1, &invoked_hook, &env, work_tree);
+	if (!invoked_hook)
 		retval = push_to_deploy(sha1, &env, work_tree);
-	else
-		retval = push_to_checkout(sha1, &env, work_tree);
 
 	strvec_clear(&env);
 	return retval;
commit.c +1 -0
index 63d7943a86d..842e47beae2 100644
@@ -1697,6 +1697,7 @@ size_t ignore_non_trailer(const char *buf, size_t len)
 }
 
 int run_commit_hook(int editor_is_used, const char *index_file,
+		    int *invoked_hook,
 		    const char *name, ...)
 {
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
commit.h +2 -1
index df42eb434f3..b5a542993c6 100644
@@ -363,7 +363,8 @@ int compare_commits_by_commit_date(const void *a_, const void *b_, void *unused)
 int compare_commits_by_gen_then_commit_date(const void *a_, const void *b_, void *unused);
 
 LAST_ARG_MUST_BE_NULL
-int run_commit_hook(int editor_is_used, const char *index_file, const char *name, ...);
+int run_commit_hook(int editor_is_used, const char *index_file,
+		    int *invoked_hook, const char *name, ...);
 
 /* Sign a commit or tag buffer, storing the result in a header. */
 int sign_with_header(struct strbuf *buf, const char *keyid);
hook.c +4 -0
index 7f93fe40dba..d045379ade8 100644
@@ -144,6 +144,9 @@ static int notify_hook_finished(int result,
 
 	hook_cb->rc |= result;
 
+	if (hook_cb->invoked_hook)
+		*hook_cb->invoked_hook = 1;
+
 	return 0;
 }
 
@@ -158,6 +161,7 @@ int run_hooks(const char *hook_name, const char *hook_path,
 		.rc = 0,
 		.hook_name = hook_name,
 		.options = options,
+		.invoked_hook = options->invoked_hook,
 	};
 	int jobs = 1;
 
hook.h +10 -0
index a85e3e6981b..f6dac75f1cc 100644
@@ -55,6 +55,15 @@ struct run_hooks_opt
 	 * for an example.
 	 */
 	consume_sideband_fn consume_sideband;
+
+	/*
+	 * A pointer which if provided will be set to 1 or 0 depending
+	 * on if a hook was invoked (i.e. existed), regardless of
+	 * whether or not that was successful. Used for avoiding
+	 * TOCTOU races in code that would otherwise call hook_exist()
+	 * after a "maybe hook run" to see if a hook was invoked.
+	 */
+	int *invoked_hook;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
@@ -68,6 +77,7 @@ struct hook_cb_data {
 	const char *hook_name;
 	struct hook *run_me;
 	struct run_hooks_opt *options;
+	int *invoked_hook;
 };
 
 /**
sequencer.c +2 -2
index 549b583b277..db8044ab47d 100644
@@ -1203,7 +1203,7 @@ static int run_prepare_commit_msg_hook(struct repository *r,
 	} else {
 		arg1 = "message";
 	}
-	if (run_commit_hook(0, r->index_file, "prepare-commit-msg", name,
+	if (run_commit_hook(0, r->index_file, NULL, "prepare-commit-msg", name,
 			    arg1, arg2, NULL))
 		ret = error(_("'prepare-commit-msg' hook failed"));
 
@@ -1533,7 +1533,7 @@ static int try_to_commit(struct repository *r,
 		goto out;
 	}
 
-	run_commit_hook(0, r->index_file, "post-commit", NULL);
+	run_commit_hook(0, r->index_file, NULL, "post-commit", NULL);
 	if (flags & AMEND_MSG)
 		commit_post_rewrite(r, current_head, oid);
 
Signature · 1 line
2.33.0.816.g1ba32acadee
#648 lore raw

RE: [PATCH v2 00/30] Minimal restart of "config-based-hooks"

In reply to Ævar Arnfjörð Bjarmason
Quoted 4 lines · Ævar Arnfjörð Bjarmason

I proposed splitting Emily's "hook config" topic[1] into at least a topic that retains all current behavior of the codebase, and merely refactors existing behavior to new APIs, and then doing behavior changes later.

This is good but can you add an introduction that is more than "Emily's hook config topic"? What is this patch series doing? (assume that Emily's patches don't exist)

Signature · 1 line
Felipe Contreras