Skip to contentSkip to conversation

propose config-based hooks

PATCH Superseded Emily Shaffer 17 participants 649 messages May 21, 2020 – Oct 12, 2021 Page 6 of 7

Subsystems (2)
lore
Share & tools
lore
#514 lore raw

PATCH v4 8/36 hook: add 'run' subcommand

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

In order to enable hooks to be run as an external process, by a standalone Git command, or by tools which wrap Git, provide an external means to run all configured hook commands for a given hook event.

Most of our hooks require more complex functionality than this, but let's start with the bare minimum required to support our simplest hooks.

In terms of implementation the usage_with_options() and "goto usage" pattern here mirrors that of builtin/{commit-graph,multi-pack-index}.c.

---
 .gitignore                 |   1 +
 Documentation/git-hook.txt |  38 +++++++++++
 Documentation/githooks.txt |   4 ++
 Makefile                   |   1 +
 builtin.h                  |   1 +
 builtin/hook.c             |  85 ++++++++++++++++++++++++
 command-list.txt           |   1 +
 git.c                      |   1 +
 hook.c                     | 111 +++++++++++++++++++++++++++++++
 hook.h                     |  41 ++++++++++++
 t/t1800-hook.sh            | 133 +++++++++++++++++++++++++++++++++++++
 11 files changed, 417 insertions(+)
 create mode 100644 Documentation/git-hook.txt
 create mode 100644 builtin/hook.c
 create mode 100755 t/t1800-hook.sh
.gitignore +1 -0
index 6be9de41ae8..66189ca3cdc 100644
@@ -77,6 +77,7 @@
 /git-grep
 /git-hash-object
 /git-help
+/git-hook
 /git-http-backend
 /git-http-fetch
 /git-http-push
Documentation/git-hook.txt new file +38 -0
new file mode 100644
index 00000000000..660d6a992a0
@@ -0,0 +1,38 @@
+git-hook(1)
+===========
+
+NAME
+----
+git-hook - run git hooks
+
+SYNOPSIS
+--------
+[verse]
+'git hook' run <hook-name> [-- <hook-args>]
+
+DESCRIPTION
+-----------
+
+This command is an interface to git hooks (see linkgit:githooks[5]).
+Currently it only provides a convenience wrapper for running hooks for
+use by git itself. In the future it might gain other functionality.
+
+SUBCOMMANDS
+-----------
+
+run::
+	Run the `<hook-name>` hook. See linkgit:githooks[5] for
+	the hook names we support.
++
+Any positional arguments to the hook should be passed after an
+optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The
+arguments (if any) differ by hook name, see linkgit:githooks[5] for
+what those are.
+
+SEE ALSO
+--------
+linkgit:githooks[5]
+
+GIT
+---
+Part of the linkgit:git[1] suite
Documentation/githooks.txt +4 -0
index b51959ff941..a16e62bc8c8 100644
@@ -698,6 +698,10 @@ and "0" meaning they were not.
 Only one parameter should be set to "1" when the hook runs.  The hook
 running passing "1", "1" should not be possible.
 
+SEE ALSO
+--------
+linkgit:git-hook[1]
+
 GIT
 ---
 Part of the linkgit:git[1] suite
Makefile +1 -0
index 1740b99db00..e68ca41507f 100644
@@ -1114,6 +1114,7 @@ BUILTIN_OBJS += builtin/get-tar-commit-id.o
 BUILTIN_OBJS += builtin/grep.o
 BUILTIN_OBJS += builtin/hash-object.o
 BUILTIN_OBJS += builtin/help.o
+BUILTIN_OBJS += builtin/hook.o
 BUILTIN_OBJS += builtin/index-pack.o
 BUILTIN_OBJS += builtin/init-db.o
 BUILTIN_OBJS += builtin/interpret-trailers.o
builtin.h +1 -0
index 16ecd5586f0..91740c15149 100644
@@ -164,6 +164,7 @@ int cmd_get_tar_commit_id(int argc, const char **argv, const char *prefix);
 int cmd_grep(int argc, const char **argv, const char *prefix);
 int cmd_hash_object(int argc, const char **argv, const char *prefix);
 int cmd_help(int argc, const char **argv, const char *prefix);
+int cmd_hook(int argc, const char **argv, const char *prefix);
 int cmd_index_pack(int argc, const char **argv, const char *prefix);
 int cmd_init_db(int argc, const char **argv, const char *prefix);
 int cmd_interpret_trailers(int argc, const char **argv, const char *prefix);
builtin/hook.c new file +85 -0

85 lines not on this page · raw message

command-list.txt +1 -0
index a289f09ed6f..9ccd8e5aebe 100644
@@ -103,6 +103,7 @@ git-grep                                mainporcelain           info
 git-gui                                 mainporcelain
 git-hash-object                         plumbingmanipulators
 git-help                                ancillaryinterrogators          complete
+git-hook                                mainporcelain
 git-http-backend                        synchingrepositories
 git-http-fetch                          synchelpers
 git-http-push                           synchelpers
git.c +1 -0
index 18bed9a9964..540909c391f 100644
@@ -538,6 +538,7 @@ static struct cmd_struct commands[] = {
 	{ "grep", cmd_grep, RUN_SETUP_GENTLY },
 	{ "hash-object", cmd_hash_object },
 	{ "help", cmd_help },
+	{ "hook", cmd_hook, RUN_SETUP },
 	{ "index-pack", cmd_index_pack, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "init", cmd_init_db },
 	{ "init-db", cmd_init_db },
hook.c +111 -0

131 lines not on this page · raw message

hook.h +41 -0

50 lines not on this page · raw message

t/t1800-hook.sh new file +133 -0

133 lines not on this page · raw message

Signature · 1 line
2.33.0.rc0.595.ge31e012651d
#515 lore raw

Re: [PATCH v4 08/36] hook: add 'run' subcommand

t/t1800-hook.sh line 2

Quoted 10 lines · Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>
[...]

t/t1800-hook.sh new file +2 -0
new file mode 100755
index 00000000000..644df0a583c
@@ -0,0 +1,133 @@
+#!/bin/bash
+

It appears that since at least May 2020 and across N re-rolls of this topic nobody (including me) has noticed that this should be #!/bin/sh, not #!/bin/bash.

I've got it fixed up locally, but thought it was not worth another 36-patch re-roll pending further feedback.

#516 lore raw

PATCH v4 9/36 gc: use hook library for pre-auto-gc hook

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Move the pre-auto-gc hook away from run-command.h to and over to the new hook.h library.

To do this introduce a simple run_hooks_oneshot() wrapper, we'll be using it extensively for these simple cases of wanting to run a single hook under a given name, and having it free the memory we allocate for us.

---
 builtin/gc.c |  3 ++-
 hook.c       | 21 +++++++++++++++++++++
 hook.h       | 13 +++++++++++++
 3 files changed, 36 insertions(+), 1 deletion(-)
builtin/gc.c +2 -1
index f05d2f0a1ac..2f74cf394d3 100644
@@ -32,6 +32,7 @@
 #include "remote.h"
 #include "object-store.h"
 #include "exec-cmd.h"
+#include "hook.h"
 
 #define FAILED_RUN "failed to run %s"
 
@@ -394,7 +395,7 @@ static int need_to_gc(void)
 	else
 		return 0;
 
-	if (run_hook_le(NULL, "pre-auto-gc", NULL))
+	if (run_hooks_oneshot("pre-auto-gc", NULL))
 		return 0;
 	return 1;
 }
hook.c +21 -0
index c8ff3975043..981a9bf46e6 100644
@@ -170,3 +170,24 @@ int run_hooks(const char *hook_name, const char *hook_path,
 
 	return cb_data.rc;
 }
+
+int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
+{
+	const char *hook_path;
+	int ret;
+	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT;
+
+	if (!options)
+		options = &hook_opt_scratch;
+
+	hook_path = find_hook(hook_name);
+	if (!hook_path) {
+		ret = 0;
+		goto cleanup;
+	}
+
+	ret = run_hooks(hook_name, hook_path, options);
+cleanup:
+	run_hooks_opt_clear(options);
+	return ret;
+}
hook.h +13 -0
index 361984c69d4..2201ab9d568 100644
@@ -51,7 +51,20 @@ void run_hooks_opt_clear(struct run_hooks_opt *o);
 /**
  * Takes an already resolved hook found via find_hook() and runs
  * it. Does not call run_hooks_opt_clear() for you.
+ *
+ * See run_hooks_oneshot() for the simpler one-shot API.
  */
 int run_hooks(const char *hookname, const char *hook_path,
 	      struct run_hooks_opt *options);
+
+/**
+ * Calls find_hook() on your "hook_name" and runs the hooks (if any)
+ * with run_hooks().
+ *
+ * If "options" is provided calls run_hooks_opt_clear() on it for
+ * you. If "options" is NULL a scratch one will be provided for you
+ * before calling run_hooks().
+ */
+int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options);
+
 #endif
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
#517 lore raw

PATCH v4 10/36 rebase: convert pre-rebase to use hook.h

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Move the pre-rebase hook away from run-command.h to and over to the new hook.h library.

---
 builtin/rebase.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
builtin/rebase.c +4 -2
index 12f093121d9..e7c668c99b1 100644
@@ -28,6 +28,7 @@
 #include "sequencer.h"
 #include "rebase-interactive.h"
 #include "reset.h"
+#include "hook.h"
 
 #define DEFAULT_REFLOG_ACTION "rebase"
 
@@ -1313,6 +1314,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	char *squash_onto_name = NULL;
 	int reschedule_failed_exec = -1;
 	int allow_preemptive_ff = 1;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
 	struct option builtin_rebase_options[] = {
 		OPT_STRING(0, "onto", &options.onto_name,
 			   N_("revision"),
@@ -2022,9 +2024,9 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	}
 
 	/* If a hook exists, give it a chance to interrupt*/
+	strvec_pushl(&hook_opt.args, options.upstream_arg, argc ? argv[0] : NULL, NULL);
 	if (!ok_to_skip_pre_rebase &&
-	    run_hook_le(NULL, "pre-rebase", options.upstream_arg,
-			argc ? argv[0] : NULL, NULL))
+	    run_hooks_oneshot("pre-rebase", &hook_opt))
 		die(_("The pre-rebase hook refused to rebase."));
 
 	if (options.flags & REBASE_DIFFSTAT) {
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
#518 lore raw

PATCH v4 11/36 am: convert applypatch to use hook.h

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Teach pre-applypatch, post-applypatch, and applypatch-msg to use the hook.h library instead of the run-command.h library.

---
 builtin/am.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
builtin/am.c +5 -3
index c603f3cebdf..e444b18b64a 100644
@@ -446,9 +446,11 @@ static void am_destroy(const struct am_state *state)
 static int run_applypatch_msg_hook(struct am_state *state)
 {
 	int ret;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 
 	assert(state->msg);
-	ret = run_hook_le(NULL, "applypatch-msg", am_path(state, "final-commit"), NULL);
+	strvec_push(&opt.args, am_path(state, "final-commit"));
+	ret = run_hooks_oneshot("applypatch-msg", &opt);
 
 	if (!ret) {
 		FREE_AND_NULL(state->msg);
@@ -1609,7 +1611,7 @@ static void do_commit(const struct am_state *state)
 	const char *reflog_msg, *author, *committer = NULL;
 	struct strbuf sb = STRBUF_INIT;
 
-	if (run_hook_le(NULL, "pre-applypatch", NULL))
+	if (run_hooks_oneshot("pre-applypatch", NULL))
 		exit(1);
 
 	if (write_cache_as_tree(&tree, 0, NULL))
@@ -1661,7 +1663,7 @@ static void do_commit(const struct am_state *state)
 		fclose(fp);
 	}
 
-	run_hook_le(NULL, "post-applypatch", NULL);
+	run_hooks_oneshot("post-applypatch", NULL);
 
 	strbuf_release(&sb);
 }
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
#519 lore raw

PATCH v4 12/36 hooks: convert 'post-checkout' hook to hook library

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Move the running of the 'post-checkout' hook away from run-command.h to the new hook.h library. For "worktree" this requires a change to it to run the hooks from a given directory.

We could strictly speaking skip the "absolute_path" flag and just check if "dir" is specified, but let's split them up for clarity, as well as for any future user who'd like to set "dir" but not implicitly change the argument to an absolute path.

---
 builtin/checkout.c | 14 +++++++++-----
 builtin/clone.c    |  6 ++++--
 builtin/worktree.c | 28 ++++++++++++----------------
 hook.c             |  9 +++++++++
 hook.h             |  8 ++++++++
 read-cache.c       |  1 +
 reset.c            | 14 ++++++++++----
 7 files changed, 53 insertions(+), 27 deletions(-)
builtin/checkout.c +9 -5
index f4cd7747d35..6d69b4c0113 100644
@@ -9,6 +9,7 @@
 #include "config.h"
 #include "diff.h"
 #include "dir.h"
+#include "hook.h"
 #include "ll-merge.h"
 #include "lockfile.h"
 #include "merge-recursive.h"
@@ -106,13 +107,16 @@ struct branch_info {
 static int post_checkout_hook(struct commit *old_commit, struct commit *new_commit,
 			      int changed)
 {
-	return run_hook_le(NULL, "post-checkout",
-			   oid_to_hex(old_commit ? &old_commit->object.oid : null_oid()),
-			   oid_to_hex(new_commit ? &new_commit->object.oid : null_oid()),
-			   changed ? "1" : "0", NULL);
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+
 	/* "new_commit" can be NULL when checking out from the index before
 	   a commit exists. */
-
+	strvec_pushl(&opt.args,
+		     oid_to_hex(old_commit ? &old_commit->object.oid : null_oid()),
+		     oid_to_hex(new_commit ? &new_commit->object.oid : null_oid()),
+		     changed ? "1" : "0",
+		     NULL);
+	return run_hooks_oneshot("post-checkout", &opt);
 }
 
 static int update_some(const struct object_id *oid, struct strbuf *base,
builtin/clone.c +4 -2
index 66fe66679c8..27fc05ee511 100644
@@ -32,6 +32,7 @@
 #include "connected.h"
 #include "packfile.h"
 #include "list-objects-filter-options.h"
+#include "hook.h"
 
 /*
  * Overall FIXMEs:
@@ -775,6 +776,7 @@ static int checkout(int submodule_progress)
 	struct tree *tree;
 	struct tree_desc t;
 	int err = 0;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
 
 	if (option_no_checkout)
 		return 0;
@@ -820,8 +822,8 @@ static int checkout(int submodule_progress)
 	if (write_locked_index(&the_index, &lock_file, COMMIT_LOCK))
 		die(_("unable to write new index file"));
 
-	err |= run_hook_le(NULL, "post-checkout", oid_to_hex(null_oid()),
-			   oid_to_hex(&oid), "1", NULL);
+	strvec_pushl(&hook_opt.args, oid_to_hex(null_oid()), oid_to_hex(&oid), "1", NULL);
+	err |= run_hooks_oneshot("post-checkout", &hook_opt);
 
 	if (!err && (option_recurse_submodules.nr > 0)) {
 		struct strvec args = STRVEC_INIT;
builtin/worktree.c +12 -16
index d22ece93e1a..330867c19bf 100644
@@ -382,22 +382,18 @@ static int add_worktree(const char *path, const char *refname,
 	 * is_junk is cleared, but do return appropriate code when hook fails.
 	 */
 	if (!ret && opts->checkout) {
-		const char *hook = find_hook("post-checkout");
-		if (hook) {
-			const char *env[] = { "GIT_DIR", "GIT_WORK_TREE", NULL };
-			cp.git_cmd = 0;
-			cp.no_stdin = 1;
-			cp.stdout_to_stderr = 1;
-			cp.dir = path;
-			cp.env = env;
-			cp.argv = NULL;
-			cp.trace2_hook_name = "post-checkout";
-			strvec_pushl(&cp.args, absolute_path(hook),
-				     oid_to_hex(null_oid()),
-				     oid_to_hex(&commit->object.oid),
-				     "1", NULL);
-			ret = run_command(&cp);
-		}
+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+
+		strvec_pushl(&opt.env, "GIT_DIR", "GIT_WORK_TREE", NULL);
+		strvec_pushl(&opt.args,
+			     oid_to_hex(null_oid()),
+			     oid_to_hex(&commit->object.oid),
+			     "1",
+			     NULL);
+		opt.dir = path;
+		opt.absolute_path = 1;
+
+		ret = run_hooks_oneshot("post-checkout", &opt);
 	}
 
 	strvec_clear(&child_env);
hook.c +9 -0
index 981a9bf46e6..5ecf79c8397 100644
@@ -92,6 +92,7 @@ static int pick_next_hook(struct child_process *cp,
 	cp->env = hook_cb->options->env.v;
 	cp->stdout_to_stderr = 1;
 	cp->trace2_hook_name = hook_cb->hook_name;
+	cp->dir = hook_cb->options->dir;
 
 	/* add command */
 	strvec_push(&cp->args, run_me->hook_path);
@@ -145,6 +146,7 @@ static int notify_hook_finished(int result,
 int run_hooks(const char *hook_name, const char *hook_path,
 	      struct run_hooks_opt *options)
 {
+	struct strbuf abs_path = STRBUF_INIT;
 	struct hook my_hook = {
 		.hook_path = hook_path,
 	};
@@ -158,6 +160,10 @@ int run_hooks(const char *hook_name, const char *hook_path,
 	if (!options)
 		BUG("a struct run_hooks_opt must be provided to run_hooks");
 
+	if (options->absolute_path) {
+		strbuf_add_absolute_path(&abs_path, hook_path);
+		my_hook.hook_path = abs_path.buf;
+	}
 	cb_data.run_me = &my_hook;
 
 	run_processes_parallel_tr2(jobs,
@@ -168,6 +174,9 @@ int run_hooks(const char *hook_name, const char *hook_path,
 				   "hook",
 				   hook_name);
 
+	if (options->absolute_path)
+		strbuf_release(&abs_path);
+
 	return cb_data.rc;
 }
 
hook.h +8 -0
index 2201ab9d568..53ea3a9649e 100644
@@ -28,6 +28,14 @@ struct run_hooks_opt
 
 	/* Args to be passed to each hook */
 	struct strvec args;
+
+	/* Resolve and run the "absolute_path(hook)" instead of
+	 * "hook". Used for "git worktree" hooks
+	 */
+	int absolute_path;
+
+	/* Path to initial working directory for subprocess */
+	const char *dir;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
read-cache.c +1 -0
index 99a174b91e6..c9e2b013972 100644
@@ -28,6 +28,7 @@
 #include "sparse-index.h"
 #include "csum-file.h"
 #include "promisor-remote.h"
+#include "hook.h"
 
 /* Mask for the name length in ce_flags in the on-disk index */
 
reset.c +10 -4
index 4bea758053b..6499bc5127d 100644
@@ -7,6 +7,7 @@
 #include "tree-walk.h"
 #include "tree.h"
 #include "unpack-trees.h"
+#include "hook.h"
 
 int reset_head(struct repository *r, struct object_id *oid, const char *action,
 	       const char *switch_to_branch, unsigned flags,
@@ -126,10 +127,15 @@ int reset_head(struct repository *r, struct object_id *oid, const char *action,
 			ret = create_symref("HEAD", switch_to_branch,
 					    reflog_head);
 	}
-	if (run_hook)
-		run_hook_le(NULL, "post-checkout",
-			    oid_to_hex(orig ? orig : null_oid()),
-			    oid_to_hex(oid), "1", NULL);
+	if (run_hook) {
+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+		strvec_pushl(&opt.args,
+			     oid_to_hex(orig ? orig : null_oid()),
+			     oid_to_hex(oid),
+			     "1",
+			     NULL);
+		run_hooks_oneshot("post-checkout", &opt);
+	}
 
 leave_reset_head:
 	strbuf_release(&msg);
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
#520 lore raw

PATCH v4 13/36 merge: convert post-merge to use hook.h

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Teach post-merge to use the hook.h library instead of the run-command.h library to run hooks.

---
 builtin/merge.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
builtin/merge.c +3 -1
index 03f244dd5a0..4965df2ac29 100644
@@ -448,6 +448,7 @@ static void finish(struct commit *head_commit,
 		   const struct object_id *new_head, const char *msg)
 {
 	struct strbuf reflog_message = STRBUF_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	const struct object_id *head = &head_commit->object.oid;
 
 	if (!msg)
@@ -489,7 +490,8 @@ static void finish(struct commit *head_commit,
 	}
 
 	/* Run a post-merge hook */
-	run_hook_le(NULL, "post-merge", squash ? "1" : "0", NULL);
+	strvec_push(&opt.args, squash ? "1" : "0");
+	run_hooks_oneshot("post-merge", &opt);
 
 	apply_autostash(git_path_merge_autostash(the_repository));
 	strbuf_release(&reflog_message);
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
#521 lore raw

PATCH v4 14/36 git hook run: add an --ignore-missing flag

In reply to Ævar Arnfjörð Bjarmason

For certain one-shot hooks we'd like to optimistically run them, and not complain if they don't exist. This will be used by send-email in a subsequent commit.

---
 Documentation/git-hook.txt | 10 +++++++++-
 builtin/hook.c             |  7 ++++++-
 t/t1800-hook.sh            |  5 +++++
 3 files changed, 20 insertions(+), 2 deletions(-)
Documentation/git-hook.txt +9 -1
index 660d6a992a0..097fb9de63b 100644
@@ -8,7 +8,7 @@ git-hook - run git hooks
 SYNOPSIS
 --------
 [verse]
-'git hook' run <hook-name> [-- <hook-args>]
+'git hook' run [--ignore-missing] <hook-name> [-- <hook-args>]
 
 DESCRIPTION
 -----------
@@ -29,6 +29,14 @@ optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The
 arguments (if any) differ by hook name, see linkgit:githooks[5] for
 what those are.
 
+OPTIONS
+-------
+
+--ignore-missing::
+	Ignore any missing hook by quietly returning zero. Used for
+	tools that want to do a blind one-shot run of a hook that may
+	or may not be present.
+
 SEE ALSO
 --------
 linkgit:githooks[5]
builtin/hook.c +6 -1
index 41dd15550cf..f33db9953c7 100644
@@ -7,7 +7,7 @@
 #include "strvec.h"
 
 #define BUILTIN_HOOK_RUN_USAGE \
-	N_("git hook run <hook-name> [-- <hook-args>]")
+	N_("git hook run [--ignore-missing] <hook-name> [-- <hook-args>]")
 
 static const char * const builtin_hook_usage[] = {
 	BUILTIN_HOOK_RUN_USAGE,
@@ -23,9 +23,12 @@ static int run(int argc, const char **argv, const char *prefix)
 {
 	int i;
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	int ignore_missing = 0;
 	const char *hook_name;
 	const char *hook_path;
 	struct option run_options[] = {
+		OPT_BOOL(0, "ignore-missing", &ignore_missing,
+			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
 		OPT_END(),
 	};
 	int ret;
@@ -53,6 +56,8 @@ static int run(int argc, const char **argv, const char *prefix)
 	git_config(git_default_config, NULL);
 
 	hook_name = argv[0];
+	if (ignore_missing)
+		return run_hooks_oneshot(hook_name, &opt);
 	hook_path = find_hook(hook_name);
 	if (!hook_path) {
 		error("cannot find a hook named %s", hook_name);
t/t1800-hook.sh +5 -0
index 644df0a583c..49df5a2cdfb 100755
@@ -25,6 +25,11 @@ test_expect_success 'git hook run: nonexistent hook' '
 	test_cmp stderr.expect stderr.actual
 '
 
+test_expect_success 'git hook run: nonexistent hook with --ignore-missing' '
+	git hook run --ignore-missing does-not-exist 2>stderr.actual &&
+	test_must_be_empty stderr.actual
+'
+
 test_expect_success 'git hook run: basic' '
 	write_script .git/hooks/test-hook <<-EOF &&
 	echo Test hook
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
#522 lore raw

PATCH v4 15/36 send-email: use 'git hook run' for 'sendemail-validate'

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Change the "sendmail-validate" hook to be run via the "git hook run" wrapper instead of via a direct invocation.

This is the smallest possibly change to get "send-email" using "git hook run". We still check the hook itself with "-x", and set a "GIT_DIR" variable, both of which are asserted by our tests. We'll need to get rid of this special behavior if we start running N hooks, but for now let's be as close to bug-for-bug compatible as possible.

---
 git-send-email.perl   | 20 ++++++++++++--------
 t/t9001-send-email.sh |  4 ++--
 2 files changed, 14 insertions(+), 10 deletions(-)
git-send-email.perl +12 -8
index e65d969d0bb..126850d974b 100755
@@ -195,13 +195,13 @@ sub format_2822_time {
 my $editor;
 
 sub system_or_msg {
-	my ($args, $msg) = @_;
+	my ($args, $msg, $cmd_name) = @_;
 	system(@$args);
 	my $signalled = $? & 127;
 	my $exit_code = $? >> 8;
 	return unless $signalled or $exit_code;
 
-	my @sprintf_args = ($args->[0], $exit_code);
+	my @sprintf_args = ($cmd_name ? $cmd_name : $args->[0], $exit_code);
 	if (defined $msg) {
 		# Quiet the 'redundant' warning category, except we
 		# need to support down to Perl 5.8, so we can't do a
@@ -2031,10 +2031,10 @@ sub validate_patch {
 	my ($fn, $xfer_encoding) = @_;
 
 	if ($repo) {
+		my $hook_name = 'sendemail-validate';
 		my $hooks_path = $repo->command_oneline('rev-parse', '--git-path', 'hooks');
 		require File::Spec;
-		my $validate_hook = File::Spec->catfile($hooks_path,
-					    'sendemail-validate');
+		my $validate_hook = File::Spec->catfile($hooks_path, $hook_name);
 		my $hook_error;
 		if (-x $validate_hook) {
 			require Cwd;
@@ -2044,13 +2044,17 @@ sub validate_patch {
 			chdir($repo->wc_path() or $repo->repo_path())
 				or die("chdir: $!");
 			local $ENV{"GIT_DIR"} = $repo->repo_path();
-			$hook_error = system_or_msg([$validate_hook, $target]);
+			my @validate_hook = ("git", "hook", "run", "--ignore-missing", $hook_name, "--", $target);
+			$hook_error = system_or_msg(\@validate_hook, undef,
+						       "git hook run $hook_name -- <patch>");
 			chdir($cwd_save) or die("chdir: $!");
 		}
 		if ($hook_error) {
-			die sprintf(__("fatal: %s: rejected by sendemail-validate hook\n" .
-				       "%s\n" .
-				       "warning: no patches were sent\n"), $fn, $hook_error);
+			$hook_error = sprintf(__("fatal: %s: rejected by %s hook\n" .
+						 $hook_error . "\n" .
+						 "warning: no patches were sent\n"),
+					      $fn, $hook_name);
+			die $hook_error;
 		}
 	}
 
t/t9001-send-email.sh +2 -2
index 57fc10e7f82..9ec7d75f0ff 100755
@@ -539,7 +539,7 @@ test_expect_success $PREREQ "--validate respects relative core.hooksPath path" '
 	test_path_is_file my-hooks.ran &&
 	cat >expect <<-EOF &&
 	fatal: longline.patch: rejected by sendemail-validate hook
-	fatal: command '"'"'my-hooks/sendemail-validate'"'"' died with exit code 1
+	fatal: command '"'"'git hook run sendemail-validate -- <patch>'"'"' died with exit code 1
 	warning: no patches were sent
 	EOF
 	test_cmp expect actual
@@ -558,7 +558,7 @@ test_expect_success $PREREQ "--validate respects absolute core.hooksPath path" '
 	test_path_is_file my-hooks.ran &&
 	cat >expect <<-EOF &&
 	fatal: longline.patch: rejected by sendemail-validate hook
-	fatal: command '"'"'$hooks_path/sendemail-validate'"'"' died with exit code 1
+	fatal: command '"'"'git hook run sendemail-validate -- <patch>'"'"' died with exit code 1
 	warning: no patches were sent
 	EOF
 	test_cmp expect actual
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
#523 lore raw

PATCH v4 16/36 git-p4: use 'git hook' to run hooks

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Instead of duplicating the behavior of run-command.h:run_hook_le() in Python, we can directly call 'git hook run'. We emulate the existence check with the --ignore-missing flag.

As this is the last hook execution in git.git to not go through "git hook run" or the hook.[ch] library we can now be absolutely sure that our assertion in hook.c that only hooks known by the generated (from githooks(5)) hook-list.h are permitted.

---
 git-p4.py | 72 ++++++-------------------------------------------------
 1 file changed, 7 insertions(+), 65 deletions(-)
git-p4.py +7 -65
index 2b4500226aa..1f24cbf0bca 100755
@@ -207,71 +207,13 @@ def decode_path(path):
         return path
 
 def run_git_hook(cmd, param=[]):
-    """Execute a hook if the hook exists."""
-    if verbose:
-        sys.stderr.write("Looking for hook: %s\n" % cmd)
-        sys.stderr.flush()
-
-    hooks_path = gitConfig("core.hooksPath")
-    if len(hooks_path) <= 0:
-        hooks_path = os.path.join(os.environ["GIT_DIR"], "hooks")
-
-    if not isinstance(param, list):
-        param=[param]
-
-    # resolve hook file name, OS depdenent
-    hook_file = os.path.join(hooks_path, cmd)
-    if platform.system() == 'Windows':
-        if not os.path.isfile(hook_file):
-            # look for the file with an extension
-            files = glob.glob(hook_file + ".*")
-            if not files:
-                return True
-            files.sort()
-            hook_file = files.pop()
-            while hook_file.upper().endswith(".SAMPLE"):
-                # The file is a sample hook. We don't want it
-                if len(files) > 0:
-                    hook_file = files.pop()
-                else:
-                    return True
-
-    if not os.path.isfile(hook_file) or not os.access(hook_file, os.X_OK):
-        return True
-
-    return run_hook_command(hook_file, param) == 0
-
-def run_hook_command(cmd, param):
-    """Executes a git hook command
-       cmd = the command line file to be executed. This can be
-       a file that is run by OS association.
-
-       param = a list of parameters to pass to the cmd command
-
-       On windows, the extension is checked to see if it should
-       be run with the Git for Windows Bash shell.  If there
-       is no file extension, the file is deemed a bash shell
-       and will be handed off to sh.exe. Otherwise, Windows
-       will be called with the shell to handle the file assocation.
-
-       For non Windows operating systems, the file is called
-       as an executable.
-    """
-    cli = [cmd] + param
-    use_shell = False
-    if platform.system() == 'Windows':
-        (root,ext) = os.path.splitext(cmd)
-        if ext == "":
-            exe_path = os.environ.get("EXEPATH")
-            if exe_path is None:
-                exe_path = ""
-            else:
-                exe_path = os.path.join(exe_path, "bin")
-            cli = [os.path.join(exe_path, "SH.EXE")] + cli
-        else:
-            use_shell = True
-    return subprocess.call(cli, shell=use_shell)
-
+    """args are specified with -a <arg> -a <arg> -a <arg>"""
+    args = ['git', 'hook', 'run', '--ignore-missing', cmd]
+    if param:
+        args.append("--")
+        for p in param:
+            args.append(p)
+    return subprocess.call(args) == 0
 
 def write_pipe(c, stdin):
     if verbose:
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
#524 lore raw

PATCH v4 17/36 commit: convert {pre-commit,prepare-commit-msg} hook to hook.h

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Move these hooks hook away from run-command.h to and over to the new hook.h library.

---
 commit.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)
commit.c +8 -7
index 143f472c0f2..63d7943a86d 100644
@@ -21,6 +21,7 @@
 #include "commit-reach.h"
 #include "run-command.h"
 #include "shallow.h"
+#include "hook.h"
 
 static struct commit_extra_header *read_commit_extra_header_lines(const char *buf, size_t len, const char **);
 
@@ -1698,22 +1699,22 @@ size_t ignore_non_trailer(const char *buf, size_t len)
 int run_commit_hook(int editor_is_used, const char *index_file,
 		    const char *name, ...)
 {
-	struct strvec hook_env = STRVEC_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	va_list args;
-	int ret;
+	const char *arg;
 
-	strvec_pushf(&hook_env, "GIT_INDEX_FILE=%s", index_file);
+	strvec_pushf(&opt.env, "GIT_INDEX_FILE=%s", index_file);
 
 	/*
 	 * Let the hook know that no editor will be launched.
 	 */
 	if (!editor_is_used)
-		strvec_push(&hook_env, "GIT_EDITOR=:");
+		strvec_push(&opt.env, "GIT_EDITOR=:");
 
 	va_start(args, name);
-	ret = run_hook_ve(hook_env.v, name, args);
+	while ((arg = va_arg(args, const char *)))
+		strvec_push(&opt.args, arg);
 	va_end(args);
-	strvec_clear(&hook_env);
 
-	return ret;
+	return run_hooks_oneshot(name, &opt);
 }
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
#525 lore raw

PATCH v4 18/36 read-cache: convert post-index-change to use hook.h

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Move the post-index-change hook away from run-command.h to and over to the new hook.h library.

This removes the last direct user of run_hook_ve(), so we can make the function static now. It'll be removed entirely soon.

---
 read-cache.c  | 10 +++++++---
 run-command.c |  2 +-
 run-command.h |  1 -
 3 files changed, 8 insertions(+), 5 deletions(-)
read-cache.c +7 -3
index c9e2b013972..90099ca14df 100644
@@ -3068,6 +3068,7 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 {
 	int ret;
 	int was_full = !istate->sparse_index;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
 
 	ret = convert_to_sparse(istate);
 
@@ -3096,9 +3097,12 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 	else
 		ret = close_lock_file_gently(lock);
 
-	run_hook_le(NULL, "post-index-change",
-			istate->updated_workdir ? "1" : "0",
-			istate->updated_skipworktree ? "1" : "0", NULL);
+	strvec_pushl(&hook_opt.args,
+		     istate->updated_workdir ? "1" : "0",
+		     istate->updated_skipworktree ? "1" : "0",
+		     NULL);
+	run_hooks_oneshot("post-index-change", &hook_opt);
+
 	istate->updated_workdir = 0;
 	istate->updated_skipworktree = 0;
 
run-command.c +1 -1
index 352f5be1646..b4341ba1c7b 100644
@@ -1320,7 +1320,7 @@ int async_with_fork(void)
 #endif
 }
 
-int run_hook_ve(const char *const *env, const char *name, va_list args)
+static int run_hook_ve(const char *const *env, const char *name, va_list args)
 {
 	struct child_process hook = CHILD_PROCESS_INIT;
 	const char *p;
run-command.h +0 -1
index f76b740f927..7a867d41217 100644
@@ -219,7 +219,6 @@ int run_command(struct child_process *);
  */
 LAST_ARG_MUST_BE_NULL
 int run_hook_le(const char *const *env, const char *name, ...);
-int run_hook_ve(const char *const *env, const char *name, va_list args);
 
 /*
  * Trigger an auto-gc
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
#526 lore raw

PATCH v4 19/36 receive-pack: convert push-to-checkout hook to hook.h

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Move the push-to-checkout hook away from run-command.h to and over to the new hook.h library.

---
 builtin/receive-pack.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
builtin/receive-pack.c +5 -2
index 91fa799b66e..a7d03bbc7d3 100644
@@ -1435,9 +1435,12 @@ static const char *push_to_checkout(unsigned char *hash,
 				    struct strvec *env,
 				    const char *work_tree)
 {
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+
 	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
-	if (run_hook_le(env->v, push_to_checkout_hook,
-			hash_to_hex(hash), NULL))
+	strvec_pushv(&opt.env, env->v);
+	strvec_push(&opt.args, hash_to_hex(hash));
+	if (run_hooks_oneshot(push_to_checkout_hook, &opt))
 		return "push-to-checkout hook declined";
 	else
 		return NULL;
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
#527 lore raw

PATCH v4 20/36 run-command: remove old run_hook_{le,ve}() hook API

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

The new hook.h library has replaced all run-command.h hook-related functionality. So let's delete this dead code.

---
 run-command.c | 32 --------------------------------
 run-command.h | 16 ----------------
 2 files changed, 48 deletions(-)
run-command.c +0 -32
index b4341ba1c7b..1399243de8a 100644
@@ -1320,38 +1320,6 @@ int async_with_fork(void)
 #endif
 }
 
-static int run_hook_ve(const char *const *env, const char *name, va_list args)
-{
-	struct child_process hook = CHILD_PROCESS_INIT;
-	const char *p;
-
-	p = find_hook(name);
-	if (!p)
-		return 0;
-
-	strvec_push(&hook.args, p);
-	while ((p = va_arg(args, const char *)))
-		strvec_push(&hook.args, p);
-	hook.env = env;
-	hook.no_stdin = 1;
-	hook.stdout_to_stderr = 1;
-	hook.trace2_hook_name = name;
-
-	return run_command(&hook);
-}
-
-int run_hook_le(const char *const *env, const char *name, ...)
-{
-	va_list args;
-	int ret;
-
-	va_start(args, name);
-	ret = run_hook_ve(env, name, args);
-	va_end(args);
-
-	return ret;
-}
-
 struct io_pump {
 	/* initialized by caller */
 	int fd;
run-command.h +0 -16
index 7a867d41217..cfb6887e4ae 100644
@@ -204,22 +204,6 @@ int finish_command_in_signal(struct child_process *);
  */
 int run_command(struct child_process *);
 
-/**
- * Run a hook.
- * The first argument is a pathname to an index file, or NULL
- * if the hook uses the default index file or no index is needed.
- * The second argument is the name of the hook.
- * The further arguments correspond to the hook arguments.
- * The last argument has to be NULL to terminate the arguments list.
- * If the hook does not exist or is not executable, the return
- * value will be zero.
- * If it is executable, the hook will be executed and the exit
- * status of the hook is returned.
- * On execution, .stdout_to_stderr and .no_stdin will be set.
- */
-LAST_ARG_MUST_BE_NULL
-int run_hook_le(const char *const *env, const char *name, ...);
-
 /*
  * Trigger an auto-gc
  */
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
#528 lore raw

PATCH v4 21/36 run-command: allow stdin for run_processes_parallel

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

While it makes sense not to inherit stdin from the parent process to avoid deadlocking, it's not necessary to completely ban stdin to children. An informed user should be able to configure stdin safely. By setting some_child.process.no_stdin=1 before calling get_next_task() we provide a reasonable default behavior but enable users to set up stdin streaming for themselves during the callback.

some_child.process.stdout_to_stderr, however, remains unmodifiable by get_next_task() - the rest of the run_processes_parallel() API depends on child output in stderr.

---
 run-command.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
run-command.c +8 -1
index 1399243de8a..482ee2d76c6 100644
@@ -1627,6 +1627,14 @@ static int pp_start_one(struct parallel_processes *pp)
 	if (i == pp->max_processes)
 		BUG("bookkeeping is hard");
 
+	/*
+	 * By default, do not inherit stdin from the parent process - otherwise,
+	 * all children would share stdin! Users may overwrite this to provide
+	 * something to the child's stdin by having their 'get_next_task'
+	 * callback assign 0 to .no_stdin and an appropriate integer to .in.
+	 */
+	pp->children[i].process.no_stdin = 1;
+
 	code = pp->get_next_task(&pp->children[i].process,
 				 &pp->children[i].err,
 				 pp->data,
@@ -1638,7 +1646,6 @@ static int pp_start_one(struct parallel_processes *pp)
 	}
 	pp->children[i].process.err = -1;
 	pp->children[i].process.stdout_to_stderr = 1;
-	pp->children[i].process.no_stdin = 1;
 
 	if (start_command(&pp->children[i].process)) {
 		code = pp->start_failure(&pp->children[i].err,
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
#529 lore raw

PATCH v4 22/36 hook: support passing stdin to hooks

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Some hooks (such as post-rewrite) need to take input via stdin. Previously, callers provided stdin to hooks by setting run-command.h:child_process.in, which takes a FD. Callers would open the file in question themselves before calling run-command(). However, since we will now need to seek to the front of the file and read it again for every hook which runs, hook.h:run_command() takes a path and handles FD management itself. Since this file is opened for read only, it should not prevent later parallel execution support.

On the frontend, this is supported by asking for a file path, rather than by reading stdin. Reading directly from stdin would involve caching the entire stdin (to memory or to disk) and reading it back from the beginning to each hook. We'd want to support cases like insufficient memory or storage for the file. While this may prove useful later, for now the path of least resistance is to just ask the user to make this interim file themselves.

---
 Documentation/git-hook.txt |  7 ++++++-
 builtin/hook.c             |  4 +++-
 hook.c                     |  8 +++++++-
 hook.h                     |  3 +++
 t/t1800-hook.sh            | 18 ++++++++++++++++++
 5 files changed, 37 insertions(+), 3 deletions(-)
Documentation/git-hook.txt +6 -1
index 097fb9de63b..fa68c1f3912 100644
@@ -8,7 +8,7 @@ git-hook - run git hooks
 SYNOPSIS
 --------
 [verse]
-'git hook' run [--ignore-missing] <hook-name> [-- <hook-args>]
+'git hook' run [--to-stdin=<path>] [--ignore-missing] <hook-name> [-- <hook-args>]
 
 DESCRIPTION
 -----------
@@ -32,6 +32,11 @@ what those are.
 OPTIONS
 -------
 
+--to-stdin::
+	For "run"; Specify a file which will be streamed into the
+	hook's stdin. The hook will receive the entire file from
+	beginning to EOF.
+
 --ignore-missing::
 	Ignore any missing hook by quietly returning zero. Used for
 	tools that want to do a blind one-shot run of a hook that may
builtin/hook.c +3 -1
index f33db9953c7..27dce6a2f0e 100644
@@ -7,7 +7,7 @@
 #include "strvec.h"
 
 #define BUILTIN_HOOK_RUN_USAGE \
-	N_("git hook run [--ignore-missing] <hook-name> [-- <hook-args>]")
+	N_("git hook run [--ignore-missing] [--to-stdin=<path>] <hook-name> [-- <hook-args>]")
 
 static const char * const builtin_hook_usage[] = {
 	BUILTIN_HOOK_RUN_USAGE,
@@ -29,6 +29,8 @@ static int run(int argc, const char **argv, const char *prefix)
 	struct option run_options[] = {
 		OPT_BOOL(0, "ignore-missing", &ignore_missing,
 			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
+		OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"),
+			   N_("file to read into hooks' stdin")),
 		OPT_END(),
 	};
 	int ret;
hook.c +7 -1
index 5ecf79c8397..63c9a60921f 100644
@@ -88,7 +88,13 @@ static int pick_next_hook(struct child_process *cp,
 	if (!run_me)
 		return 0;
 
-	cp->no_stdin = 1;
+	/* reopen the file for stdin; run_command closes it. */
+	if (hook_cb->options->path_to_stdin) {
+		cp->no_stdin = 0;
+		cp->in = xopen(hook_cb->options->path_to_stdin, O_RDONLY);
+	} else {
+		cp->no_stdin = 1;
+	}
 	cp->env = hook_cb->options->env.v;
 	cp->stdout_to_stderr = 1;
 	cp->trace2_hook_name = hook_cb->hook_name;
hook.h +3 -0
index 53ea3a9649e..cd6a68a3b50 100644
@@ -36,6 +36,9 @@ struct run_hooks_opt
 
 	/* Path to initial working directory for subprocess */
 	const char *dir;
+
+	/* Path to file which should be piped to stdin for each hook */
+	const char *path_to_stdin;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
t/t1800-hook.sh +18 -0
index 49df5a2cdfb..217db848b3f 100755
@@ -135,4 +135,22 @@ test_expect_success 'git -c core.hooksPath=<PATH> hook run' '
 	test_cmp expect actual
 '
 
+test_expect_success 'stdin to hooks' '
+	write_script .git/hooks/test-hook <<-\EOF &&
+	echo BEGIN stdin
+	cat
+	echo END stdin
+	EOF
+
+	cat >expect <<-EOF &&
+	BEGIN stdin
+	hello
+	END stdin
+	EOF
+
+	echo hello >input &&
+	git hook run --to-stdin=input test-hook 2>actual &&
+	test_cmp expect actual
+'
+
 test_done
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
#530 lore raw

PATCH v4 23/36 am: convert 'post-rewrite' hook to hook.h

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

---
 builtin/am.c | 20 ++++----------------
 1 file changed, 4 insertions(+), 16 deletions(-)
builtin/am.c +4 -16
index e444b18b64a..9e3d4d9ab44 100644
@@ -467,24 +467,12 @@ static int run_applypatch_msg_hook(struct am_state *state)
  */
 static int run_post_rewrite_hook(const struct am_state *state)
 {
-	struct child_process cp = CHILD_PROCESS_INIT;
-	const char *hook = find_hook("post-rewrite");
-	int ret;
-
-	if (!hook)
-		return 0;
-
-	strvec_push(&cp.args, hook);
-	strvec_push(&cp.args, "rebase");
-
-	cp.in = xopen(am_path(state, "rewritten"), O_RDONLY);
-	cp.stdout_to_stderr = 1;
-	cp.trace2_hook_name = "post-rewrite";
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 
-	ret = run_command(&cp);
+	strvec_push(&opt.args, "rebase");
+	opt.path_to_stdin = am_path(state, "rewritten");
 
-	close(cp.in);
-	return ret;
+	return run_hooks_oneshot("post-rewrite", &opt);
 }
 
 /**
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
#531 lore raw

PATCH v4 24/36 run-command: add stdin callback for parallelization

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

If a user of the run_processes_parallel() API wants to pipe a large amount of information to stdin of each parallel command, that information could exceed the buffer of the pipe allocated for that process's stdin. Generally this is solved by repeatedly writing to child_process.in between calls to start_command() and finish_command(); run_processes_parallel() did not provide users an opportunity to access child_process at that time.

Because the data might be extremely large (for example, a list of all refs received during a push from a client) simply taking a string_list or strbuf is not as scalable as using a callback; the rest of the run_processes_parallel() API also uses callbacks, so making this feature match the rest of the API reduces mental load on the user.

---
 builtin/fetch.c             |  1 +
 builtin/submodule--helper.c |  2 +-
 hook.c                      |  1 +
 run-command.c               | 54 +++++++++++++++++++++++++++++++++++--
 run-command.h               | 17 +++++++++++-
 submodule.c                 |  1 +
 t/helper/test-run-command.c | 31 ++++++++++++++++++---
 t/t0061-run-command.sh      | 30 +++++++++++++++++++++
 8 files changed, 129 insertions(+), 8 deletions(-)
builtin/fetch.c +1 -0
index 25740c13df1..fef6e85d003 100644
@@ -1817,6 +1817,7 @@ static int fetch_multiple(struct string_list *list, int max_children)
 		result = run_processes_parallel_tr2(max_children,
 						    &fetch_next_remote,
 						    &fetch_failed_to_start,
+						    NULL,
 						    &fetch_finished,
 						    &state,
 						    "fetch", "parallel/fetch");
builtin/submodule--helper.c +1 -1
index f73963ad67d..f42ded548bf 100644
@@ -2294,7 +2294,7 @@ static int update_submodules(struct submodule_update_clone *suc)
 	int i;
 
 	run_processes_parallel_tr2(suc->max_jobs, update_clone_get_next_task,
-				   update_clone_start_failure,
+				   update_clone_start_failure, NULL,
 				   update_clone_task_finished, suc, "submodule",
 				   "parallel/update");
 
hook.c +1 -0
index 63c9a60921f..a7462639d97 100644
@@ -175,6 +175,7 @@ int run_hooks(const char *hook_name, const char *hook_path,
 	run_processes_parallel_tr2(jobs,
 				   pick_next_hook,
 				   notify_start_failure,
+				   NULL,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
run-command.c +52 -2

125 lines not on this page · raw message

run-command.h +16 -1
index cfb6887e4ae..80d394664ae 100644
@@ -422,6 +422,20 @@ typedef int (*start_failure_fn)(struct strbuf *out,
 				void *pp_cb,
 				void *pp_task_cb);
 
+/**
+ * This callback is called repeatedly on every child process who requests
+ * start_command() to create a pipe by setting child_process.in < 0.
+ *
+ * pp_cb is the callback cookie as passed into run_processes_parallel, and
+ * pp_task_cb is the callback cookie as passed into get_next_task_fn.
+ * The contents of 'send' will be read into the pipe and passed to the pipe.
+ *
+ * Return nonzero to close the pipe.
+ */
+typedef int (*feed_pipe_fn)(struct strbuf *pipe,
+			    void *pp_cb,
+			    void *pp_task_cb);
+
 /**
  * This callback is called on every child process that finished processing.
  *
@@ -456,10 +470,11 @@ typedef int (*task_finished_fn)(int result,
 int run_processes_parallel(int n,
 			   get_next_task_fn,
 			   start_failure_fn,
+			   feed_pipe_fn,
 			   task_finished_fn,
 			   void *pp_cb);
 int run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn,
-			       task_finished_fn, void *pp_cb,
+			       feed_pipe_fn, task_finished_fn, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label);
 
 /**
submodule.c +1 -0
index 8e611fe1dbf..db1700a502d 100644
@@ -1632,6 +1632,7 @@ int fetch_populated_submodules(struct repository *r,
 	run_processes_parallel_tr2(max_parallel_jobs,
 				   get_next_submodule,
 				   fetch_start_failure,
+				   NULL,
 				   fetch_finish,
 				   &spf,
 				   "submodule", "parallel/fetch");
t/helper/test-run-command.c +27 -4
index 7ae03dc7123..9348184d303 100644
@@ -32,8 +32,13 @@ static int parallel_next(struct child_process *cp,
 		return 0;
 
 	strvec_pushv(&cp->args, d->argv);
+	cp->in = d->in;
+	cp->no_stdin = d->no_stdin;
 	strbuf_addstr(err, "preloaded output of a child\n");
 	number_callbacks++;
+
+	*task_cb = xmalloc(sizeof(int));
+	*(int*)(*task_cb) = 2;
 	return 1;
 }
 
@@ -55,6 +60,17 @@ static int task_finished(int result,
 	return 1;
 }
 
+static int test_stdin(struct strbuf *pipe, void *cb, void *task_cb)
+{
+	int *lines_remaining = task_cb;
+
+	if (*lines_remaining)
+		strbuf_addf(pipe, "sample stdin %d\n", --(*lines_remaining));
+
+	return !(*lines_remaining);
+}
+
+
 struct testsuite {
 	struct string_list tests, failed;
 	int next;
@@ -185,7 +201,7 @@ static int testsuite(int argc, const char **argv)
 		suite.tests.nr, max_jobs);
 
 	ret = run_processes_parallel(max_jobs, next_test, test_failed,
-				     test_finished, &suite);
+				     test_stdin, test_finished, &suite);
 
 	if (suite.failed.nr > 0) {
 		ret = 1;
@@ -413,15 +429,22 @@ int cmd__run_command(int argc, const char **argv)
 
 	if (!strcmp(argv[1], "run-command-parallel"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, &proc));
+					    NULL, NULL, NULL, &proc));
 
 	if (!strcmp(argv[1], "run-command-abort"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, task_finished, &proc));
+					    NULL, NULL, task_finished, &proc));
 
 	if (!strcmp(argv[1], "run-command-no-jobs"))
 		exit(run_processes_parallel(jobs, no_job,
-					    NULL, task_finished, &proc));
+					    NULL, NULL, task_finished, &proc));
+
+	if (!strcmp(argv[1], "run-command-stdin")) {
+		proc.in = -1;
+		proc.no_stdin = 0;
+		exit (run_processes_parallel(jobs, parallel_next, NULL,
+					     test_stdin, NULL, &proc));
+	}
 
 	fprintf(stderr, "check usage\n");
 	return 1;
t/t0061-run-command.sh +30 -0
index 7d599675e35..87759482ad1 100755
@@ -143,6 +143,36 @@ test_expect_success 'run_command runs in parallel with more tasks than jobs avai
 	test_cmp expect actual
 '
 
+cat >expect <<-EOF
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+EOF
+
+test_expect_success 'run_command listens to stdin' '
+	write_script stdin-script <<-\EOF &&
+	echo "listening for stdin:"
+	while read line; do
+		echo "$line"
+	done
+	EOF
+	test-tool run-command run-command-stdin 2 ./stdin-script 2>actual &&
+	test_cmp expect actual
+'
+
 cat >expect <<-EOF
 preloaded output of a child
 asking for a quick stop
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
#532 lore raw

PATCH v4 25/36 hook: provide stdin by string_list or callback

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

In cases where a hook requires only a small amount of information via stdin, it should be simple for users to provide a string_list alone. But in more complicated cases where the stdin is too large to hold in memory, let's instead provide a callback the users can populate line after line.

---
 hook.c | 33 ++++++++++++++++++++++++++++++++-
 hook.h | 27 +++++++++++++++++++++++++++
 2 files changed, 59 insertions(+), 1 deletion(-)
hook.c +32 -1
index a7462639d97..44e73827800 100644
@@ -77,6 +77,29 @@ void run_hooks_opt_clear(struct run_hooks_opt *o)
 	strvec_clear(&o->args);
 }
 
+int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb)
+{
+	int *item_idx;
+	struct hook *ctx = pp_task_cb;
+	struct hook_cb_data *hook_cb = pp_cb;
+	struct string_list *to_pipe = hook_cb->options->feed_pipe_ctx;
+
+	/* Bootstrap the state manager if necessary. */
+	if (!ctx->feed_pipe_cb_data) {
+		ctx->feed_pipe_cb_data = xmalloc(sizeof(unsigned int));
+		*(int*)ctx->feed_pipe_cb_data = 0;
+	}
+
+	item_idx = ctx->feed_pipe_cb_data;
+
+	if (*item_idx < to_pipe->nr) {
+		strbuf_addf(pipe, "%s\n", to_pipe->items[*item_idx].string);
+		(*item_idx)++;
+		return 0;
+	}
+	return 1;
+}
+
 static int pick_next_hook(struct child_process *cp,
 			  struct strbuf *out,
 			  void *pp_cb,
@@ -92,6 +115,10 @@ static int pick_next_hook(struct child_process *cp,
 	if (hook_cb->options->path_to_stdin) {
 		cp->no_stdin = 0;
 		cp->in = xopen(hook_cb->options->path_to_stdin, O_RDONLY);
+	} else if (hook_cb->options->feed_pipe) {
+		/* ask for start_command() to make a pipe for us */
+		cp->in = -1;
+		cp->no_stdin = 0;
 	} else {
 		cp->no_stdin = 1;
 	}
@@ -175,7 +202,7 @@ int run_hooks(const char *hook_name, const char *hook_path,
 	run_processes_parallel_tr2(jobs,
 				   pick_next_hook,
 				   notify_start_failure,
-				   NULL,
+				   options->feed_pipe,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
@@ -183,6 +210,7 @@ int run_hooks(const char *hook_name, const char *hook_path,
 
 	if (options->absolute_path)
 		strbuf_release(&abs_path);
+	free(my_hook.feed_pipe_cb_data);
 
 	return cb_data.rc;
 }
@@ -196,6 +224,9 @@ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 	if (!options)
 		options = &hook_opt_scratch;
 
+	if (options->path_to_stdin && options->feed_pipe)
+		BUG("choose only one method to populate stdin");
+
 	hook_path = find_hook(hook_name);
 	if (!hook_path) {
 		ret = 0;
hook.h +27 -0
index cd6a68a3b50..b55f283f90b 100644
@@ -19,6 +19,12 @@ int hook_exists(const char *hookname);
 struct hook {
 	/* The path to the hook */
 	const char *hook_path;
+
+	/*
+	 * Use this to keep state for your feed_pipe_fn if you are using
+	 * run_hooks_opt.feed_pipe. Otherwise, do not touch it.
+	 */
+	void *feed_pipe_cb_data;
 };
 
 struct run_hooks_opt
@@ -39,6 +45,19 @@ struct run_hooks_opt
 
 	/* Path to file which should be piped to stdin for each hook */
 	const char *path_to_stdin;
+
+	/*
+	 * Callback and state pointer to ask for more content to pipe to stdin.
+	 * Will be called repeatedly, for each hook. See
+	 * hook.c:pipe_from_stdin() for an example. Keep per-hook state in
+	 * hook.feed_pipe_cb_data (per process). Keep initialization context in
+	 * feed_pipe_ctx (shared by all processes).
+	 *
+	 * See 'pipe_from_string_list()' for info about how to specify a
+	 * string_list as the stdin input instead of writing your own handler.
+	 */
+	feed_pipe_fn feed_pipe;
+	void *feed_pipe_ctx;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
@@ -46,6 +65,14 @@ struct run_hooks_opt
 	.args = STRVEC_INIT, \
 }
 
+/*
+ * To specify a 'struct string_list', set 'run_hooks_opt.feed_pipe_ctx' to the
+ * string_list and set 'run_hooks_opt.feed_pipe' to 'pipe_from_string_list()'.
+ * This will pipe each string in the list to stdin, separated by newlines.  (Do
+ * not inject your own newlines.)
+ */
+int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb);
+
 /*
  * Callback provided to feed_pipe_fn and consume_sideband_fn.
  */
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
#533 lore raw

PATCH v4 26/36 hook: convert 'post-rewrite' hook in sequencer.c to hook.h

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

By using 'hook.h' for 'post-rewrite', we simplify hook invocations by not needing to put together our own 'struct child_process'.

The signal handling that's being removed by this commit now takes place in run-command.h:run_processes_parallel(), so it is OK to remove them here.

---
 sequencer.c | 79 ++++++++++++++++++++++-------------------------------
 1 file changed, 32 insertions(+), 47 deletions(-)
sequencer.c +32 -47
index 9aac08c1545..77f809c00e4 100644
@@ -35,6 +35,7 @@
 #include "commit-reach.h"
 #include "rebase-interactive.h"
 #include "reset.h"
+#include "string-list.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -1147,33 +1148,27 @@ int update_head_with_reflog(const struct commit *old_head,
 static int run_rewrite_hook(const struct object_id *oldoid,
 			    const struct object_id *newoid)
 {
-	struct child_process proc = CHILD_PROCESS_INIT;
-	const char *argv[3];
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct strbuf tmp = STRBUF_INIT;
+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 	int code;
-	struct strbuf sb = STRBUF_INIT;
 
-	argv[0] = find_hook("post-rewrite");
-	if (!argv[0])
-		return 0;
+	strvec_push(&opt.args, "amend");
 
-	argv[1] = "amend";
-	argv[2] = NULL;
-
-	proc.argv = argv;
-	proc.in = -1;
-	proc.stdout_to_stderr = 1;
-	proc.trace2_hook_name = "post-rewrite";
-
-	code = start_command(&proc);
-	if (code)
-		return code;
-	strbuf_addf(&sb, "%s %s\n", oid_to_hex(oldoid), oid_to_hex(newoid));
-	sigchain_push(SIGPIPE, SIG_IGN);
-	write_in_full(proc.in, sb.buf, sb.len);
-	close(proc.in);
-	strbuf_release(&sb);
-	sigchain_pop(SIGPIPE);
-	return finish_command(&proc);
+	strbuf_addf(&tmp,
+		    "%s %s",
+		    oid_to_hex(oldoid),
+		    oid_to_hex(newoid));
+	string_list_append(&to_stdin, tmp.buf);
+
+	opt.feed_pipe = pipe_from_string_list;
+	opt.feed_pipe_ctx = &to_stdin;
+
+	code = run_hooks_oneshot("post-rewrite", &opt);
+
+	strbuf_release(&tmp);
+	string_list_clear(&to_stdin, 0);
+	return code;
 }
 
 void commit_post_rewrite(struct repository *r,
@@ -4526,30 +4521,20 @@ static int pick_commits(struct repository *r,
 		flush_rewritten_pending();
 		if (!stat(rebase_path_rewritten_list(), &st) &&
 				st.st_size > 0) {
-			struct child_process child = CHILD_PROCESS_INIT;
-			const char *post_rewrite_hook =
-				find_hook("post-rewrite");
-
-			child.in = open(rebase_path_rewritten_list(), O_RDONLY);
-			child.git_cmd = 1;
-			strvec_push(&child.args, "notes");
-			strvec_push(&child.args, "copy");
-			strvec_push(&child.args, "--for-rewrite=rebase");
+			struct child_process notes_cp = CHILD_PROCESS_INIT;
+			struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+
+			notes_cp.in = open(rebase_path_rewritten_list(), O_RDONLY);
+			notes_cp.git_cmd = 1;
+			strvec_push(&notes_cp.args, "notes");
+			strvec_push(&notes_cp.args, "copy");
+			strvec_push(&notes_cp.args, "--for-rewrite=rebase");
 			/* we don't care if this copying failed */
-			run_command(&child);
-
-			if (post_rewrite_hook) {
-				struct child_process hook = CHILD_PROCESS_INIT;
-
-				hook.in = open(rebase_path_rewritten_list(),
-					O_RDONLY);
-				hook.stdout_to_stderr = 1;
-				hook.trace2_hook_name = "post-rewrite";
-				strvec_push(&hook.args, post_rewrite_hook);
-				strvec_push(&hook.args, "rebase");
-				/* we don't care if this hook failed */
-				run_command(&hook);
-			}
+			run_command(&notes_cp);
+
+			hook_opt.path_to_stdin = rebase_path_rewritten_list();
+			strvec_push(&hook_opt.args, "rebase");
+			run_hooks_oneshot("post-rewrite", &hook_opt);
 		}
 		apply_autostash(rebase_path_autostash());
 
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
#534 lore raw

PATCH v4 27/36 transport: convert pre-push hook to hook.h

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Move the pre-push hook away from run-command.h to and over to the new hook.h library.

---
 transport.c | 56 ++++++++++++++---------------------------------------
 1 file changed, 14 insertions(+), 42 deletions(-)
transport.c +14 -42
index 77e196f75f5..4ca8fc0391d 100644
@@ -1203,63 +1203,35 @@ static void die_with_unpushed_submodules(struct string_list *needs_pushing)
 static int run_pre_push_hook(struct transport *transport,
 			     struct ref *remote_refs)
 {
-	int ret = 0, x;
+	int ret = 0;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	struct ref *r;
-	struct child_process proc = CHILD_PROCESS_INIT;
-	struct strbuf buf;
-	const char *argv[4];
-
-	if (!(argv[0] = find_hook("pre-push")))
-		return 0;
-
-	argv[1] = transport->remote->name;
-	argv[2] = transport->url;
-	argv[3] = NULL;
-
-	proc.argv = argv;
-	proc.in = -1;
-	proc.trace2_hook_name = "pre-push";
-
-	if (start_command(&proc)) {
-		finish_command(&proc);
-		return -1;
-	}
+	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
 
-	sigchain_push(SIGPIPE, SIG_IGN);
-
-	strbuf_init(&buf, 256);
+	strvec_push(&opt.args, transport->remote->name);
+	strvec_push(&opt.args, transport->url);
 
 	for (r = remote_refs; r; r = r->next) {
+		struct strbuf buf = STRBUF_INIT;
+
 		if (!r->peer_ref) continue;
 		if (r->status == REF_STATUS_REJECT_NONFASTFORWARD) continue;
 		if (r->status == REF_STATUS_REJECT_STALE) continue;
 		if (r->status == REF_STATUS_REJECT_REMOTE_UPDATED) continue;
 		if (r->status == REF_STATUS_UPTODATE) continue;
 
-		strbuf_reset(&buf);
-		strbuf_addf( &buf, "%s %s %s %s\n",
+		strbuf_addf(&buf, "%s %s %s %s",
 			 r->peer_ref->name, oid_to_hex(&r->new_oid),
 			 r->name, oid_to_hex(&r->old_oid));
-
-		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
-			/* We do not mind if a hook does not read all refs. */
-			if (errno != EPIPE)
-				ret = -1;
-			break;
-		}
+		string_list_append(&to_stdin, strbuf_detach(&buf, NULL));
 	}
 
-	strbuf_release(&buf);
-
-	x = close(proc.in);
-	if (!ret)
-		ret = x;
-
-	sigchain_pop(SIGPIPE);
+	opt.feed_pipe = pipe_from_string_list;
+	opt.feed_pipe_ctx = &to_stdin;
 
-	x = finish_command(&proc);
-	if (!ret)
-		ret = x;
+	ret = run_hooks_oneshot("pre-push", &opt);
+	to_stdin.strdup_strings = 1;
+	string_list_clear(&to_stdin, 0);
 
 	return ret;
 }
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
#535 lore raw

PATCH v4 28/36 hook tests: test for exact "pre-push" hook input

In reply to Ævar Arnfjörð Bjarmason

Extend the tests added in ec55559f937 (push: Add support for pre-push hooks, 2013-01-13) to exhaustively test for the exact input we're expecting. This helps a parallel series that's refactoring how the hook is called, to e.g. make sure that we don't miss a trailing newline.

---
 t/t5571-pre-push-hook.sh | 23 ++++++++++++++++++-----
 1 file changed, 18 insertions(+), 5 deletions(-)
t/t5571-pre-push-hook.sh +18 -5
index ad8d5804f7b..d2857a6fbc0 100755
@@ -11,7 +11,7 @@ HOOKDIR="$(git rev-parse --git-dir)/hooks"
 HOOK="$HOOKDIR/pre-push"
 mkdir -p "$HOOKDIR"
 write_script "$HOOK" <<EOF
-cat >/dev/null
+cat >actual
 exit 0
 EOF
 
@@ -20,10 +20,16 @@ test_expect_success 'setup' '
 	git init --bare repo1 &&
 	git remote add parent1 repo1 &&
 	test_commit one &&
-	git push parent1 HEAD:foreign
+	cat >expect <<-EOF &&
+	HEAD $(git rev-parse HEAD) refs/heads/foreign $(test_oid zero)
+	EOF
+
+	test_when_finished "rm actual" &&
+	git push parent1 HEAD:foreign &&
+	test_cmp expect actual
 '
 write_script "$HOOK" <<EOF
-cat >/dev/null
+cat >actual
 exit 1
 EOF
 
@@ -32,11 +38,18 @@ export COMMIT1
 
 test_expect_success 'push with failing hook' '
 	test_commit two &&
-	test_must_fail git push parent1 HEAD
+	cat >expect <<-EOF &&
+	HEAD $(git rev-parse HEAD) refs/heads/main $(test_oid zero)
+	EOF
+
+	test_when_finished "rm actual" &&
+	test_must_fail git push parent1 HEAD &&
+	test_cmp expect actual
 '
 
 test_expect_success '--no-verify bypasses hook' '
-	git push --no-verify parent1 HEAD
+	git push --no-verify parent1 HEAD &&
+	test_path_is_missing actual
 '
 
 COMMIT2="$(git rev-parse HEAD)"
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
author Reviewed-by on v4 28/36
#536 lore raw

Re: [PATCH v4 28/36] hook tests: test for exact "pre-push" hook input

Quoted 6 lines · Ævar Arnfjörð Bjarmason

Extend the tests added in ec55559f937 (push: Add support for pre-push hooks, 2013-01-13) to exhaustively test for the exact input we're expecting. This helps a parallel series that's refactoring how the hook is called, to e.g. make sure that we don't miss a trailing newline

The reference to "a parallel series" I think didn't belong in the commit-msg in the first place, and doesn't make sense now (because this is in said series, right?).

Quoted 25 lines
---
 t/t5571-pre-push-hook.sh | 23 ++++++++++++++++++-----
 1 file changed, 18 insertions(+), 5 deletions(-)
t/t5571-pre-push-hook.sh +2 -2
index ad8d5804f7b..d2857a6fbc0 100755
@@ -11,7 +11,7 @@ HOOKDIR="$(git rev-parse --git-dir)/hooks"
 HOOK="$HOOKDIR/pre-push"
 mkdir -p "$HOOKDIR"
 write_script "$HOOK" <<EOF
-cat >/dev/null
+cat >actual
 exit 0
 EOF
 
@@ -20,10 +20,16 @@ test_expect_success 'setup' '
 	git init --bare repo1 &&
 	git remote add parent1 repo1 &&
 	test_commit one &&
-	git push parent1 HEAD:foreign
+	cat >expect <<-EOF &&

t/t5571-pre-push-hook.sh test_expect_success 'setup' ' · line 32

Quoted 10 lines
t/t5571-pre-push-hook.sh +7 -1
+	HEAD $(git rev-parse HEAD) refs/heads/foreign $(test_oid zero)
+	EOF
+
+	test_when_finished "rm actual" &&
+	git push parent1 HEAD:foreign &&
+	test_cmp expect actual
 '
 write_script "$HOOK" <<EOF
-cat >/dev/null
+cat >actual

Hm, I am not sure I like this. It upsets the usual convention of what 'actual' means ("output I captured from a 'git' command"). It is nitpicky, but I would be happier to see it cat to some other filename, like 'received-input'.

Quoted 11 lines
t/t5571-pre-push-hook.sh +3 -1
 exit 1
 EOF
 
@@ -32,11 +38,18 @@ export COMMIT1
 
 test_expect_success 'push with failing hook' '
 	test_commit two &&
-	test_must_fail git push parent1 HEAD
+	cat >expect <<-EOF &&
+	HEAD $(git rev-parse HEAD) refs/heads/main $(test_oid zero)
+	EOF

t/t5571-pre-push-hook.sh export COMMIT1 · line 52

Quoted 10 lines
t/t5571-pre-push-hook.sh +6 -1
+
+	test_when_finished "rm actual" &&
+	test_must_fail git push parent1 HEAD &&
+	test_cmp expect actual
 '
 
 test_expect_success '--no-verify bypasses hook' '
-	git push --no-verify parent1 HEAD
+	git push --no-verify parent1 HEAD &&
+	test_path_is_missing actual

Other than the naming nit, though, the test changes look reasonable to me to ensure we don't goof up the stdin support in hook.c. Thanks.

With (or, I guess, without) changes,

Quoted 6 lines
t/t5571-pre-push-hook.sh +0 -0
 '
 
 COMMIT2="$(git rev-parse HEAD)"
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
#537 lore raw

PATCH v4 29/36 hook tests: use a modern style for "pre-push" tests

In reply to Ævar Arnfjörð Bjarmason

Indent the here-docs and use "test_cmp" instead of "diff" in tests added in ec55559f937 (push: Add support for pre-push hooks, 2013-01-13). Let's also use the more typical "expect" instead of "expected" to be consistent with the rest of the test file.

---
 t/t5571-pre-push-hook.sh | 71 ++++++++++++++++++++--------------------
 1 file changed, 35 insertions(+), 36 deletions(-)
t/t5571-pre-push-hook.sh +35 -36
index d2857a6fbc0..6d0d5b854ea 100755
@@ -61,15 +61,15 @@ echo "$2" >>actual
 cat >>actual
 EOF
 
-cat >expected <<EOF
-parent1
-repo1
-refs/heads/main $COMMIT2 refs/heads/foreign $COMMIT1
-EOF
-
 test_expect_success 'push with hook' '
+	cat >expected <<-EOF &&
+	parent1
+	repo1
+	refs/heads/main $COMMIT2 refs/heads/foreign $COMMIT1
+	EOF
+
 	git push parent1 main:foreign &&
-	diff expected actual
+	test_cmp expected actual
 '
 
 test_expect_success 'add a branch' '
@@ -80,49 +80,48 @@ test_expect_success 'add a branch' '
 COMMIT3="$(git rev-parse HEAD)"
 export COMMIT3
 
-cat >expected <<EOF
-parent1
-repo1
-refs/heads/other $COMMIT3 refs/heads/foreign $COMMIT2
-EOF
-
 test_expect_success 'push to default' '
+	cat >expect <<-EOF &&
+	parent1
+	repo1
+	refs/heads/other $COMMIT3 refs/heads/foreign $COMMIT2
+	EOF
 	git push &&
-	diff expected actual
+	test_cmp expect actual
 '
 
-cat >expected <<EOF
-parent1
-repo1
-refs/tags/one $COMMIT1 refs/tags/tag1 $ZERO_OID
-HEAD~ $COMMIT2 refs/heads/prev $ZERO_OID
-EOF
-
 test_expect_success 'push non-branches' '
+	cat >expect <<-EOF &&
+	parent1
+	repo1
+	refs/tags/one $COMMIT1 refs/tags/tag1 $ZERO_OID
+	HEAD~ $COMMIT2 refs/heads/prev $ZERO_OID
+	EOF
+
 	git push parent1 one:tag1 HEAD~:refs/heads/prev &&
-	diff expected actual
+	test_cmp expect actual
 '
 
-cat >expected <<EOF
-parent1
-repo1
-(delete) $ZERO_OID refs/heads/prev $COMMIT2
-EOF
-
 test_expect_success 'push delete' '
+	cat >expect <<-EOF &&
+	parent1
+	repo1
+	(delete) $ZERO_OID refs/heads/prev $COMMIT2
+	EOF
+
 	git push parent1 :prev &&
-	diff expected actual
+	test_cmp expect actual
 '
 
-cat >expected <<EOF
-repo1
-repo1
-HEAD $COMMIT3 refs/heads/other $ZERO_OID
-EOF
-
 test_expect_success 'push to URL' '
+	cat >expect <<-EOF &&
+	repo1
+	repo1
+	HEAD $COMMIT3 refs/heads/other $ZERO_OID
+	EOF
+
 	git push repo1 HEAD &&
-	diff expected actual
+	test_cmp expect actual
 '
 
 test_expect_success 'set up many-ref tests' '
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
author Reviewed-by on v4 29/36
#538 lore raw

Re: [PATCH v4 29/36] hook tests: use a modern style for "pre-push" tests

Quoted 25 lines · Ævar Arnfjörð Bjarmason

Indent the here-docs and use "test_cmp" instead of "diff" in tests added in ec55559f937 (push: Add support for pre-push hooks, 2013-01-13). Let's also use the more typical "expect" instead of "expected" to be consistent with the rest of the test file.

---
 t/t5571-pre-push-hook.sh | 71 ++++++++++++++++++++--------------------
 1 file changed, 35 insertions(+), 36 deletions(-)
t/t5571-pre-push-hook.sh +0 -6
index d2857a6fbc0..6d0d5b854ea 100755
@@ -61,15 +61,15 @@ echo "$2" >>actual
 cat >>actual
 EOF
 
-cat >expected <<EOF
-parent1
-repo1
-refs/heads/main $COMMIT2 refs/heads/foreign $COMMIT1
-EOF
-

t/t5571-pre-push-hook.sh echo "$2" >>actual · line 72

Quoted 10 lines
t/t5571-pre-push-hook.sh +7 -1
 test_expect_success 'push with hook' '
+	cat >expected <<-EOF &&
+	parent1
+	repo1
+	refs/heads/main $COMMIT2 refs/heads/foreign $COMMIT1
+	EOF
+
 	git push parent1 main:foreign &&
-	diff expected actual
+	test_cmp expected actual

Your commit message tells me you're getting rid of 'expected', but here it is. Should that just be 'expect'?

Otherwise it looks fine.

#539 lore raw

PATCH v4 30/36 reference-transaction: use hook.h to run hooks

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

---
 refs.c | 40 +++++++++++++---------------------------
 1 file changed, 13 insertions(+), 27 deletions(-)
refs.c +13 -27
index 6211692eaae..73d4a939267 100644
@@ -2062,47 +2062,33 @@ int ref_update_reject_duplicates(struct string_list *refnames,
 static int run_transaction_hook(struct ref_transaction *transaction,
 				const char *state)
 {
-	struct child_process proc = CHILD_PROCESS_INIT;
-	struct strbuf buf = STRBUF_INIT;
-	const char *hook;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
 	int ret = 0, i;
 
-	hook = find_hook("reference-transaction");
-	if (!hook)
-		return ret;
-
-	strvec_pushl(&proc.args, hook, state, NULL);
-	proc.in = -1;
-	proc.stdout_to_stderr = 1;
-	proc.trace2_hook_name = "reference-transaction";
-
-	ret = start_command(&proc);
-	if (ret)
+	if (!hook_exists("reference-transaction"))
 		return ret;
 
-	sigchain_push(SIGPIPE, SIG_IGN);
+	strvec_push(&opt.args, state);
 
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
+		struct strbuf buf = STRBUF_INIT;
 
-		strbuf_reset(&buf);
-		strbuf_addf(&buf, "%s %s %s\n",
+		strbuf_addf(&buf, "%s %s %s",
 			    oid_to_hex(&update->old_oid),
 			    oid_to_hex(&update->new_oid),
 			    update->refname);
-
-		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
-			if (errno != EPIPE)
-				ret = -1;
-			break;
-		}
+		string_list_append(&to_stdin, strbuf_detach(&buf, NULL));
 	}
 
-	close(proc.in);
-	sigchain_pop(SIGPIPE);
-	strbuf_release(&buf);
+	opt.feed_pipe = pipe_from_string_list;
+	opt.feed_pipe_ctx = &to_stdin;
+
+	ret = run_hooks_oneshot("reference-transaction", &opt);
+	to_stdin.strdup_strings = 1;
+	string_list_clear(&to_stdin, 0);
 
-	ret |= finish_command(&proc);
 	return ret;
 }
 
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
#540 lore raw

PATCH v4 31/36 run-command: allow capturing of collated output

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Some callers, for example server-side hooks which wish to relay hook output to clients across a transport, want to capture what would normally print to stderr and do something else with it. Allow that via a callback.

By calling the callback regardless of whether there's output available, we allow clients to send e.g. a keepalive if necessary.

Because we expose a strbuf, not a fd or FILE*, there's no need to create a temporary pipe or similar - we can just skip the print to stderr and instead hand it to the caller.

---
 builtin/fetch.c             |  2 +-
 builtin/submodule--helper.c |  2 +-
 hook.c                      |  1 +
 run-command.c               | 33 +++++++++++++++++++++++++--------
 run-command.h               | 18 +++++++++++++++++-
 submodule.c                 |  2 +-
 t/helper/test-run-command.c | 25 ++++++++++++++++++++-----
 t/t0061-run-command.sh      |  7 +++++++
 8 files changed, 73 insertions(+), 17 deletions(-)
builtin/fetch.c +1 -1
index fef6e85d003..de14df5085c 100644
@@ -1817,7 +1817,7 @@ static int fetch_multiple(struct string_list *list, int max_children)
 		result = run_processes_parallel_tr2(max_children,
 						    &fetch_next_remote,
 						    &fetch_failed_to_start,
-						    NULL,
+						    NULL, NULL,
 						    &fetch_finished,
 						    &state,
 						    "fetch", "parallel/fetch");
builtin/submodule--helper.c +1 -1
index f42ded548bf..e5e5a8bcfb5 100644
@@ -2294,7 +2294,7 @@ static int update_submodules(struct submodule_update_clone *suc)
 	int i;
 
 	run_processes_parallel_tr2(suc->max_jobs, update_clone_get_next_task,
-				   update_clone_start_failure, NULL,
+				   update_clone_start_failure, NULL, NULL,
 				   update_clone_task_finished, suc, "submodule",
 				   "parallel/update");
 
hook.c +1 -0
index 44e73827800..9ae3007cdd3 100644
@@ -203,6 +203,7 @@ int run_hooks(const char *hook_name, const char *hook_path,
 				   pick_next_hook,
 				   notify_start_failure,
 				   options->feed_pipe,
+				   NULL,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
run-command.c +25 -8
index f1616858d18..aacc336f951 100644
@@ -1493,6 +1493,7 @@ struct parallel_processes {
 	get_next_task_fn get_next_task;
 	start_failure_fn start_failure;
 	feed_pipe_fn feed_pipe;
+	consume_sideband_fn consume_sideband;
 	task_finished_fn task_finished;
 
 	struct {
@@ -1558,6 +1559,7 @@ static void pp_init(struct parallel_processes *pp,
 		    get_next_task_fn get_next_task,
 		    start_failure_fn start_failure,
 		    feed_pipe_fn feed_pipe,
+		    consume_sideband_fn consume_sideband,
 		    task_finished_fn task_finished,
 		    void *data)
 {
@@ -1578,6 +1580,7 @@ static void pp_init(struct parallel_processes *pp,
 	pp->start_failure = start_failure ? start_failure : default_start_failure;
 	pp->feed_pipe = feed_pipe ? feed_pipe : default_feed_pipe;
 	pp->task_finished = task_finished ? task_finished : default_task_finished;
+	pp->consume_sideband = consume_sideband;
 
 	pp->nr_processes = 0;
 	pp->output_owner = 0;
@@ -1614,7 +1617,10 @@ static void pp_cleanup(struct parallel_processes *pp)
 	 * When get_next_task added messages to the buffer in its last
 	 * iteration, the buffered output is non empty.
 	 */
-	strbuf_write(&pp->buffered_output, stderr);
+	if (pp->consume_sideband)
+		pp->consume_sideband(&pp->buffered_output, pp->data);
+	else
+		strbuf_write(&pp->buffered_output, stderr);
 	strbuf_release(&pp->buffered_output);
 
 	sigchain_pop_common();
@@ -1735,9 +1741,13 @@ static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout)
 static void pp_output(struct parallel_processes *pp)
 {
 	int i = pp->output_owner;
+
 	if (pp->children[i].state == GIT_CP_WORKING &&
 	    pp->children[i].err.len) {
-		strbuf_write(&pp->children[i].err, stderr);
+		if (pp->consume_sideband)
+			pp->consume_sideband(&pp->children[i].err, pp->data);
+		else
+			strbuf_write(&pp->children[i].err, stderr);
 		strbuf_reset(&pp->children[i].err);
 	}
 }
@@ -1776,11 +1786,15 @@ static int pp_collect_finished(struct parallel_processes *pp)
 			strbuf_addbuf(&pp->buffered_output, &pp->children[i].err);
 			strbuf_reset(&pp->children[i].err);
 		} else {
-			strbuf_write(&pp->children[i].err, stderr);
+			/* Output errors, then all other finished child processes */
+			if (pp->consume_sideband) {
+				pp->consume_sideband(&pp->children[i].err, pp->data);
+				pp->consume_sideband(&pp->buffered_output, pp->data);
+			} else {
+				strbuf_write(&pp->children[i].err, stderr);
+				strbuf_write(&pp->buffered_output, stderr);
+			}
 			strbuf_reset(&pp->children[i].err);
-
-			/* Output all other finished child processes */
-			strbuf_write(&pp->buffered_output, stderr);
 			strbuf_reset(&pp->buffered_output);
 
 			/*
@@ -1804,6 +1818,7 @@ int run_processes_parallel(int n,
 			   get_next_task_fn get_next_task,
 			   start_failure_fn start_failure,
 			   feed_pipe_fn feed_pipe,
+			   consume_sideband_fn consume_sideband,
 			   task_finished_fn task_finished,
 			   void *pp_cb)
 {
@@ -1814,7 +1829,7 @@ int run_processes_parallel(int n,
 
 	sigchain_push(SIGPIPE, SIG_IGN);
 
-	pp_init(&pp, n, get_next_task, start_failure, feed_pipe, task_finished, pp_cb);
+	pp_init(&pp, n, get_next_task, start_failure, feed_pipe, consume_sideband, task_finished, pp_cb);
 	while (1) {
 		for (i = 0;
 		    i < spawn_cap && !pp.shutdown &&
@@ -1852,6 +1867,7 @@ int run_processes_parallel(int n,
 int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 			       start_failure_fn start_failure,
 			       feed_pipe_fn feed_pipe,
+			       consume_sideband_fn consume_sideband,
 			       task_finished_fn task_finished, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label)
 {
@@ -1861,7 +1877,8 @@ int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 				   ((n < 1) ? online_cpus() : n));
 
 	result = run_processes_parallel(n, get_next_task, start_failure,
-					feed_pipe, task_finished, pp_cb);
+					feed_pipe, consume_sideband,
+					task_finished, pp_cb);
 
 	trace2_region_leave(tr2_category, tr2_label, NULL);
 
run-command.h +17 -1
index 80d394664ae..e321d23bbd2 100644
@@ -436,6 +436,20 @@ typedef int (*feed_pipe_fn)(struct strbuf *pipe,
 			    void *pp_cb,
 			    void *pp_task_cb);
 
+/**
+ * If this callback is provided, instead of collating process output to stderr,
+ * they will be collated into a new pipe. consume_sideband_fn will be called
+ * repeatedly. When output is available on that pipe, it will be contained in
+ * 'output'. But it will be called with an empty 'output' too, to allow for
+ * keepalives or similar operations if necessary.
+ *
+ * pp_cb is the callback cookie as passed into run_processes_parallel.
+ *
+ * Since this callback is provided with the collated output, no task cookie is
+ * provided.
+ */
+typedef void (*consume_sideband_fn)(struct strbuf *output, void *pp_cb);
+
 /**
  * This callback is called on every child process that finished processing.
  *
@@ -471,10 +485,12 @@ int run_processes_parallel(int n,
 			   get_next_task_fn,
 			   start_failure_fn,
 			   feed_pipe_fn,
+			   consume_sideband_fn,
 			   task_finished_fn,
 			   void *pp_cb);
 int run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn,
-			       feed_pipe_fn, task_finished_fn, void *pp_cb,
+			       feed_pipe_fn, consume_sideband_fn,
+			       task_finished_fn, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label);
 
 /**
submodule.c +1 -1
index db1700a502d..32364d8bd56 100644
@@ -1632,7 +1632,7 @@ int fetch_populated_submodules(struct repository *r,
 	run_processes_parallel_tr2(max_parallel_jobs,
 				   get_next_submodule,
 				   fetch_start_failure,
-				   NULL,
+				   NULL, NULL,
 				   fetch_finish,
 				   &spf,
 				   "submodule", "parallel/fetch");
t/helper/test-run-command.c +20 -5
index 9348184d303..d53db6d11c4 100644
@@ -51,6 +51,16 @@ static int no_job(struct child_process *cp,
 	return 0;
 }
 
+static void test_consume_sideband(struct strbuf *output, void *cb)
+{
+	FILE *sideband;
+
+	sideband = fopen("./sideband", "a");
+
+	strbuf_write(output, sideband);
+	fclose(sideband);
+}
+
 static int task_finished(int result,
 			 struct strbuf *err,
 			 void *pp_cb,
@@ -201,7 +211,7 @@ static int testsuite(int argc, const char **argv)
 		suite.tests.nr, max_jobs);
 
 	ret = run_processes_parallel(max_jobs, next_test, test_failed,
-				     test_stdin, test_finished, &suite);
+				     test_stdin, NULL, test_finished, &suite);
 
 	if (suite.failed.nr > 0) {
 		ret = 1;
@@ -429,23 +439,28 @@ int cmd__run_command(int argc, const char **argv)
 
 	if (!strcmp(argv[1], "run-command-parallel"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, NULL, &proc));
+					    NULL, NULL, NULL, NULL, &proc));
 
 	if (!strcmp(argv[1], "run-command-abort"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, task_finished, &proc));
+					    NULL, NULL, NULL, task_finished, &proc));
 
 	if (!strcmp(argv[1], "run-command-no-jobs"))
 		exit(run_processes_parallel(jobs, no_job,
-					    NULL, NULL, task_finished, &proc));
+					    NULL, NULL, NULL, task_finished, &proc));
 
 	if (!strcmp(argv[1], "run-command-stdin")) {
 		proc.in = -1;
 		proc.no_stdin = 0;
 		exit (run_processes_parallel(jobs, parallel_next, NULL,
-					     test_stdin, NULL, &proc));
+					     test_stdin, NULL, NULL, &proc));
 	}
 
+	if (!strcmp(argv[1], "run-command-sideband"))
+		exit(run_processes_parallel(jobs, parallel_next, NULL, NULL,
+					    test_consume_sideband, NULL,
+					    &proc));
+
 	fprintf(stderr, "check usage\n");
 	return 1;
 }
t/t0061-run-command.sh +7 -0
index 87759482ad1..e99f6c7f445 100755
@@ -143,6 +143,13 @@ test_expect_success 'run_command runs in parallel with more tasks than jobs avai
 	test_cmp expect actual
 '
 
+test_expect_success 'run_command can divert output' '
+	test_when_finished rm sideband &&
+	test-tool run-command run-command-sideband 3 sh -c "printf \"%s\n%s\n\" Hello World" 2>actual &&
+	test_must_be_empty actual &&
+	test_cmp expect sideband
+'
+
 cat >expect <<-EOF
 preloaded output of a child
 listening for stdin:
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
#541 lore raw

PATCH v4 32/36 hooks: allow callers to capture output

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Some server-side hooks will require capturing output to send over sideband instead of printing directly to stderr. Expose that capability.

---
 hook.c | 3 ++-
 hook.h | 8 ++++++++
 2 files changed, 10 insertions(+), 1 deletion(-)
hook.c +2 -1
index 9ae3007cdd3..efed5f73b4d 100644
@@ -203,12 +203,13 @@ int run_hooks(const char *hook_name, const char *hook_path,
 				   pick_next_hook,
 				   notify_start_failure,
 				   options->feed_pipe,
-				   NULL,
+				   options->consume_sideband,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
 				   hook_name);
 
+
 	if (options->absolute_path)
 		strbuf_release(&abs_path);
 	free(my_hook.feed_pipe_cb_data);
hook.h +8 -0
index b55f283f90b..37a9690c2ca 100644
@@ -58,6 +58,14 @@ struct run_hooks_opt
 	 */
 	feed_pipe_fn feed_pipe;
 	void *feed_pipe_ctx;
+
+	/*
+	 * Populate this to capture output and prevent it from being printed to
+	 * stderr. This will be passed directly through to
+	 * run_command:run_parallel_processes(). See t/helper/test-run-command.c
+	 * for an example.
+	 */
+	consume_sideband_fn consume_sideband;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
#542 lore raw

PATCH v4 33/36 receive-pack: convert 'update' hook to hook.h

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

This makes use of the new sideband API in hook.h added in the preceding commit.

---
 builtin/receive-pack.c | 62 ++++++++++++++++++++++++++++--------------
 1 file changed, 41 insertions(+), 21 deletions(-)
builtin/receive-pack.c +41 -21
index a7d03bbc7d3..31ce4ece4e7 100644
@@ -937,33 +937,53 @@ static int run_receive_hook(struct command *commands,
 	return status;
 }
 
-static int run_update_hook(struct command *cmd)
+static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
 {
-	const char *argv[5];
-	struct child_process proc = CHILD_PROCESS_INIT;
-	int code;
+	int keepalive_active = 0;
 
-	argv[0] = find_hook("update");
-	if (!argv[0])
-		return 0;
+	if (keepalive_in_sec <= 0)
+		use_keepalive = KEEPALIVE_NEVER;
+	if (use_keepalive == KEEPALIVE_ALWAYS)
+		keepalive_active = 1;
 
-	argv[1] = cmd->ref_name;
-	argv[2] = oid_to_hex(&cmd->old_oid);
-	argv[3] = oid_to_hex(&cmd->new_oid);
-	argv[4] = NULL;
+	/* send a keepalive if there is no data to write */
+	if (keepalive_active && !output->len) {
+		static const char buf[] = "0005\1";
+		write_or_die(1, buf, sizeof(buf) - 1);
+		return;
+	}
 
-	proc.no_stdin = 1;
-	proc.stdout_to_stderr = 1;
-	proc.err = use_sideband ? -1 : 0;
-	proc.argv = argv;
-	proc.trace2_hook_name = "update";
+	if (use_keepalive == KEEPALIVE_AFTER_NUL && !keepalive_active) {
+		const char *first_null = memchr(output->buf, '\0', output->len);
+		if (first_null) {
+			/* The null bit is excluded. */
+			size_t before_null = first_null - output->buf;
+			size_t after_null = output->len - (before_null + 1);
+			keepalive_active = 1;
+			send_sideband(1, 2, output->buf, before_null, use_sideband);
+			send_sideband(1, 2, first_null + 1, after_null, use_sideband);
+
+			return;
+		}
+	}
+
+	send_sideband(1, 2, output->buf, output->len, use_sideband);
+}
+
+static int run_update_hook(struct command *cmd)
+{
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+
+	strvec_pushl(&opt.args,
+		     cmd->ref_name,
+		     oid_to_hex(&cmd->old_oid),
+		     oid_to_hex(&cmd->new_oid),
+		     NULL);
 
-	code = start_command(&proc);
-	if (code)
-		return code;
 	if (use_sideband)
-		copy_to_sideband(proc.err, -1, NULL);
-	return finish_command(&proc);
+		opt.consume_sideband = hook_output_to_sideband;
+
+	return run_hooks_oneshot("update", &opt);
 }
 
 static struct command *find_command_by_refname(struct command *list,
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
#543 lore raw

PATCH v4 34/36 post-update: use hook.h library

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

---
 builtin/receive-pack.c | 25 ++++++-------------------
 1 file changed, 6 insertions(+), 19 deletions(-)
builtin/receive-pack.c +6 -19
index 31ce4ece4e7..26e302aab85 100644
@@ -1650,33 +1650,20 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 static void run_update_post_hook(struct command *commands)
 {
 	struct command *cmd;
-	struct child_process proc = CHILD_PROCESS_INIT;
-	const char *hook;
-
-	hook = find_hook("post-update");
-	if (!hook)
-		return;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 
 	for (cmd = commands; cmd; cmd = cmd->next) {
 		if (cmd->error_string || cmd->did_not_exist)
 			continue;
-		if (!proc.args.nr)
-			strvec_push(&proc.args, hook);
-		strvec_push(&proc.args, cmd->ref_name);
+		strvec_push(&opt.args, cmd->ref_name);
 	}
-	if (!proc.args.nr)
+	if (!opt.args.nr)
 		return;
 
-	proc.no_stdin = 1;
-	proc.stdout_to_stderr = 1;
-	proc.err = use_sideband ? -1 : 0;
-	proc.trace2_hook_name = "post-update";
+	if (use_sideband)
+		opt.consume_sideband = hook_output_to_sideband;
 
-	if (!start_command(&proc)) {
-		if (use_sideband)
-			copy_to_sideband(proc.err, -1, NULL);
-		finish_command(&proc);
-	}
+	run_hooks_oneshot("post-update", &opt);
 }
 
 static void check_aliased_update_internal(struct command *cmd,
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
#544 lore raw

PATCH v4 35/36 receive-pack: convert receive hooks to hook.h

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

---
 builtin/receive-pack.c | 190 ++++++++++++++++++-----------------------
 1 file changed, 83 insertions(+), 107 deletions(-)
builtin/receive-pack.c +83 -107
index 26e302aab85..c3984680d7f 100644
@@ -747,7 +747,7 @@ static int check_cert_push_options(const struct string_list *push_options)
 	return retval;
 }
 
-static void prepare_push_cert_sha1(struct child_process *proc)
+static void prepare_push_cert_sha1(struct run_hooks_opt *opt)
 {
 	static int already_done;
 
@@ -771,110 +771,42 @@ static void prepare_push_cert_sha1(struct child_process *proc)
 		nonce_status = check_nonce(push_cert.buf, bogs);
 	}
 	if (!is_null_oid(&push_cert_oid)) {
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT=%s",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT=%s",
 			     oid_to_hex(&push_cert_oid));
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_SIGNER=%s",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT_SIGNER=%s",
 			     sigcheck.signer ? sigcheck.signer : "");
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_KEY=%s",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT_KEY=%s",
 			     sigcheck.key ? sigcheck.key : "");
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_STATUS=%c",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT_STATUS=%c",
 			     sigcheck.result);
 		if (push_cert_nonce) {
-			strvec_pushf(&proc->env_array,
+			strvec_pushf(&opt->env,
 				     "GIT_PUSH_CERT_NONCE=%s",
 				     push_cert_nonce);
-			strvec_pushf(&proc->env_array,
+			strvec_pushf(&opt->env,
 				     "GIT_PUSH_CERT_NONCE_STATUS=%s",
 				     nonce_status);
 			if (nonce_status == NONCE_SLOP)
-				strvec_pushf(&proc->env_array,
+				strvec_pushf(&opt->env,
 					     "GIT_PUSH_CERT_NONCE_SLOP=%ld",
 					     nonce_stamp_slop);
 		}
 	}
 }
 
+struct receive_hook_feed_context {
+	struct command *cmd;
+	int skip_broken;
+};
+
 struct receive_hook_feed_state {
 	struct command *cmd;
 	struct ref_push_report *report;
 	int skip_broken;
 	struct strbuf buf;
-	const struct string_list *push_options;
 };
 
-typedef int (*feed_fn)(void *, const char **, size_t *);
-static int run_and_feed_hook(const char *hook_name, feed_fn feed,
-			     struct receive_hook_feed_state *feed_state)
-{
-	struct child_process proc = CHILD_PROCESS_INIT;
-	struct async muxer;
-	const char *argv[2];
-	int code;
-
-	argv[0] = find_hook(hook_name);
-	if (!argv[0])
-		return 0;
-
-	argv[1] = NULL;
-
-	proc.argv = argv;
-	proc.in = -1;
-	proc.stdout_to_stderr = 1;
-	proc.trace2_hook_name = hook_name;
-
-	if (feed_state->push_options) {
-		int i;
-		for (i = 0; i < feed_state->push_options->nr; i++)
-			strvec_pushf(&proc.env_array,
-				     "GIT_PUSH_OPTION_%d=%s", i,
-				     feed_state->push_options->items[i].string);
-		strvec_pushf(&proc.env_array, "GIT_PUSH_OPTION_COUNT=%d",
-			     feed_state->push_options->nr);
-	} else
-		strvec_pushf(&proc.env_array, "GIT_PUSH_OPTION_COUNT");
-
-	if (tmp_objdir)
-		strvec_pushv(&proc.env_array, tmp_objdir_env(tmp_objdir));
-
-	if (use_sideband) {
-		memset(&muxer, 0, sizeof(muxer));
-		muxer.proc = copy_to_sideband;
-		muxer.in = -1;
-		code = start_async(&muxer);
-		if (code)
-			return code;
-		proc.err = muxer.in;
-	}
-
-	prepare_push_cert_sha1(&proc);
-
-	code = start_command(&proc);
-	if (code) {
-		if (use_sideband)
-			finish_async(&muxer);
-		return code;
-	}
-
-	sigchain_push(SIGPIPE, SIG_IGN);
-
-	while (1) {
-		const char *buf;
-		size_t n;
-		if (feed(feed_state, &buf, &n))
-			break;
-		if (write_in_full(proc.in, buf, n) < 0)
-			break;
-	}
-	close(proc.in);
-	if (use_sideband)
-		finish_async(&muxer);
-
-	sigchain_pop(SIGPIPE);
-
-	return finish_command(&proc);
-}
-
-static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep)
+static int feed_receive_hook(void *state_)
 {
 	struct receive_hook_feed_state *state = state_;
 	struct command *cmd = state->cmd;
@@ -883,9 +815,7 @@ static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep)
 	       state->skip_broken && (cmd->error_string || cmd->did_not_exist))
 		cmd = cmd->next;
 	if (!cmd)
-		return -1; /* EOF */
-	if (!bufp)
-		return 0; /* OK, can feed something. */
+		return 1; /* EOF - close the pipe*/
 	strbuf_reset(&state->buf);
 	if (!state->report)
 		state->report = cmd->report;
@@ -909,32 +839,36 @@ static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep)
 			    cmd->ref_name);
 		state->cmd = cmd->next;
 	}
-	if (bufp) {
-		*bufp = state->buf.buf;
-		*sizep = state->buf.len;
-	}
 	return 0;
 }
 
-static int run_receive_hook(struct command *commands,
-			    const char *hook_name,
-			    int skip_broken,
-			    const struct string_list *push_options)
+static int feed_receive_hook_cb(struct strbuf *pipe, void *pp_cb, void *pp_task_cb)
 {
-	struct receive_hook_feed_state state;
-	int status;
-
-	strbuf_init(&state.buf, 0);
-	state.cmd = commands;
-	state.skip_broken = skip_broken;
-	state.report = NULL;
-	if (feed_receive_hook(&state, NULL, NULL))
-		return 0;
-	state.cmd = commands;
-	state.push_options = push_options;
-	status = run_and_feed_hook(hook_name, feed_receive_hook, &state);
-	strbuf_release(&state.buf);
-	return status;
+	struct hook *hook = pp_task_cb;
+	struct receive_hook_feed_state *feed_state = hook->feed_pipe_cb_data;
+	int rc;
+
+	/* first-time setup */
+	if (!feed_state) {
+		struct hook_cb_data *hook_cb = pp_cb;
+		struct run_hooks_opt *opt = hook_cb->options;
+		struct receive_hook_feed_context *ctx = opt->feed_pipe_ctx;
+		if (!ctx)
+			BUG("run_hooks_opt.feed_pipe_ctx required for receive hook");
+
+		feed_state = xmalloc(sizeof(struct receive_hook_feed_state));
+		strbuf_init(&feed_state->buf, 0);
+		feed_state->cmd = ctx->cmd;
+		feed_state->skip_broken = ctx->skip_broken;
+		feed_state->report = NULL;
+
+		hook->feed_pipe_cb_data = feed_state;
+	}
+
+	rc = feed_receive_hook(feed_state);
+	if (!rc)
+		strbuf_addbuf(pipe, &feed_state->buf);
+	return rc;
 }
 
 static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
@@ -970,6 +904,48 @@ static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
 	send_sideband(1, 2, output->buf, output->len, use_sideband);
 }
 
+static int run_receive_hook(struct command *commands,
+			    const char *hook_name,
+			    int skip_broken,
+			    const struct string_list *push_options)
+{
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct receive_hook_feed_context ctx;
+	struct command *iter = commands;
+
+	/* if there are no valid commands, don't invoke the hook at all. */
+	while (iter && skip_broken && (iter->error_string || iter->did_not_exist))
+		iter = iter->next;
+	if (!iter)
+		return 0;
+
+	if (push_options) {
+		int i;
+		for (i = 0; i < push_options->nr; i++)
+			strvec_pushf(&opt.env, "GIT_PUSH_OPTION_%d=%s", i,
+				     push_options->items[i].string);
+		strvec_pushf(&opt.env, "GIT_PUSH_OPTION_COUNT=%d", push_options->nr);
+	} else
+		strvec_push(&opt.env, "GIT_PUSH_OPTION_COUNT");
+
+	if (tmp_objdir)
+		strvec_pushv(&opt.env, tmp_objdir_env(tmp_objdir));
+
+	prepare_push_cert_sha1(&opt);
+
+	/* set up sideband printer */
+	if (use_sideband)
+		opt.consume_sideband = hook_output_to_sideband;
+
+	/* set up stdin callback */
+	ctx.cmd = commands;
+	ctx.skip_broken = skip_broken;
+	opt.feed_pipe = feed_receive_hook_cb;
+	opt.feed_pipe_ctx = &ctx;
+
+	return run_hooks_oneshot(hook_name, &opt);
+}
+
 static int run_update_hook(struct command *cmd)
 {
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
#545 lore raw

PATCH v4 36/36 hooks: fix a TOCTOU in "did we run a hook?" heuristic

In reply to Ævar Arnfjörð Bjarmason

Fix a Time-of-check to time-of-use (TOCTOU) race in code added in 680ee550d72 (commit: skip discarding the index if there is no pre-commit hook, 2017-08-14).

We can fix the race passing around information about whether or not we ran the hook in question, instead of running hook_exists() after the fact to check if the hook in question exists. This problem has been noted on-list when 680ee550d72 was discussed[1], but had not been fixed.

In addition to fixing this for the pre-commit hook as suggested there I'm also fixing this for the pre-merge-commit hook. See 6098817fd7f (git-merge: honor pre-merge-commit hook, 2019-08-07) for the introduction of its previous behavior.

Let's also change this for the push-to-checkout hook. Now instead of checking if the hook exists and either doing a push to checkout or a push to deploy we'll always attempt a push to checkout. If the hook doesn't exist we'll fall back on push to deploy. The same behavior as before, without the TOCTOU race. See 0855331941b (receive-pack:
support push-to-checkout hook, 2014-12-01) for the introduction of the previous behavior.

This leaves uses of hook_exists() in two places that matter. The "reference-transaction" check in refs.c, see 67541597670 (refs:
implement reference transaction hook, 2020-06-19), and the prepare-commit-msg hook, see 66618a50f9c (sequencer: run 'prepare-commit-msg' hook, 2018-01-24).

In both of those cases we're saving ourselves CPU time by not preparing data for the hook that we'll then do nothing with if we don't have the hook. So using this "invoked_hook" pattern doesn't make sense in those cases.

More importantly, in those cases the worst we'll do is miss that we "should" run the hook because a new hook appeared, whereas in the pre-commit and pre-merge-commit cases we'll skip an important discard_cache() on the bases of our faulty guess.

I do think none of these races really matter in practice. It would be some one-off issue as a hook was added or removed. I did think it was stupid that we didn't pass a "did this run?" flag instead of doing this guessing at a distance though, so now we're not guessing anymore.

1. https://lore.kernel.org/git/[email protected]/

---
 builtin/commit.c       | 18 +++++++++++-------
 builtin/merge.c        | 16 ++++++++++------
 builtin/receive-pack.c |  8 +++++---
 commit.c               |  1 +
 commit.h               |  3 ++-
 hook.c                 |  4 ++++
 hook.h                 | 10 ++++++++++
 sequencer.c            |  4 ++--
 8 files changed, 45 insertions(+), 19 deletions(-)
builtin/commit.c +11 -7
index aa3c741efa9..0b7642e8324 100644
@@ -725,11 +725,13 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 	int clean_message_contents = (cleanup_mode != COMMIT_MSG_CLEANUP_NONE);
 	int old_display_comment_prefix;
 	int merge_contains_scissors = 0;
+	int invoked_hook = 0;
 
 	/* This checks and barfs if author is badly specified */
 	determine_author_info(author_ident);
 
-	if (!no_verify && run_commit_hook(use_editor, index_file, "pre-commit", NULL))
+	if (!no_verify && run_commit_hook(use_editor, index_file, &invoked_hook,
+					  "pre-commit", NULL))
 		return 0;
 
 	if (squash_message) {
@@ -1052,10 +1054,10 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (!no_verify && hook_exists("pre-commit")) {
+	if (!no_verify && invoked_hook) {
 		/*
-		 * Re-read the index as pre-commit hook could have updated it,
-		 * and write it out as a tree.  We must do this before we invoke
+		 * Re-read the index as the pre-commit-commit hook was invoked
+		 * and could have updated it. We must do this before we invoke
 		 * the editor and after we invoke run_status above.
 		 */
 		discard_cache();
@@ -1067,7 +1069,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (run_commit_hook(use_editor, index_file, "prepare-commit-msg",
+	if (run_commit_hook(use_editor, index_file, NULL, "prepare-commit-msg",
 			    git_path_commit_editmsg(), hook_arg1, hook_arg2, NULL))
 		return 0;
 
@@ -1084,7 +1086,8 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 	}
 
 	if (!no_verify &&
-	    run_commit_hook(use_editor, index_file, "commit-msg", git_path_commit_editmsg(), NULL)) {
+	    run_commit_hook(use_editor, index_file, NULL, "commit-msg",
+			    git_path_commit_editmsg(), NULL)) {
 		return 0;
 	}
 
@@ -1840,7 +1843,8 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 
 	repo_rerere(the_repository, 0);
 	run_auto_maintenance(quiet);
-	run_commit_hook(use_editor, get_index_file(), "post-commit", NULL);
+	run_commit_hook(use_editor, get_index_file(), NULL, "post-commit",
+			NULL);
 	if (amend && !no_post_rewrite) {
 		commit_post_rewrite(the_repository, current_head, &oid);
 	}
builtin/merge.c +10 -6
index 4965df2ac29..9bd4a2532c3 100644
@@ -843,15 +843,18 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 {
 	struct strbuf msg = STRBUF_INIT;
 	const char *index_file = get_index_file();
+	int invoked_hook = 0;
 
-	if (!no_verify && run_commit_hook(0 < option_edit, index_file, "pre-merge-commit", NULL))
+	if (!no_verify && run_commit_hook(0 < option_edit, index_file,
+					  &invoked_hook, "pre-merge-commit",
+					  NULL))
 		abort_commit(remoteheads, NULL);
 	/*
-	 * Re-read the index as pre-merge-commit hook could have updated it,
-	 * and write it out as a tree.  We must do this before we invoke
+	 * Re-read the index as the pre-merge-commit hook was invoked
+	 * and could have updated it. We must do this before we invoke
 	 * the editor and after we invoke run_status above.
 	 */
-	if (hook_exists("pre-merge-commit"))
+	if (invoked_hook)
 		discard_cache();
 	read_cache_from(index_file);
 	strbuf_addbuf(&msg, &merge_msg);
@@ -872,7 +875,8 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 		append_signoff(&msg, ignore_non_trailer(msg.buf, msg.len), 0);
 	write_merge_heads(remoteheads);
 	write_file_buf(git_path_merge_msg(the_repository), msg.buf, msg.len);
-	if (run_commit_hook(0 < option_edit, get_index_file(), "prepare-commit-msg",
+	if (run_commit_hook(0 < option_edit, get_index_file(), NULL,
+			    "prepare-commit-msg",
 			    git_path_merge_msg(the_repository), "merge", NULL))
 		abort_commit(remoteheads, NULL);
 	if (0 < option_edit) {
@@ -881,7 +885,7 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 	}
 
 	if (!no_verify && run_commit_hook(0 < option_edit, get_index_file(),
-					  "commit-msg",
+					  NULL, "commit-msg",
 					  git_path_merge_msg(the_repository), NULL))
 		abort_commit(remoteheads, NULL);
 
builtin/receive-pack.c +5 -3
index c3984680d7f..ebec6f3bb10 100644
@@ -1428,10 +1428,12 @@ static const char *push_to_deploy(unsigned char *sha1,
 static const char *push_to_checkout_hook = "push-to-checkout";
 
 static const char *push_to_checkout(unsigned char *hash,
+				    int *invoked_hook,
 				    struct strvec *env,
 				    const char *work_tree)
 {
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	opt.invoked_hook = invoked_hook;
 
 	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
 	strvec_pushv(&opt.env, env->v);
@@ -1446,6 +1448,7 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *w
 {
 	const char *retval, *work_tree, *git_dir = NULL;
 	struct strvec env = STRVEC_INIT;
+	int invoked_hook = 0;
 
 	if (worktree && worktree->path)
 		work_tree = worktree->path;
@@ -1463,10 +1466,9 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *w
 
 	strvec_pushf(&env, "GIT_DIR=%s", absolute_path(git_dir));
 
-	if (!hook_exists(push_to_checkout_hook))
+	retval = push_to_checkout(sha1, &invoked_hook, &env, work_tree);
+	if (!invoked_hook)
 		retval = push_to_deploy(sha1, &env, work_tree);
-	else
-		retval = push_to_checkout(sha1, &env, work_tree);
 
 	strvec_clear(&env);
 	return retval;
commit.c +1 -0
index 63d7943a86d..842e47beae2 100644
@@ -1697,6 +1697,7 @@ size_t ignore_non_trailer(const char *buf, size_t len)
 }
 
 int run_commit_hook(int editor_is_used, const char *index_file,
+		    int *invoked_hook,
 		    const char *name, ...)
 {
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
commit.h +2 -1
index df42eb434f3..b5a542993c6 100644
@@ -363,7 +363,8 @@ int compare_commits_by_commit_date(const void *a_, const void *b_, void *unused)
 int compare_commits_by_gen_then_commit_date(const void *a_, const void *b_, void *unused);
 
 LAST_ARG_MUST_BE_NULL
-int run_commit_hook(int editor_is_used, const char *index_file, const char *name, ...);
+int run_commit_hook(int editor_is_used, const char *index_file,
+		    int *invoked_hook, const char *name, ...);
 
 /* Sign a commit or tag buffer, storing the result in a header. */
 int sign_with_header(struct strbuf *buf, const char *keyid);
hook.c +4 -0
index efed5f73b4d..ee20b2e3658 100644
@@ -173,6 +173,9 @@ static int notify_hook_finished(int result,
 
 	hook_cb->rc |= result;
 
+	if (hook_cb->invoked_hook)
+		*hook_cb->invoked_hook = 1;
+
 	return 0;
 }
 
@@ -187,6 +190,7 @@ int run_hooks(const char *hook_name, const char *hook_path,
 		.rc = 0,
 		.hook_name = hook_name,
 		.options = options,
+		.invoked_hook = options->invoked_hook,
 	};
 	int jobs = 1;
 
hook.h +10 -0
index 37a9690c2ca..58dfbf474c9 100644
@@ -66,6 +66,15 @@ struct run_hooks_opt
 	 * for an example.
 	 */
 	consume_sideband_fn consume_sideband;
+
+	/*
+	 * A pointer which if provided will be set to 1 or 0 depending
+	 * on if a hook was invoked (i.e. existed), regardless of
+	 * whether or not that was successful. Used for avoiding
+	 * TOCTOU races in code that would otherwise call hook_exist()
+	 * after a "maybe hook run" to see if a hook was invoked.
+	 */
+	int *invoked_hook;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
@@ -90,6 +99,7 @@ struct hook_cb_data {
 	const char *hook_name;
 	struct hook *run_me;
 	struct run_hooks_opt *options;
+	int *invoked_hook;
 };
 
 void run_hooks_opt_clear(struct run_hooks_opt *o);
sequencer.c +2 -2
index 77f809c00e4..f451e23d0c1 100644
@@ -1203,7 +1203,7 @@ static int run_prepare_commit_msg_hook(struct repository *r,
 	} else {
 		arg1 = "message";
 	}
-	if (run_commit_hook(0, r->index_file, "prepare-commit-msg", name,
+	if (run_commit_hook(0, r->index_file, NULL, "prepare-commit-msg", name,
 			    arg1, arg2, NULL))
 		ret = error(_("'prepare-commit-msg' hook failed"));
 
@@ -1533,7 +1533,7 @@ static int try_to_commit(struct repository *r,
 		goto out;
 	}
 
-	run_commit_hook(0, r->index_file, "post-commit", NULL);
+	run_commit_hook(0, r->index_file, NULL, "post-commit", NULL);
 	if (flags & AMEND_MSG)
 		commit_post_rewrite(r, current_head, oid);
 
Signature · 1 line
2.33.0.rc0.595.ge31e012651d
#546 lore raw

v2 cover config-based hooks restarted

In reply to Ævar Arnfjörð Bjarmason

This is the config-based hooks topic rebased onto v4 of Ævar's branch[1]. There is a happy CI build of it on GitHub[2].

The topic overall adds the ability to set up hooks by modifying the config, in addition to placing specially named hooks into the hookdir. This enables users to specify multiple hooks for a given event, and so this topic also fleshes out the use of the run_processes_parallel() API which is now introduced in Ævar's reordering of prior patches.

Patches 1-4 make some minor changes to prepare Ævar's series to handle more than one hook at a time. With the exception of patch 4, there should be no behavior change for existing hooks.

Patch 2 is opinionated about which hooks should and shouldn't be allowed to run in parallel; if you care about a specific hook, please take a look there.

Patch 5 is the motivating feature - it begins to parse the config looking for hooks.

Patch 6 takes advantage of the decoupling of hooks and GITDIR to allow out-of-repo hook runs, which would only run hooks specified in the system or global config. This mainly targets 'sendemail-validate', but 'git-sendemail.perl' still explicitly disallows out-of-repo hook execution on that hook for now. (Maybe that change should be added in this series? Or maybe patch 6 belongs with that kind of change?)

Since v1:

The largest change is that the config schema is different, following the discussion from [3] and on. 'hookcmd' has gone away entirely, and all config-specified hooks need a user-provided name associated with them.

I have also dropped the 'skip' config from this series (also discussed in [3]) as users can reset their 'hook.myhook.command' to 'true' or some other noop in a later config to effectively disable any hook. The 'skip' feature may be added later or may not, depending on if users ask us for it.

The 'from_hookdir' field has been removed entirely. When storing hooks by name instead of by command, it makes sense enough to store the one from the hookdir in a special way - NULL name - because it does not have a name to specify. This way we can use that field as an indicator that the hook came from the hookdir, and we also do not need to reserve a name to use for hookdir hooks (as opposed to setting hookdir hooks to have name = "FROM_HOOKDIR" or something).

The tests have been moved into t1800-hook.sh and simplified.

The 'struct run_hooks_opt' initializer has been moved to macros (as opposed to functions), and .jobs=0 indicates that we should look up the configured job count or nproc at hook run time. Ævar asked me to change these macros to RUN_HOOKS_OPT_INIT and RUN_HOOKS_OPT_INIT_SYNC or something to simplify the merge, but I did not do that - I removed all instances of RUN_HOOKS_OPT_INIT and now everybody uses RUN_HOOKS_OPT_INIT_SYNC or RUN_HOOKS_OPT_INIT_ASYNC. I think it's important for contributors to be able to tell at a glance whether the hook event expects resource contention that would prevent parallelism. I also think it makes the review easier (if longer) for folks who are reviewing trying to decide whether the parallelism is appropriate for each hook event. I do think that run_hooks_oneshot() also is a step backwards in this regard, but I didn't get a chance to say so on the commit that introduces it yet.

Thanks in advance, all.

- Emily

1: https://lore.kernel.org/git/cover-v4-00.36-00000000000-20210803T191505Z-avarab%40gmail.com
2: https://github.com/nasamuffin/git/actions/runs/1122126800 (which
points to an earlier successul run before I messed with the commit
messages)
3: https://lore.kernel.org/git/87fswey5wd.fsf%40evledraar.gmail.com

Emily Shaffer (6):
hook: run a list of hooks instead
hook: allow parallel hook execution
hook: introduce "git hook list"
hook: allow running non-native hooks
hook: include hooks from the config
hook: allow out-of-repo 'git hook' invocations

 Documentation/config/hook.txt |   9 ++
 Documentation/git-hook.txt    |  48 +++++-
 builtin/am.c                  |   4 +-
 builtin/checkout.c            |   2 +-
 builtin/clone.c               |   2 +-
 builtin/hook.c                |  68 +++++++-
 builtin/merge.c               |   2 +-
 builtin/rebase.c              |   2 +-
 builtin/receive-pack.c        |   9 +-
 builtin/worktree.c            |   2 +-
 commit.c                      |   2 +-
 git.c                         |   2 +-
 hook.c                        | 289 +++++++++++++++++++++++++++++-----
 hook.h                        |  61 +++++--
 read-cache.c                  |   2 +-
 refs.c                        |   2 +-
 reset.c                       |   3 +-
 sequencer.c                   |   4 +-
 t/t1800-hook.sh               | 161 ++++++++++++++++++-
 transport.c                   |   2 +-
 20 files changed, 596 insertions(+), 80 deletions(-)
 create mode 100644 Documentation/config/hook.txt

Range-diff against v1:

1:  c4b95fb08a < -:  ---------- hook: treat hookdir hook specially
-:  ---------- > 1:  5177e8ba2c hook: run a list of hooks instead
-:  ---------- > 2:  eda439cd57 hook: allow parallel hook execution
-:  ---------- > 3:  cdfe3b6e16 hook: introduce "git hook list"
2:  e6a56ac674 = 4:  eb4e03e22b hook: allow running non-native hooks
3:  32ad49ea9b ! 5:  2c8e874158 hook: include hooks from the config
    @@ Commit message
         hooks to run for a given event.
     
         Multiple commands can be specified for a given hook by providing
    -    multiple "hook.<hookname>.command = <path-to-hook>" lines. Hooks will be
    -    run in config order.
    +    multiple "hook.<friendly-name>.command = <path-to-hook>" and
    +    "hook.<friendly-name>.event = <hook-event>" lines. Hooks will be run in
    +    config order of the "hook.<name>.event" lines.
     
         For example:
     
           $ git config --list | grep ^hook
    -      hook.pre-commit.command=~/bar.sh
    +      hook.bar.command=~/bar.sh
    +      hook.bar.event=pre-commit
     
           $ git hook run
           # Runs ~/bar.sh
    @@ hook.c: const char *find_hook_gently(const char *name)
     -		struct hook *to_add = xmalloc(sizeof(*to_add));
     -		to_add->hook_path = hook_path;
     -		to_add->feed_pipe_cb_data = NULL;
    --		to_add->from_hookdir = 1;
     -		list_add_tail(&to_add->list, hook_head);
     -	}
     +	/* Add the hook from the hookdir. The placeholder makes it easier to
    @@ hook.c: static int pick_next_hook(struct child_process *cp,
     +	cp->use_shell = !!run_me->name;
     +
      	/* add command */
    --	if (run_me->from_hookdir && hook_cb->options->absolute_path)
    +-	if (hook_cb->options->absolute_path)
     -		strvec_push(&cp->args, absolute_path(run_me->hook_path));
     -	else
     -		strvec_push(&cp->args, run_me->hook_path);
    @@ hook.h: int hook_exists(const char *hookname);
      	struct list_head list;
     -	/* The path to the hook */
     -	const char *hook_path;
    --
    --	unsigned from_hookdir : 1;
     +	/*
     +	 * The friendly name of the hook. NULL indicates the hook is from the
     +	 * hookdir.

4: 91e54185b3 = 6: 3216e51b6b hook: allow out-of-repo 'git hook' invocations

Signature · 1 line
2.32.0.605.g8dce9f2422-goog
#547 lore raw

PATCH v2 1/6 hook: run a list of hooks instead

To prepare for multihook support, teach hook.[hc] to take a list of hooks at run_hooks and run_found_hooks. Right now the list is always one entry, but in the future we will allow users to supply more than one executable for a single hook event.

---
 builtin/hook.c |  14 ++++---
 hook.c         | 103 +++++++++++++++++++++++++++++++++++--------------
 hook.h         |  16 +++++++-
 3 files changed, 96 insertions(+), 37 deletions(-)
builtin/hook.c +8 -6
index 5eb7cf73a4..4d39c9e75e 100644
@@ -25,7 +25,8 @@ static int run(int argc, const char **argv, const char *prefix)
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	int ignore_missing = 0;
 	const char *hook_name;
-	const char *hook_path;
+	struct list_head *hooks;
+
 	struct option run_options[] = {
 		OPT_BOOL(0, "ignore-missing", &ignore_missing,
 			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
@@ -58,15 +59,16 @@ static int run(int argc, const char **argv, const char *prefix)
 	git_config(git_default_config, NULL);
 
 	hook_name = argv[0];
-	if (ignore_missing)
-		return run_hooks_oneshot(hook_name, &opt);
-	hook_path = find_hook(hook_name);
-	if (!hook_path) {
+	hooks = hook_list(hook_name);
+	if (list_empty(hooks)) {
+		/* ... act like run_hooks_oneshot() under --ignore-missing */
+		if (ignore_missing)
+			return 0;
 		error("cannot find a hook named %s", hook_name);
 		return 1;
 	}
 
-	ret = run_hooks(hook_name, hook_path, &opt);
+	ret = run_hooks(hook_name, hooks, &opt);
 	run_hooks_opt_clear(&opt);
 	return ret;
 usage:
hook.c +74 -29

161 lines not on this page · raw message

hook.h +14 -2
index 58dfbf474c..7705e6a529 100644
@@ -3,6 +3,7 @@
 #include "strbuf.h"
 #include "strvec.h"
 #include "run-command.h"
+#include "list.h"
 
 /*
  * Returns the path to the hook file, or NULL if the hook is missing
@@ -17,6 +18,7 @@ const char *find_hook(const char *name);
 int hook_exists(const char *hookname);
 
 struct hook {
+	struct list_head list;
 	/* The path to the hook */
 	const char *hook_path;
 
@@ -27,6 +29,12 @@ struct hook {
 	void *feed_pipe_cb_data;
 };
 
+/*
+ * Provides a linked list of 'struct hook' detailing commands which should run
+ * in response to the 'hookname' event, in execution order.
+ */
+struct list_head* hook_list(const char *hookname);
+
 struct run_hooks_opt
 {
 	/* Environment vars to be set for each hook */
@@ -97,6 +105,7 @@ struct hook_cb_data {
 	/* rc reflects the cumulative failure state */
 	int rc;
 	const char *hook_name;
+	struct list_head *head;
 	struct hook *run_me;
 	struct run_hooks_opt *options;
 	int *invoked_hook;
@@ -110,8 +119,8 @@ void run_hooks_opt_clear(struct run_hooks_opt *o);
  *
  * See run_hooks_oneshot() for the simpler one-shot API.
  */
-int run_hooks(const char *hookname, const char *hook_path,
-	      struct run_hooks_opt *options);
+int run_hooks(const char *hookname, struct list_head *hooks,
+		    struct run_hooks_opt *options);
 
 /**
  * Calls find_hook() on your "hook_name" and runs the hooks (if any)
@@ -123,4 +132,7 @@ int run_hooks(const char *hookname, const char *hook_path,
  */
 int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options);
 
+/* Empties the list at 'head', calling 'free_hook()' on each entry */
+void clear_hook_list(struct list_head *head);
+
 #endif
Signature · 1 line
2.32.0.605.g8dce9f2422-goog
#548 lore raw

Re: [PATCH v2 1/6] hook: run a list of hooks instead

Quoted 17 lines · Emily Shaffer

To prepare for multihook support, teach hook.[hc] to take a list of hooks at run_hooks and run_found_hooks. Right now the list is always one entry, but in the future we will allow users to supply more than one executable for a single hook event.

---
 builtin/hook.c |  14 ++++---
 hook.c         | 103 +++++++++++++++++++++++++++++++++++--------------
 hook.h         |  16 +++++++-
 3 files changed, 96 insertions(+), 37 deletions(-)
builtin/hook.c +0 -0
index 5eb7cf73a4..4d39c9e75e 100644

builtin/hook.c static int run(int argc, const char **argv, const char *prefix) · line 29

Quoted 6 lines
builtin/hook.c +2 -1
@@ -25,7 +25,8 @@ static int run(int argc, const char **argv, const char *prefix)
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	int ignore_missing = 0;
 	const char *hook_name;
-	const char *hook_path;
+	struct list_head *hooks;
+

Natural. We used to use the path to the hook because we were expecting only one. We now use the name to find a list of hooks.

All the caller sees is just list_head without any direct visibility into it, which feels like a great abstraction. Presumably everything will go through the API functions taking this opaque "list of hooks" thing (or "the first one in the list" if the API function does not iterate over it, perhaps?).

builtin/hook.c static int run(int argc, const char **argv, const char *prefix) · line 62

Quoted 12 lines
builtin/hook.c +1 -4
 	struct option run_options[] = {
 		OPT_BOOL(0, "ignore-missing", &ignore_missing,
 			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
@@ -58,15 +59,16 @@ static int run(int argc, const char **argv, const char *prefix)
 	git_config(git_default_config, NULL);
 
 	hook_name = argv[0];
-	if (ignore_missing)
-		return run_hooks_oneshot(hook_name, &opt);
-	hook_path = find_hook(hook_name);
-	if (!hook_path) {
+	hooks = hook_list(hook_name);

OK.

builtin/hook.c static int run(int argc, const char **argv, const char *prefix) · line 66

Quoted 4 lines
builtin/hook.c +4 -0
+	if (list_empty(hooks)) {
+		/* ... act like run_hooks_oneshot() under --ignore-missing */
+		if (ignore_missing)
+			return 0;

OK.

Quoted 9 lines
builtin/hook.c +0 -0
 		error("cannot find a hook named %s", hook_name);
 		return 1;
 	}
 
hook.c +0 -0
index ee20b2e365..80e150548c 100644

hook.c line 11

Quoted 8 lines
hook.c +5 -0
@@ -4,6 +4,28 @@
 #include "hook-list.h"
 #include "config.h"
 
+static void free_hook(struct hook *ptr)
+{
+	if (ptr) {
+		free(ptr->feed_pipe_cb_data);
+	}

Lose the extra {}, as we do not do more than the above free() even at the end of the series?

Quoted 6 lines
hook.c +6 -0
+	free(ptr);
+}
+
+static void remove_hook(struct list_head *to_remove)
+{
+	struct hook *hook_to_remove = list_entry(to_remove, struct hook, list);

hook.c line 27

Quoted 10 lines
hook.c +10 -0
+	list_del(to_remove);
+	free_hook(hook_to_remove);
+}
+
+void clear_hook_list(struct list_head *head)
+{
+	struct list_head *pos, *tmp;
+	list_for_each_safe(pos, tmp, head)
+		remove_hook(pos);
+}

OK.

hook.c int hook_exists(const char *name) · line 96

Quoted 1 line
hook.c +1 -0
+struct list_head* hook_list(const char* hookname)

Shift both of the asterisks; our asterisks do not stick to the type but to the identifier.

hook.c int hook_exists(const char *name) · line 103

Quoted 7 lines
hook.c +7 -0
+{
+	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
+
+	INIT_LIST_HEAD(hook_head);
+
+	if (!hookname)
+		return NULL;

Checking for invalid hookname first would avoid leaking hook_head, no? The caller of hook_list() we saw earlier immediately calls list_empty() which will segfault. The caller need to be tightened, but I wonder if it would be a programmer's error to pass a NULL hookname to this function. If so, this can simply be a BUG() and the earlier allocation and initialization of hook_head can stay where they are. Otherwise, the caller should see if this returns a NULL.

hook.c int hook_exists(const char *name) · line 115

Quoted 11 lines
hook.c +11 -0
+	if (have_git_dir()) {
+		const char *hook_path = find_hook(hookname);
+
+		/* Add the hook from the hookdir */
+		if (hook_path) {
+			struct hook *to_add = xmalloc(sizeof(*to_add));
+			to_add->hook_path = hook_path;
+			to_add->feed_pipe_cb_data = NULL;
+			list_add_tail(&to_add->list, hook_head);
+		}
+	}

Calling this function to grab a list of hooks when we are not in any repository is not an error but just we get "there is nothing to run". Does the design give us a more useful behaviour, compared to alternatives like "you have to be in a repository or calling this function is an error"?

Not an objection wrapped in a rhetorical question, but a genuine question. "It would help this and that usecase" would be an ideal answer, "We could do either way, but we happened to have written the code this way first, and at the end of the series we did not see any practical downsides" would also be a great answer.

hook.c

Quoted 9 lines
hook.c +8 -0
+	return hook_head;
+}
+
 
+/*
+ * Provides a linked list of 'struct hook' detailing commands which should run
+ * in response to the 'hookname' event, in execution order.
+ */
+struct list_head* hook_list(const char *hookname);

struct list_head *hook_list(const char *hookname);

#549 lore raw

Re: [PATCH v2 1/6] hook: run a list of hooks instead

Quoted 30 lines · Junio C Hamano
Quoted 17 lines · Emily Shaffer

To prepare for multihook support, teach hook.[hc] to take a list of hooks at run_hooks and run_found_hooks. Right now the list is always one entry, but in the future we will allow users to supply more than one executable for a single hook event.

---
 builtin/hook.c |  14 ++++---
 hook.c         | 103 +++++++++++++++++++++++++++++++++++--------------
 hook.h         |  16 +++++++-
 3 files changed, 96 insertions(+), 37 deletions(-)
builtin/hook.c +0 -0
index 5eb7cf73a4..4d39c9e75e 100644

builtin/hook.c static int run(int argc, const char **argv, const char *prefix) · line 29

Quoted 6 lines
builtin/hook.c +2 -1
@@ -25,7 +25,8 @@ static int run(int argc, const char **argv, const char *prefix)
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	int ignore_missing = 0;
 	const char *hook_name;
-	const char *hook_path;
+	struct list_head *hooks;
+

Natural. We used to use the path to the hook because we were expecting only one. We now use the name to find a list of hooks.

Quoted 5 lines

All the caller sees is just list_head without any direct visibility into it, which feels like a great abstraction. Presumably everything will go through the API functions taking this opaque "list of hooks" thing (or "the first one in the list" if the API function does not iterate over it, perhaps?).

Hum, I guess that in a later patch builtin/hook.c does learn to take apart the list_head into a 'struct hook' to print the output of 'git hook list'. I haven't read your review of that patch yet though.

Quoted 8 lines
Quoted 8 lines
hook.c +0 -0
index ee20b2e365..80e150548c 100644
@@ -4,6 +4,28 @@
 #include "hook-list.h"
 #include "config.h"
 
Quoted 8 lines

hook.c line 11

Quoted 5 lines
hook.c +5 -0
+static void free_hook(struct hook *ptr)
+{
+	if (ptr) {
+		free(ptr->feed_pipe_cb_data);
+	}

Lose the extra {}, as we do not do more than the above free() even at the end of the series?

ACK

Quoted 5 lines

hook.c int hook_exists(const char *name) · line 96

Quoted 2 lines
hook.c +1 -0
+struct list_head* hook_list(const char* hookname)

Shift both of the asterisks; our asterisks do not stick to the type but to the identifier.

ACK

Quoted 17 lines

hook.c int hook_exists(const char *name) · line 103

Quoted 8 lines
hook.c +7 -0
+{
+	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
+
+	INIT_LIST_HEAD(hook_head);
+
+	if (!hookname)
+		return NULL;

Checking for invalid hookname first would avoid leaking hook_head, no? The caller of hook_list() we saw earlier immediately calls list_empty() which will segfault. The caller need to be tightened, but I wonder if it would be a programmer's error to pass a NULL hookname to this function. If so, this can simply be a BUG() and the earlier allocation and initialization of hook_head can stay where they are. Otherwise, the caller should see if this returns a NULL.

Ah, good point. I think this makes sense to BUG().

Quoted 10 lines
Quoted 10 lines
hook.c +9 -0
+	if (have_git_dir()) {
+		const char *hook_path = find_hook(hookname);
+
+		/* Add the hook from the hookdir */
+		if (hook_path) {
+			struct hook *to_add = xmalloc(sizeof(*to_add));
+			to_add->hook_path = hook_path;
+			to_add->feed_pipe_cb_data = NULL;
+			list_add_tail(&to_add->list, hook_head);
Quoted 8 lines

hook.c line 30

Quoted 2 lines
hook.c +2 -0
+		}
+	}

Calling this function to grab a list of hooks when we are not in any repository is not an error but just we get "there is nothing to run". Does the design give us a more useful behaviour, compared to alternatives like "you have to be in a repository or calling this function is an error"?

Later we enable calling hook_list without a gitdir, in patch 6
(https://lore.kernel.org/git/20210812004258.74318-7-emilyshaffer%40google.com).
So maybe the behavior as it is now is premature?

But leaving the hook list empty means we can behave gracefully if anybody is calling a hook without necessarily being sure they have gitdir. I would need to audit callsites to check if they are checking whether they have one or not. I think in most cases they probably are checking that.

Quoted 6 lines

Not an objection wrapped in a rhetorical question, but a genuine question. "It would help this and that usecase" would be an ideal answer, "We could do either way, but we happened to have written the code this way first, and at the end of the series we did not see any practical downsides" would also be a great answer.

The main use case, today, for letting this work nicely even outside of a gitdir, is sendemail-validate hook. But mostly, I was thinking that when we're freed from dependence on .git/hooks/, there's no reason to disallow out-of-repo hooks in case someone wants to add a new one in the future - for example to 'git maintenance' daemon runs.

Quoted 5 lines
Quoted 4 lines
hook.c +3 -0
+	return hook_head;
+}
+
Quoted 7 lines

hook.c

Quoted 5 lines
hook.c +5 -0
+/*
+ * Provides a linked list of 'struct hook' detailing commands which should run
+ * in response to the 'hookname' event, in execution order.
+ */
+struct list_head* hook_list(const char *hookname);

struct list_head *hook_list(const char *hookname);

ACK

Thanks for the thoughts.
- Emily

#550 lore raw

PATCH v2 2/6 hook: allow parallel hook execution

In reply to Emily Shaffer

In many cases, there's no reason not to allow hooks to execute in parallel. run_processes_parallel() is well-suited - it's a task queue that runs its housekeeping in series, which means users don't need to worry about thread safety on their callback data. True multithreaded execution with the async_* functions isn't necessary here. Synchronous hook execution can be achieved by only allowing 1 job to run at a time.

Teach run_hooks() to use that function for simple hooks which don't require stdin or capture of stderr.

---
 Documentation/config/hook.txt |  4 ++++
 Documentation/git-hook.txt    | 17 ++++++++++++++++-
 builtin/am.c                  |  4 ++--
 builtin/checkout.c            |  2 +-
 builtin/clone.c               |  2 +-
 builtin/hook.c                |  4 +++-
 builtin/merge.c               |  2 +-
 builtin/rebase.c              |  2 +-
 builtin/receive-pack.c        |  9 +++++----
 builtin/worktree.c            |  2 +-
 commit.c                      |  2 +-
 hook.c                        | 36 +++++++++++++++++++++++++++++++----
 hook.h                        | 24 ++++++++++++++++++-----
 read-cache.c                  |  2 +-
 refs.c                        |  2 +-
 reset.c                       |  3 ++-
 sequencer.c                   |  4 ++--
 transport.c                   |  2 +-
 18 files changed, 94 insertions(+), 29 deletions(-)
 create mode 100644 Documentation/config/hook.txt
Documentation/config/hook.txt new file +4 -0
new file mode 100644
index 0000000000..96d3d6572c
@@ -0,0 +1,4 @@
+hook.jobs::
+	Specifies how many hooks can be run simultaneously during parallelized
+	hook execution. If unspecified, defaults to the number of processors on
+	the current system.
Documentation/git-hook.txt +16 -1
index fa68c1f391..8bf82b5dd4 100644
@@ -8,7 +8,8 @@ git-hook - run git hooks
 SYNOPSIS
 --------
 [verse]
-'git hook' run [--to-stdin=<path>] [--ignore-missing] <hook-name> [-- <hook-args>]
+'git hook' run [--to-stdin=<path>] [--ignore-missing] [(-j|--jobs) <n>]
+	<hook-name> [-- <hook-args>]
 
 DESCRIPTION
 -----------
@@ -42,6 +43,20 @@ OPTIONS
 	tools that want to do a blind one-shot run of a hook that may
 	or may not be present.
 
+-j::
+--jobs::
+	Only valid for `run`.
++
+Specify how many hooks to run simultaneously. If this flag is not specified, use
+the value of the `hook.jobs` config. If the config is not specified, use the
+number of CPUs on the current system. Some hooks may be ineligible for
+parallelization: for example, 'commit-msg' intends hooks modify the commit
+message body and cannot be parallelized.
+
+CONFIGURATION
+-------------
+include::config/hook.txt[]
+
 SEE ALSO
 --------
 linkgit:githooks[5]
builtin/am.c +2 -2
index 9e3d4d9ab4..c24a27d6a1 100644
@@ -446,7 +446,7 @@ static void am_destroy(const struct am_state *state)
 static int run_applypatch_msg_hook(struct am_state *state)
 {
 	int ret;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SYNC;
 
 	assert(state->msg);
 	strvec_push(&opt.args, am_path(state, "final-commit"));
@@ -467,7 +467,7 @@ static int run_applypatch_msg_hook(struct am_state *state)
  */
 static int run_post_rewrite_hook(const struct am_state *state)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
 
 	strvec_push(&opt.args, "rebase");
 	opt.path_to_stdin = am_path(state, "rewritten");
builtin/checkout.c +1 -1
index 6d69b4c011..27166c0bb8 100644
@@ -107,7 +107,7 @@ struct branch_info {
 static int post_checkout_hook(struct commit *old_commit, struct commit *new_commit,
 			      int changed)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SYNC;
 
 	/* "new_commit" can be NULL when checking out from the index before
 	   a commit exists. */
builtin/clone.c +1 -1
index 27fc05ee51..599e7a7936 100644
@@ -776,7 +776,7 @@ static int checkout(int submodule_progress)
 	struct tree *tree;
 	struct tree_desc t;
 	int err = 0;
-	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_SYNC;
 
 	if (option_no_checkout)
 		return 0;
builtin/hook.c +3 -1
index 4d39c9e75e..12c9126032 100644
@@ -22,7 +22,7 @@ static const char * const builtin_hook_run_usage[] = {
 static int run(int argc, const char **argv, const char *prefix)
 {
 	int i;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SYNC;
 	int ignore_missing = 0;
 	const char *hook_name;
 	struct list_head *hooks;
@@ -32,6 +32,8 @@ static int run(int argc, const char **argv, const char *prefix)
 			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
 		OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"),
 			   N_("file to read into hooks' stdin")),
+		OPT_INTEGER('j', "jobs", &opt.jobs,
+			    N_("run up to <n> hooks simultaneously")),
 		OPT_END(),
 	};
 	int ret;
builtin/merge.c +1 -1
index 9bd4a2532c..c749c382c3 100644
@@ -448,7 +448,7 @@ static void finish(struct commit *head_commit,
 		   const struct object_id *new_head, const char *msg)
 {
 	struct strbuf reflog_message = STRBUF_INIT;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
 	const struct object_id *head = &head_commit->object.oid;
 
 	if (!msg)
builtin/rebase.c +1 -1
index e7c668c99b..fecf248ed9 100644
@@ -1314,7 +1314,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	char *squash_onto_name = NULL;
 	int reschedule_failed_exec = -1;
 	int allow_preemptive_ff = 1;
-	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_ASYNC;
 	struct option builtin_rebase_options[] = {
 		OPT_STRING(0, "onto", &options.onto_name,
 			   N_("revision"),
builtin/receive-pack.c +5 -4
index ebec6f3bb1..b32dcc9000 100644
@@ -909,7 +909,7 @@ static int run_receive_hook(struct command *commands,
 			    int skip_broken,
 			    const struct string_list *push_options)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
 	struct receive_hook_feed_context ctx;
 	struct command *iter = commands;
 
@@ -948,7 +948,7 @@ static int run_receive_hook(struct command *commands,
 
 static int run_update_hook(struct command *cmd)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
 
 	strvec_pushl(&opt.args,
 		     cmd->ref_name,
@@ -1432,7 +1432,8 @@ static const char *push_to_checkout(unsigned char *hash,
 				    struct strvec *env,
 				    const char *work_tree)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SYNC;
+
 	opt.invoked_hook = invoked_hook;
 
 	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
@@ -1628,7 +1629,7 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 static void run_update_post_hook(struct command *commands)
 {
 	struct command *cmd;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
 
 	for (cmd = commands; cmd; cmd = cmd->next) {
 		if (cmd->error_string || cmd->did_not_exist)
builtin/worktree.c +1 -1
index 330867c19b..efead564c1 100644
@@ -382,7 +382,7 @@ static int add_worktree(const char *path, const char *refname,
 	 * is_junk is cleared, but do return appropriate code when hook fails.
 	 */
 	if (!ret && opts->checkout) {
-		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SYNC;
 
 		strvec_pushl(&opt.env, "GIT_DIR", "GIT_WORK_TREE", NULL);
 		strvec_pushl(&opt.args,
commit.c +1 -1
index 842e47beae..0e6e5a5b27 100644
@@ -1700,7 +1700,7 @@ int run_commit_hook(int editor_is_used, const char *index_file,
 		    int *invoked_hook,
 		    const char *name, ...)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SYNC;
 	va_list args;
 	const char *arg;
 
hook.c +32 -4
index 80e150548c..37f682c6d8 100644
@@ -228,6 +228,28 @@ static int notify_hook_finished(int result,
 	return 0;
 }
 
+/*
+ * Determines how many jobs to use after we know we want to parallelize. First
+ * priority is the config 'hook.jobs' and second priority is the number of CPUs.
+ */
+static int configured_hook_jobs(void)
+{
+	/*
+	 * The config and the CPU count probably won't change during the process
+	 * lifetime, so cache the result in case we invoke multiple hooks during
+	 * one process.
+	 */
+	static int jobs = 0;
+	if (jobs)
+		return jobs;
+
+	if (git_config_get_int("hook.jobs", &jobs))
+		/* if the config isn't set, fall back to CPU count. */
+		jobs = online_cpus();
+
+	return jobs;
+}
+
 int run_hooks(const char *hook_name, struct list_head *hooks,
 		    struct run_hooks_opt *options)
 {
@@ -237,16 +259,18 @@ int run_hooks(const char *hook_name, struct list_head *hooks,
 		.options = options,
 		.invoked_hook = options->invoked_hook,
 	};
-	int jobs = 1;
 
 	if (!options)
 		BUG("a struct run_hooks_opt must be provided to run_hooks");
 
-
 	cb_data.head = hooks;
 	cb_data.run_me = list_first_entry(hooks, struct hook, list);
 
-	run_processes_parallel_tr2(jobs,
+	/* INIT_ASYNC sets jobs to 0, so go look up how many to use. */
+	if (!options->jobs)
+		options->jobs = configured_hook_jobs();
+
+	run_processes_parallel_tr2(options->jobs,
 				   pick_next_hook,
 				   notify_start_failure,
 				   options->feed_pipe,
@@ -265,7 +289,11 @@ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 {
 	struct list_head *hooks;
 	int ret = 0;
-	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT;
+	/*
+	 * Turn on parallelism by default. Hooks which don't want it should
+	 * specify their options accordingly.
+	 */
+	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT_ASYNC;
 
 	if (!options)
 		options = &hook_opt_scratch;
hook.h +19 -5
index 7705e6a529..4f90228a0c 100644
@@ -43,6 +43,13 @@ struct run_hooks_opt
 	/* Args to be passed to each hook */
 	struct strvec args;
 
+	/*
+	 * Number of threads to parallelize across. Set to 0 to use the
+	 * 'hook.jobs' config or, if that config is unset, the number of cores
+	 * on the system.
+	 */
+	int jobs;
+
 	/* Resolve and run the "absolute_path(hook)" instead of
 	 * "hook". Used for "git worktree" hooks
 	 */
@@ -85,11 +92,6 @@ struct run_hooks_opt
 	int *invoked_hook;
 };
 
-#define RUN_HOOKS_OPT_INIT { \
-	.env = STRVEC_INIT, \
-	.args = STRVEC_INIT, \
-}
-
 /*
  * To specify a 'struct string_list', set 'run_hooks_opt.feed_pipe_ctx' to the
  * string_list and set 'run_hooks_opt.feed_pipe' to 'pipe_from_string_list()'.
@@ -111,6 +113,18 @@ struct hook_cb_data {
 	int *invoked_hook;
 };
 
+#define RUN_HOOKS_OPT_INIT_SYNC { \
+	.jobs = 1, \
+	.env = STRVEC_INIT, \
+	.args = STRVEC_INIT, \
+}
+
+#define RUN_HOOKS_OPT_INIT_ASYNC { \
+	.jobs = 0, \
+	.env = STRVEC_INIT, \
+	.args = STRVEC_INIT, \
+}
+
 void run_hooks_opt_clear(struct run_hooks_opt *o);
 
 /**
read-cache.c +1 -1
index 90099ca14d..fd2bc67667 100644
@@ -3068,7 +3068,7 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 {
 	int ret;
 	int was_full = !istate->sparse_index;
-	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_ASYNC;
 
 	ret = convert_to_sparse(istate);
 
refs.c +1 -1
index 73d4a93926..305a075746 100644
@@ -2062,7 +2062,7 @@ int ref_update_reject_duplicates(struct string_list *refnames,
 static int run_transaction_hook(struct ref_transaction *transaction,
 				const char *state)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
 	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
 	int ret = 0, i;
 
reset.c +2 -1
index 6499bc5127..b93fe6a783 100644
@@ -128,7 +128,8 @@ int reset_head(struct repository *r, struct object_id *oid, const char *action,
 					    reflog_head);
 	}
 	if (run_hook) {
-		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
+
 		strvec_pushl(&opt.args,
 			     oid_to_hex(orig ? orig : null_oid()),
 			     oid_to_hex(oid),
sequencer.c +2 -2
index f451e23d0c..f13a0cbfbe 100644
@@ -1148,7 +1148,7 @@ int update_head_with_reflog(const struct commit *old_head,
 static int run_rewrite_hook(const struct object_id *oldoid,
 			    const struct object_id *newoid)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
 	struct strbuf tmp = STRBUF_INIT;
 	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 	int code;
@@ -4522,7 +4522,7 @@ static int pick_commits(struct repository *r,
 		if (!stat(rebase_path_rewritten_list(), &st) &&
 				st.st_size > 0) {
 			struct child_process notes_cp = CHILD_PROCESS_INIT;
-			struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+			struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_ASYNC;
 
 			notes_cp.in = open(rebase_path_rewritten_list(), O_RDONLY);
 			notes_cp.git_cmd = 1;
transport.c +1 -1
index 4ca8fc0391..abf8785885 100644
@@ -1204,7 +1204,7 @@ static int run_pre_push_hook(struct transport *transport,
 			     struct ref *remote_refs)
 {
 	int ret = 0;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
 	struct ref *r;
 	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
 
Signature · 1 line
2.32.0.605.g8dce9f2422-goog
#551 lore raw

Re: [PATCH v2 2/6] hook: allow parallel hook execution

Quoted 10 lines · Emily Shaffer

In many cases, there's no reason not to allow hooks to execute in parallel. run_processes_parallel() is well-suited - it's a task queue that runs its housekeeping in series, which means users don't need to worry about thread safety on their callback data. True multithreaded execution with the async_* functions isn't necessary here. Synchronous hook execution can be achieved by only allowing 1 job to run at a time.

Teach run_hooks() to use that function for simple hooks which don't require stdin or capture of stderr.

Since run_hooks() has been using run_processes_parallel_tr2() already in the ab/config-based-hooks-base topic, the above description puzzled me quite a bit. I think you meant to say with this step you update the callers to pass either .jobs=1 or .jobs=0 in the hooks_opt, so that hooks _can_ run in parallel (as opposed to using hardcoded jobs=1 in run_hooks() when calling the underlying run_processes_parallel_tr2() function).

I do not think SYNC and ASYNC are great adjectives for what you are doing in this step. Wouldn't ASYNC mean "the caller tells the hooks to run, continues without waiting for them to finish"? In order to let more than one hooks to run at the same time, the caller has to continue without waiting for the first one it started so that it can start the second one before the first one finishes (otherwise these two hooks will not run at the same time), but ASYNC implys a lot more than that. After starting all of these hooks, the caller may continue doing things that are not even related to these spawned hooks if a hook is run asynchronously.

But I do not think that is what is happening here. After all, run_processes_parallel_tr2() will not return until all of the subprocesses are culled.

Perhaps the distinction you are trying to express is easier to convey to readers if you instead used the verb TO RUN with the adverb SERIALLY vs IN PARALLEL (as opposed to SYNCHRONOUSLY vs ASYNCHRONOUSLY)?

Quoted 14 lines
builtin/hook.c +1 -1
index 4d39c9e75e..12c9126032 100644
@@ -22,7 +22,7 @@ static const char * const builtin_hook_run_usage[] = {
 static int run(int argc, const char **argv, const char *prefix)
 {
 	int i;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SYNC;
 	int ignore_missing = 0;
 	const char *hook_name;
 	struct list_head *hooks;

builtin/hook.c static int run(int argc, const char **argv, const char *prefix) · line 39

Quoted 8 lines
builtin/hook.c +2 -0
@@ -32,6 +32,8 @@ static int run(int argc, const char **argv, const char *prefix)
 			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
 		OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"),
 			   N_("file to read into hooks' stdin")),
+		OPT_INTEGER('j', "jobs", &opt.jobs,
+			    N_("run up to <n> hooks simultaneously")),
 		OPT_END(),
 	};
 	int ret;

OK, so by default "git hook" runs 1 at a time but we can instruct to run the optimal number of jobs via "git hook -j0" etc.

Quoted 19 lines
hook.c +11 -0
index 80e150548c..37f682c6d8 100644
@@ -228,6 +228,28 @@ static int notify_hook_finished(int result,
 	return 0;
 }
 
+/*
+ * Determines how many jobs to use after we know we want to parallelize. First
+ * priority is the config 'hook.jobs' and second priority is the number of CPUs.
+ */
+static int configured_hook_jobs(void)
+{
+	/*
+	 * The config and the CPU count probably won't change during the process
+	 * lifetime, so cache the result in case we invoke multiple hooks during
+	 * one process.
+	 */

hook.c static int notify_hook_finished(int result, · line 251

Quoted 10 lines
hook.c +10 -0
+	static int jobs = 0;
+	if (jobs)
+		return jobs;
+
+	if (git_config_get_int("hook.jobs", &jobs))
+		/* if the config isn't set, fall back to CPU count. */
+		jobs = online_cpus();
+
+	return jobs;
+}

Not a suggestion to make this improvement as a part of this series, but there already are more than several codepaths doing essentially the same thing as above, taking their own configuration or environment variables and falling back to online_cpus().

It may be worth introducing a shared helper function so that this can become

	static int configured_hook_jobs(void)
	{
		static int jobs; /* no need for "= 0" */

		if (!jobs)
			jobs = default_parallelism("hook.jobs", NULL);
		return jobs;
	}

where the new helper takes the configuration and environment variable names.

Thanks.

#552 lore raw

Re: [PATCH v2 2/6] hook: allow parallel hook execution

Quoted 14 lines · Junio C Hamano
Quoted 10 lines · Emily Shaffer

In many cases, there's no reason not to allow hooks to execute in parallel. run_processes_parallel() is well-suited - it's a task queue that runs its housekeeping in series, which means users don't need to worry about thread safety on their callback data. True multithreaded execution with the async_* functions isn't necessary here. Synchronous hook execution can be achieved by only allowing 1 job to run at a time.

Teach run_hooks() to use that function for simple hooks which don't require stdin or capture of stderr.

Quoted 7 lines

Since run_hooks() has been using run_processes_parallel_tr2() already in the ab/config-based-hooks-base topic, the above description puzzled me quite a bit. I think you meant to say with this step you update the callers to pass either .jobs=1 or .jobs=0 in the hooks_opt, so that hooks _can_ run in parallel (as opposed to using hardcoded jobs=1 in run_hooks() when calling the underlying run_processes_parallel_tr2() function).

I think you are right, and this commit message is from a much older version of the series. Ok; I will reword it. I think it can be much simpler.

Quoted 12 lines

I do not think SYNC and ASYNC are great adjectives for what you are doing in this step. Wouldn't ASYNC mean "the caller tells the hooks to run, continues without waiting for them to finish"? In order to let more than one hooks to run at the same time, the caller has to continue without waiting for the first one it started so that it can start the second one before the first one finishes (otherwise these two hooks will not run at the same time), but ASYNC implys a lot more than that. After starting all of these hooks, the caller may continue doing things that are not even related to these spawned hooks if a hook is run asynchronously.

Quoted 8 lines

But I do not think that is what is happening here. After all, run_processes_parallel_tr2() will not return until all of the subprocesses are culled.

Perhaps the distinction you are trying to express is easier to convey to readers if you instead used the verb TO RUN with the adverb SERIALLY vs IN PARALLEL (as opposed to SYNCHRONOUSLY vs ASYNCHRONOUSLY)?

Good point. I think you are right and I'll change the init macros to SERIAL/PARALLEL instead.

Quoted 18 lines
Quoted 18 lines
builtin/hook.c +1 -1
index 4d39c9e75e..12c9126032 100644
@@ -22,7 +22,7 @@ static const char * const builtin_hook_run_usage[] = {
 static int run(int argc, const char **argv, const char *prefix)
 {
 	int i;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SYNC;
 	int ignore_missing = 0;
 	const char *hook_name;
 	struct list_head *hooks;
@@ -32,6 +32,8 @@ static int run(int argc, const char **argv, const char *prefix)
 			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
 		OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"),
 			   N_("file to read into hooks' stdin")),
Quoted 8 lines

builtin/hook.c static int run(int argc, const char **argv, const char *prefix) · line 39

Quoted 5 lines
builtin/hook.c +2 -0
+		OPT_INTEGER('j', "jobs", &opt.jobs,
+			    N_("run up to <n> hooks simultaneously")),
 		OPT_END(),
 	};
 	int ret;

OK, so by default "git hook" runs 1 at a time but we can instruct to run the optimal number of jobs via "git hook -j0" etc.

Is it surprising for an explicitly provided "-j0" to still be overridden by a config if one was provided? That is, "-j0" doesn't mean "-j$(nproc)" as it does in many other Unixy tools, here. It means "-j(hook.jobs OR $(nproc))". I guess you could also say that "if you configured hook.jobs, it is because you decided nproc was less optimal than the value you put into hook.jobs" - so maybe it is fine?

Quoted 48 lines
Quoted 20 lines
hook.c +11 -0
index 80e150548c..37f682c6d8 100644
@@ -228,6 +228,28 @@ static int notify_hook_finished(int result,
 	return 0;
 }
 
+/*
+ * Determines how many jobs to use after we know we want to parallelize. First
+ * priority is the config 'hook.jobs' and second priority is the number of CPUs.
+ */
+static int configured_hook_jobs(void)
+{
+	/*
+	 * The config and the CPU count probably won't change during the process
+	 * lifetime, so cache the result in case we invoke multiple hooks during
+	 * one process.
+	 */

hook.c static int notify_hook_finished(int result, · line 251

Quoted 10 lines
hook.c +10 -0
+	static int jobs = 0;
+	if (jobs)
+		return jobs;
+
+	if (git_config_get_int("hook.jobs", &jobs))
+		/* if the config isn't set, fall back to CPU count. */
+		jobs = online_cpus();
+
+	return jobs;
+}

Not a suggestion to make this improvement as a part of this series, but there already are more than several codepaths doing essentially the same thing as above, taking their own configuration or environment variables and falling back to online_cpus().

It may be worth introducing a shared helper function so that this can become

	static int configured_hook_jobs(void)
	{
		static int jobs; /* no need for "= 0" */

		if (!jobs)
			jobs = default_parallelism("hook.jobs", NULL);
		return jobs;
	}
Quoted 2 lines

where the new helper takes the configuration and environment variable names.

Sounds handy.

- Emily

#553 lore raw

PATCH v2 3/6 hook: introduce "git hook list"

In reply to Emily Shaffer

If more than one hook will be run, it may be useful to see a list of which hooks should be run. At very least, it will be useful for us to test the semantics of multihooks ourselves.

For now, only list the hooks which will run in the order they will run in; later, it might be useful to include more information like where the hooks were configured and whether or not they will run.

---
 builtin/hook.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++++
 hook.c         | 18 ++++++++----------
 2 files changed, 57 insertions(+), 10 deletions(-)
builtin/hook.c +49 -0
index 12c9126032..c36b05376c 100644
@@ -8,8 +8,11 @@
 
 #define BUILTIN_HOOK_RUN_USAGE \
 	N_("git hook run [--to-stdin=<path>] <hook-name> [-- <hook-args>]")
+#define BUILTIN_HOOK_LIST_USAGE \
+	N_("git hook list <hook-name>")
 
 static const char * const builtin_hook_usage[] = {
+	BUILTIN_HOOK_LIST_USAGE,
 	BUILTIN_HOOK_RUN_USAGE,
 	NULL
 };
@@ -19,6 +22,50 @@ static const char * const builtin_hook_run_usage[] = {
 	NULL
 };
 
+static const char *const builtin_hook_list_usage[] = {
+	BUILTIN_HOOK_LIST_USAGE,
+	NULL
+};
+
+static int list(int argc, const char **argv, const char *prefix)
+{
+	struct list_head *head, *pos;
+	const char *hookname = NULL;
+	struct strbuf hookdir_annotation = STRBUF_INIT;
+
+	struct option list_options[] = {
+		OPT_END(),
+	};
+
+	argc = parse_options(argc, argv, prefix, list_options,
+			     builtin_hook_list_usage, 0);
+
+	if (argc < 1)
+		usage_msg_opt(_("You must specify a hook event name to list."),
+			      builtin_hook_list_usage, list_options);
+
+	hookname = argv[0];
+
+	head = hook_list(hookname);
+
+	if (list_empty(head)) {
+		printf(_("no commands configured for hook '%s'\n"),
+		       hookname);
+		return 0;
+	}
+
+	list_for_each(pos, head) {
+		struct hook *item = list_entry(pos, struct hook, list);
+		item = list_entry(pos, struct hook, list);
+		if (item)
+			printf("%s\n", item->hook_path);
+	}
+
+	clear_hook_list(head);
+	strbuf_release(&hookdir_annotation);
+
+	return 0;
+}
 static int run(int argc, const char **argv, const char *prefix)
 {
 	int i;
@@ -88,6 +135,8 @@ int cmd_hook(int argc, const char **argv, const char *prefix)
 	if (!argc)
 		goto usage;
 
+	if (!strcmp(argv[0], "list"))
+		return list(argc, argv, prefix);
 	if (!strcmp(argv[0], "run"))
 		return run(argc, argv, prefix);
 
hook.c +8 -10
index 37f682c6d8..2714b63473 100644
@@ -96,22 +96,20 @@ int hook_exists(const char *name)
 struct list_head* hook_list(const char* hookname)
 {
 	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
+	const char *hook_path = find_hook(hookname);
+
 
 	INIT_LIST_HEAD(hook_head);
 
 	if (!hookname)
 		return NULL;
 
-	if (have_git_dir()) {
-		const char *hook_path = find_hook(hookname);
-
-		/* Add the hook from the hookdir */
-		if (hook_path) {
-			struct hook *to_add = xmalloc(sizeof(*to_add));
-			to_add->hook_path = hook_path;
-			to_add->feed_pipe_cb_data = NULL;
-			list_add_tail(&to_add->list, hook_head);
-		}
+	/* Add the hook from the hookdir */
+	if (hook_path) {
+		struct hook *to_add = xmalloc(sizeof(*to_add));
+		to_add->hook_path = hook_path;
+		to_add->feed_pipe_cb_data = NULL;
+		list_add_tail(&to_add->list, hook_head);
 	}
 
 	return hook_head;
Signature · 1 line
2.32.0.605.g8dce9f2422-goog
#554 lore raw

Re: [PATCH v2 3/6] hook: introduce "git hook list"

Quoted 12 lines · Emily Shaffer
builtin/hook.c +12 -0
+static int list(int argc, const char **argv, const char *prefix)
+{
+	struct list_head *head, *pos;
+	const char *hookname = NULL;
+	struct strbuf hookdir_annotation = STRBUF_INIT;
+
+	struct option list_options[] = {
+		OPT_END(),
+	};
+
+	argc = parse_options(argc, argv, prefix, list_options,
+			     builtin_hook_list_usage, 0);

builtin/hook.c static const char * const builtin_hook_run_usage[] = { · line 51

Quoted 10 lines
builtin/hook.c +10 -0
+
+	if (argc < 1)
+		usage_msg_opt(_("You must specify a hook event name to list."),
+			      builtin_hook_list_usage, list_options);
+
+	hookname = argv[0];
+
+	head = hook_list(hookname);
+
+	if (list_empty(head)) {

The same "can't hook_list() signal an error by returning NULL?" comment applies here.

	head = hook_list(hookname);
	if (!head)
		die(_("no such hook '%s'"), hookname);

or something?

builtin/hook.c static const char * const builtin_hook_run_usage[] = { · line 54

Quoted 3 lines
builtin/hook.c +3 -0
+		printf(_("no commands configured for hook '%s'\n"),
+		       hookname);
+		return 0;

If it is a normally expected state that there is no hook for the given name, signalling success by returning 0 here may be sensible, but then the message should at least go to the standard error stream to leave the standard output empty, so that a caller can reasonably do something like

	for path in $(git hooks list "$1")
	do
		ls -l "$path"
	done

If we really want to show such a message, perhaps

	if (list_empty(head)) {
        	if (!quiet)
			warning(_("no commands configured"));
		return 0;
	}

The normal display just shows the path without saying "command %s will run for hook %s"; the warning probably should do the same.

Having said that, if it truly is a normal and expected state that no hook is defined for the given name, I actually think there should be no message.

Quoted 32 lines
builtin/hook.c +8 -0
+	}
+
+	list_for_each(pos, head) {
+		struct hook *item = list_entry(pos, struct hook, list);
+		item = list_entry(pos, struct hook, list);
+		if (item)
+			printf("%s\n", item->hook_path);
+	}
hook.c +2 -7
index 37f682c6d8..2714b63473 100644
@@ -96,22 +96,20 @@ int hook_exists(const char *name)
 struct list_head* hook_list(const char* hookname)
 {
 	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
+	const char *hook_path = find_hook(hookname);
+
 
 	INIT_LIST_HEAD(hook_head);
 
 	if (!hookname)
 		return NULL;
 
-	if (have_git_dir()) {
-		const char *hook_path = find_hook(hookname);
-
-		/* Add the hook from the hookdir */
-		if (hook_path) {
-			struct hook *to_add = xmalloc(sizeof(*to_add));
-			to_add->hook_path = hook_path;

hook.c int hook_exists(const char *name) · line 113

Quoted 10 lines
hook.c +6 -3
-			to_add->feed_pipe_cb_data = NULL;
-			list_add_tail(&to_add->list, hook_head);
-		}
+	/* Add the hook from the hookdir */
+	if (hook_path) {
+		struct hook *to_add = xmalloc(sizeof(*to_add));
+		to_add->hook_path = hook_path;
+		to_add->feed_pipe_cb_data = NULL;
+		list_add_tail(&to_add->list, hook_head);
 	}

I do not think this belongs to the step to add "list" command. The log message does not explain or justify why have-git-dir goes away, either.

#555 lore raw

Re: [PATCH v2 3/6] hook: introduce "git hook list"

Quoted 58 lines · Junio C Hamano
Quoted 12 lines · Emily Shaffer
builtin/hook.c +12 -0
+static int list(int argc, const char **argv, const char *prefix)
+{
+	struct list_head *head, *pos;
+	const char *hookname = NULL;
+	struct strbuf hookdir_annotation = STRBUF_INIT;
+
+	struct option list_options[] = {
+		OPT_END(),
+	};
+
+	argc = parse_options(argc, argv, prefix, list_options,
+			     builtin_hook_list_usage, 0);

builtin/hook.c static const char * const builtin_hook_run_usage[] = { · line 51

Quoted 10 lines
builtin/hook.c +10 -0
+
+	if (argc < 1)
+		usage_msg_opt(_("You must specify a hook event name to list."),
+			      builtin_hook_list_usage, list_options);
+
+	hookname = argv[0];
+
+	head = hook_list(hookname);
+
+	if (list_empty(head)) {

The same "can't hook_list() signal an error by returning NULL?" comment applies here.

	head = hook_list(hookname);
	if (!head)
		die(_("no such hook '%s'"), hookname);

or something?

builtin/hook.c static const char * const builtin_hook_run_usage[] = { · line 54

Quoted 3 lines
builtin/hook.c +3 -0
+		printf(_("no commands configured for hook '%s'\n"),
+		       hookname);
+		return 0;

If it is a normally expected state that there is no hook for the given name, signalling success by returning 0 here may be sensible, but then the message should at least go to the standard error stream to leave the standard output empty, so that a caller can reasonably do something like

	for path in $(git hooks list "$1")
	do
		ls -l "$path"
	done

If we really want to show such a message, perhaps

	if (list_empty(head)) {
        	if (!quiet)
			warning(_("no commands configured"));
		return 0;
	}
Quoted 6 lines

The normal display just shows the path without saying "command %s will run for hook %s"; the warning probably should do the same.

Having said that, if it truly is a normal and expected state that no hook is defined for the given name, I actually think there should be no message.

Ah, I think you are saying "either return an error code and be chatty if you want, or return an empty list and a success code, but pick one". Makes sense to me.

No message + well-defined return code sounds fine. I'll do that.

Quoted 39 lines
Quoted 39 lines
builtin/hook.c +8 -0
+	}
+
+	list_for_each(pos, head) {
+		struct hook *item = list_entry(pos, struct hook, list);
+		item = list_entry(pos, struct hook, list);
+		if (item)
+			printf("%s\n", item->hook_path);
+	}
hook.c +5 -10
index 37f682c6d8..2714b63473 100644
@@ -96,22 +96,20 @@ int hook_exists(const char *name)
 struct list_head* hook_list(const char* hookname)
 {
 	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
+	const char *hook_path = find_hook(hookname);
+
 
 	INIT_LIST_HEAD(hook_head);
 
 	if (!hookname)
 		return NULL;
 
-	if (have_git_dir()) {
-		const char *hook_path = find_hook(hookname);
-
-		/* Add the hook from the hookdir */
-		if (hook_path) {
-			struct hook *to_add = xmalloc(sizeof(*to_add));
-			to_add->hook_path = hook_path;
-			to_add->feed_pipe_cb_data = NULL;
-			list_add_tail(&to_add->list, hook_head);
-		}
+	/* Add the hook from the hookdir */
+	if (hook_path) {
+		struct hook *to_add = xmalloc(sizeof(*to_add));
Quoted 8 lines

hook.c int hook_exists(const char *name) · line 113

Quoted 4 lines
hook.c +3 -0
+		to_add->hook_path = hook_path;
+		to_add->feed_pipe_cb_data = NULL;
+		list_add_tail(&to_add->list, hook_head);
 	}

I do not think this belongs to the step to add "list" command. The log message does not explain or justify why have-git-dir goes away, either.

Ah, sure.

It seems like I also didn't update the documentation for 'git hook' command during this commit. Will fix that as well.

#556 lore raw

PATCH v2 4/6 hook: allow running non-native hooks

In reply to Emily Shaffer

As the hook architecture and 'git hook run' become more featureful, we may find wrappers wanting to use the hook architecture to run their own hooks, thereby getting nice things like parallelism and idiomatic Git configuration for free. Enable this by letting 'git hook run' bypass the known_hooks() check.

We do still want to keep known_hooks() around, though - by die()ing when an internal Git call asks for run_hooks("my-new-hook"), we can remind Git developers to update Documentation/githooks.txt with their new hook, which in turn helps Git users discover this new hook.

---
 Documentation/git-hook.txt |  8 ++++++++
 builtin/hook.c             |  4 ++--
 hook.c                     | 32 ++++++++++++++++++++++++++++----
 hook.h                     | 16 +++++++++++++++-
 4 files changed, 53 insertions(+), 7 deletions(-)
Documentation/git-hook.txt +8 -0
index 8bf82b5dd4..11a8b87c60 100644
@@ -18,6 +18,14 @@ This command is an interface to git hooks (see linkgit:githooks[5]).
 Currently it only provides a convenience wrapper for running hooks for
 use by git itself. In the future it might gain other functionality.
 
+It's possible to use this command to refer to hooks which are not native to Git,
+for example if a wrapper around Git wishes to expose hooks into its own
+operation in a way which is already familiar to Git users. However, wrappers
+invoking such hooks should be careful to name their hook events something which
+Git is unlikely to use for a native hook later on. For example, Git is much less
+likely to create a `mytool-validate-commit` hook than it is to create a
+`validate-commit` hook.
+
 SUBCOMMANDS
 -----------
 
builtin/hook.c +2 -2
index c36b05376c..3aa65dd791 100644
@@ -46,7 +46,7 @@ static int list(int argc, const char **argv, const char *prefix)
 
 	hookname = argv[0];
 
-	head = hook_list(hookname);
+	head = hook_list(hookname, 1);
 
 	if (list_empty(head)) {
 		printf(_("no commands configured for hook '%s'\n"),
@@ -108,7 +108,7 @@ static int run(int argc, const char **argv, const char *prefix)
 	git_config(git_default_config, NULL);
 
 	hook_name = argv[0];
-	hooks = hook_list(hook_name);
+	hooks = hook_list(hook_name, 1);
 	if (list_empty(hooks)) {
 		/* ... act like run_hooks_oneshot() under --ignore-missing */
 		if (ignore_missing)
hook.c +28 -4
index 2714b63473..e5acd02a50 100644
@@ -52,12 +52,21 @@ static int known_hook(const char *name)
 
 const char *find_hook(const char *name)
 {
-	static struct strbuf path = STRBUF_INIT;
+	const char *hook_path;
 
 	if (!known_hook(name))
 		die(_("the hook '%s' is not known to git, should be in hook-list.h via githooks(5)"),
 		    name);
 
+	hook_path = find_hook_gently(name);
+
+	return hook_path;
+}
+
+const char *find_hook_gently(const char *name)
+{
+	static struct strbuf path = STRBUF_INIT;
+
 	strbuf_reset(&path);
 	strbuf_git_path(&path, "hooks/%s", name);
 	if (access(path.buf, X_OK) < 0) {
@@ -93,10 +102,16 @@ int hook_exists(const char *name)
 	return !!find_hook(name);
 }
 
-struct list_head* hook_list(const char* hookname)
+struct hook_config_cb
+{
+	struct strbuf *hook_key;
+	struct list_head *list;
+};
+
+struct list_head* hook_list(const char* hookname, int allow_unknown)
 {
 	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
-	const char *hook_path = find_hook(hookname);
+	const char *hook_path;
 
 
 	INIT_LIST_HEAD(hook_head);
@@ -104,6 +119,11 @@ struct list_head* hook_list(const char* hookname)
 	if (!hookname)
 		return NULL;
 
+	if (allow_unknown)
+		hook_path = find_hook_gently(hookname);
+	else
+		hook_path = find_hook(hookname);
+
 	/* Add the hook from the hookdir */
 	if (hook_path) {
 		struct hook *to_add = xmalloc(sizeof(*to_add));
@@ -299,7 +319,11 @@ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 	if (options->path_to_stdin && options->feed_pipe)
 		BUG("choose only one method to populate stdin");
 
-	hooks = hook_list(hook_name);
+	/*
+	 * 'git hooks run <hookname>' uses run_found_hooks, so we don't need to
+	 * allow unknown hooknames here.
+	 */
+	hooks = hook_list(hook_name, 0);
 
 	/*
 	 * If you need to act on a missing hook, use run_found_hooks()
hook.h +15 -1
index 4f90228a0c..ffa96c6e4d 100644
@@ -9,8 +9,16 @@
  * Returns the path to the hook file, or NULL if the hook is missing
  * or disabled. Note that this points to static storage that will be
  * overwritten by further calls to find_hook and run_hook_*.
+ *
+ * If the hook is not a native hook (e.g. present in Documentation/githooks.txt)
+ * find_hook() will die(). find_hook_gently() does not consult the native hook
+ * list and will check for any hook name in the hooks directory regardless of
+ * whether it is known. find_hook() should be used by internal calls to hooks,
+ * and find_hook_gently() should only be used when the hookname was provided by
+ * the user, such as by 'git hook run my-wrapper-hook'.
  */
 const char *find_hook(const char *name);
+const char *find_hook_gently(const char *name);
 
 /*
  * A boolean version of find_hook()
@@ -32,8 +40,14 @@ struct hook {
 /*
  * Provides a linked list of 'struct hook' detailing commands which should run
  * in response to the 'hookname' event, in execution order.
+ *
+ * If allow_unknown is unset, hooks will be checked against the hook list
+ * generated from Documentation/githooks.txt. Otherwise, any hook name will be
+ * allowed. allow_unknown should only be set when the hook name is provided by
+ * the user; internal calls to hook_list should make sure the hook they are
+ * invoking is present in Documentation/githooks.txt.
  */
-struct list_head* hook_list(const char *hookname);
+struct list_head* hook_list(const char *hookname, int allow_unknown);
 
 struct run_hooks_opt
 {
Signature · 1 line
2.32.0.605.g8dce9f2422-goog
#557 lore raw

Re: [PATCH v2 4/6] hook: allow running non-native hooks

Quoted 14 lines · Emily Shaffer
builtin/hook.c +1 -1
index c36b05376c..3aa65dd791 100644
@@ -46,7 +46,7 @@ static int list(int argc, const char **argv, const char *prefix)
 
 	hookname = argv[0];
 
-	head = hook_list(hookname);
+	head = hook_list(hookname, 1);
 
 	if (list_empty(head)) {
 		printf(_("no commands configured for hook '%s'\n"),

builtin/hook.c static int run(int argc, const char **argv, const char *prefix) · line 114

Quoted 8 lines
builtin/hook.c +1 -1
@@ -108,7 +108,7 @@ static int run(int argc, const char **argv, const char *prefix)
 	git_config(git_default_config, NULL);
 
 	hook_name = argv[0];
-	hooks = hook_list(hook_name);
+	hooks = hook_list(hook_name, 1);
 	if (list_empty(hooks)) {
 		/* ... act like run_hooks_oneshot() under --ignore-missing */
 		if (ignore_missing)

This is minor, as I expect that the callers of hook_list() will always confined in builtin/hook.c, but it probably is easier to read if you gave two functions, just like you have the pair of helpers find_hook() and find_hook_gently(), as the literal "1" forces the readers to remember if that means "die if not found", or "ok if it is a bogus name".

In addition, it may make more sense to keep hook_list() signal failure by returning NULL and leave the dying to the caller. In-code callers (as opposed to "git hook run" that can throw any random string that came from the user at the API) will never throw a bogus name unless there is a bug, and they'll have to check for an error return from hook_list() anyway and the error message they would give may have to be different from the one that is given against a hook name randomly thrown at us by the user.

#558 lore raw

Re: [PATCH v2 4/6] hook: allow running non-native hooks

Quoted 19 lines · Junio C Hamano
Quoted 16 lines · Emily Shaffer
builtin/hook.c +1 -1
index c36b05376c..3aa65dd791 100644
@@ -46,7 +46,7 @@ static int list(int argc, const char **argv, const char *prefix)
 
 	hookname = argv[0];
 
-	head = hook_list(hookname);
+	head = hook_list(hookname, 1);
 
 	if (list_empty(head)) {
 		printf(_("no commands configured for hook '%s'\n"),
@@ -108,7 +108,7 @@ static int run(int argc, const char **argv, const char *prefix)
 	git_config(git_default_config, NULL);
 
Quoted 13 lines

builtin/hook.c static int run(int argc, const char **argv, const char *prefix) · line 114

Quoted 6 lines
builtin/hook.c +1 -1
 	hook_name = argv[0];
-	hooks = hook_list(hook_name);
+	hooks = hook_list(hook_name, 1);
 	if (list_empty(hooks)) {
 		/* ... act like run_hooks_oneshot() under --ignore-missing */
 		if (ignore_missing)

This is minor, as I expect that the callers of hook_list() will always confined in builtin/hook.c, but it probably is easier to read if you gave two functions, just like you have the pair of helpers find_hook() and find_hook_gently(), as the literal "1" forces the readers to remember if that means "die if not found", or "ok if it is a bogus name".

Yes, I see what you mean. Ok. I have been wanting to change the naming anyways - most functions in hook.h are verb-y ("find hook", "run hooks", so on) but hook_list stands out as the only noun-y function.

So I considered changing it to "list_hooks" and "list_hooks_gently", to align with find_hook(_gently)....

Quoted 9 lines

In addition, it may make more sense to keep hook_list() signal failure by returning NULL and leave the dying to the caller. In-code callers (as opposed to "git hook run" that can throw any random string that came from the user at the API) will never throw a bogus name unless there is a bug, and they'll have to check for an error return from hook_list() anyway and the error message they would give may have to be different from the one that is given against a hook name randomly thrown at us by the user.

Sure, that makes sense enough... but then I wonder if it would be better to let the caller check whether the name is allowed at all, first, separately from the hook_list() call.

On the one hand, having hook_list() do the validation of the hook name makes it harder for a hook doing something very unusual to neglect to add documentation. (I'm thinking of, for example, a hook doing something equally weird to the proc-receive hook, which cannot use the hook library because it needs to be able to do this weird two-way communication thing.
(https://lore.kernel.org/git/20210527000856.695702-31-emilyshaffer%40google.com))
It would be pretty bad for a hook which is already complicated to also forget to include documentation.

On the other hand, as it is now - builtin/hook.c hardcodes "I don't care if the hook is unknown" and hook.c hardcodes "reject if the hook is unknown" and nobody else calls hook_list at all - it isn't so bad to bail early, before even calling hook_list() in the first place, if the hook is unknown.

I also think that approach would make a callsite easier to understand than checking for null from hook_list().

  const char *hookname = "my-new-hook";

/* Here it's pretty clear what the reason for the error was... */ if (!known_hook(hookname))

    BUG("is hook '%s' in Documentation/githooks.txt?", hookname);
  hooks = hook_list(hookname);
  ...

vs.

  const char *hookname = "my-new-hook";
  hooks = hook_list(hookname);
  /*
   * But here, I have to go and look at the hook_list() source to
   * understand why null 'hooks' means I missed some doc step.
   */
  if (!hookname)
    BUG("is hook '%s' in Documentation/githooks.txt?", hookname);
  ...

Maybe others disagree with me, but I would guess the first example is more easily understandable to someone unfamiliar with the hook code. So I think I will go with that approach, and include some notice in the doc comment over hook_list().

- Emily

#559 lore raw

Re: [PATCH v2 4/6] hook: allow running non-native hooks

Quoted 96 lines · Emily Shaffer
Quoted 19 lines · Junio C Hamano
Quoted 16 lines · Emily Shaffer
builtin/hook.c +1 -1
index c36b05376c..3aa65dd791 100644
@@ -46,7 +46,7 @@ static int list(int argc, const char **argv, const char *prefix)
 
 	hookname = argv[0];
 
-	head = hook_list(hookname);
+	head = hook_list(hookname, 1);
 
 	if (list_empty(head)) {
 		printf(_("no commands configured for hook '%s'\n"),
@@ -108,7 +108,7 @@ static int run(int argc, const char **argv, const char *prefix)
 	git_config(git_default_config, NULL);
 
Quoted 13 lines

builtin/hook.c static int run(int argc, const char **argv, const char *prefix) · line 114

Quoted 6 lines
builtin/hook.c +1 -1
 	hook_name = argv[0];
-	hooks = hook_list(hook_name);
+	hooks = hook_list(hook_name, 1);
 	if (list_empty(hooks)) {
 		/* ... act like run_hooks_oneshot() under --ignore-missing */
 		if (ignore_missing)

This is minor, as I expect that the callers of hook_list() will always confined in builtin/hook.c, but it probably is easier to read if you gave two functions, just like you have the pair of helpers find_hook() and find_hook_gently(), as the literal "1" forces the readers to remember if that means "die if not found", or "ok if it is a bogus name".

Yes, I see what you mean. Ok. I have been wanting to change the naming anyways - most functions in hook.h are verb-y ("find hook", "run hooks", so on) but hook_list stands out as the only noun-y function.

So I considered changing it to "list_hooks" and "list_hooks_gently", to align with find_hook(_gently)....

Quoted 9 lines

In addition, it may make more sense to keep hook_list() signal failure by returning NULL and leave the dying to the caller. In-code callers (as opposed to "git hook run" that can throw any random string that came from the user at the API) will never throw a bogus name unless there is a bug, and they'll have to check for an error return from hook_list() anyway and the error message they would give may have to be different from the one that is given against a hook name randomly thrown at us by the user.

Sure, that makes sense enough... but then I wonder if it would be better to let the caller check whether the name is allowed at all, first, separately from the hook_list() call.

On the one hand, having hook_list() do the validation of the hook name makes it harder for a hook doing something very unusual to neglect to add documentation. (I'm thinking of, for example, a hook doing something equally weird to the proc-receive hook, which cannot use the hook library because it needs to be able to do this weird two-way communication thing.
(https://lore.kernel.org/git/20210527000856.695702-31-emilyshaffer%40google.com))
It would be pretty bad for a hook which is already complicated to also forget to include documentation.

On the other hand, as it is now - builtin/hook.c hardcodes "I don't care if the hook is unknown" and hook.c hardcodes "reject if the hook is unknown" and nobody else calls hook_list at all - it isn't so bad to bail early, before even calling hook_list() in the first place, if the hook is unknown.

I also think that approach would make a callsite easier to understand than checking for null from hook_list().

  const char *hookname = "my-new-hook";

/* Here it's pretty clear what the reason for the error was... */ if (!known_hook(hookname))

    BUG("is hook '%s' in Documentation/githooks.txt?", hookname);
  hooks = hook_list(hookname);
  ...

vs.

  const char *hookname = "my-new-hook";
  hooks = hook_list(hookname);
  /*
   * But here, I have to go and look at the hook_list() source to
   * understand why null 'hooks' means I missed some doc step.
   */
  if (!hookname)
    BUG("is hook '%s' in Documentation/githooks.txt?", hookname);
  ...
Quoted 4 lines

Maybe others disagree with me, but I would guess the first example is more easily understandable to someone unfamiliar with the hook code. So I think I will go with that approach, and include some notice in the doc comment over hook_list().

Hm. Now that I sit to type it, I guess putting the onus on the strange-new-hook caller to also type "if known_hook()" is about the same as just expecting the strange-new-hook caller to know they are supposed to document their hook. Plus, known_hook() is static right now.

I think it still makes sense to BUG() instead of error() or die() in 'list_hooks()' (non-gently) - the failure of that call is a developer error, either in not having documented their hook correctly or in calling 'list_hooks()' instead of 'list_hooks_gently()' when they meant the latter. So I will not take the NULL return approach.

- Emily

#560 lore raw

Re: [PATCH v2 4/6] hook: allow running non-native hooks

Quoted 6 lines · Emily Shaffer
Quoted 6 lines

Yes, I see what you mean. Ok. I have been wanting to change the naming anyways - most functions in hook.h are verb-y ("find hook", "run hooks", so on) but hook_list stands out as the only noun-y function.

So I considered changing it to "list_hooks" and "list_hooks_gently", to align with find_hook(_gently)....

I do not claim I am goot at naming (or better than you at it anyway), but list-hooks sounds to me like it is calling printf() to show the hooks to the user, not computing a list of hooks and returning it to the caller.

Quoted 8 lines
Quoted 8 lines

I also think that approach would make a callsite easier to understand than checking for null from hook_list().

  const char *hookname = "my-new-hook";

/* Here it's pretty clear what the reason for the error was... */ if (!known_hook(hookname))

    BUG("is hook '%s' in Documentation/githooks.txt?", hookname);

Yes. The callsite becomes easier to understand, and it separates the responsibility between the helper to respond to "please give me list of defined hooks" and its caller that may react to the returned list with "ok, among these hooks, this does not look kosher for such and such reason, so I'd die/warn/error" much cleaner.

#561 lore raw

PATCH v2 5/6 hook: include hooks from the config

In reply to Emily Shaffer

Teach the hook.[hc] library to parse configs to populare the list of hooks to run for a given event.

Multiple commands can be specified for a given hook by providing multiple "hook.<friendly-name>.command = <path-to-hook>" and "hook.<friendly-name>.event = <hook-event>" lines. Hooks will be run in config order of the "hook.<name>.event" lines.

For example:

  $ git config --list | grep ^hook
  hook.bar.command=~/bar.sh
  hook.bar.event=pre-commit
  $ git hook run
  # Runs ~/bar.sh
  # Runs .git/hooks/pre-commit
---
 Documentation/config/hook.txt |   5 +
 Documentation/git-hook.txt    |  23 ++++-
 builtin/hook.c                |   5 +-
 hook.c                        | 166 +++++++++++++++++++++++++++++-----
 hook.h                        |   7 +-
 t/t1800-hook.sh               | 141 ++++++++++++++++++++++++++++-
 6 files changed, 318 insertions(+), 29 deletions(-)
Documentation/config/hook.txt +5 -0
index 96d3d6572c..a97b980cca 100644
@@ -1,3 +1,8 @@
+hook.<command>.command::
+	A command to execute during the <command> hook event. This can be an
+	executable on your device, a oneliner for your shell, or the name of a
+	hookcmd. See linkgit:git-hook[1].
+
 hook.jobs::
 	Specifies how many hooks can be run simultaneously during parallelized
 	hook execution. If unspecified, defaults to the number of processors on
Documentation/git-hook.txt +21 -2
index 11a8b87c60..c610ed9583 100644
@@ -26,12 +26,31 @@ Git is unlikely to use for a native hook later on. For example, Git is much less
 likely to create a `mytool-validate-commit` hook than it is to create a
 `validate-commit` hook.
 
+This command parses the default configuration files for pairs of configs like
+so:
+
+  [hook "linter"]
+    event = pre-commit
+    command = ~/bin/linter --c
+
+Conmmands are run in the order Git encounters their associated
+`hook.<name>.event` configs during the configuration parse (see
+linkgit:git-config[1]).
+
+In general, when instructions suggest adding a script to
+`.git/hooks/<hook-event>`, you can specify it in the config instead by running
+`git config --add hook.<some-name>.command <path-to-script> && git config --add
+hook.<some-name>.event <hook-event>` - this way you can share the script between
+multiple repos. That is, `cp ~/my-script.sh ~/project/.git/hooks/pre-commit`
+would become `git config --add hook.my-script.command ~/my-script.sh && git
+config --add hook.my-script.event pre-commit`.
+
 SUBCOMMANDS
 -----------
 
 run::
-	Run the `<hook-name>` hook. See linkgit:githooks[5] for
-	the hook names we support.
+	Runs hooks configured for `<hook-name>`, in the order they are
+	discovered during the config parse.
 +
 Any positional arguments to the hook should be passed after an
 optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The
builtin/hook.c +3 -2
index 3aa65dd791..ea49dc4ef6 100644
@@ -49,7 +49,7 @@ static int list(int argc, const char **argv, const char *prefix)
 	head = hook_list(hookname, 1);
 
 	if (list_empty(head)) {
-		printf(_("no commands configured for hook '%s'\n"),
+		printf(_("no hooks configured for event '%s'\n"),
 		       hookname);
 		return 0;
 	}
@@ -58,7 +58,8 @@ static int list(int argc, const char **argv, const char *prefix)
 		struct hook *item = list_entry(pos, struct hook, list);
 		item = list_entry(pos, struct hook, list);
 		if (item)
-			printf("%s\n", item->hook_path);
+			printf("%s\n", item->name ? item->name
+						  : _("hook from hookdir"));
 	}
 
 	clear_hook_list(head);
hook.c +144 -22

214 lines not on this page · raw message

hook.h +5 -2
index ffa96c6e4d..a6263864b9 100644
@@ -27,8 +27,11 @@ int hook_exists(const char *hookname);
 
 struct hook {
 	struct list_head list;
-	/* The path to the hook */
-	const char *hook_path;
+	/*
+	 * The friendly name of the hook. NULL indicates the hook is from the
+	 * hookdir.
+	 */
+	const char *name;
 
 	/*
 	 * Use this to keep state for your feed_pipe_fn if you are using
t/t1800-hook.sh +140 -1

157 lines not on this page · raw message

Signature · 1 line
2.32.0.605.g8dce9f2422-goog
#562 lore raw

Re: [PATCH v2 5/6] hook: include hooks from the config

Quoted 8 lines · Emily Shaffer

Teach the hook.[hc] library to parse configs to populare the list of hooks to run for a given event.

Multiple commands can be specified for a given hook by providing multiple "hook.<friendly-name>.command = <path-to-hook>" and "hook.<friendly-name>.event = <hook-event>" lines. Hooks will be run in config order of the "hook.<name>.event" lines.

Quoted 5 lines

For example:

  $ git config --list | grep ^hook
  hook.bar.command=~/bar.sh
  hook.bar.event=pre-commit

Your answer might be "read the design doc", but it is unclear to me why "bar" (friendly-name) is needed in this picture at all. Is it because you may want to fire more than one command for pre-commit event? IOW,

[hook "bar"]
command = bar1.sh
command = bar2.sh
event = pre-commit

is easier to manage with an extra level of redirection? I doubt it as

[hook "pre-commit"]
command = bar1.sh
command = bar2.sh

would be equally expressive and shorter. Or would it help use case for multiple "friendly-name" to refer to the same "event", e.g.

[hook "xyzzy"]
event = pre-commit
command = xyzzy1

[hook "frotz"]
event = pre-commit
command = frotz1
command = frotz2

or something? I am not sure if this gives us useful extra flexibility, and if so, the extra flexibility helps us more than it confuses us.

And moving the "event" to the second level in the configuration hierarchy, getting rid of "friendly-name" from the design, would not make this example unworkable, either:

Quoted 3 lines
  $ git hook run
  # Runs ~/bar.sh
  # Runs .git/hooks/pre-commit

Again, this is not an objection wrapped in a rhetorical question. It just is that I do not see how the extra level of redirection helps us.

Documentation/config/hook.txt line 4

Quoted 9 lines
Documentation/config/hook.txt +4 -0
index 96d3d6572c..a97b980cca 100644
@@ -1,3 +1,8 @@
+hook.<command>.command::
+	A command to execute during the <command> hook event. This can be an
+	executable on your device, a oneliner for your shell, or the name of a
+	hookcmd. See linkgit:git-hook[1].

Please make sure you use the terminology consistently. If the second level is "friendly name", hook.<name>.command should be described, instead of hook.<command>.command.

Also, to help those who are familiar with the current Git from their use in the past 10 years or so, giving an example name from the current system may help, e.g. when describing hook.<name>.event, you may want to say the values are things like "pre-commit", "receive", etc.

Documentation/config/hook.txt

Quoted 6 lines
Documentation/config/hook.txt +6 -0
+This command parses the default configuration files for pairs of configs like
+so:
+
+  [hook "linter"]
+    event = pre-commit
+    command = ~/bin/linter --c

The above addition of .command should also have hook.<name>.event next to it, no?

Quoted 1 line

+Conmmands are run in the order Git encounters their associated

"Conmmands -> Commands", I would think.

Quoted 2 lines

+hook.<name>.event configs during the configuration parse (see +linkgit:git-config[1]).

Here you use <name>, which should be matched by the description in the first hunk of the patch to this file.

Documentation/config/hook.txt

Quoted 7 lines
Documentation/config/hook.txt +7 -0
+In general, when instructions suggest adding a script to
+`.git/hooks/<hook-event>`, you can specify it in the config instead by running
+`git config --add hook.<some-name>.command <path-to-script> && git config --add
+hook.<some-name>.event <hook-event>` - this way you can share the script between
+multiple repos. That is, `cp ~/my-script.sh ~/project/.git/hooks/pre-commit`
+would become `git config --add hook.my-script.command ~/my-script.sh && git
+config --add hook.my-script.event pre-commit`.

One repository may use a friendly name "xyzzy" while the other may use "frotz" to group the hooks that trigger upon "pre-commit" event, but unless one of the repositories change the friendly name to match, they cannot share these configurations, no? It seems that an extra level of indirection is hindering sharing, rather than helping.

Quoted 13 lines
builtin/hook.c +1 -1
index 3aa65dd791..ea49dc4ef6 100644
@@ -49,7 +49,7 @@ static int list(int argc, const char **argv, const char *prefix)
 	head = hook_list(hookname, 1);
 
 	if (list_empty(head)) {
-		printf(_("no commands configured for hook '%s'\n"),
+		printf(_("no hooks configured for event '%s'\n"),
 		       hookname);
...

builtin/hook.c static int list(int argc, const char **argv, const char *prefix) · line 63

Quoted 7 lines
builtin/hook.c +2 -1
@@ -58,7 +58,8 @@ static int list(int argc, const char **argv, const char *prefix)
 		struct hook *item = list_entry(pos, struct hook, list);
 		item = list_entry(pos, struct hook, list);
 		if (item)
-			printf("%s\n", item->hook_path);
+			printf("%s\n", item->name ? item->name
+						  : _("hook from hookdir"));
 	}

I won't comment on this part as my comments on earlier patches would probably have butchered the preimage already for this change to survive intact ;-)

#563 lore raw

Re: [PATCH v2 5/6] hook: include hooks from the config

Quoted 54 lines · Junio C Hamano
Quoted 8 lines · Emily Shaffer

Teach the hook.[hc] library to parse configs to populare the list of hooks to run for a given event.

Multiple commands can be specified for a given hook by providing multiple "hook.<friendly-name>.command = <path-to-hook>" and "hook.<friendly-name>.event = <hook-event>" lines. Hooks will be run in config order of the "hook.<name>.event" lines.

Quoted 5 lines

For example:

  $ git config --list | grep ^hook
  hook.bar.command=~/bar.sh
  hook.bar.event=pre-commit

Your answer might be "read the design doc", but it is unclear to me why "bar" (friendly-name) is needed in this picture at all. Is it because you may want to fire more than one command for pre-commit event? IOW,

[hook "bar"]
command = bar1.sh
command = bar2.sh
event = pre-commit

is easier to manage with an extra level of redirection? I doubt it as

[hook "pre-commit"]
command = bar1.sh
command = bar2.sh

would be equally expressive and shorter. Or would it help use case for multiple "friendly-name" to refer to the same "event", e.g.

[hook "xyzzy"]
event = pre-commit
command = xyzzy1

[hook "frotz"]
event = pre-commit
command = frotz1
command = frotz2

or something? I am not sure if this gives us useful extra flexibility, and if so, the extra flexibility helps us more than it confuses us.

And moving the "event" to the second level in the configuration hierarchy, getting rid of "friendly-name" from the design, would not make this example unworkable, either:

Quoted 7 lines
Quoted 3 lines
  $ git hook run
  # Runs ~/bar.sh
  # Runs .git/hooks/pre-commit

Again, this is not an objection wrapped in a rhetorical question. It just is that I do not see how the extra level of redirection helps us.

Please have a look at
https://lore.kernel.org/git/87fswey5wd.fsf%40evledraar.gmail.com and replies, where Ævar and I discussed the schema change at length. I know it is a lot of back and forth but I think it is useful to understand why I ended up changing the schema this way.

Quoted 14 lines

Documentation/config/hook.txt line 4

Quoted 10 lines
Documentation/config/hook.txt +4 -0
index 96d3d6572c..a97b980cca 100644
@@ -1,3 +1,8 @@
+hook.<command>.command::
+	A command to execute during the <command> hook event. This can be an
+	executable on your device, a oneliner for your shell, or the name of a
+	hookcmd. See linkgit:git-hook[1].

Please make sure you use the terminology consistently. If the second level is "friendly name", hook.<name>.command should be described, instead of hook.<command>.command.

Thanks, this is an oversight. Will update the config/hook.txt doc in next reroll.

Quoted 6 lines

Also, to help those who are familiar with the current Git from their use in the past 10 years or so, giving an example name from the current system may help, e.g. when describing hook.<name>.event, you may want to say the values are things like "pre-commit", "receive", etc.

Sure.

Quoted 10 lines

Documentation/config/hook.txt

Quoted 7 lines
Documentation/config/hook.txt +6 -0
+This command parses the default configuration files for pairs of configs like
+so:
+
+  [hook "linter"]
+    event = pre-commit
+    command = ~/bin/linter --c

The above addition of .command should also have hook.<name>.event next to it, no?

I don't understand the question. Doesn't this config snippet equate to """
hook.linter.event=pre-commit hook.linter.command=~/bin/linter --c
"""
? So in this case, '<name>' is 'linter', as that's not a native Git hook.

Quoted 4 lines
Quoted 2 lines

+Conmmands are run in the order Git encounters their associated

"Conmmands -> Commands", I would think.

ACK

Quoted 6 lines
Quoted 3 lines

+hook.<name>.event configs during the configuration parse (see +linkgit:git-config[1]).

Here you use <name>, which should be matched by the description in the first hunk of the patch to this file.

Yep.

Quoted 15 lines

Documentation/config/hook.txt

Quoted 8 lines
Documentation/config/hook.txt +7 -0
+In general, when instructions suggest adding a script to
+`.git/hooks/<hook-event>`, you can specify it in the config instead by running
+`git config --add hook.<some-name>.command <path-to-script> && git config --add
+hook.<some-name>.event <hook-event>` - this way you can share the script between
+multiple repos. That is, `cp ~/my-script.sh ~/project/.git/hooks/pre-commit`
+would become `git config --add hook.my-script.command ~/my-script.sh && git
+config --add hook.my-script.event pre-commit`.

One repository may use a friendly name "xyzzy" while the other may use "frotz" to group the hooks that trigger upon "pre-commit" event, but unless one of the repositories change the friendly name to match, they cannot share these configurations, no? It seems that an extra level of indirection is hindering sharing, rather than helping.

Ah, I think this means the documentation isn't sufficient, if you are asking that. Instead of explaining it in ephemeral email, I think it is better for me to explain it in documentation reroll, and for you to then tell me how you interpret it. I expect to send the reroll before I go home today, since I didn't receive comments from anybody besides you so far.

Thanks very much for the feedback on the doc - this is very useful.

- Emily

#564 lore raw

PATCH v2 6/6 hook: allow out-of-repo 'git hook' invocations

In reply to Emily Shaffer

Since hooks can now be supplied via the config, and a config can be present without a gitdir via the global and system configs, we can start to allow 'git hook run' to occur without a gitdir. This enables us to do things like run sendemail-validate hooks when running 'git send-email' from a nongit directory.

It still doesn't make sense to look for hooks in the hookdir in nongit repos, though, as there is no hookdir.

---

Notes:
For hookdir hooks, do we want to run them in nongit dir when core.hooksPath is set? For example, if someone set core.hooksPath in their global config and then ran 'git hook run sendemail-validate' in a nongit dir?

 git.c           |  2 +-
 hook.c          |  2 +-
 t/t1800-hook.sh | 20 +++++++++++++++-----
 3 files changed, 17 insertions(+), 7 deletions(-)
git.c +1 -1
index 540909c391..39988ee3b0 100644
@@ -538,7 +538,7 @@ static struct cmd_struct commands[] = {
 	{ "grep", cmd_grep, RUN_SETUP_GENTLY },
 	{ "hash-object", cmd_hash_object },
 	{ "help", cmd_help },
-	{ "hook", cmd_hook, RUN_SETUP },
+	{ "hook", cmd_hook, RUN_SETUP_GENTLY },
 	{ "index-pack", cmd_index_pack, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "init", cmd_init_db },
 	{ "init-db", cmd_init_db },
hook.c +1 -1
index 51ada266bc..87d57f4118 100644
@@ -215,7 +215,7 @@ struct list_head* hook_list(const char *hookname, int allow_unknown)
 
 	/* Add the hook from the hookdir. The placeholder makes it easier to
 	 * allocate work in pick_next_hook. */
-	if (find_hook_gently(hookname))
+	if (have_git_dir() && find_hook_gently(hookname))
 		append_or_move_hook(hook_head, NULL);
 
 	return hook_head;
t/t1800-hook.sh +15 -5
index ef2432f53a..a7e45c0d16 100755
@@ -118,15 +118,25 @@ test_expect_success 'git hook run -- pass arguments' '
 	test_cmp expect actual
 '
 
-test_expect_success 'git hook run -- out-of-repo runs excluded' '
-	write_script .git/hooks/test-hook <<-EOF &&
-	echo Test hook
-	EOF
+test_expect_success 'git hook run: out-of-repo runs execute global hooks' '
+	test_config_global hook.global-hook.event test-hook --add &&
+	test_config_global hook.global-hook.command "echo no repo no problems" --add &&
+
+	echo "global-hook" >expect &&
+	nongit git hook list test-hook >actual &&
+	test_cmp expect actual &&
+
+	echo "no repo no problems" >expect &&
 
-	nongit test_must_fail git hook run test-hook
+	nongit git hook run test-hook 2>actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'git -c core.hooksPath=<PATH> hook run' '
+	write_script .git/hooks/test-hook <<-EOF &&
+	echo Test hook
+	EOF
+
 	mkdir my-hooks &&
 	write_script my-hooks/test-hook <<-\EOF &&
 	echo Hook ran $1 >>actual
Signature · 1 line
2.32.0.605.g8dce9f2422-goog
#565 lore raw

Re: [PATCH v2 0/6] config-based hooks restarted

In reply to Emily Shaffer
Quoted 2 lines · Emily Shaffer

This is the config-based hooks topic rebased onto v4 of Ævar's branch[1].

I have [1] in my tree, but these patches do not seem to apply cleanly; I see a failure in "git hooks list" step.

Should I perhaps try merging [1] to one of the v2.33-rc and then queue these on top?

#566 lore raw

Re: [PATCH v2 0/6] config-based hooks restarted

Quoted 10 lines · Junio C Hamano
Quoted 2 lines · Emily Shaffer

This is the config-based hooks topic rebased onto v4 of Ævar's branch[1].

I have [1] in my tree, but these patches do not seem to apply cleanly; I see a failure in "git hooks list" step.

Should I perhaps try merging [1] to one of the v2.33-rc and then queue these on top?

Ah, I figured it out. Your 3/6 seems to be stale wrt 4787177b (hook: support passing stdin to hooks, 2021-08-03) that came from
https://lore.kernel.org/git/[email protected]/

That patch did this to the early part of builtin/hook.c:

        diff --git a/builtin/hook.c b/builtin/hook.c
        index f33db9953c..27dce6a2f0 100644
        --- a/builtin/hook.c
        +++ b/builtin/hook.c
        @@ -7,7 +7,7 @@
         #include "strvec.h"

         #define BUILTIN_HOOK_RUN_USAGE \
        -	N_("git hook run [--ignore-missing] <hook-name> [-- <hook-args>]")
        +	N_("git hook run [--ignore-missing] [--to-stdin=<path>] <hook-name> [-- <hook-args>]")

         static const char * const builtin_hook_usage[] = {
                BUILTIN_HOOK_RUN_USAGE,

but your [v2 3/6] here expects something entirely different.

        diff --git a/builtin/hook.c b/builtin/hook.c
        index 12c9126032..c36b05376c 100644
        --- a/builtin/hook.c
        +++ b/builtin/hook.c
        @@ -8,8 +8,11 @@

         #define BUILTIN_HOOK_RUN_USAGE \
                N_("git hook run [--to-stdin=<path>] <hook-name> [-- <hook-args>]")
        +#define BUILTIN_HOOK_LIST_USAGE \
        +	N_("git hook list <hook-name>")

I've wiggled the patch in, as there wasn't any other funny inconsistency like this one, but please double check the result after I push it out perhaps tomorrow morning.

Thanks.

#567 lore raw

Re: [PATCH v2 0/6] config-based hooks restarted

Quoted 11 lines · Junio C Hamano
Quoted 8 lines · Junio C Hamano
Quoted 2 lines · Emily Shaffer

This is the config-based hooks topic rebased onto v4 of Ævar's branch[1].

I have [1] in my tree, but these patches do not seem to apply cleanly; I see a failure in "git hooks list" step.

Quoted 6 lines
Quoted 2 lines

Should I perhaps try merging [1] to one of the v2.33-rc and then queue these on top?

Ah, I figured it out. Your 3/6 seems to be stale wrt 4787177b (hook: support passing stdin to hooks, 2021-08-03) that came from
https://lore.kernel.org/git/[email protected]/

Ah, I am sorry!

Quoted 30 lines

That patch did this to the early part of builtin/hook.c:

        diff --git a/builtin/hook.c b/builtin/hook.c
        index f33db9953c..27dce6a2f0 100644
        --- a/builtin/hook.c
        +++ b/builtin/hook.c
        @@ -7,7 +7,7 @@
         #include "strvec.h"

         #define BUILTIN_HOOK_RUN_USAGE \
        -	N_("git hook run [--ignore-missing] <hook-name> [-- <hook-args>]")
        +	N_("git hook run [--ignore-missing] [--to-stdin=<path>] <hook-name> [-- <hook-args>]")

         static const char * const builtin_hook_usage[] = {
                BUILTIN_HOOK_RUN_USAGE,

but your [v2 3/6] here expects something entirely different.

        diff --git a/builtin/hook.c b/builtin/hook.c
        index 12c9126032..c36b05376c 100644
        --- a/builtin/hook.c
        +++ b/builtin/hook.c
        @@ -8,8 +8,11 @@

         #define BUILTIN_HOOK_RUN_USAGE \
                N_("git hook run [--to-stdin=<path>] <hook-name> [-- <hook-args>]")
        +#define BUILTIN_HOOK_LIST_USAGE \
        +	N_("git hook list <hook-name>")
Quoted 3 lines

I've wiggled the patch in, as there wasn't any other funny inconsistency like this one, but please double check the result after I push it out perhaps tomorrow morning.

I'll take a look and update local accordingly. Sorry for the trouble.

- Emily

#568 lore raw

v3 cover config-based hooks restarted

In reply to Emily Shaffer

This is the config-based hooks topic rebased onto v4 of Ævar's branch[1]. There is a happy CI build of it on GitHub[2].

The topic overall adds the ability to set up hooks by modifying the config, in addition to placing specially named hooks into the hookdir. This enables users to specify multiple hooks for a given event, and so this topic also fleshes out the use of the run_processes_parallel() API which is now introduced in Ævar's reordering of prior patches.

Patches 1-4 make some minor changes to prepare Ævar's series to handle more than one hook at a time. With the exception of patch 4, there should be no behavior change for existing hooks.

Patch 2 is opinionated about which hooks should and shouldn't be allowed to run in parallel; if you care about a specific hook, please take a look there.

Patch 5 is the motivating feature - it begins to parse the config looking for hooks.

Patch 6 takes advantage of the decoupling of hooks and GITDIR to allow out-of-repo hook runs, which would only run hooks specified in the system or global config. This mainly targets 'sendemail-validate', but 'git-sendemail.perl' still explicitly disallows out-of-repo hook execution on that hook for now. (Maybe that change should be added in this series? Or maybe patch 6 belongs with that kind of change?)

Since v2, I have addressed comments left by Junio - thanks for the review. Mostly these were small nits, but two somewhat larger changes came out:

- 'hook_list' becomes 'list_hooks' and 'list_hooks_gently'. It might still be better to name it something else, and it might still be better to decouple the "is this a known hook" check from the "give me all the hooks" function; comments welcome.
- The documentation in patch 5, which starts using the config, has been expanded to hopefully become more clear. I am especially interested in feedback in this doc change, as it's the main place users should be able to learn how to use the new feature.

Everything else should be pretty minor.

Right now I'm trying to focus on this series first and foremost, hence sending two rerolls based on the same version of Ævar's base restart. I'll try to perform a code review on Ævar's latest tomorrow.

Thanks!
- Emily

1: https://lore.kernel.org/git/cover-v4-00.36-00000000000-20210803T191505Z-avarab%40gmail.com
2: https://github.com/nasamuffin/git/actions/runs/1145128560

Emily Shaffer (6):
hook: run a list of hooks instead
hook: allow parallel hook execution
hook: introduce "git hook list"
hook: allow running non-native hooks
hook: include hooks from the config
hook: allow out-of-repo 'git hook' invocations

 Documentation/config/hook.txt |  22 +++
 Documentation/git-hook.txt    |  87 +++++++++-
 builtin/am.c                  |   4 +-
 builtin/checkout.c            |   2 +-
 builtin/clone.c               |   2 +-
 builtin/hook.c                |  65 +++++++-
 builtin/merge.c               |   2 +-
 builtin/rebase.c              |   2 +-
 builtin/receive-pack.c        |   9 +-
 builtin/worktree.c            |   2 +-
 commit.c                      |   2 +-
 git.c                         |   2 +-
 hook.c                        | 293 +++++++++++++++++++++++++++++-----
 hook.h                        |  61 +++++--
 read-cache.c                  |   2 +-
 refs.c                        |   2 +-
 reset.c                       |   3 +-
 sequencer.c                   |   4 +-
 t/t1800-hook.sh               | 161 ++++++++++++++++++-
 transport.c                   |   2 +-
 20 files changed, 648 insertions(+), 81 deletions(-)
 create mode 100644 Documentation/config/hook.txt

Range-diff against v2:

 1:  81fe1ed90d <  -:  ---------- Makefile: mark "check" target as .PHONY
 2:  b32abc81fb <  -:  ---------- Makefile: stop hardcoding {command,config}-list.h
 3:  fa46bd1154 <  -:  ---------- Makefile: remove an out-of-date comment
 4:  715fd63089 <  -:  ---------- hook.[ch]: move find_hook() to this new library
 5:  ac6c018d27 <  -:  ---------- hook.c: add a hook_exists() wrapper and use it in bugreport.c
 6:  53619d87a2 <  -:  ---------- hook.c users: use "hook_exists()" insted of "find_hook()"
 7:  39dbb89620 <  -:  ---------- hook-list.h: add a generated list of hooks, like config-list.h
 8:  7d2c1cec74 <  -:  ---------- hook: add 'run' subcommand
 9:  3fea1cdaf8 <  -:  ---------- gc: use hook library for pre-auto-gc hook
10:  c86a9c6f95 <  -:  ---------- rebase: convert pre-rebase to use hook.h
11:  31dff1f274 <  -:  ---------- am: convert applypatch to use hook.h
12:  a2e27d745c <  -:  ---------- hooks: convert 'post-checkout' hook to hook library
13:  1a71fa8b75 <  -:  ---------- merge: convert post-merge to use hook.h
14:  95e19fa468 <  -:  ---------- git hook run: add an --ignore-missing flag
15:  371f581e3b <  -:  ---------- send-email: use 'git hook run' for 'sendemail-validate'
16:  99306a4bd3 <  -:  ---------- git-p4: use 'git hook' to run hooks
17:  59d1a563ca <  -:  ---------- commit: convert {pre-commit,prepare-commit-msg} hook to hook.h
18:  7c8de04707 <  -:  ---------- read-cache: convert post-index-change to use hook.h
19:  a2e7176609 <  -:  ---------- receive-pack: convert push-to-checkout hook to hook.h
20:  f03ef24df7 <  -:  ---------- run-command: remove old run_hook_{le,ve}() hook API
21:  18c1dfe677 <  -:  ---------- run-command: allow stdin for run_processes_parallel
22:  eff713fc19 <  -:  ---------- hook: support passing stdin to hooks
23:  2cbec65834 <  -:  ---------- am: convert 'post-rewrite' hook to hook.h
24:  5d34d18bee <  -:  ---------- run-command: add stdin callback for parallelization
25:  b989d910ed <  -:  ---------- hook: provide stdin by string_list or callback
26:  ce6929493b <  -:  ---------- hook: convert 'post-rewrite' hook in sequencer.c to hook.h
27:  13affda962 <  -:  ---------- transport: convert pre-push hook to hook.h
28:  c74720029a <  -:  ---------- hook tests: test for exact "pre-push" hook input
29:  1252a251bb <  -:  ---------- hook tests: use a modern style for "pre-push" tests
30:  3f2488fea6 <  -:  ---------- reference-transaction: use hook.h to run hooks
31:  2e78c47ead <  -:  ---------- run-command: allow capturing of collated output
32:  95132acbb0 <  -:  ---------- hooks: allow callers to capture output
33:  f2510ca910 <  -:  ---------- receive-pack: convert 'update' hook to hook.h
34:  8686308864 <  -:  ---------- post-update: use hook.h library
35:  b10f5142a0 <  -:  ---------- receive-pack: convert receive hooks to hook.h
36:  5c29c932c3 <  -:  ---------- hooks: fix a TOCTOU in "did we run a hook?" heuristic
37:  5177e8ba2c !  1:  6d6400329c hook: run a list of hooks instead
    @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
     -		return run_hooks_oneshot(hook_name, &opt);
     -	hook_path = find_hook(hook_name);
     -	if (!hook_path) {
    -+	hooks = hook_list(hook_name);
    ++	hooks = list_hooks(hook_name);
     +	if (list_empty(hooks)) {
     +		/* ... act like run_hooks_oneshot() under --ignore-missing */
     +		if (ignore_missing)
    @@ hook.c
      
     +static void free_hook(struct hook *ptr)
     +{
    -+	if (ptr) {
    ++	if (ptr)
     +		free(ptr->feed_pipe_cb_data);
    -+	}
     +	free(ptr);
     +}
     +
    @@ hook.c
      static int known_hook(const char *name)
      {
      	const char **p;
    -@@ hook.c: int hook_exists(const char *name)
    - 	return !!find_hook(name);
    - }
    +@@ hook.c: const char *find_hook(const char *name)
      
    -+struct list_head* hook_list(const char* hookname)
    + int hook_exists(const char *name)
    + {
    +-	return !!find_hook(name);
    ++	return !list_empty(list_hooks(name));
    ++}
    ++
    ++struct list_head *list_hooks(const char *hookname)
     +{
     +	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
     +
     +	INIT_LIST_HEAD(hook_head);
     +
     +	if (!hookname)
    -+		return NULL;
    ++		BUG("null hookname was provided to hook_list()!");
     +
     +	if (have_git_dir()) {
     +		const char *hook_path = find_hook(hookname);
    @@ hook.c: int hook_exists(const char *name)
     +	}
     +
     +	return hook_head;
    -+}
    -+
    + }
    + 
      void run_hooks_opt_clear(struct run_hooks_opt *o)
    - {
    - 	strvec_clear(&o->env);
     @@ hook.c: static int pick_next_hook(struct child_process *cp,
      	cp->dir = hook_cb->options->dir;
      
    @@ hook.c: int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *optio
     -	hook_path = find_hook(hook_name);
     -	if (!hook_path) {
     -		ret = 0;
    -+	hooks = hook_list(hook_name);
    ++	hooks = list_hooks(hook_name);
     +
     +	/*
     +	 * If you need to act on a missing hook, use run_found_hooks()
    @@ hook.h: struct hook {
     + * Provides a linked list of 'struct hook' detailing commands which should run
     + * in response to the 'hookname' event, in execution order.
     + */
    -+struct list_head* hook_list(const char *hookname);
    ++struct list_head *list_hooks(const char *hookname);
     +
      struct run_hooks_opt
      {
38:  eda439cd57 !  2:  dfb995ce4d hook: allow parallel hook execution
    @@ Commit message
         hook: allow parallel hook execution
     
         In many cases, there's no reason not to allow hooks to execute in
    -    parallel. run_processes_parallel() is well-suited - it's a task queue
    -    that runs its housekeeping in series, which means users don't
    -    need to worry about thread safety on their callback data. True
    -    multithreaded execution with the async_* functions isn't necessary here.
    -    Synchronous hook execution can be achieved by only allowing 1 job to run
    -    at a time.
    +    parallel, if more than one was provided. hook.c already calls
    +    run_processes_parallel(), so all we need to do is allow the job count we
    +    hand to run_processes_parallel() to be greater than 1.
     
    -    Teach run_hooks() to use that function for simple hooks which don't
    -    require stdin or capture of stderr.
    +    If users have specified no alternative, we can use the processor count
    +    from online_cpus() to run an efficient number of tasks at once. However,
    +    users can also override this number if they want by configuring
    +    'hook.jobs'.
    +
    +    To avoid looking up 'hook.jobs' in cases where we don't end up with any
    +    hooks to run anyways, teach the hook runner commands to notice if
    +    .jobs==0 and do a config or online_cpus() lookup if so, when we already
    +    know we have jobs to run.
    +
    +    Serial execution can still be performed by setting .jobs == 1.
     
         Signed-off-by: Emily Shaffer <[email protected]>
         Helped-by: Ævar Arnfjörð Bjarmason <[email protected]>
    @@ builtin/am.c: static void am_destroy(const struct am_state *state)
      {
      	int ret;
     -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SYNC;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
      
      	assert(state->msg);
      	strvec_push(&opt.args, am_path(state, "final-commit"));
    @@ builtin/am.c: static int run_applypatch_msg_hook(struct am_state *state)
      static int run_post_rewrite_hook(const struct am_state *state)
      {
     -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
      
      	strvec_push(&opt.args, "rebase");
      	opt.path_to_stdin = am_path(state, "rewritten");
    @@ builtin/checkout.c: struct branch_info {
      			      int changed)
      {
     -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SYNC;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
      
      	/* "new_commit" can be NULL when checking out from the index before
      	   a commit exists. */
    @@ builtin/clone.c: static int checkout(int submodule_progress)
      	struct tree_desc t;
      	int err = 0;
     -	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_SYNC;
    ++	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_SERIAL;
      
      	if (option_no_checkout)
      		return 0;
    @@ builtin/hook.c: static const char * const builtin_hook_run_usage[] = {
      {
      	int i;
     -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SYNC;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
      	int ignore_missing = 0;
      	const char *hook_name;
      	struct list_head *hooks;
    @@ builtin/merge.c: static void finish(struct commit *head_commit,
      {
      	struct strbuf reflog_message = STRBUF_INIT;
     -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
      	const struct object_id *head = &head_commit->object.oid;
      
      	if (!msg)
    @@ builtin/rebase.c: int cmd_rebase(int argc, const char **argv, const char *prefix
      	int reschedule_failed_exec = -1;
      	int allow_preemptive_ff = 1;
     -	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_ASYNC;
    ++	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_PARALLEL;
      	struct option builtin_rebase_options[] = {
      		OPT_STRING(0, "onto", &options.onto_name,
      			   N_("revision"),
    @@ builtin/receive-pack.c: static int run_receive_hook(struct command *commands,
      			    const struct string_list *push_options)
      {
     -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
      	struct receive_hook_feed_context ctx;
      	struct command *iter = commands;
      
    @@ builtin/receive-pack.c: static int run_receive_hook(struct command *commands,
      static int run_update_hook(struct command *cmd)
      {
     -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
      
      	strvec_pushl(&opt.args,
      		     cmd->ref_name,
    @@ builtin/receive-pack.c: static const char *push_to_checkout(unsigned char *hash,
      				    const char *work_tree)
      {
     -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SYNC;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
     +
      	opt.invoked_hook = invoked_hook;
      
    @@ builtin/receive-pack.c: static const char *update(struct command *cmd, struct sh
      {
      	struct command *cmd;
     -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
      
      	for (cmd = commands; cmd; cmd = cmd->next) {
      		if (cmd->error_string || cmd->did_not_exist)
    @@ builtin/worktree.c: static int add_worktree(const char *path, const char *refnam
      	 */
      	if (!ret && opts->checkout) {
     -		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SYNC;
    ++		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
      
      		strvec_pushl(&opt.env, "GIT_DIR", "GIT_WORK_TREE", NULL);
      		strvec_pushl(&opt.args,
    @@ commit.c: int run_commit_hook(int editor_is_used, const char *index_file,
      		    const char *name, ...)
      {
     -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SYNC;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
      	va_list args;
      	const char *arg;
      
    @@ hook.c: int run_hooks(const char *hook_name, struct list_head *hooks,
      	cb_data.run_me = list_first_entry(hooks, struct hook, list);
      
     -	run_processes_parallel_tr2(jobs,
    -+	/* INIT_ASYNC sets jobs to 0, so go look up how many to use. */
    ++	/* INIT_PARALLEL sets jobs to 0, so go look up how many to use. */
     +	if (!options->jobs)
     +		options->jobs = configured_hook_jobs();
     +
    @@ hook.c: int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *optio
     +	 * Turn on parallelism by default. Hooks which don't want it should
     +	 * specify their options accordingly.
     +	 */
    -+	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT_ASYNC;
    ++	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT_PARALLEL;
      
      	if (!options)
      		options = &hook_opt_scratch;
    @@ hook.h: struct hook_cb_data {
      	int *invoked_hook;
      };
      
    -+#define RUN_HOOKS_OPT_INIT_SYNC { \
    ++#define RUN_HOOKS_OPT_INIT_SERIAL { \
     +	.jobs = 1, \
     +	.env = STRVEC_INIT, \
     +	.args = STRVEC_INIT, \
     +}
     +
    -+#define RUN_HOOKS_OPT_INIT_ASYNC { \
    ++#define RUN_HOOKS_OPT_INIT_PARALLEL { \
     +	.jobs = 0, \
     +	.env = STRVEC_INIT, \
     +	.args = STRVEC_INIT, \
    @@ read-cache.c: static int do_write_locked_index(struct index_state *istate, struc
      	int ret;
      	int was_full = !istate->sparse_index;
     -	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_ASYNC;
    ++	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_PARALLEL;
      
      	ret = convert_to_sparse(istate);
      
    @@ refs.c: int ref_update_reject_duplicates(struct string_list *refnames,
      				const char *state)
      {
     -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
      	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
      	int ret = 0, i;
      
    @@ reset.c: int reset_head(struct repository *r, struct object_id *oid, const char
      	}
      	if (run_hook) {
     -		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
    ++		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
     +
      		strvec_pushl(&opt.args,
      			     oid_to_hex(orig ? orig : null_oid()),
    @@ sequencer.c: int update_head_with_reflog(const struct commit *old_head,
      			    const struct object_id *newoid)
      {
     -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
      	struct strbuf tmp = STRBUF_INIT;
      	struct string_list to_stdin = STRING_LIST_INIT_DUP;
      	int code;
    @@ sequencer.c: static int pick_commits(struct repository *r,
      				st.st_size > 0) {
      			struct child_process notes_cp = CHILD_PROCESS_INIT;
     -			struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
    -+			struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_ASYNC;
    ++			struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_PARALLEL;
      
      			notes_cp.in = open(rebase_path_rewritten_list(), O_RDONLY);
      			notes_cp.git_cmd = 1;
    @@ transport.c: static int run_pre_push_hook(struct transport *transport,
      {
      	int ret = 0;
     -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
      	struct ref *r;
      	struct string_list to_stdin = STRING_LIST_INIT_NODUP;

39: cdfe3b6e16 ! 3: c8a04306e9 hook: introduce "git hook list"

    @@ Commit message
     
         Signed-off-by: Emily Shaffer <[email protected]>
     
    + ## Documentation/git-hook.txt ##
    +@@ Documentation/git-hook.txt: SYNOPSIS
    + [verse]
    + 'git hook' run [--to-stdin=<path>] [--ignore-missing] [(-j|--jobs) <n>]
    + 	<hook-name> [-- <hook-args>]
    ++'git hook' list <hook-name>
    + 
    + DESCRIPTION
    + -----------
    +@@ Documentation/git-hook.txt: optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The
    + arguments (if any) differ by hook name, see linkgit:githooks[5] for
    + what those are.
    + 
    ++list::
    ++	Print a list of hooks which will be run on `<hook-name>` event. If no
    ++	hooks are configured for that event, print nothing and return 1.
    ++
    + OPTIONS
    + -------
    + 
    +
      ## builtin/hook.c ##
     @@
      
      #define BUILTIN_HOOK_RUN_USAGE \
    - 	N_("git hook run [--to-stdin=<path>] <hook-name> [-- <hook-args>]")
    + 	N_("git hook run [--ignore-missing] [--to-stdin=<path>] <hook-name> [-- <hook-args>]")
     +#define BUILTIN_HOOK_LIST_USAGE \
     +	N_("git hook list <hook-name>")
      
    @@ builtin/hook.c: static const char * const builtin_hook_run_usage[] = {
     +
     +	head = hook_list(hookname);
     +
    -+	if (list_empty(head)) {
    -+		printf(_("no commands configured for hook '%s'\n"),
    -+		       hookname);
    -+		return 0;
    -+	}
    ++	if (list_empty(head))
    ++		return 1;
     +
     +	list_for_each(pos, head) {
     +		struct hook *item = list_entry(pos, struct hook, list);
    @@ builtin/hook.c: int cmd_hook(int argc, const char **argv, const char *prefix)
      
     
      ## hook.c ##
    -@@ hook.c: int hook_exists(const char *name)
    - struct list_head* hook_list(const char* hookname)
    +@@ hook.c: struct list_head *list_hooks(const char *hookname)
      {
      	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
    -+	const char *hook_path = find_hook(hookname);
    -+
      
    ++
      	INIT_LIST_HEAD(hook_head);
      
      	if (!hookname)
    - 		return NULL;
    +@@ hook.c: struct list_head *list_hooks(const char *hookname)
      
    --	if (have_git_dir()) {
    --		const char *hook_path = find_hook(hookname);
    + 	if (have_git_dir()) {
    + 		const char *hook_path = find_hook(hookname);
     -
     -		/* Add the hook from the hookdir */
    --		if (hook_path) {
    --			struct hook *to_add = xmalloc(sizeof(*to_add));
    --			to_add->hook_path = hook_path;
    --			to_add->feed_pipe_cb_data = NULL;
    --			list_add_tail(&to_add->list, hook_head);
    --		}
    -+	/* Add the hook from the hookdir */
    -+	if (hook_path) {
    -+		struct hook *to_add = xmalloc(sizeof(*to_add));
    -+		to_add->hook_path = hook_path;
    -+		to_add->feed_pipe_cb_data = NULL;
    -+		list_add_tail(&to_add->list, hook_head);
    - 	}
    - 
    - 	return hook_head;
    + 		if (hook_path) {
    + 			struct hook *to_add = xmalloc(sizeof(*to_add));
    + 			to_add->hook_path = hook_path;
40:  eb4e03e22b !  4:  af14116d0f hook: allow running non-native hooks
    @@ builtin/hook.c: static int list(int argc, const char **argv, const char *prefix)
      	hookname = argv[0];
      
     -	head = hook_list(hookname);
    -+	head = hook_list(hookname, 1);
    ++	head = list_hooks_gently(hookname);
      
    - 	if (list_empty(head)) {
    - 		printf(_("no commands configured for hook '%s'\n"),
    + 	if (list_empty(head))
    + 		return 1;
     @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
      	git_config(git_default_config, NULL);
      
      	hook_name = argv[0];
    --	hooks = hook_list(hook_name);
    -+	hooks = hook_list(hook_name, 1);
    +-	hooks = list_hooks(hook_name);
    ++	hooks = list_hooks_gently(hook_name);
      	if (list_empty(hooks)) {
      		/* ... act like run_hooks_oneshot() under --ignore-missing */
      		if (ignore_missing)
    @@ hook.c: static int known_hook(const char *name)
     +	const char *hook_path;
      
      	if (!known_hook(name))
    - 		die(_("the hook '%s' is not known to git, should be in hook-list.h via githooks(5)"),
    +-		die(_("the hook '%s' is not known to git, should be in hook-list.h via githooks(5)"),
    ++		BUG(_("the hook '%s' is not known to git, should be in hook-list.h via githooks(5)"),
      		    name);
      
     +	hook_path = find_hook_gently(name);
    @@ hook.c: static int known_hook(const char *name)
      	strbuf_git_path(&path, "hooks/%s", name);
      	if (access(path.buf, X_OK) < 0) {
     @@ hook.c: int hook_exists(const char *name)
    - 	return !!find_hook(name);
    + 	return !list_empty(list_hooks(name));
      }
      
    --struct list_head* hook_list(const char* hookname)
     +struct hook_config_cb
     +{
     +	struct strbuf *hook_key;
     +	struct list_head *list;
     +};
     +
    -+struct list_head* hook_list(const char* hookname, int allow_unknown)
    + struct list_head *list_hooks(const char *hookname)
      {
    - 	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
    --	const char *hook_path = find_hook(hookname);
    -+	const char *hook_path;
    +-	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
    ++	if (!known_hook(hookname))
    ++		BUG("Don't recognize hook event '%s'! "
    ++		    "Is it documented in Documentation/githooks.txt?",
    ++		    hookname);
    ++	return list_hooks_gently(hookname);
    ++}
      
    ++struct list_head *list_hooks_gently(const char *hookname)
    ++{
    ++	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
      
      	INIT_LIST_HEAD(hook_head);
    -@@ hook.c: struct list_head* hook_list(const char* hookname)
    - 	if (!hookname)
    - 		return NULL;
    - 
    -+	if (allow_unknown)
    -+		hook_path = find_hook_gently(hookname);
    -+	else
    -+		hook_path = find_hook(hookname);
    -+
    - 	/* Add the hook from the hookdir */
    - 	if (hook_path) {
    - 		struct hook *to_add = xmalloc(sizeof(*to_add));
    + 
    +@@ hook.c: struct list_head *list_hooks(const char *hookname)
    + 		BUG("null hookname was provided to hook_list()!");
    + 
    + 	if (have_git_dir()) {
    +-		const char *hook_path = find_hook(hookname);
    ++		const char *hook_path = find_hook_gently(hookname);
    + 		if (hook_path) {
    + 			struct hook *to_add = xmalloc(sizeof(*to_add));
    + 			to_add->hook_path = hook_path;
     @@ hook.c: int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
      	if (options->path_to_stdin && options->feed_pipe)
      		BUG("choose only one method to populate stdin");
      
    --	hooks = hook_list(hook_name);
     +	/*
    -+	 * 'git hooks run <hookname>' uses run_found_hooks, so we don't need to
    ++	 * 'git hooks run <hookname>' uses run_hooks, so we don't need to
     +	 * allow unknown hooknames here.
     +	 */
    -+	hooks = hook_list(hook_name, 0);
    + 	hooks = list_hooks(hook_name);
      
      	/*
    - 	 * If you need to act on a missing hook, use run_found_hooks()
     
      ## hook.h ##
     @@
    @@ hook.h
       * overwritten by further calls to find_hook and run_hook_*.
     + *
     + * If the hook is not a native hook (e.g. present in Documentation/githooks.txt)
    -+ * find_hook() will die(). find_hook_gently() does not consult the native hook
    ++ * find_hook() will BUG(). find_hook_gently() does not consult the native hook
     + * list and will check for any hook name in the hooks directory regardless of
     + * whether it is known. find_hook() should be used by internal calls to hooks,
     + * and find_hook_gently() should only be used when the hookname was provided by
    @@ hook.h: struct hook {
       * Provides a linked list of 'struct hook' detailing commands which should run
       * in response to the 'hookname' event, in execution order.
     + *
    -+ * If allow_unknown is unset, hooks will be checked against the hook list
    -+ * generated from Documentation/githooks.txt. Otherwise, any hook name will be
    -+ * allowed. allow_unknown should only be set when the hook name is provided by
    -+ * the user; internal calls to hook_list should make sure the hook they are
    -+ * invoking is present in Documentation/githooks.txt.
    ++ * list_hooks() checks the provided hookname against Documentation/githooks.txt
    ++ * and BUG()s if it is not found.  list_hooks_gently() allows any hookname. The
    ++ * latter should only be used when the hook name is provided by the user, and
    ++ * the former should be used by internal callers.
       */
    --struct list_head* hook_list(const char *hookname);
    -+struct list_head* hook_list(const char *hookname, int allow_unknown);
    + struct list_head *list_hooks(const char *hookname);
    ++struct list_head *list_hooks_gently(const char *hookname);
      
      struct run_hooks_opt
      {

41: 2c8e874158 ! 5: 2bbb179962 hook: include hooks from the config
@@ Commit message

## Documentation/config/hook.txt ##
@@
-+hook.<command>.command::
-+ A command to execute during the <command> hook event. This can be an
-+ executable on your device, a oneliner for your shell, or the name of a
-+ hookcmd. See linkgit:git-hook[1].
++hook.<name>.command::
++ A command to execute whenever hook.<name> is invoked. <name> should
++ be a unique "friendly" name which you can use to identify this hook
++ command. (You can specify when to invoke this command with
++ hook.<name>.event.) The value can be an executable on your device or a
++ oneliner for your shell. If more than one value is specified for the
++ same <name>, the last value parsed will be the only command executed.
++ See linkgit:git-hook[1].
++
++hook.<name>.event::
++ The hook events which should invoke hook.<name>. <name> should be a
++ unique "friendly" name which you can use to identify this hook. The
++ value should be the name of a hook event, like "pre-commit" or "update".
++ (See linkgit:githooks[5] for a complete list of hooks Git knows about.)
++ On the specified event, the associated hook.<name>.command will be
++ executed. More than one event can be specified if you wish for
++ hook.<name> to execute on multiple events. See linkgit:git-hook[1].
+
hook.jobs::
Specifies how many hooks can be run simultaneously during parallelized
@@ Documentation/git-hook.txt: Git is unlikely to use for a native hook later on. F
+ event = pre-commit
+ command = ~/bin/linter --c
+
-+Conmmands are run in the order Git encounters their associated
++In this example, [hook "linter"] represents one script - ~/bin/linter --c - ++which can be shared by many repos, and even by many hook events, if appropriate. ++
++Commands are run in the order Git encounters their associated
+hook.<name>.event configs during the configuration parse (see
-+linkgit:git-config[1]).
++linkgit:git-config[1]). Although multiple hook.linter.event configs can be ++added, only one hook.linter.command event is valid - Git uses "last-one-wins" ++to determine which command to run.
++
++So if you wanted your linter to run when you commit as well as when you push, ++you would configure it like so:
++
++ [hook "linter"]
++ event = pre-commit
++ event = pre-push
++ command = ~/bin/linter --c
++
++With this config, ~/bin/linter --c would be run by Git before a commit is ++generated (during pre-commit) as well as before a push is performed (during ++pre-push).
++
++And if you wanted to run your linter as well as a secret-leak detector during ++only the "pre-commit" hook event, you would configure it instead like so:
++
++ [hook "linter"]
++ event = pre-commit
++ command = ~/bin/linter --c
++ [hook "no-leaks"]
++ event = pre-commit
++ command = ~/bin/leak-detector
++
++With this config, before a commit is generated (during pre-commit), Git would ++first start ~/bin/linter --c and second start ~/bin/leak-detector. It would ++evaluate the output of each when deciding whether to proceed with the commit. ++
++For a full list of hook events which you can set your hook.<name>.event to, ++and how hooks are invoked during those events, see linkgit:githooks[5].
+
+In general, when instructions suggest adding a script to +.git/hooks/<hook-event>, you can specify it in the config instead by running
@@ Documentation/git-hook.txt: Git is unlikely to use for a native hook later on. F
optional -- (or --end-of-options, see linkgit:gitcli[7]). The

## builtin/hook.c ##
-@@ builtin/hook.c: static int list(int argc, const char **argv, const char *prefix)
- head = hook_list(hookname, 1);
-
- if (list_empty(head)) {
-- printf(_("no commands configured for hook '%s'\n"),
-+ printf(_("no hooks configured for event '%s'\n"),
- hookname);
- return 0;
- }
@@ builtin/hook.c: static int list(int argc, const char **argv, const char *prefix)
struct hook *item = list_entry(pos, struct hook, list); item = list_entry(pos, struct hook, list);
@@ hook.c: static void free_hook(struct hook *ptr)
static void remove_hook(struct list_head *to_remove)
{
struct hook *hook_to_remove = list_entry(to_remove, struct hook, list);
-@@ hook.c: const char *find_hook_gently(const char *name)
-
- int hook_exists(const char *name)
- {
-- return !!find_hook(name);
-+ return !list_empty(hook_list(name, 0));
- }
+@@ hook.c: int hook_exists(const char *name)

struct hook_config_cb
{
@@ hook.c: const char *find_hook_gently(const char *name)
struct list_head *list;
};

--struct list_head* hook_list(const char* hookname, int allow_unknown)
+/*
+ * Callback for git_config which adds configured hooks to a hook list. Hooks
+ * can be configured by specifying both hook.<friend-name>.command = <path> and
@@ hook.c: const char *find_hook_gently(const char *name)
+ return 0;
+}
+
-+struct list_head* hook_list(const char *hookname, int allow_unknown)
+ struct list_head *list_hooks(const char *hookname)
+ {
+ if (!known_hook(hookname))
+@@ hook.c: struct list_head *list_hooks(const char *hookname)
+ struct list_head *list_hooks_gently(const char *hookname)
{
struct list_head *hook_head = xmalloc(sizeof(struct list_head));
-- const char *hook_path;
+ struct hook_config_cb cb_data = {
+ .hook_event = hookname,
+ .list = hook_head,
+ };

-+ if (!allow_unknown && !known_hook(hookname))
-+ die(_("Don't recognize hook event '%s'. "
-+ "Is it documented in 'githooks.txt'?"),
-+ hookname);
-
INIT_LIST_HEAD(hook_head);

if (!hookname)
- return NULL;
+ BUG("null hookname was provided to hook_list()!");

-- if (allow_unknown)
-- hook_path = find_hook_gently(hookname);
-- else
-- hook_path = find_hook(hookname);
+- if (have_git_dir()) {
+- const char *hook_path = find_hook_gently(hookname);
+- if (hook_path) {
+- struct hook *to_add = xmalloc(sizeof(*to_add));
+- to_add->hook_path = hook_path;
+- to_add->feed_pipe_cb_data = NULL;
+- list_add_tail(&to_add->list, hook_head);
+- }
+- }
+ /* Add the hooks from the config, e.g. hook.myhook.event = pre-commit */
+ git_config(hook_config_lookup, &cb_data);
-
-- /* Add the hook from the hookdir */
-- if (hook_path) {
-- struct hook *to_add = xmalloc(sizeof(*to_add));
-- to_add->hook_path = hook_path;
-- to_add->feed_pipe_cb_data = NULL;
-- list_add_tail(&to_add->list, hook_head);
-- }
++
+ /* Add the hook from the hookdir. The placeholder makes it easier to
+ * allocate work in pick_next_hook. */
+ if (find_hook_gently(hookname))
@@ hook.c: static int notify_start_failure(struct strbuf *out,

return 1;
}
-@@ hook.c: int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
- BUG("choose only one method to populate stdin");
-
- /*
-- * 'git hooks run <hookname>' uses run_found_hooks, so we don't need to
-+ * 'git hooks run <hookname>' uses run_found_hooks, and we want to make
-+ * sure internal callers are using known hooks, so we don't need to
- * allow unknown hooknames here.
- */
- hooks = hook_list(hook_name, 0);

## hook.h ##
@@ hook.h: int hook_exists(const char *hookname);
42: 3216e51b6b ! 6: 30ffe98601 hook: allow out-of-repo 'git hook' invocations

    @@ git.c: static struct cmd_struct commands[] = {
      	{ "init-db", cmd_init_db },
     
      ## hook.c ##
    -@@ hook.c: struct list_head* hook_list(const char *hookname, int allow_unknown)
    +@@ hook.c: struct list_head *list_hooks_gently(const char *hookname)
      
      	/* Add the hook from the hookdir. The placeholder makes it easier to
      	 * allocate work in pick_next_hook. */
Signature · 1 line
2.33.0.rc2.250.ged5fa647cd-goog
#569 lore raw

PATCH v3 1/6 hook: run a list of hooks instead

To prepare for multihook support, teach hook.[hc] to take a list of hooks at run_hooks and run_found_hooks. Right now the list is always one entry, but in the future we will allow users to supply more than one executable for a single hook event.

---
 builtin/hook.c |  14 ++++---
 hook.c         | 104 +++++++++++++++++++++++++++++++++++--------------
 hook.h         |  16 +++++++-
 3 files changed, 96 insertions(+), 38 deletions(-)
builtin/hook.c +8 -6
index 27dce6a2f0..e18357ba1f 100644
@@ -25,7 +25,8 @@ static int run(int argc, const char **argv, const char *prefix)
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	int ignore_missing = 0;
 	const char *hook_name;
-	const char *hook_path;
+	struct list_head *hooks;
+
 	struct option run_options[] = {
 		OPT_BOOL(0, "ignore-missing", &ignore_missing,
 			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
@@ -58,15 +59,16 @@ static int run(int argc, const char **argv, const char *prefix)
 	git_config(git_default_config, NULL);
 
 	hook_name = argv[0];
-	if (ignore_missing)
-		return run_hooks_oneshot(hook_name, &opt);
-	hook_path = find_hook(hook_name);
-	if (!hook_path) {
+	hooks = list_hooks(hook_name);
+	if (list_empty(hooks)) {
+		/* ... act like run_hooks_oneshot() under --ignore-missing */
+		if (ignore_missing)
+			return 0;
 		error("cannot find a hook named %s", hook_name);
 		return 1;
 	}
 
-	ret = run_hooks(hook_name, hook_path, &opt);
+	ret = run_hooks(hook_name, hooks, &opt);
 	run_hooks_opt_clear(&opt);
 	return ret;
 usage:
hook.c +74 -30

162 lines not on this page · raw message

hook.h +14 -2
index 58dfbf474c..12b56616bb 100644
@@ -3,6 +3,7 @@
 #include "strbuf.h"
 #include "strvec.h"
 #include "run-command.h"
+#include "list.h"
 
 /*
  * Returns the path to the hook file, or NULL if the hook is missing
@@ -17,6 +18,7 @@ const char *find_hook(const char *name);
 int hook_exists(const char *hookname);
 
 struct hook {
+	struct list_head list;
 	/* The path to the hook */
 	const char *hook_path;
 
@@ -27,6 +29,12 @@ struct hook {
 	void *feed_pipe_cb_data;
 };
 
+/*
+ * Provides a linked list of 'struct hook' detailing commands which should run
+ * in response to the 'hookname' event, in execution order.
+ */
+struct list_head *list_hooks(const char *hookname);
+
 struct run_hooks_opt
 {
 	/* Environment vars to be set for each hook */
@@ -97,6 +105,7 @@ struct hook_cb_data {
 	/* rc reflects the cumulative failure state */
 	int rc;
 	const char *hook_name;
+	struct list_head *head;
 	struct hook *run_me;
 	struct run_hooks_opt *options;
 	int *invoked_hook;
@@ -110,8 +119,8 @@ void run_hooks_opt_clear(struct run_hooks_opt *o);
  *
  * See run_hooks_oneshot() for the simpler one-shot API.
  */
-int run_hooks(const char *hookname, const char *hook_path,
-	      struct run_hooks_opt *options);
+int run_hooks(const char *hookname, struct list_head *hooks,
+		    struct run_hooks_opt *options);
 
 /**
  * Calls find_hook() on your "hook_name" and runs the hooks (if any)
@@ -123,4 +132,7 @@ int run_hooks(const char *hookname, const char *hook_path,
  */
 int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options);
 
+/* Empties the list at 'head', calling 'free_hook()' on each entry */
+void clear_hook_list(struct list_head *head);
+
 #endif
Signature · 1 line
2.33.0.rc2.250.ged5fa647cd-goog
#570 lore raw

Re: [PATCH v3 1/6] hook: run a list of hooks instead

builtin/hook.c static int run(int argc, const char **argv, const char *prefix) · line 32

Quoted 10 lines · Emily Shaffer
builtin/hook.c +2 -1
@@ -25,7 +25,8 @@ static int run(int argc, const char **argv, const char *prefix)
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	int ignore_missing = 0;
 	const char *hook_name;
-	const char *hook_path;
+	struct list_head *hooks;
+
 	struct option run_options[] = {
 		OPT_BOOL(0, "ignore-missing", &ignore_missing,
 			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),

In general in this patch series there's a bunch of little whitespace changes like that along with other changes. I think it's probably best if I just absorb that in the "base" topic instead of doing them here. E.g. in this case we could also add a line between "struct option" and the rest.

I don't mind either way, but the whitespace churn makes for distracting reading...

Quoted 11 lines
builtin/hook.c +3 -4
@@ -58,15 +59,16 @@ static int run(int argc, const char **argv, const char *prefix)
 	git_config(git_default_config, NULL);
 
 	hook_name = argv[0];
-	if (ignore_missing)
-		return run_hooks_oneshot(hook_name, &opt);
-	hook_path = find_hook(hook_name);
-	if (!hook_path) {
+	hooks = list_hooks(hook_name);
+	if (list_empty(hooks)) {
+		/* ... act like run_hooks_oneshot() under --ignore-missing */

builtin/hook.c static int run(int argc, const char **argv, const char *prefix) · line 73

Quoted 10 lines
builtin/hook.c +3 -1
+		if (ignore_missing)
+			return 0;
 		error("cannot find a hook named %s", hook_name);
 		return 1;
 	}
 
-	ret = run_hooks(hook_name, hook_path, &opt);
+	ret = run_hooks(hook_name, hooks, &opt);
 	run_hooks_opt_clear(&opt);
 	return ret;

This memory management is a bit inconsistent. So here we list_hooks(), pass it to run_hooks(), which clears it for us, OK...

Quoted 29 lines
hook.c +5 -11
-int run_hooks(const char *hook_name, const char *hook_path,
-	      struct run_hooks_opt *options)
+int run_hooks(const char *hook_name, struct list_head *hooks,
+		    struct run_hooks_opt *options)
 {
-	struct strbuf abs_path = STRBUF_INIT;
-	struct hook my_hook = {
-		.hook_path = hook_path,
-	};
 	struct hook_cb_data cb_data = {
 		.rc = 0,
 		.hook_name = hook_name,
@@ -197,11 +241,9 @@ int run_hooks(const char *hook_name, const char *hook_path,
 	if (!options)
 		BUG("a struct run_hooks_opt must be provided to run_hooks");
 
-	if (options->absolute_path) {
-		strbuf_add_absolute_path(&abs_path, hook_path);
-		my_hook.hook_path = abs_path.buf;
-	}
-	cb_data.run_me = &my_hook;
+
+	cb_data.head = hooks;
+	cb_data.run_me = list_first_entry(hooks, struct hook, list);
 
 	run_processes_parallel_tr2(jobs,
 				   pick_next_hook,
@@ -213,18 +255,15 @@ int run_hooks(const char *hook_name, const char *hook_path,
 				   "hook",

hook.c int run_hooks(const char *hook_name, const char *hook_path, · line 261

Quoted 10 lines
hook.c +1 -4
 				   hook_name);
 
-
-	if (options->absolute_path)
-		strbuf_release(&abs_path);
-	free(my_hook.feed_pipe_cb_data);
+	clear_hook_list(hooks);
 
 	return cb_data.rc;
 }

Which we can see here will call clear_hook_list(), so far so good, but then...

Quoted 22 lines
hook.c +8 -5
 int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 {
-	const char *hook_path;
-	int ret;
+	struct list_head *hooks;
+	int ret = 0;
 	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT;
 
 	if (!options)
@@ -233,14 +272,19 @@ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 	if (options->path_to_stdin && options->feed_pipe)
 		BUG("choose only one method to populate stdin");
 
-	hook_path = find_hook(hook_name);
-	if (!hook_path) {
-		ret = 0;
+	hooks = list_hooks(hook_name);
+
+	/*
+	 * If you need to act on a missing hook, use run_found_hooks()
+	 * instead
+	 */

hook.c int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options) · line 288

Quoted 10 lines
hook.c +4 -2
+	if (list_empty(hooks))
 		goto cleanup;
-	}
 
-	ret = run_hooks(hook_name, hook_path, options);
+	ret = run_hooks(hook_name, hooks, options);
+
 cleanup:
 	run_hooks_opt_clear(options);
+	clear_hook_list(hooks);

...the oneshot command also does clear_hook_list(), after calling run_hooks() which cleared it already. That looks like a mistake, i.e. we should always trust run_hooks() to clear it, no?

#571 lore raw

Re: [PATCH v3 1/6] hook: run a list of hooks instead

Quoted 21 lines · Ævar Arnfjörð Bjarmason

builtin/hook.c static int run(int argc, const char **argv, const char *prefix) · line 32

Quoted 10 lines · Emily Shaffer
builtin/hook.c +2 -1
@@ -25,7 +25,8 @@ static int run(int argc, const char **argv, const char *prefix)
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	int ignore_missing = 0;
 	const char *hook_name;
-	const char *hook_path;
+	struct list_head *hooks;
+
 	struct option run_options[] = {
 		OPT_BOOL(0, "ignore-missing", &ignore_missing,
 			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),

In general in this patch series there's a bunch of little whitespace changes like that along with other changes. I think it's probably best if I just absorb that in the "base" topic instead of doing them here. E.g. in this case we could also add a line between "struct option" and the rest.

Quoted 2 lines

I don't mind either way, but the whitespace churn makes for distracting reading...

Ah, hm. I don't know if in this specific case it's necessary for me to even have this whitespace change, since 'run_options' is still a struct declaration. I'll just drop this one, but in general whichever whitespace bits you like from this topic, feel free to absorb. Will make a note to scan through the diff when I rebase onto your next reroll checking for spurious whitespace changes.

Quoted 94 lines
Quoted 12 lines
builtin/hook.c +3 -4
@@ -58,15 +59,16 @@ static int run(int argc, const char **argv, const char *prefix)
 	git_config(git_default_config, NULL);
 
 	hook_name = argv[0];
-	if (ignore_missing)
-		return run_hooks_oneshot(hook_name, &opt);
-	hook_path = find_hook(hook_name);
-	if (!hook_path) {
+	hooks = list_hooks(hook_name);
+	if (list_empty(hooks)) {
+		/* ... act like run_hooks_oneshot() under --ignore-missing */

builtin/hook.c static int run(int argc, const char **argv, const char *prefix) · line 73

Quoted 10 lines
builtin/hook.c +3 -1
+		if (ignore_missing)
+			return 0;
 		error("cannot find a hook named %s", hook_name);
 		return 1;
 	}
 
-	ret = run_hooks(hook_name, hook_path, &opt);
+	ret = run_hooks(hook_name, hooks, &opt);
 	run_hooks_opt_clear(&opt);
 	return ret;

This memory management is a bit inconsistent. So here we list_hooks(), pass it to run_hooks(), which clears it for us, OK...

Quoted 29 lines
hook.c +5 -11
-int run_hooks(const char *hook_name, const char *hook_path,
-	      struct run_hooks_opt *options)
+int run_hooks(const char *hook_name, struct list_head *hooks,
+		    struct run_hooks_opt *options)
 {
-	struct strbuf abs_path = STRBUF_INIT;
-	struct hook my_hook = {
-		.hook_path = hook_path,
-	};
 	struct hook_cb_data cb_data = {
 		.rc = 0,
 		.hook_name = hook_name,
@@ -197,11 +241,9 @@ int run_hooks(const char *hook_name, const char *hook_path,
 	if (!options)
 		BUG("a struct run_hooks_opt must be provided to run_hooks");
 
-	if (options->absolute_path) {
-		strbuf_add_absolute_path(&abs_path, hook_path);
-		my_hook.hook_path = abs_path.buf;
-	}
-	cb_data.run_me = &my_hook;
+
+	cb_data.head = hooks;
+	cb_data.run_me = list_first_entry(hooks, struct hook, list);
 
 	run_processes_parallel_tr2(jobs,
 				   pick_next_hook,
@@ -213,18 +255,15 @@ int run_hooks(const char *hook_name, const char *hook_path,
 				   "hook",

hook.c int run_hooks(const char *hook_name, const char *hook_path, · line 261

Quoted 10 lines
hook.c +1 -4
 				   hook_name);
 
-
-	if (options->absolute_path)
-		strbuf_release(&abs_path);
-	free(my_hook.feed_pipe_cb_data);
+	clear_hook_list(hooks);
 
 	return cb_data.rc;
 }

Which we can see here will call clear_hook_list(), so far so good, but then...

Quoted 26 lines
hook.c +9 -6
 int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 {
-	const char *hook_path;
-	int ret;
+	struct list_head *hooks;
+	int ret = 0;
 	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT;
 
 	if (!options)
@@ -233,14 +272,19 @@ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 	if (options->path_to_stdin && options->feed_pipe)
 		BUG("choose only one method to populate stdin");
 
-	hook_path = find_hook(hook_name);
-	if (!hook_path) {
-		ret = 0;
+	hooks = list_hooks(hook_name);
+
+	/*
+	 * If you need to act on a missing hook, use run_found_hooks()
+	 * instead
+	 */
+	if (list_empty(hooks))
 		goto cleanup;
-	}
 
Quoted 10 lines

hook.c int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options) · line 288

Quoted 6 lines
hook.c +3 -1
-	ret = run_hooks(hook_name, hook_path, options);
+	ret = run_hooks(hook_name, hooks, options);
+
 cleanup:
 	run_hooks_opt_clear(options);
+	clear_hook_list(hooks);

...the oneshot command also does clear_hook_list(), after calling run_hooks() which cleared it already. That looks like a mistake, i.e. we should always trust run_hooks() to clear it, no?

Ah, good catch. I will update the comment on run_hooks() and fix _oneshot() :)

- Emily

#572 lore raw

Re: [PATCH v3 1/6] hook: run a list of hooks instead

Quoted 137 lines · Emily Shaffer
Quoted 21 lines · Ævar Arnfjörð Bjarmason

Review static int run(int argc, const char **argv, const char *prefix) · line 32

Quoted 10 lines · Emily Shaffer
(quoted diff) +2 -1
@@ -25,7 +25,8 @@ static int run(int argc, const char **argv, const char *prefix)
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	int ignore_missing = 0;
 	const char *hook_name;
-	const char *hook_path;
+	struct list_head *hooks;
+
 	struct option run_options[] = {
 		OPT_BOOL(0, "ignore-missing", &ignore_missing,
 			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),

In general in this patch series there's a bunch of little whitespace changes like that along with other changes. I think it's probably best if I just absorb that in the "base" topic instead of doing them here. E.g. in this case we could also add a line between "struct option" and the rest.

Quoted 2 lines

I don't mind either way, but the whitespace churn makes for distracting reading...

Ah, hm. I don't know if in this specific case it's necessary for me to even have this whitespace change, since 'run_options' is still a struct declaration. I'll just drop this one, but in general whichever whitespace bits you like from this topic, feel free to absorb. Will make a note to scan through the diff when I rebase onto your next reroll checking for spurious whitespace changes.

Quoted 94 lines
Quoted 12 lines
(quoted diff) +3 -4
@@ -58,15 +59,16 @@ static int run(int argc, const char **argv, const char *prefix)
 	git_config(git_default_config, NULL);
 
 	hook_name = argv[0];
-	if (ignore_missing)
-		return run_hooks_oneshot(hook_name, &opt);
-	hook_path = find_hook(hook_name);
-	if (!hook_path) {
+	hooks = list_hooks(hook_name);
+	if (list_empty(hooks)) {
+		/* ... act like run_hooks_oneshot() under --ignore-missing */

Review static int run(int argc, const char **argv, const char *prefix) · line 73

Quoted 10 lines
(quoted diff) +3 -1
+		if (ignore_missing)
+			return 0;
 		error("cannot find a hook named %s", hook_name);
 		return 1;
 	}
 
-	ret = run_hooks(hook_name, hook_path, &opt);
+	ret = run_hooks(hook_name, hooks, &opt);
 	run_hooks_opt_clear(&opt);
 	return ret;

This memory management is a bit inconsistent. So here we list_hooks(), pass it to run_hooks(), which clears it for us, OK...

Quoted 29 lines
(quoted diff) +5 -11
-int run_hooks(const char *hook_name, const char *hook_path,
-	      struct run_hooks_opt *options)
+int run_hooks(const char *hook_name, struct list_head *hooks,
+		    struct run_hooks_opt *options)
 {
-	struct strbuf abs_path = STRBUF_INIT;
-	struct hook my_hook = {
-		.hook_path = hook_path,
-	};
 	struct hook_cb_data cb_data = {
 		.rc = 0,
 		.hook_name = hook_name,
@@ -197,11 +241,9 @@ int run_hooks(const char *hook_name, const char *hook_path,
 	if (!options)
 		BUG("a struct run_hooks_opt must be provided to run_hooks");
 
-	if (options->absolute_path) {
-		strbuf_add_absolute_path(&abs_path, hook_path);
-		my_hook.hook_path = abs_path.buf;
-	}
-	cb_data.run_me = &my_hook;
+
+	cb_data.head = hooks;
+	cb_data.run_me = list_first_entry(hooks, struct hook, list);
 
 	run_processes_parallel_tr2(jobs,
 				   pick_next_hook,
@@ -213,18 +255,15 @@ int run_hooks(const char *hook_name, const char *hook_path,
 				   "hook",

Review int run_hooks(const char *hook_name, const char *hook_path, · line 261

Quoted 10 lines
(quoted diff) +1 -4
 				   hook_name);
 
-
-	if (options->absolute_path)
-		strbuf_release(&abs_path);
-	free(my_hook.feed_pipe_cb_data);
+	clear_hook_list(hooks);
 
 	return cb_data.rc;
 }

Which we can see here will call clear_hook_list(), so far so good, but then...

Quoted 26 lines
(quoted diff) +9 -6
 int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 {
-	const char *hook_path;
-	int ret;
+	struct list_head *hooks;
+	int ret = 0;
 	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT;
 
 	if (!options)
@@ -233,14 +272,19 @@ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 	if (options->path_to_stdin && options->feed_pipe)
 		BUG("choose only one method to populate stdin");
 
-	hook_path = find_hook(hook_name);
-	if (!hook_path) {
-		ret = 0;
+	hooks = list_hooks(hook_name);
+
+	/*
+	 * If you need to act on a missing hook, use run_found_hooks()
+	 * instead
+	 */
+	if (list_empty(hooks))
 		goto cleanup;
-	}
 
Quoted 10 lines

Review int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options) · line 288

Quoted 6 lines
(quoted diff) +3 -1
-	ret = run_hooks(hook_name, hook_path, options);
+	ret = run_hooks(hook_name, hooks, options);
+
 cleanup:
 	run_hooks_opt_clear(options);
+	clear_hook_list(hooks);

...the oneshot command also does clear_hook_list(), after calling run_hooks() which cleared it already. That looks like a mistake, i.e. we should always trust run_hooks() to clear it, no?

Quoted 4 lines

Ah, good catch. I will update the comment on run_hooks() and fix _oneshot() :)

- Emily

I found a further memory issue with this, on "seen" running e.g. t0001 when compiled with SANITIZE=leak is broken by this series.

It's because in clear_hook_list() you clear the list of hooks, but forget to clear the malloc'd container, so a missing free() fixes it. As in the POC patch at the end of this mail.

But e.g. "git hook list <name>" is still broken, easy enough to fix, just also needs fixing of the list_hooks_gently() callsites to e.g. this:

    diff --git a/builtin/hook.c b/builtin/hook.c
    index 50233366a8..2cd1831075 100644
    --- a/builtin/hook.c
    +++ b/builtin/hook.c
    @@ -48,8 +48,10 @@ static int list(int argc, const char **argv, const char *prefix)
     
     	head = list_hooks_gently(hookname);
     
    -	if (list_empty(head))
    +	if (list_empty(head)) {
    +		clear_hook_list(head);
     		return 1;
    +	}
     
     	list_for_each(pos, head) {
     		struct hook *item = list_entry(pos, struct hook, list);

Although going to that length to make the SANITIZE=leak run clean is arguably overdoing it...

hook.c +11 -1
index 23af86b9ea..e6e1e4173a 100644
@@ -67,6 +67,7 @@ void clear_hook_list(struct list_head *head)
 	struct list_head *pos, *tmp;
 	list_for_each_safe(pos, tmp, head)
 		remove_hook(pos);
+	free(head);
 }
 
 static int known_hook(const char *name)
@@ -142,7 +143,16 @@ const char *find_hook_gently(const char *name)
 
 int hook_exists(const char *name)
 {
-	return !list_empty(list_hooks(name));
+	struct list_head *hooks;
+	int exists;
+
+	hooks = list_hooks(name);
+
+	exists = !list_empty(hooks);
+
+	clear_hook_list(hooks);
+
+	return exists;
 }
 
 struct hook_config_cb
#573 lore raw

PATCH v3 2/6 hook: allow parallel hook execution

In reply to Emily Shaffer

In many cases, there's no reason not to allow hooks to execute in parallel, if more than one was provided. hook.c already calls run_processes_parallel(), so all we need to do is allow the job count we hand to run_processes_parallel() to be greater than 1.

If users have specified no alternative, we can use the processor count from online_cpus() to run an efficient number of tasks at once. However, users can also override this number if they want by configuring 'hook.jobs'.

To avoid looking up 'hook.jobs' in cases where we don't end up with any hooks to run anyways, teach the hook runner commands to notice if .jobs==0 and do a config or online_cpus() lookup if so, when we already know we have jobs to run.

Serial execution can still be performed by setting .jobs == 1.

---
 Documentation/config/hook.txt |  4 ++++
 Documentation/git-hook.txt    | 17 ++++++++++++++++-
 builtin/am.c                  |  4 ++--
 builtin/checkout.c            |  2 +-
 builtin/clone.c               |  2 +-
 builtin/hook.c                |  4 +++-
 builtin/merge.c               |  2 +-
 builtin/rebase.c              |  2 +-
 builtin/receive-pack.c        |  9 +++++----
 builtin/worktree.c            |  2 +-
 commit.c                      |  2 +-
 hook.c                        | 36 +++++++++++++++++++++++++++++++----
 hook.h                        | 24 ++++++++++++++++++-----
 read-cache.c                  |  2 +-
 refs.c                        |  2 +-
 reset.c                       |  3 ++-
 sequencer.c                   |  4 ++--
 transport.c                   |  2 +-
 18 files changed, 94 insertions(+), 29 deletions(-)
 create mode 100644 Documentation/config/hook.txt
Documentation/config/hook.txt new file +4 -0
new file mode 100644
index 0000000000..96d3d6572c
@@ -0,0 +1,4 @@
+hook.jobs::
+	Specifies how many hooks can be run simultaneously during parallelized
+	hook execution. If unspecified, defaults to the number of processors on
+	the current system.
Documentation/git-hook.txt +16 -1
index fa68c1f391..8bf82b5dd4 100644
@@ -8,7 +8,8 @@ git-hook - run git hooks
 SYNOPSIS
 --------
 [verse]
-'git hook' run [--to-stdin=<path>] [--ignore-missing] <hook-name> [-- <hook-args>]
+'git hook' run [--to-stdin=<path>] [--ignore-missing] [(-j|--jobs) <n>]
+	<hook-name> [-- <hook-args>]
 
 DESCRIPTION
 -----------
@@ -42,6 +43,20 @@ OPTIONS
 	tools that want to do a blind one-shot run of a hook that may
 	or may not be present.
 
+-j::
+--jobs::
+	Only valid for `run`.
++
+Specify how many hooks to run simultaneously. If this flag is not specified, use
+the value of the `hook.jobs` config. If the config is not specified, use the
+number of CPUs on the current system. Some hooks may be ineligible for
+parallelization: for example, 'commit-msg' intends hooks modify the commit
+message body and cannot be parallelized.
+
+CONFIGURATION
+-------------
+include::config/hook.txt[]
+
 SEE ALSO
 --------
 linkgit:githooks[5]
builtin/am.c +2 -2
index 9e3d4d9ab4..c7ffc7eafc 100644
@@ -446,7 +446,7 @@ static void am_destroy(const struct am_state *state)
 static int run_applypatch_msg_hook(struct am_state *state)
 {
 	int ret;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
 
 	assert(state->msg);
 	strvec_push(&opt.args, am_path(state, "final-commit"));
@@ -467,7 +467,7 @@ static int run_applypatch_msg_hook(struct am_state *state)
  */
 static int run_post_rewrite_hook(const struct am_state *state)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 
 	strvec_push(&opt.args, "rebase");
 	opt.path_to_stdin = am_path(state, "rewritten");
builtin/checkout.c +1 -1
index 6d69b4c011..2cb5c67f64 100644
@@ -107,7 +107,7 @@ struct branch_info {
 static int post_checkout_hook(struct commit *old_commit, struct commit *new_commit,
 			      int changed)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
 
 	/* "new_commit" can be NULL when checking out from the index before
 	   a commit exists. */
builtin/clone.c +1 -1
index 27fc05ee51..986c3b1932 100644
@@ -776,7 +776,7 @@ static int checkout(int submodule_progress)
 	struct tree *tree;
 	struct tree_desc t;
 	int err = 0;
-	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_SERIAL;
 
 	if (option_no_checkout)
 		return 0;
builtin/hook.c +3 -1
index e18357ba1f..4dd3617876 100644
@@ -22,7 +22,7 @@ static const char * const builtin_hook_run_usage[] = {
 static int run(int argc, const char **argv, const char *prefix)
 {
 	int i;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
 	int ignore_missing = 0;
 	const char *hook_name;
 	struct list_head *hooks;
@@ -32,6 +32,8 @@ static int run(int argc, const char **argv, const char *prefix)
 			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
 		OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"),
 			   N_("file to read into hooks' stdin")),
+		OPT_INTEGER('j', "jobs", &opt.jobs,
+			    N_("run up to <n> hooks simultaneously")),
 		OPT_END(),
 	};
 	int ret;
builtin/merge.c +1 -1
index 9bd4a2532c..81d7ebbbf6 100644
@@ -448,7 +448,7 @@ static void finish(struct commit *head_commit,
 		   const struct object_id *new_head, const char *msg)
 {
 	struct strbuf reflog_message = STRBUF_INIT;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 	const struct object_id *head = &head_commit->object.oid;
 
 	if (!msg)
builtin/rebase.c +1 -1
index e7c668c99b..ad118c983c 100644
@@ -1314,7 +1314,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	char *squash_onto_name = NULL;
 	int reschedule_failed_exec = -1;
 	int allow_preemptive_ff = 1;
-	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 	struct option builtin_rebase_options[] = {
 		OPT_STRING(0, "onto", &options.onto_name,
 			   N_("revision"),
builtin/receive-pack.c +5 -4
index ebec6f3bb1..7460124b74 100644
@@ -909,7 +909,7 @@ static int run_receive_hook(struct command *commands,
 			    int skip_broken,
 			    const struct string_list *push_options)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 	struct receive_hook_feed_context ctx;
 	struct command *iter = commands;
 
@@ -948,7 +948,7 @@ static int run_receive_hook(struct command *commands,
 
 static int run_update_hook(struct command *cmd)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 
 	strvec_pushl(&opt.args,
 		     cmd->ref_name,
@@ -1432,7 +1432,8 @@ static const char *push_to_checkout(unsigned char *hash,
 				    struct strvec *env,
 				    const char *work_tree)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
+
 	opt.invoked_hook = invoked_hook;
 
 	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
@@ -1628,7 +1629,7 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 static void run_update_post_hook(struct command *commands)
 {
 	struct command *cmd;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 
 	for (cmd = commands; cmd; cmd = cmd->next) {
 		if (cmd->error_string || cmd->did_not_exist)
builtin/worktree.c +1 -1
index 330867c19b..3090506790 100644
@@ -382,7 +382,7 @@ static int add_worktree(const char *path, const char *refname,
 	 * is_junk is cleared, but do return appropriate code when hook fails.
 	 */
 	if (!ret && opts->checkout) {
-		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
 
 		strvec_pushl(&opt.env, "GIT_DIR", "GIT_WORK_TREE", NULL);
 		strvec_pushl(&opt.args,
commit.c +1 -1
index 842e47beae..a38bd04752 100644
@@ -1700,7 +1700,7 @@ int run_commit_hook(int editor_is_used, const char *index_file,
 		    int *invoked_hook,
 		    const char *name, ...)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
 	va_list args;
 	const char *arg;
 
hook.c +32 -4
index 333cfd633a..b8420353fa 100644
@@ -227,6 +227,28 @@ static int notify_hook_finished(int result,
 	return 0;
 }
 
+/*
+ * Determines how many jobs to use after we know we want to parallelize. First
+ * priority is the config 'hook.jobs' and second priority is the number of CPUs.
+ */
+static int configured_hook_jobs(void)
+{
+	/*
+	 * The config and the CPU count probably won't change during the process
+	 * lifetime, so cache the result in case we invoke multiple hooks during
+	 * one process.
+	 */
+	static int jobs = 0;
+	if (jobs)
+		return jobs;
+
+	if (git_config_get_int("hook.jobs", &jobs))
+		/* if the config isn't set, fall back to CPU count. */
+		jobs = online_cpus();
+
+	return jobs;
+}
+
 int run_hooks(const char *hook_name, struct list_head *hooks,
 		    struct run_hooks_opt *options)
 {
@@ -236,16 +258,18 @@ int run_hooks(const char *hook_name, struct list_head *hooks,
 		.options = options,
 		.invoked_hook = options->invoked_hook,
 	};
-	int jobs = 1;
 
 	if (!options)
 		BUG("a struct run_hooks_opt must be provided to run_hooks");
 
-
 	cb_data.head = hooks;
 	cb_data.run_me = list_first_entry(hooks, struct hook, list);
 
-	run_processes_parallel_tr2(jobs,
+	/* INIT_PARALLEL sets jobs to 0, so go look up how many to use. */
+	if (!options->jobs)
+		options->jobs = configured_hook_jobs();
+
+	run_processes_parallel_tr2(options->jobs,
 				   pick_next_hook,
 				   notify_start_failure,
 				   options->feed_pipe,
@@ -264,7 +288,11 @@ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 {
 	struct list_head *hooks;
 	int ret = 0;
-	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT;
+	/*
+	 * Turn on parallelism by default. Hooks which don't want it should
+	 * specify their options accordingly.
+	 */
+	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT_PARALLEL;
 
 	if (!options)
 		options = &hook_opt_scratch;
hook.h +19 -5
index 12b56616bb..cd3123d290 100644
@@ -43,6 +43,13 @@ struct run_hooks_opt
 	/* Args to be passed to each hook */
 	struct strvec args;
 
+	/*
+	 * Number of threads to parallelize across. Set to 0 to use the
+	 * 'hook.jobs' config or, if that config is unset, the number of cores
+	 * on the system.
+	 */
+	int jobs;
+
 	/* Resolve and run the "absolute_path(hook)" instead of
 	 * "hook". Used for "git worktree" hooks
 	 */
@@ -85,11 +92,6 @@ struct run_hooks_opt
 	int *invoked_hook;
 };
 
-#define RUN_HOOKS_OPT_INIT { \
-	.env = STRVEC_INIT, \
-	.args = STRVEC_INIT, \
-}
-
 /*
  * To specify a 'struct string_list', set 'run_hooks_opt.feed_pipe_ctx' to the
  * string_list and set 'run_hooks_opt.feed_pipe' to 'pipe_from_string_list()'.
@@ -111,6 +113,18 @@ struct hook_cb_data {
 	int *invoked_hook;
 };
 
+#define RUN_HOOKS_OPT_INIT_SERIAL { \
+	.jobs = 1, \
+	.env = STRVEC_INIT, \
+	.args = STRVEC_INIT, \
+}
+
+#define RUN_HOOKS_OPT_INIT_PARALLEL { \
+	.jobs = 0, \
+	.env = STRVEC_INIT, \
+	.args = STRVEC_INIT, \
+}
+
 void run_hooks_opt_clear(struct run_hooks_opt *o);
 
 /**
read-cache.c +1 -1
index 90099ca14d..f157e62531 100644
@@ -3068,7 +3068,7 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 {
 	int ret;
 	int was_full = !istate->sparse_index;
-	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 
 	ret = convert_to_sparse(istate);
 
refs.c +1 -1
index 73d4a93926..5543b8cdab 100644
@@ -2062,7 +2062,7 @@ int ref_update_reject_duplicates(struct string_list *refnames,
 static int run_transaction_hook(struct ref_transaction *transaction,
 				const char *state)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
 	int ret = 0, i;
 
reset.c +2 -1
index 6499bc5127..1941adb771 100644
@@ -128,7 +128,8 @@ int reset_head(struct repository *r, struct object_id *oid, const char *action,
 					    reflog_head);
 	}
 	if (run_hook) {
-		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
+
 		strvec_pushl(&opt.args,
 			     oid_to_hex(orig ? orig : null_oid()),
 			     oid_to_hex(oid),
sequencer.c +2 -2
index f451e23d0c..30fbe79b8a 100644
@@ -1148,7 +1148,7 @@ int update_head_with_reflog(const struct commit *old_head,
 static int run_rewrite_hook(const struct object_id *oldoid,
 			    const struct object_id *newoid)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 	struct strbuf tmp = STRBUF_INIT;
 	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 	int code;
@@ -4522,7 +4522,7 @@ static int pick_commits(struct repository *r,
 		if (!stat(rebase_path_rewritten_list(), &st) &&
 				st.st_size > 0) {
 			struct child_process notes_cp = CHILD_PROCESS_INIT;
-			struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+			struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 
 			notes_cp.in = open(rebase_path_rewritten_list(), O_RDONLY);
 			notes_cp.git_cmd = 1;
transport.c +1 -1
index 4ca8fc0391..33da71a108 100644
@@ -1204,7 +1204,7 @@ static int run_pre_push_hook(struct transport *transport,
 			     struct ref *remote_refs)
 {
 	int ret = 0;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 	struct ref *r;
 	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
 
Signature · 1 line
2.33.0.rc2.250.ged5fa647cd-goog
#574 lore raw

Re: [PATCH v3 2/6] hook: allow parallel hook execution

Documentation/git-hook.txt git-hook - run git hooks · line 12

Quoted 3 lines · Emily Shaffer

-'git hook' run [--to-stdin=<path>] [--ignore-missing] <hook-name> [-- <hook-args>] +'git hook' run [--to-stdin=<path>] [--ignore-missing] [(-j|--jobs) <n>]

Documentation/git-hook.txt +1 -0
+	<hook-name> [-- <hook-args>]

As an aside I wondered if it shouldn't be [[-j|--jobs] <n>], but grepped around and found that (x|y|z) means a mandatory pick of x, y or z, but [x|y|z] means that, plus possibly picking none, I think.

So this is fine, just something I wondered about...

Documentation/git-hook.txt OPTIONS · line 51

Quoted 6 lines
+-j::
+--jobs::
Documentation/git-hook.txt +4 -0
+	Only valid for `run`.
++
+Specify how many hooks to run simultaneously. If this flag is not specified, use
+the value of the `hook.jobs` config. If the config is not specified, use the

s/use the value/uses the value/

Also we usually say "of the XYZ config, see linkgit:git-config[1]", or something to that effect when we mention config variables. Perhaps we should do the same here.

Documentation/git-hook.txt OPTIONS · line 54

Quoted 3 lines
Documentation/git-hook.txt +3 -0
+number of CPUs on the current system. Some hooks may be ineligible for
+parallelization: for example, 'commit-msg' intends hooks modify the commit
+message body and cannot be parallelized.

Not something that *needs* to happen in this series, but I wonder if we shouldn't have per-type config here too, so users could force it even for those hook types if they want.

Quoted 14 lines
hook.h +2 -5
-#define RUN_HOOKS_OPT_INIT { \
-	.env = STRVEC_INIT, \
-	.args = STRVEC_INIT, \
-}
-
 /*
  * To specify a 'struct string_list', set 'run_hooks_opt.feed_pipe_ctx' to the
  * string_list and set 'run_hooks_opt.feed_pipe' to 'pipe_from_string_list()'.
@@ -111,6 +113,18 @@ struct hook_cb_data {
 	int *invoked_hook;
 };
 
+#define RUN_HOOKS_OPT_INIT_SERIAL { \
+	.jobs = 1, \

hook.h struct hook_cb_data { · line 127

Quoted 10 lines
hook.h +10 -0
+	.env = STRVEC_INIT, \
+	.args = STRVEC_INIT, \
+}
+
+#define RUN_HOOKS_OPT_INIT_PARALLEL { \
+	.jobs = 0, \
+	.env = STRVEC_INIT, \
+	.args = STRVEC_INIT, \
+}
+

Ditto earlier comments about whitespace churn, i.e. I can just move this around in the base topic, so the diff here is the change/addition, not also moving things around.

#575 lore raw

Re: [PATCH v3 2/6] hook: allow parallel hook execution

Quoted 9 lines · Ævar Arnfjörð Bjarmason
Quoted 2 lines · Emily Shaffer

+'git hook' run [--to-stdin=<path>] [--ignore-missing] [(-j|--jobs) <n>]
+ <hook-name> [-- <hook-args>]

As an aside I wondered if it shouldn't be [[-j|--jobs] <n>], but grepped around and found that (x|y|z) means a mandatory pick of x, y or z, but [x|y|z] means that, plus possibly picking none, I think.

So this is fine, just something I wondered about...

Indeed, the text is fine as-is.

Quoted 4 lines

Documentation/git-hook.txt OPTIONS · line 51

Quoted 2 lines
Documentation/git-hook.txt +2 -0
+Specify how many hooks to run simultaneously. If this flag is not specified, use
+the value of the `hook.jobs` config. If the config is not specified, use the

s/use the value/uses the value/

As a native English speaker, the text as written (even if a bit stilted) sounds better than the suggested replacement. A small tweak might improve it slightly:

If this flag is not specified, the value of the hooks.job configuration is used instead (see linkgit:git-config[1]).

but is not at all worth a re-roll or a lot of discussion.

Quoted 3 lines

Also we usually say "of the XYZ config, see linkgit:git-config[1]", or something to that effect when we mention config variables. Perhaps we should do the same here.

#576 lore raw

Re: [PATCH v3 2/6] hook: allow parallel hook execution

In reply to Ævar Arnfjörð Bjarmason
Quoted 12 lines · Ævar Arnfjörð Bjarmason

Documentation/git-hook.txt git-hook - run git hooks · line 12

Quoted 3 lines · Emily Shaffer

-'git hook' run [--to-stdin=<path>] [--ignore-missing] <hook-name> [-- <hook-args>] +'git hook' run [--to-stdin=<path>] [--ignore-missing] [(-j|--jobs) <n>]

Documentation/git-hook.txt +1 -0
+	<hook-name> [-- <hook-args>]

As an aside I wondered if it shouldn't be [[-j|--jobs] <n>], but grepped around and found that (x|y|z) means a mandatory pick of x, y or z, but [x|y|z] means that, plus possibly picking none, I think.

Quoted 1 line

So this is fine, just something I wondered about...

Hm, I guess I hadn't see [x|y|z] (or paid attention to it). Anyway, yes, I think [(-j|--jobs) <n>] is probably most correct here: "entirely optionally you can provide a jobs arg; but if you do it must start with -j or --jobs and it must contain a number". I guess you could also say [-j <n> | --jobs <n>] but that might leave the reader to consider whether <n> has a different meaning in each? Anyway, doesn't matter really, and thanks for completely nerd-sniping me ;)

Quoted 10 lines

Documentation/git-hook.txt OPTIONS · line 51

Quoted 7 lines
+-j::
+--jobs::
Documentation/git-hook.txt +4 -0
+	Only valid for `run`.
++
+Specify how many hooks to run simultaneously. If this flag is not specified, use
+the value of the `hook.jobs` config. If the config is not specified, use the

s/use the value/uses the value/

Quoted 3 lines

Also we usually say "of the XYZ config, see linkgit:git-config[1]", or something to that effect when we mention config variables. Perhaps we should do the same here.

Done, thanks.

Quoted 8 lines

Documentation/git-hook.txt OPTIONS · line 54

Quoted 4 lines
Documentation/git-hook.txt +3 -0
+number of CPUs on the current system. Some hooks may be ineligible for
+parallelization: for example, 'commit-msg' intends hooks modify the commit
+message body and cannot be parallelized.

Not something that *needs* to happen in this series, but I wonder if we shouldn't have per-type config here too, so users could force it even for those hook types if they want.

Yeah, I'm happy to implement that later when someone complains... ;)

Quoted 13 lines
Quoted 13 lines
hook.h +0 -5
-#define RUN_HOOKS_OPT_INIT { \
-	.env = STRVEC_INIT, \
-	.args = STRVEC_INIT, \
-}
-
 /*
  * To specify a 'struct string_list', set 'run_hooks_opt.feed_pipe_ctx' to the
  * string_list and set 'run_hooks_opt.feed_pipe' to 'pipe_from_string_list()'.
@@ -111,6 +113,18 @@ struct hook_cb_data {
 	int *invoked_hook;
 };
 
Quoted 16 lines

hook.h struct hook_cb_data { · line 127

Quoted 12 lines
hook.h +12 -0
+#define RUN_HOOKS_OPT_INIT_SERIAL { \
+	.jobs = 1, \
+	.env = STRVEC_INIT, \
+	.args = STRVEC_INIT, \
+}
+
+#define RUN_HOOKS_OPT_INIT_PARALLEL { \
+	.jobs = 0, \
+	.env = STRVEC_INIT, \
+	.args = STRVEC_INIT, \
+}
+

Ditto earlier comments about whitespace churn, i.e. I can just move this around in the base topic, so the diff here is the change/addition, not also moving things around.

Nah, I think I just wasn't paying much attention to where it goes. It makes more sense to put this by the 'struct run_hooks_opt' decl, I'll move mine instead.

- Emily

#577 lore raw

PATCH v3 3/6 hook: introduce "git hook list"

In reply to Emily Shaffer

If more than one hook will be run, it may be useful to see a list of which hooks should be run. At very least, it will be useful for us to test the semantics of multihooks ourselves.

For now, only list the hooks which will run in the order they will run in; later, it might be useful to include more information like where the hooks were configured and whether or not they will run.

---
 Documentation/git-hook.txt |  5 +++++
 builtin/hook.c             | 46 ++++++++++++++++++++++++++++++++++++++
 hook.c                     |  3 +--
 3 files changed, 52 insertions(+), 2 deletions(-)
Documentation/git-hook.txt +5 -0
index 8bf82b5dd4..701ada9fc0 100644
@@ -10,6 +10,7 @@ SYNOPSIS
 [verse]
 'git hook' run [--to-stdin=<path>] [--ignore-missing] [(-j|--jobs) <n>]
 	<hook-name> [-- <hook-args>]
+'git hook' list <hook-name>
 
 DESCRIPTION
 -----------
@@ -30,6 +31,10 @@ optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The
 arguments (if any) differ by hook name, see linkgit:githooks[5] for
 what those are.
 
+list::
+	Print a list of hooks which will be run on `<hook-name>` event. If no
+	hooks are configured for that event, print nothing and return 1.
+
 OPTIONS
 -------
 
builtin/hook.c +46 -0

66 lines not on this page · raw message

hook.c +1 -2
index b8420353fa..b1ea372e15 100644
@@ -96,6 +96,7 @@ struct list_head *list_hooks(const char *hookname)
 {
 	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
 
+
 	INIT_LIST_HEAD(hook_head);
 
 	if (!hookname)
@@ -103,8 +104,6 @@ struct list_head *list_hooks(const char *hookname)
 
 	if (have_git_dir()) {
 		const char *hook_path = find_hook(hookname);
-
-		/* Add the hook from the hookdir */
 		if (hook_path) {
 			struct hook *to_add = xmalloc(sizeof(*to_add));
 			to_add->hook_path = hook_path;
Signature · 1 line
2.33.0.rc2.250.ged5fa647cd-goog
#578 lore raw

Re: [PATCH v3 3/6] hook: introduce "git hook list"

Quoted 6 lines · Emily Shaffer
builtin/hook.c +6 -0
+static int list(int argc, const char **argv, const char *prefix)
+{
+	struct list_head *head, *pos;
+	const char *hookname = NULL;
+	struct strbuf hookdir_annotation = STRBUF_INIT;
+

builtin/hook.c static const char * const builtin_hook_run_usage[] = { · line 45

Quoted 10 lines
builtin/hook.c +10 -0
+	struct option list_options[] = {
+		OPT_END(),
+	};
+
+	argc = parse_options(argc, argv, prefix, list_options,
+			     builtin_hook_list_usage, 0);
+
+	if (argc < 1)
+		usage_msg_opt(_("You must specify a hook event name to list."),
+			      builtin_hook_list_usage, list_options);

Untested, but aren't we silently ignoring:

git hook list pre-receive some extra gar bage here

I.e. shouldn't this be an "argc != 1" check?

builtin/hook.c static const char * const builtin_hook_run_usage[] = { · line 58

Quoted 13 lines
builtin/hook.c +13 -0
+
+	hookname = argv[0];
+
+	head = hook_list(hookname);
+
+	if (list_empty(head))
+		return 1;
+
+	list_for_each(pos, head) {
+		struct hook *item = list_entry(pos, struct hook, list);
+		item = list_entry(pos, struct hook, list);
+		if (item)
+			printf("%s\n", item->hook_path);

Nit/suggestion: use puts(x) instead of printf("%s\n", x), but that's also a bikeshedding/style preference, so ignore if you disagree...

builtin/hook.c static const char * const builtin_hook_run_usage[] = { · line 61

Quoted 3 lines
builtin/hook.c +3 -0
+	}
+
+	clear_hook_list(head);

Nit/API suggestion: Maybe s/list_for_each/list_for_each_safe/ and remove_hook() in the loop would make more sense for this one-shot caller than iterating over the list twice?

Anyway, currently remove_hook() is static, and it's probably good to not peek behind the curtain here, so on second thought clear_hook_list() is probably best...

builtin/hook.c static const char * const builtin_hook_run_usage[] = { · line 62

Quoted 1 line
builtin/hook.c +1 -0
+	strbuf_release(&hookdir_annotation);

This function did nothing with hookdir_annotation. Looks like leftover WIP code, but maybe it's used in (and should be moved to) a later commit, let's keep reading...

hook.c struct list_head *list_hooks(const char *hookname) · line 100

Quoted 5 lines

[...]

hook.c +1 -0
 	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
 
+
 	INIT_LIST_HEAD(hook_head);

..ditto...

hook.c struct list_head *list_hooks(const char *hookname) · line 106

Quoted 7 lines
hook.c +0 -1
 	if (!hookname)
@@ -103,8 +104,6 @@ struct list_head *list_hooks(const char *hookname)
 
 	if (have_git_dir()) {
 		const char *hook_path = find_hook(hookname);
-

... earlier notes about whitespace churn...

Quoted 4 lines
hook.c +0 -1
-		/* Add the hook from the hookdir */
 		if (hook_path) {
 			struct hook *to_add = xmalloc(sizeof(*to_add));
 			to_add->hook_path = hook_path;
#579 lore raw

Re: [PATCH v3 3/6] hook: introduce "git hook list"

Quoted 23 lines · Ævar Arnfjörð Bjarmason
Quoted 6 lines · Emily Shaffer
builtin/hook.c +6 -0
+static int list(int argc, const char **argv, const char *prefix)
+{
+	struct list_head *head, *pos;
+	const char *hookname = NULL;
+	struct strbuf hookdir_annotation = STRBUF_INIT;
+

builtin/hook.c static const char * const builtin_hook_run_usage[] = { · line 45

Quoted 10 lines
builtin/hook.c +10 -0
+	struct option list_options[] = {
+		OPT_END(),
+	};
+
+	argc = parse_options(argc, argv, prefix, list_options,
+			     builtin_hook_list_usage, 0);
+
+	if (argc < 1)
+		usage_msg_opt(_("You must specify a hook event name to list."),
+			      builtin_hook_list_usage, list_options);

Untested, but aren't we silently ignoring:

Quoted 3 lines

git hook list pre-receive some extra gar bage here

I.e. shouldn't this be an "argc != 1" check?

Yeah, I think you are right. Will switch.

Quoted 17 lines
Quoted 4 lines
builtin/hook.c +3 -0
+
+	hookname = argv[0];
+

builtin/hook.c static const char * const builtin_hook_run_usage[] = { · line 58

Quoted 10 lines
builtin/hook.c +10 -0
+	head = hook_list(hookname);
+
+	if (list_empty(head))
+		return 1;
+
+	list_for_each(pos, head) {
+		struct hook *item = list_entry(pos, struct hook, list);
+		item = list_entry(pos, struct hook, list);
+		if (item)
+			printf("%s\n", item->hook_path);

Nit/suggestion: use puts(x) instead of printf("%s\n", x), but that's also a bikeshedding/style preference, so ignore if you disagree...

I was curious, because today I learned about puts() ;), so I checked (sorry for escape gore):

  $ gg puts\( | wc -l
  217
  $ gg "printf(\"%s\(\\\\n\)\?\"" | wc -l
  96

So looks like it is indeed more idiomatic by about 2x to just use puts(). Will switch.

Quoted 9 lines

builtin/hook.c static const char * const builtin_hook_run_usage[] = { · line 61

Quoted 4 lines
builtin/hook.c +3 -0
+	}
+
+	clear_hook_list(head);

Nit/API suggestion: Maybe s/list_for_each/list_for_each_safe/ and remove_hook() in the loop would make more sense for this one-shot caller than iterating over the list twice?

Quoted 3 lines

Anyway, currently remove_hook() is static, and it's probably good to not peek behind the curtain here, so on second thought clear_hook_list() is probably best...

Sounds like you talked yourself out of it before I could. Noop ;)

Quoted 6 lines

builtin/hook.c static const char * const builtin_hook_run_usage[] = { · line 62

Quoted 2 lines
builtin/hook.c +1 -0
+	strbuf_release(&hookdir_annotation);

This function did nothing with hookdir_annotation. Looks like leftover WIP code, but maybe it's used in (and should be moved to) a later commit, let's keep reading...

Ah, leftover WIP code indeed. Will drop it. I do think, though, that "hook from hookdir" is an ugly thing to say in list(), so any better suggestions welcome.

Quoted 8 lines

hook.c struct list_head *list_hooks(const char *hookname) · line 100

Quoted 6 lines

[...]

hook.c +1 -0
 	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
 
+
 	INIT_LIST_HEAD(hook_head);

..ditto...

ACK

Quoted 10 lines

hook.c struct list_head *list_hooks(const char *hookname) · line 106

Quoted 7 lines
hook.c +0 -1
 	if (!hookname)
@@ -103,8 +104,6 @@ struct list_head *list_hooks(const char *hookname)
 
 	if (have_git_dir()) {
 		const char *hook_path = find_hook(hookname);
-

... earlier notes about whitespace churn...

Quoted 5 lines

hook.c struct list_head *list_hooks(const char *hookname) · line 109

Quoted 4 lines
hook.c +0 -1
-		/* Add the hook from the hookdir */
 		if (hook_path) {
 			struct hook *to_add = xmalloc(sizeof(*to_add));
 			to_add->hook_path = hook_path;

Thanks.
- Emily

#580 lore raw

Re: [PATCH v3 3/6] hook: introduce "git hook list"

In reply to Emily Shaffer

builtin/hook.c static const char * const builtin_hook_run_usage[] = { · line 49

Quoted 1 line · Emily Shaffer
builtin/hook.c +1 -0
+	head = hook_list(hookname);

This doesn't compile, as hook_list() is added in a later commit. Something something earlier suggestion of "git rebase -i --exec 'make test'" :)

#581 lore raw

Re: [PATCH v3 3/6] hook: introduce "git hook list"

Quoted 9 lines · Ævar Arnfjörð Bjarmason

builtin/hook.c static const char * const builtin_hook_run_usage[] = { · line 49

Quoted 1 line · Emily Shaffer
builtin/hook.c +1 -0
+	head = hook_list(hookname);

This doesn't compile, as hook_list() is added in a later commit. Something something earlier suggestion of "git rebase -i --exec 'make test'" :)

Aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaargh. Thanks.

#582 lore raw

PATCH v3 4/6 hook: allow running non-native hooks

In reply to Emily Shaffer

As the hook architecture and 'git hook run' become more featureful, we may find wrappers wanting to use the hook architecture to run their own hooks, thereby getting nice things like parallelism and idiomatic Git configuration for free. Enable this by letting 'git hook run' bypass the known_hooks() check.

We do still want to keep known_hooks() around, though - by die()ing when an internal Git call asks for run_hooks("my-new-hook"), we can remind Git developers to update Documentation/githooks.txt with their new hook, which in turn helps Git users discover this new hook.

---
 Documentation/git-hook.txt |  8 ++++++++
 builtin/hook.c             |  4 ++--
 hook.c                     | 35 +++++++++++++++++++++++++++++++----
 hook.h                     | 14 ++++++++++++++
 4 files changed, 55 insertions(+), 6 deletions(-)
Documentation/git-hook.txt +8 -0
index 701ada9fc0..d1db084e4f 100644
@@ -19,6 +19,14 @@ This command is an interface to git hooks (see linkgit:githooks[5]).
 Currently it only provides a convenience wrapper for running hooks for
 use by git itself. In the future it might gain other functionality.
 
+It's possible to use this command to refer to hooks which are not native to Git,
+for example if a wrapper around Git wishes to expose hooks into its own
+operation in a way which is already familiar to Git users. However, wrappers
+invoking such hooks should be careful to name their hook events something which
+Git is unlikely to use for a native hook later on. For example, Git is much less
+likely to create a `mytool-validate-commit` hook than it is to create a
+`validate-commit` hook.
+
 SUBCOMMANDS
 -----------
 
builtin/hook.c +2 -2
index d21f303eca..80397d39f5 100644
@@ -46,7 +46,7 @@ static int list(int argc, const char **argv, const char *prefix)
 
 	hookname = argv[0];
 
-	head = hook_list(hookname);
+	head = list_hooks_gently(hookname);
 
 	if (list_empty(head))
 		return 1;
@@ -105,7 +105,7 @@ static int run(int argc, const char **argv, const char *prefix)
 	git_config(git_default_config, NULL);
 
 	hook_name = argv[0];
-	hooks = list_hooks(hook_name);
+	hooks = list_hooks_gently(hook_name);
 	if (list_empty(hooks)) {
 		/* ... act like run_hooks_oneshot() under --ignore-missing */
 		if (ignore_missing)
hook.c +31 -4
index b1ea372e15..ab1e86ddcf 100644
@@ -51,12 +51,21 @@ static int known_hook(const char *name)
 
 const char *find_hook(const char *name)
 {
-	static struct strbuf path = STRBUF_INIT;
+	const char *hook_path;
 
 	if (!known_hook(name))
-		die(_("the hook '%s' is not known to git, should be in hook-list.h via githooks(5)"),
+		BUG(_("the hook '%s' is not known to git, should be in hook-list.h via githooks(5)"),
 		    name);
 
+	hook_path = find_hook_gently(name);
+
+	return hook_path;
+}
+
+const char *find_hook_gently(const char *name)
+{
+	static struct strbuf path = STRBUF_INIT;
+
 	strbuf_reset(&path);
 	strbuf_git_path(&path, "hooks/%s", name);
 	if (access(path.buf, X_OK) < 0) {
@@ -92,10 +101,24 @@ int hook_exists(const char *name)
 	return !list_empty(list_hooks(name));
 }
 
+struct hook_config_cb
+{
+	struct strbuf *hook_key;
+	struct list_head *list;
+};
+
 struct list_head *list_hooks(const char *hookname)
 {
-	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
+	if (!known_hook(hookname))
+		BUG("Don't recognize hook event '%s'! "
+		    "Is it documented in Documentation/githooks.txt?",
+		    hookname);
+	return list_hooks_gently(hookname);
+}
 
+struct list_head *list_hooks_gently(const char *hookname)
+{
+	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
 
 	INIT_LIST_HEAD(hook_head);
 
@@ -103,7 +126,7 @@ struct list_head *list_hooks(const char *hookname)
 		BUG("null hookname was provided to hook_list()!");
 
 	if (have_git_dir()) {
-		const char *hook_path = find_hook(hookname);
+		const char *hook_path = find_hook_gently(hookname);
 		if (hook_path) {
 			struct hook *to_add = xmalloc(sizeof(*to_add));
 			to_add->hook_path = hook_path;
@@ -299,6 +322,10 @@ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 	if (options->path_to_stdin && options->feed_pipe)
 		BUG("choose only one method to populate stdin");
 
+	/*
+	 * 'git hooks run <hookname>' uses run_hooks, so we don't need to
+	 * allow unknown hooknames here.
+	 */
 	hooks = list_hooks(hook_name);
 
 	/*
hook.h +14 -0
index cd3123d290..6b7b2d14d2 100644
@@ -9,8 +9,16 @@
  * Returns the path to the hook file, or NULL if the hook is missing
  * or disabled. Note that this points to static storage that will be
  * overwritten by further calls to find_hook and run_hook_*.
+ *
+ * If the hook is not a native hook (e.g. present in Documentation/githooks.txt)
+ * find_hook() will BUG(). find_hook_gently() does not consult the native hook
+ * list and will check for any hook name in the hooks directory regardless of
+ * whether it is known. find_hook() should be used by internal calls to hooks,
+ * and find_hook_gently() should only be used when the hookname was provided by
+ * the user, such as by 'git hook run my-wrapper-hook'.
  */
 const char *find_hook(const char *name);
+const char *find_hook_gently(const char *name);
 
 /*
  * A boolean version of find_hook()
@@ -32,8 +40,14 @@ struct hook {
 /*
  * Provides a linked list of 'struct hook' detailing commands which should run
  * in response to the 'hookname' event, in execution order.
+ *
+ * list_hooks() checks the provided hookname against Documentation/githooks.txt
+ * and BUG()s if it is not found.  list_hooks_gently() allows any hookname. The
+ * latter should only be used when the hook name is provided by the user, and
+ * the former should be used by internal callers.
  */
 struct list_head *list_hooks(const char *hookname);
+struct list_head *list_hooks_gently(const char *hookname);
 
 struct run_hooks_opt
 {
Signature · 1 line
2.33.0.rc2.250.ged5fa647cd-goog
#583 lore raw

Re: [PATCH v3 4/6] hook: allow running non-native hooks

Quoted 13 lines · Emily Shaffer

As the hook architecture and 'git hook run' become more featureful, we may find wrappers wanting to use the hook architecture to run their own hooks, thereby getting nice things like parallelism and idiomatic Git configuration for free. Enable this by letting 'git hook run' bypass the known_hooks() check.

We do still want to keep known_hooks() around, though - by die()ing when an internal Git call asks for run_hooks("my-new-hook"), we can remind Git developers to update Documentation/githooks.txt with their new hook, which in turn helps Git users discover this new hook.

[...]

Documentation/git-hook.txt This command is an interface to git hooks (see linkgit:githooks[5]). · line 31

Quoted 10 lines
Documentation/git-hook.txt +8 -0
+It's possible to use this command to refer to hooks which are not native to Git,
+for example if a wrapper around Git wishes to expose hooks into its own
+operation in a way which is already familiar to Git users. However, wrappers
+invoking such hooks should be careful to name their hook events something which
+Git is unlikely to use for a native hook later on. For example, Git is much less
+likely to create a `mytool-validate-commit` hook than it is to create a
+`validate-commit` hook.
+
 SUBCOMMANDS
 -----------

The goal here makes sense, but...

Quoted 14 lines
builtin/hook.c +1 -1
index d21f303eca..80397d39f5 100644
@@ -46,7 +46,7 @@ static int list(int argc, const char **argv, const char *prefix)
 
 	hookname = argv[0];
 
-	head = hook_list(hookname);
+	head = list_hooks_gently(hookname);
 
 	if (list_empty(head))
 		return 1;

builtin/hook.c static int run(int argc, const char **argv, const char *prefix) · line 111

Quoted 8 lines
builtin/hook.c +1 -1
@@ -105,7 +105,7 @@ static int run(int argc, const char **argv, const char *prefix)
 	git_config(git_default_config, NULL);
 
 	hook_name = argv[0];
-	hooks = list_hooks(hook_name);
+	hooks = list_hooks_gently(hook_name);
 	if (list_empty(hooks)) {
 		/* ... act like run_hooks_oneshot() under --ignore-missing */
 		if (ignore_missing)

This introduces a bug v.s. the previous state, e.g. before:

    $ git hook run --ignore-missing foobar
    fatal: the hook 'foobar' is not known to git, should be in hook-list.h via githooks(5)

But after we'll silently ignore it. I.e. we've conflated --ignore-missing with a new and hypothetical (and this is now a synonym of) --ignore-missing-and-allow-unknown-hook-names.

So we've conflated the user's one-shot "foobar" script with wanting to catch a typo in e.g. git-send-email.perl.

Also instead of the user's typos being caught with a die (here using your BUG(...) version):

    $ git hook list pre-recive
    BUG: hook.c:115: Don't recognize hook event 'pre-recive'! Is it documented in Documentation/githooks.txt?
    Aborted

We'll now silently return 1, so indistinguishabl from typing it properly as pre-receive.

All that being said I think it's arguable that if we're going to allow "git hook run blahblah" that the die() in the base topic in my "hook-list.h: add a generated list of hooks, like config-list.h" is more trouble than it's worth.

I.e. do we really need to be concerned about new hooks being added and someone forgetting a githooks.txt update, or a typo in the git.git code that nobody notices? Probably not.

But I think the change here is clearly broken vis-a-vis the stated goals of its commit message as it stands, i.e. "[...]we do still want to keep known_hooks() around, though[...]". Should we fix it by adding a new internal-only flag to the command, or just saying we shouldn't have the behavior at all? What do you think.

Aside from that, this change seems to be untested, I tried making this non-gentle for testing, and all tests still passed. I.e. we don't have any tests for running such a hook like mytool-validate-commit, but should as part of this change.

#584 lore raw

Re: [PATCH v3 4/6] hook: allow running non-native hooks

Quoted 84 lines · Ævar Arnfjörð Bjarmason
Quoted 13 lines · Emily Shaffer

As the hook architecture and 'git hook run' become more featureful, we may find wrappers wanting to use the hook architecture to run their own hooks, thereby getting nice things like parallelism and idiomatic Git configuration for free. Enable this by letting 'git hook run' bypass the known_hooks() check.

We do still want to keep known_hooks() around, though - by die()ing when an internal Git call asks for run_hooks("my-new-hook"), we can remind Git developers to update Documentation/githooks.txt with their new hook, which in turn helps Git users discover this new hook.

[...]

Documentation/git-hook.txt This command is an interface to git hooks (see linkgit:githooks[5]). · line 31

Quoted 10 lines
Documentation/git-hook.txt +8 -0
+It's possible to use this command to refer to hooks which are not native to Git,
+for example if a wrapper around Git wishes to expose hooks into its own
+operation in a way which is already familiar to Git users. However, wrappers
+invoking such hooks should be careful to name their hook events something which
+Git is unlikely to use for a native hook later on. For example, Git is much less
+likely to create a `mytool-validate-commit` hook than it is to create a
+`validate-commit` hook.
+
 SUBCOMMANDS
 -----------

The goal here makes sense, but...

Quoted 14 lines
builtin/hook.c +1 -1
index d21f303eca..80397d39f5 100644
@@ -46,7 +46,7 @@ static int list(int argc, const char **argv, const char *prefix)
 
 	hookname = argv[0];
 
-	head = hook_list(hookname);
+	head = list_hooks_gently(hookname);
 
 	if (list_empty(head))
 		return 1;

builtin/hook.c static int run(int argc, const char **argv, const char *prefix) · line 111

Quoted 8 lines
builtin/hook.c +1 -1
@@ -105,7 +105,7 @@ static int run(int argc, const char **argv, const char *prefix)
 	git_config(git_default_config, NULL);
 
 	hook_name = argv[0];
-	hooks = list_hooks(hook_name);
+	hooks = list_hooks_gently(hook_name);
 	if (list_empty(hooks)) {
 		/* ... act like run_hooks_oneshot() under --ignore-missing */
 		if (ignore_missing)

This introduces a bug v.s. the previous state, e.g. before:

    $ git hook run --ignore-missing foobar
    fatal: the hook 'foobar' is not known to git, should be in hook-list.h via githooks(5)

But after we'll silently ignore it. I.e. we've conflated --ignore-missing with a new and hypothetical (and this is now a synonym of) --ignore-missing-and-allow-unknown-hook-names.

So we've conflated the user's one-shot "foobar" script with wanting to catch a typo in e.g. git-send-email.perl.

Also instead of the user's typos being caught with a die (here using your BUG(...) version):

    $ git hook list pre-recive
    BUG: hook.c:115: Don't recognize hook event 'pre-recive'! Is it documented in Documentation/githooks.txt?
    Aborted

We'll now silently return 1, so indistinguishabl from typing it properly as pre-receive.

All that being said I think it's arguable that if we're going to allow "git hook run blahblah" that the die() in the base topic in my "hook-list.h: add a generated list of hooks, like config-list.h" is more trouble than it's worth.

I.e. do we really need to be concerned about new hooks being added and someone forgetting a githooks.txt update, or a typo in the git.git code that nobody notices? Probably not.

Quoted 5 lines

But I think the change here is clearly broken vis-a-vis the stated goals of its commit message as it stands, i.e. "[...]we do still want to keep known_hooks() around, though[...]". Should we fix it by adding a new internal-only flag to the command, or just saying we shouldn't have the behavior at all? What do you think.

I think it's A) pretty important to make it easy for users to run whatever not-necessarily-git-native hook they want, and B) useful for script Git commands to take advantage of the typo check. So, I'll add a --enforce-known-hookname (or maybe a better named one, this isn't my strong suit) and switch git-send-email and friends to use it. Like we discussed off-list, I think it's a good idea to drop the envvar for exceptional test names from the codebase entirely.

Quoted 5 lines

Aside from that, this change seems to be untested, I tried making this non-gentle for testing, and all tests still passed. I.e. we don't have any tests for running such a hook like mytool-validate-commit, but should as part of this change.

Sure.

Actually, I was in the middle of typing about how I wouldn't change your 'test-hook' and so on tests, and it occurs to me that it might actually be a better fit for your series to add this --reject-unknown (or whatever) flag, instead of the envvar magics. So I'll hold off on making any changes unless I hear from you.

- Emily

#585 lore raw

Re: [PATCH v3 4/6] hook: allow running non-native hooks

Quoted 5 lines · Emily Shaffer

I think it's A) pretty important to make it easy for users to run whatever not-necessarily-git-native hook they want, and B) useful for script Git commands to take advantage of the typo check. So, I'll add a --enforce-known-hookname (or maybe a better named one, this isn't my strong suit) and switch git-send-email and friends to use it.

I somehow feel this is backwards.

Once you write the invocation of "git hook run <hookname>" into your script and tested it, there is no further need for typo checking.

What's the use case you are trying to help with typo checking? When a script takes a hookname from the user and runs "git hook run $1", then passing --this-must-be-a-known-hook option that errors out when the named hook does not exist and unrecognised (there is no need to error out if a hook with unusual name the user gave us does exist---the user asked us to run it, so we just can run it) might make sense. But I am somehow not getting the sense that it is the expected use case you are worried about.

If the reason why you are making the typo-checking an opt-in feature is because you want to allow users to run "git hook run" with minimum typing, I suspect that you may be optimizing for a wrong case. Interactive users are the ones that benefit from typo-checking the most, and if they are interactive (as opposed to being a script), they are flexible enough not to say "git hook run foobar" when they know foobar does not exist and they know foobar is not a generally accepted hook, no? So, I think it makes more sense to by default allow a hook with a recognizable name to be missing, but complain when a randomly named hook is missing, and to have an option that permits a hook to be unrecognised and missing.

#586 lore raw

PATCH v3 5/6 hook: include hooks from the config

In reply to Emily Shaffer

Teach the hook.[hc] library to parse configs to populare the list of hooks to run for a given event.

Multiple commands can be specified for a given hook by providing multiple "hook.<friendly-name>.command = <path-to-hook>" and "hook.<friendly-name>.event = <hook-event>" lines. Hooks will be run in config order of the "hook.<name>.event" lines.

For example:

  $ git config --list | grep ^hook
  hook.bar.command=~/bar.sh
  hook.bar.event=pre-commit
  $ git hook run
  # Runs ~/bar.sh
  # Runs .git/hooks/pre-commit
---
 Documentation/config/hook.txt |  18 ++++
 Documentation/git-hook.txt    |  57 ++++++++++++-
 builtin/hook.c                |   3 +-
 hook.c                        | 153 ++++++++++++++++++++++++++++++----
 hook.h                        |   7 +-
 t/t1800-hook.sh               | 141 ++++++++++++++++++++++++++++++-
 6 files changed, 357 insertions(+), 22 deletions(-)
Documentation/config/hook.txt +18 -0
index 96d3d6572c..c394756328 100644
@@ -1,3 +1,21 @@
+hook.<name>.command::
+	A command to execute whenever `hook.<name>` is invoked. `<name>` should
+	be a unique "friendly" name which you can use to identify this hook
+	command. (You can specify when to invoke this command with
+	`hook.<name>.event`.) The value can be an executable on your device or a
+	oneliner for your shell. If more than one value is specified for the
+	same `<name>`, the last value parsed will be the only command executed.
+	See linkgit:git-hook[1].
+
+hook.<name>.event::
+	The hook events which should invoke `hook.<name>`. `<name>` should be a
+	unique "friendly" name which you can use to identify this hook. The
+	value should be the name of a hook event, like "pre-commit" or "update".
+	(See linkgit:githooks[5] for a complete list of hooks Git knows about.)
+	On the specified event, the associated `hook.<name>.command` will be
+	executed. More than one event can be specified if you wish for
+	`hook.<name>` to execute on multiple events. See linkgit:git-hook[1].
+
 hook.jobs::
 	Specifies how many hooks can be run simultaneously during parallelized
 	hook execution. If unspecified, defaults to the number of processors on
Documentation/git-hook.txt +55 -2

67 lines not on this page · raw message

builtin/hook.c +2 -1
index 80397d39f5..50233366a8 100644
@@ -55,7 +55,8 @@ static int list(int argc, const char **argv, const char *prefix)
 		struct hook *item = list_entry(pos, struct hook, list);
 		item = list_entry(pos, struct hook, list);
 		if (item)
-			printf("%s\n", item->hook_path);
+			printf("%s\n", item->name ? item->name
+						  : _("hook from hookdir"));
 	}
 
 	clear_hook_list(head);
hook.c +137 -16

193 lines not on this page · raw message

hook.h +5 -2
index 6b7b2d14d2..621bd2cde1 100644
@@ -27,8 +27,11 @@ int hook_exists(const char *hookname);
 
 struct hook {
 	struct list_head list;
-	/* The path to the hook */
-	const char *hook_path;
+	/*
+	 * The friendly name of the hook. NULL indicates the hook is from the
+	 * hookdir.
+	 */
+	const char *name;
 
 	/*
 	 * Use this to keep state for your feed_pipe_fn if you are using
t/t1800-hook.sh +140 -1

157 lines not on this page · raw message

Signature · 1 line
2.33.0.rc2.250.ged5fa647cd-goog
#587 lore raw

Re: [PATCH v3 5/6] hook: include hooks from the config

Quoted 12 lines · Emily Shaffer
Documentation/config/hook.txt +7 -0
index 96d3d6572c..c394756328 100644
@@ -1,3 +1,21 @@
+hook.<name>.command::
+	A command to execute whenever `hook.<name>` is invoked. `<name>` should
+	be a unique "friendly" name which you can use to identify this hook
+	command. (You can specify when to invoke this command with
+	`hook.<name>.event`.) The value can be an executable on your device or a
+	oneliner for your shell. If more than one value is specified for the
+	same `<name>`, the last value parsed will be the only command executed.

Documentation/config/hook.txt line 17

Quoted 10 lines
Documentation/config/hook.txt +10 -0
+	See linkgit:git-hook[1].
+
+hook.<name>.event::
+	The hook events which should invoke `hook.<name>`. `<name>` should be a
+	unique "friendly" name which you can use to identify this hook. The
+	value should be the name of a hook event, like "pre-commit" or "update".
+	(See linkgit:githooks[5] for a complete list of hooks Git knows about.)
+	On the specified event, the associated `hook.<name>.command` will be
+	executed. More than one event can be specified if you wish for
+	`hook.<name>` to execute on multiple events. See linkgit:git-hook[1].

Looking much better. It now gives enough information to readers to understand (if not enough to convince that it is a good idea) why an indirection with "friendly name" between the event and command is there. In short, <name> names the command to be run and without indirection, you'd end up having to write:

[hook "check-whitespace && spellcheck-log-message"]
event = pre-commit
[hook "check-whitespace && spellcheck-log-message"]
event = another-hookable-event

which may give the same expressiveness (and may even be workable if the configuration were machine generated) but it is typo-prone, and a single typo, or even an insignificant whitespace change in the command, would destroy the grouping of "this command fires upon these events".

It becomes much less typo prone with the indirection, i.e.

[hook "logcheck"]
command = check-whitespace && spellcheck-log-message

[hook "logcheck"]
event = pre-commit

[hook "logcheck"]
event = another-hookable-event

using the "friendly name", especially if these entries are spread across different configuration files.

My original question was primarily because I thought the second-level <name> corresponded to <event>. If that were the case, it can trivially be made simpler without making it typo-prone, i.e.

    [hook "pre-commit"]
	command = check-whitespace && spellcheck-log-message

    [hook "another-hookable-event"]
	command = check-whitespace && spellcheck-log-message

since the name of the event would be much shorter than the command line. But since we are not grouping per hookable event (to apply the "last one wins" rule to determine which single command is the one that gets to run).

Thanks.

#588 lore raw

Re: [PATCH v3 5/6] hook: include hooks from the config

Quoted 67 lines · Junio C Hamano
Quoted 12 lines · Emily Shaffer
Documentation/config/hook.txt +7 -0
index 96d3d6572c..c394756328 100644
@@ -1,3 +1,21 @@
+hook.<name>.command::
+	A command to execute whenever `hook.<name>` is invoked. `<name>` should
+	be a unique "friendly" name which you can use to identify this hook
+	command. (You can specify when to invoke this command with
+	`hook.<name>.event`.) The value can be an executable on your device or a
+	oneliner for your shell. If more than one value is specified for the
+	same `<name>`, the last value parsed will be the only command executed.

Documentation/config/hook.txt line 17

Quoted 10 lines
Documentation/config/hook.txt +10 -0
+	See linkgit:git-hook[1].
+
+hook.<name>.event::
+	The hook events which should invoke `hook.<name>`. `<name>` should be a
+	unique "friendly" name which you can use to identify this hook. The
+	value should be the name of a hook event, like "pre-commit" or "update".
+	(See linkgit:githooks[5] for a complete list of hooks Git knows about.)
+	On the specified event, the associated `hook.<name>.command` will be
+	executed. More than one event can be specified if you wish for
+	`hook.<name>` to execute on multiple events. See linkgit:git-hook[1].

Looking much better. It now gives enough information to readers to understand (if not enough to convince that it is a good idea) why an indirection with "friendly name" between the event and command is there. In short, <name> names the command to be run and without indirection, you'd end up having to write:

[hook "check-whitespace && spellcheck-log-message"]
event = pre-commit
[hook "check-whitespace && spellcheck-log-message"]
event = another-hookable-event

which may give the same expressiveness (and may even be workable if the configuration were machine generated) but it is typo-prone, and a single typo, or even an insignificant whitespace change in the command, would destroy the grouping of "this command fires upon these events".

It becomes much less typo prone with the indirection, i.e.

[hook "logcheck"]
command = check-whitespace && spellcheck-log-message

[hook "logcheck"]
event = pre-commit

[hook "logcheck"]
event = another-hookable-event

using the "friendly name", especially if these entries are spread across different configuration files.

My original question was primarily because I thought the second-level <name> corresponded to <event>. If that were the case, it can trivially be made simpler without making it typo-prone, i.e.

    [hook "pre-commit"]
	command = check-whitespace && spellcheck-log-message

    [hook "another-hookable-event"]
	command = check-whitespace && spellcheck-log-message
Quoted 5 lines

since the name of the event would be much shorter than the command line. But since we are not grouping per hookable event (to apply the "last one wins" rule to determine which single command is the one that gets to run).

To be clear, the config schema did work the way you describe until this revision. Ævar suggested the change and it seemed like a good idea to me (and the rest of the Google folks) so I changed between v2 and v3 of the restart.

- Emily

#589 lore raw

Re: [PATCH v3 5/6] hook: include hooks from the config

Quoted 18 lines · Emily Shaffer
Quoted 11 lines · Junio C Hamano

My original question was primarily because I thought the second-level <name> corresponded to <event>. If that were the case, it can trivially be made simpler without making it typo-prone, i.e.

    [hook "pre-commit"]
	command = check-whitespace && spellcheck-log-message

    [hook "another-hookable-event"]
	command = check-whitespace && spellcheck-log-message
Quoted 5 lines

since the name of the event would be much shorter than the command line. But since we are not grouping per hookable event (to apply the "last one wins" rule to determine which single command is the one that gets to run).

Quoted 4 lines

To be clear, the config schema did work the way you describe until this revision. Ævar suggested the change and it seemed like a good idea to me (and the rest of the Google folks) so I changed between v2 and v3 of the restart.

To be clear, you do not want to take the above as my suggestion to go back to the previous one, since that is not what I meant. As long as you and others are happy with what you folks ended up with, i.e. the current one that uses <name> that is a short-hand for <command>, that is what matters.

Thanks.

#590 lore raw

Re: [PATCH v3 5/6] hook: include hooks from the config

In reply to Emily Shaffer
Quoted 2 lines · Emily Shaffer

Teach the hook.[hc] library to parse configs to populare the list of hooks to run for a given event.

s/populare/populate/

Quoted 4 lines

Multiple commands can be specified for a given hook by providing multiple "hook.<friendly-name>.command = <path-to-hook>" and "hook.<friendly-name>.event = <hook-event>" lines. Hooks will be run in config order of the "hook.<name>.event" lines.

The "will be run in order" probably needs some amending to account for --jobs, no?

Quoted 5 lines

For example:

  $ git config --list | grep ^hook
  hook.bar.command=~/bar.sh
  hook.bar.event=pre-commit

Perhaps the example should use:

    git config --get-regexp '^hook\.'
Quoted 3 lines
  $ git hook run
  # Runs ~/bar.sh
  # Runs .git/hooks/pre-commit

And this "# Runs" is not actual output by git, but just an explanation for what happens, better to reword it somehow so it doesn't give that impression.

But the example also seems to be broken, surely it should be "git hook run bar", not "git hook run"? Reading ahead, but presumably no-arg doesn't run all known hooks...

Quoted 15 lines
---
 Documentation/config/hook.txt |  18 ++++
 Documentation/git-hook.txt    |  57 ++++++++++++-
 builtin/hook.c                |   3 +-
 hook.c                        | 153 ++++++++++++++++++++++++++++++----
 hook.h                        |   7 +-
 t/t1800-hook.sh               | 141 ++++++++++++++++++++++++++++++-
 6 files changed, 357 insertions(+), 22 deletions(-)
Documentation/config/hook.txt +0 -0
index 96d3d6572c..c394756328 100644

Documentation/config/hook.txt line 8

Quoted 8 lines
Documentation/config/hook.txt +8 -0
@@ -1,3 +1,21 @@
+hook.<name>.command::
+	A command to execute whenever `hook.<name>` is invoked. `<name>` should
+	be a unique "friendly" name which you can use to identify this hook
+	command. (You can specify when to invoke this command with
+	`hook.<name>.event`.) The value can be an executable on your device or a
+	oneliner for your shell. If more than one value is specified for the
+	same `<name>`, the last value parsed will be the only command executed.
+	See linkgit:git-hook[1].

Hrm, so here we say "If more than one value is specified for ... the last value parsed will be the only command executed", but in the commit message it's:

Multiple commands can be specified for a given hook by providing multiple "hook.<friendly-name>.command = <path-to-hook>" and "hook.<friendly-name>.event = <hook-event>" lines.

As we've discussed earlier I've got a preference for the former, but just reading this commit message & doc the for the first time I still don't know what you went for, looks like one or the other needs updating. I'm intentionally not reading ahead as I review this.

Saying that it's a "unique name", but also discussing what happens if it's not unique in the sense that we have multiple "hook.<name>.*" is a bit confusing. I think I know what you're going for, perhaps something like this would be better to describe it:

For a "hook.<name>.{command,event}" hook entry you'll need to pick a "<name>" that's not shared with any other hook, if you do normal single-value config semantics apply and git will use the last provided value.

Or something...

Documentation/config/hook.txt line 12

Quoted 3 lines
Documentation/config/hook.txt +3 -0
+hook.<name>.event::
+	The hook events which should invoke `hook.<name>`. `<name>` should be a
+	unique "friendly" name which you can use to identify this hook. The

p> + value should be the name of a hook event, like "pre-commit" or "update".

Quoted 15 lines
Documentation/config/hook.txt +5 -0
+	(See linkgit:githooks[5] for a complete list of hooks Git knows about.)
+	On the specified event, the associated `hook.<name>.command` will be
+	executed. More than one event can be specified if you wish for
+	`hook.<name>` to execute on multiple events. See linkgit:git-hook[1].
+
 hook.jobs::
 	Specifies how many hooks can be run simultaneously during parallelized
 	hook execution. If unspecified, defaults to the number of processors on
Documentation/git-hook.txt +0 -0
index d1db084e4f..9c6cbdc2eb 100644
@@ -27,12 +27,65 @@ Git is unlikely to use for a native hook later on. For example, Git is much less
 likely to create a `mytool-validate-commit` hook than it is to create a
 `validate-commit` hook.

Documentation/git-hook.txt Git is unlikely to use for a native hook later on. For example, Git is much less · line 38

Quoted 10 lines
Documentation/git-hook.txt +9 -0
 
+This command parses the default configuration files for pairs of configs like
+so:
+
+  [hook "linter"]
+    event = pre-commit
+    command = ~/bin/linter --c
+
+In this example, `[hook "linter"]` represents one script - `~/bin/linter --c` -
+which can be shared by many repos, and even by many hook events, if appropriate.

OK, so now it seems like "hook.<name>.command" is 1=1 for "hook.<name>" and "command", and "hook.<name>.event" is 1=many, maybe that's correct, but reading on...

Documentation/git-hook.txt Git is unlikely to use for a native hook later on. For example, Git is much less · line 44

Quoted 5 lines
Documentation/git-hook.txt +5 -0
+Commands are run in the order Git encounters their associated
+`hook.<name>.event` configs during the configuration parse (see
+linkgit:git-config[1]). Although multiple `hook.linter.event` configs can be
+added, only one `hook.linter.command` event is valid - Git uses "last-one-wins"
+to determine which command to run.

...ah, and confirmed here...

Documentation/git-hook.txt Git is unlikely to use for a native hook later on. For example, Git is much less · line 56

Quoted 11 lines
Documentation/git-hook.txt +11 -0
+So if you wanted your linter to run when you commit as well as when you push,
+you would configure it like so:
+
+  [hook "linter"]
+    event = pre-commit
+    event = pre-push
+    command = ~/bin/linter --c
+
+With this config, `~/bin/linter --c` would be run by Git before a commit is
+generated (during `pre-commit`) as well as before a push is performed (during
+`pre-push`).

Aside: I know we're discussing a presumably imaginary linter, but it's a bit jarring to see "--" for a one-letter short-option, perhaps "-c" or "--check" for the example...

Documentation/git-hook.txt Git is unlikely to use for a native hook later on. For example, Git is much less · line 66

Quoted 9 lines
Documentation/git-hook.txt +9 -0
+And if you wanted to run your linter as well as a secret-leak detector during
+only the "pre-commit" hook event, you would configure it instead like so:
+
+  [hook "linter"]
+    event = pre-commit
+    command = ~/bin/linter --c
+  [hook "no-leaks"]
+    event = pre-commit
+    command = ~/bin/leak-detector

I think these examples would flow a bit more naturally if we started by discussing how to setup one configured hook, then two unrelated hooks (perhaps a "commit-msg" and "pre-commit" hook), and then finally the cases where a given "hook.<name>.command" has multiple "hook.<name>.event" entries. My assumption in suggesting that is that that's, respectively, the most common to less common use cases.

Documentation/git-hook.txt Git is unlikely to use for a native hook later on. For example, Git is much less · line 73

Quoted 6 lines
Documentation/git-hook.txt +6 -0
+With this config, before a commit is generated (during `pre-commit`), Git would
+first start `~/bin/linter --c` and second start `~/bin/leak-detector`. It would
+evaluate the output of each when deciding whether to proceed with the commit.
+
+For a full list of hook events which you can set your `hook.<name>.event` to,
+and how hooks are invoked during those events, see linkgit:githooks[5].

Let's discuss what happens with unknown values for hook.<name>.event here, i.e. just "are ignored".

[I'll discuss my opinions on the new and revised config schema in another mail, just commenting on the patch here in terms of its stated goals].

Documentation/git-hook.txt Git is unlikely to use for a native hook later on. For example, Git is much less · line 81

Quoted 7 lines
Documentation/git-hook.txt +7 -0
+In general, when instructions suggest adding a script to
+`.git/hooks/<hook-event>`, you can specify it in the config instead by running
+`git config --add hook.<some-name>.command <path-to-script> && git config --add
+hook.<some-name>.event <hook-event>` - this way you can share the script between
+multiple repos. That is, `cp ~/my-script.sh ~/project/.git/hooks/pre-commit`
+would become `git config --add hook.my-script.command ~/my-script.sh && git
+config --add hook.my-script.event pre-commit`.

I think this would benefit a lot from being split up into code example prose, so:

    You can [...]

    ----
    git config -add hook.<some-name>.command <path-to-script> &&
    git config -add hook.<some-name>.event <hook-event>
    ----

It's more lines, but I think a lot more readable.

I think the part of this that says "You can share the script between multiple repos" could really use some elaboration.

I.e. if the user is following an example where they'd otherwise edit .git/hooks/ and then just run "git config", they'll add it to .git/config.

So then it won't be shared between multiple repos, what I think this paragraph is trying to go for is that instead of say:

ln -s ~/my-script.sh .git/hooks/pre-commit

You can instead do, with configurable hooks:

git config hook.my-pre-commit.command ~/my-script.sh git config --add hook.my-pre-commit.event pre-commit

Notice how I omitted the --add for the first one. It's also confusing if we're documenting "*.command" as "last one wins" to use --add with it, in an example that's discussing how to add it to some local repo, no?

Or is this example really trying to discuss what would happen if you:

    cat >~/my-script.sh
    ...
    ^C
    chmod +x ~/my-script.sh
    git config --global hook.my-pre-commit.command ~/my-script.sh
    git config hook.my-pre-commit.event pre-commit

That's not clear to me, in any case, I think we'd do well to lead with the former, and then discuss the latter. I.e. "here's what you'd symlink before, now it's in config like this", and then discuss how you'd set "global" or perhaps included config, which isn't possible with the .git/hooks/<script> mechanism.

Documentation/git-hook.txt Git is unlikely to use for a native hook later on. For example, Git is much less · line 88

Quoted 8 lines
Documentation/git-hook.txt +2 -2
 SUBCOMMANDS
 -----------
 
 run::
-	Run the `<hook-name>` hook. See linkgit:githooks[5] for
-	the hook names we support.
+	Runs hooks configured for `<hook-name>`, in the order they are
+	discovered during the config parse.

This and any other things that discuss how hooks are run in detail should really talk about the thing running .git/hooks/<name> *and* any configured hooks, and on the subject of --jobs and ordering we should also talk about what the order of config v.s. .git/hooks/<name> is.

Documentation/git-hook.txt Git is unlikely to use for a native hook later on. For example, Git is much less · line 84

Quoted 2 lines
Documentation/git-hook.txt +2 -0
+	/* to enable oneliners, let config-specified hooks run in shell.
+	 * config-specified hooks have a name. */

nit: usual style is multi-line comments like:

/*
* some text[...]
* some more...
*/

Not:

/* text here right away[...]
* some more ... */

Documentation/git-hook.txt Git is unlikely to use for a native hook later on. For example, Git is much less

Quoted 13 lines
Documentation/git-hook.txt +8 -4
+	cp->use_shell = !!run_me->name;
+
 	/* add command */
-	if (hook_cb->options->absolute_path)
-		strvec_push(&cp->args, absolute_path(run_me->hook_path));
-	else
-		strvec_push(&cp->args, run_me->hook_path);
+	if (run_me->name) {
+		/* ...from config */
+		struct strbuf cmd_key = STRBUF_INIT;
+		char *command = NULL;
+
+		strbuf_addf(&cmd_key, "hook.%s.command", run_me->name);

Missing strbuf_release() for this later?

Documentation/git-hook.txt

Quoted 2 lines
Documentation/git-hook.txt +2 -0
+		if (git_config_get_string(cmd_key.buf, &command)) {
+			/* TODO test me! */

...seems easy enough to just have a test for..., i.e. an *.event entry with no *.command.

Documentation/git-hook.txt

Quoted 11 lines
Documentation/git-hook.txt +11 -0
+			die(_("'hook.%s.command' must be configured "
+			      "or 'hook.%s.event' must be removed; aborting.\n"),
+			    run_me->name, run_me->name);
+		}
+
+		strvec_push(&cp->args, command);
+	} else {
+		/* ...from hookdir. */
+		const char *hook_path = NULL;
+		/*
+		 *

Nit: Too few \n before the text in a comment earlier, too many here :)

Documentation/git-hook.txt

Quoted 2 lines
Documentation/git-hook.txt +2 -0
+		 * At this point we are already running, so don't validate
+		 * whether the hook name is known or not.

...because it was done earlier somewhere, or...?

Quoted 53 lines
Documentation/git-hook.txt +16 -2
+		 */
+		hook_path = find_hook_gently(hook_cb->hook_name);
+		if (!hook_path)
+			BUG("hookdir hook in hook list but no hookdir hook present in filesystem");
+
+		if (hook_cb->options->absolute_path)
+			hook_path = absolute_path(hook_path);
+
+		strvec_push(&cp->args, hook_path);
+	}
+
 
 	/*
 	 * add passed-in argv, without expanding - let the user get back
@@ -228,8 +346,11 @@ static int notify_start_failure(struct strbuf *out,
 
 	hook_cb->rc |= 1;
 
-	strbuf_addf(out, _("Couldn't start hook '%s'\n"),
-		    attempted->hook_path);
+	if (attempted->name)
+		strbuf_addf(out, _("Couldn't start hook '%s'\n"),
+		    attempted->name);
+	else
+		strbuf_addstr(out, _("Couldn't start hook from hooks directory\n"));
 
 	return 1;
 }
hook.h +5 -2
index 6b7b2d14d2..621bd2cde1 100644
@@ -27,8 +27,11 @@ int hook_exists(const char *hookname);
 
 struct hook {
 	struct list_head list;
-	/* The path to the hook */
-	const char *hook_path;
+	/*
+	 * The friendly name of the hook. NULL indicates the hook is from the
+	 * hookdir.
+	 */
+	const char *name;
 
 	/*
 	 * Use this to keep state for your feed_pipe_fn if you are using
t/t1800-hook.sh +0 -0
index 217db848b3..ef2432f53a 100755
@@ -1,13 +1,29 @@
 #!/bin/bash
 

t/t1800-hook.sh line 11

Quoted 10 lines
t/t1800-hook.sh +6 -1
-test_description='git-hook command'
+test_description='git-hook command and config-managed multihooks'
 
 . ./test-lib.sh
 
+setup_hooks () {
+	test_config hook.ghi.event pre-commit --add
+	test_config hook.ghi.command "/path/ghi" --add
+	test_config_global hook.def.event pre-commit --add
+	test_config_global hook.def.command "/path/def" --add

Isn't --add redundant here? Seems no test is stressing multi-value hook.{ghi,def}.* from a quick glance...

Quoted 5 lines
t/t1800-hook.sh +5 -0
+}
+
+setup_hookdir () {
+	mkdir .git/hooks
+	write_script .git/hooks/pre-commit <<-EOF

t/t1800-hook.sh line 26

Quoted 10 lines
t/t1800-hook.sh +6 -0
+	echo \"Legacy Hook\"
+	EOF
+	test_when_finished rm -rf .git/hooks
+}
+
 test_expect_success 'git hook usage' '
 	test_expect_code 129 git hook &&
 	test_expect_code 129 git hook run &&
 	test_expect_code 129 git hook run -h &&
+	test_expect_code 129 git hook list -h &&

Doesn't this belong in a previous commit that added "git hook list", not here?

Quoted 15 lines
t/t1800-hook.sh +8 -0
 	test_expect_code 129 git hook run --unknown 2>err &&
 	grep "unknown option" err
 '
@@ -153,4 +169,127 @@ test_expect_success 'stdin to hooks' '
 	test_cmp expect actual
 '
 
+test_expect_success 'git hook list orders by config order' '
+	setup_hooks &&
+
+	cat >expected <<-EOF &&
+	def
+	ghi
+	EOF
+

t/t1800-hook.sh test_expect_success 'stdin to hooks' ' · line 189

Quoted 10 lines
t/t1800-hook.sh +10 -0
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook list reorders on duplicate event declarations' '
+	setup_hooks &&
+
+	# 'def' is usually configured globally; move it to the end by
+	# configuring it locally.
+	test_config hook.def.event "pre-commit" --add &&

Ah, well the --add belongs here, but not needed in setup_hooks, right?

Quoted 69 lines
t/t1800-hook.sh +69 -0
+
+	cat >expected <<-EOF &&
+	ghi
+	def
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook list shows hooks from the hookdir' '
+	setup_hookdir &&
+
+	cat >expected <<-EOF &&
+	hook from hookdir
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'inline hook definitions execute oneliners' '
+	test_config hook.oneliner.event "pre-commit" &&
+	test_config hook.oneliner.command "echo \"Hello World\"" &&
+
+	echo "Hello World" >expected &&
+
+	# hooks are run with stdout_to_stderr = 1
+	git hook run pre-commit 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'inline hook definitions resolve paths' '
+	write_script sample-hook.sh <<-EOF &&
+	echo \"Sample Hook\"
+	EOF
+
+	test_when_finished "rm sample-hook.sh" &&
+
+	test_config hook.sample-hook.event pre-commit &&
+	test_config hook.sample-hook.command "\"$(pwd)/sample-hook.sh\"" &&
+
+	echo \"Sample Hook\" >expected &&
+
+	# hooks are run with stdout_to_stderr = 1
+	git hook run pre-commit 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'hookdir hook included in git hook run' '
+	setup_hookdir &&
+
+	echo \"Legacy Hook\" >expected &&
+
+	# hooks are run with stdout_to_stderr = 1
+	git hook run pre-commit 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'stdin to multiple hooks' '
+	test_config hook.stdin-a.event "test-hook" --add &&
+	test_config hook.stdin-a.command "xargs -P1 -I% echo a%" --add &&
+	test_config hook.stdin-b.event "test-hook" --add &&
+	test_config hook.stdin-b.command "xargs -P1 -I% echo b%" --add &&
+
+	cat >input <<-EOF &&
+	1
+	2
+	3

t/t1800-hook.sh test_expect_success 'stdin to hooks' ' · line 268

Quoted 10 lines
t/t1800-hook.sh +10 -0
+	EOF
+
+	cat >expected <<-EOF &&
+	a1
+	a2
+	a3
+	b1
+	b2
+	b3
+	EOF

For any here-docs without variables, use <<-\EOF, note the backslash.

#591 lore raw

Re: [PATCH v3 5/6] hook: include hooks from the config

On Tue, Aug 24, 2021 at 09:30:25PM +0200, Ævar Arnfjörð Bjarmason wrote:

Disclaimer: I was writing a pretty involved reply to this and my tmux session crashed, so hopefully I can recall it well enough. Sorry if anything is confusing :)

Quoted 7 lines · Emily Shaffer
Quoted 3 lines

Teach the hook.[hc] library to parse configs to populare the list of hooks to run for a given event.

s/populare/populate/

ack

Quoted 8 lines
Quoted 5 lines

Multiple commands can be specified for a given hook by providing multiple "hook.<friendly-name>.command = <path-to-hook>" and "hook.<friendly-name>.event = <hook-event>" lines. Hooks will be run in config order of the "hook.<name>.event" lines.

The "will be run in order" probably needs some amending to account for --jobs, no?

I changed it to "started in order", and tacked on "(but may be run in parallel)".

Quoted 10 lines
Quoted 6 lines

For example:

  $ git config --list | grep ^hook
  hook.bar.command=~/bar.sh
  hook.bar.event=pre-commit

Perhaps the example should use:

    git config --get-regexp '^hook\.'

Sure, I better not inflict my crappy habits on readers ;)

Quoted 5 lines
Quoted 4 lines
  $ git hook run
  # Runs ~/bar.sh
  # Runs .git/hooks/pre-commit
Quoted 7 lines

And this "# Runs" is not actual output by git, but just an explanation for what happens, better to reword it somehow so it doesn't give that impression.

But the example also seems to be broken, surely it should be "git hook run bar", not "git hook run"? Reading ahead, but presumably no-arg doesn't run all known hooks...

Ah, even the suggestion was wrong - it should be git hook run pre-commit. Zzzz. :)

Quoted 43 lines
Quoted 16 lines
---
 Documentation/config/hook.txt |  18 ++++
 Documentation/git-hook.txt    |  57 ++++++++++++-
 builtin/hook.c                |   3 +-
 hook.c                        | 153 ++++++++++++++++++++++++++++++----
 hook.h                        |   7 +-
 t/t1800-hook.sh               | 141 ++++++++++++++++++++++++++++++-
 6 files changed, 357 insertions(+), 22 deletions(-)
Documentation/config/hook.txt +0 -0
index 96d3d6572c..c394756328 100644

Documentation/config/hook.txt line 8

Quoted 8 lines
Documentation/config/hook.txt +8 -0
@@ -1,3 +1,21 @@
+hook.<name>.command::
+	A command to execute whenever `hook.<name>` is invoked. `<name>` should
+	be a unique "friendly" name which you can use to identify this hook
+	command. (You can specify when to invoke this command with
+	`hook.<name>.event`.) The value can be an executable on your device or a
+	oneliner for your shell. If more than one value is specified for the
+	same `<name>`, the last value parsed will be the only command executed.
+	See linkgit:git-hook[1].

Hrm, so here we say "If more than one value is specified for ... the last value parsed will be the only command executed", but in the commit message it's:

Multiple commands can be specified for a given hook by providing multiple "hook.<friendly-name>.command = <path-to-hook>" and "hook.<friendly-name>.event = <hook-event>" lines.

As we've discussed earlier I've got a preference for the former, but just reading this commit message & doc the for the first time I still don't know what you went for, looks like one or the other needs updating. I'm intentionally not reading ahead as I review this.

Saying that it's a "unique name", but also discussing what happens if it's not unique in the sense that we have multiple "hook.<name>.*" is a bit confusing. I think I know what you're going for, perhaps something like this would be better to describe it:

Quoted 6 lines

For a "hook.<name>.{command,event}" hook entry you'll need to pick a "<name>" that's not shared with any other hook, if you do normal single-value config semantics apply and git will use the last provided value.

Or something...

Yeah, I ended up reworking this a lot. I think the manpage needs some structuring work, to be honest - I wrote a lot about how to use 'git hook run' to add hooks to your wrapper around Git, for example, and stuck it in a section.

While I'm waiting for your next reroll, I'm planning to send the doc to a tech writer we've got on loan internally and see if they've got any tips for us here. Hopefully they can help us out :)

I'm going to snip the rest of the doc comments, because while I know I took action on them last week, I'm not sure I recall what I changed; and I'm hoping to get a third party to take a look. I did read them all, I promise :)

Quoted 10 lines

Documentation/config/hook.txt line 10

Quoted 2 lines
Documentation/config/hook.txt +2 -0
+	/* to enable oneliners, let config-specified hooks run in shell.
+	 * config-specified hooks have a name. */

nit: usual style is multi-line comments like:

/*
* some text[...]
* some more...
*/

Quoted 4 lines

Not:

/* text here right away[...]
* some more ... */

ACK. By the way, anybody have tips for making Vim gracefully transition from

/* happily typing a single line comment

to

/*
* happily typing a single line comment that
* became super long

Wonder if anything is working quite well for anybody, because I mess this up all the time ;)

I guess I could just check for this kind of thing at pre-commit time. Maybe with a hook. ;) ;)

Quoted 16 lines
Quoted 4 lines
Documentation/config/hook.txt +2 -0
+	cp->use_shell = !!run_me->name;
+
 	/* add command */

Documentation/config/hook.txt

Quoted 10 lines
Documentation/config/hook.txt +6 -4
-	if (hook_cb->options->absolute_path)
-		strvec_push(&cp->args, absolute_path(run_me->hook_path));
-	else
-		strvec_push(&cp->args, run_me->hook_path);
+	if (run_me->name) {
+		/* ...from config */
+		struct strbuf cmd_key = STRBUF_INIT;
+		char *command = NULL;
+
+		strbuf_addf(&cmd_key, "hook.%s.command", run_me->name);

Missing strbuf_release() for this later?

Yep, thanks.

Quoted 6 lines

Documentation/config/hook.txt

Quoted 3 lines
Documentation/config/hook.txt +2 -0
+		if (git_config_get_string(cmd_key.buf, &command)) {
+			/* TODO test me! */

...seems easy enough to just have a test for..., i.e. an *.event entry with no *.command.

Yeah, this TODO was probably for myself, so I wouldn't push the reroll without writing the test. That worked really well....

Quoted 14 lines

Documentation/config/hook.txt

Quoted 12 lines
Documentation/config/hook.txt +11 -0
+			die(_("'hook.%s.command' must be configured "
+			      "or 'hook.%s.event' must be removed; aborting.\n"),
+			    run_me->name, run_me->name);
+		}
+
+		strvec_push(&cp->args, command);
+	} else {
+		/* ...from hookdir. */
+		const char *hook_path = NULL;
+		/*
+		 *

Nit: Too few \n before the text in a comment earlier, too many here :)

*facepalm*

Quoted 5 lines

Documentation/config/hook.txt

Quoted 3 lines
Documentation/config/hook.txt +2 -0
+		 * At this point we are already running, so don't validate
+		 * whether the hook name is known or not.

...because it was done earlier somewhere, or...?

Yeah, that validation occurs in list_hooks() instead. I'll make the comment more clear.

Quoted 59 lines
Quoted 59 lines
Documentation/config/hook.txt +16 -2
+		 */
+		hook_path = find_hook_gently(hook_cb->hook_name);
+		if (!hook_path)
+			BUG("hookdir hook in hook list but no hookdir hook present in filesystem");
+
+		if (hook_cb->options->absolute_path)
+			hook_path = absolute_path(hook_path);
+
+		strvec_push(&cp->args, hook_path);
+	}
+
 
 	/*
 	 * add passed-in argv, without expanding - let the user get back
@@ -228,8 +346,11 @@ static int notify_start_failure(struct strbuf *out,
 
 	hook_cb->rc |= 1;
 
-	strbuf_addf(out, _("Couldn't start hook '%s'\n"),
-		    attempted->hook_path);
+	if (attempted->name)
+		strbuf_addf(out, _("Couldn't start hook '%s'\n"),
+		    attempted->name);
+	else
+		strbuf_addstr(out, _("Couldn't start hook from hooks directory\n"));
 
 	return 1;
 }
hook.h +5 -2
index 6b7b2d14d2..621bd2cde1 100644
@@ -27,8 +27,11 @@ int hook_exists(const char *hookname);
 
 struct hook {
 	struct list_head list;
-	/* The path to the hook */
-	const char *hook_path;
+	/*
+	 * The friendly name of the hook. NULL indicates the hook is from the
+	 * hookdir.
+	 */
+	const char *name;
 
 	/*
 	 * Use this to keep state for your feed_pipe_fn if you are using
t/t1800-hook.sh +1 -1
index 217db848b3..ef2432f53a 100755
@@ -1,13 +1,29 @@
 #!/bin/bash
 
-test_description='git-hook command'
+test_description='git-hook command and config-managed multihooks'
 
 . ./test-lib.sh
 
Quoted 8 lines

t/t1800-hook.sh line 11

Quoted 5 lines
t/t1800-hook.sh +5 -0
+setup_hooks () {
+	test_config hook.ghi.event pre-commit --add
+	test_config hook.ghi.command "/path/ghi" --add
+	test_config_global hook.def.event pre-commit --add
+	test_config_global hook.def.command "/path/def" --add

Isn't --add redundant here? Seems no test is stressing multi-value hook.{ghi,def}.* from a quick glance...

Sounds like a failing in the test suite ;)

Will remove --add from the command configs, and will stick in test for a hook to run in multiple places.

I went back and forth over whether to add the extra .event config in the setup vs. in a specific test, and I decided that by adding it in the setup, we get some implicit "this is fine" assurance, as well as the one explicit test (which I added just now) that the hook shows up in both places.

Quoted 11 lines
Quoted 11 lines
t/t1800-hook.sh +10 -0
+}
+
+setup_hookdir () {
+	mkdir .git/hooks
+	write_script .git/hooks/pre-commit <<-EOF
+	echo \"Legacy Hook\"
+	EOF
+	test_when_finished rm -rf .git/hooks
+}
+
Quoted 8 lines

t/t1800-hook.sh line 26

Quoted 5 lines
t/t1800-hook.sh +1 -0
 test_expect_success 'git hook usage' '
 	test_expect_code 129 git hook &&
 	test_expect_code 129 git hook run &&
 	test_expect_code 129 git hook run -h &&
+	test_expect_code 129 git hook list -h &&

Doesn't this belong in a previous commit that added "git hook list", not here?

Yes, nice. Thanks.

Quoted 20 lines
Quoted 20 lines
t/t1800-hook.sh +12 -0
 	test_expect_code 129 git hook run --unknown 2>err &&
 	grep "unknown option" err
 '
@@ -153,4 +169,127 @@ test_expect_success 'stdin to hooks' '
 	test_cmp expect actual
 '
 
+test_expect_success 'git hook list orders by config order' '
+	setup_hooks &&
+
+	cat >expected <<-EOF &&
+	def
+	ghi
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
Quoted 8 lines

t/t1800-hook.sh test_expect_success 'stdin to hooks' ' · line 189

Quoted 6 lines
t/t1800-hook.sh +6 -0
+test_expect_success 'git hook list reorders on duplicate event declarations' '
+	setup_hooks &&
+
+	# 'def' is usually configured globally; move it to the end by
+	# configuring it locally.
+	test_config hook.def.event "pre-commit" --add &&

Ah, well the --add belongs here, but not needed in setup_hooks, right?

Addressed above.

Quoted 9 lines
Quoted 9 lines
t/t1800-hook.sh +9 -0
+
+	cat >input <<-EOF &&
+	1
+	2
+	3
+	EOF
+
+	cat >expected <<-EOF &&
+	a1
Quoted 8 lines

t/t1800-hook.sh test_expect_success 'stdin to hooks' ' · line 268

Quoted 6 lines
t/t1800-hook.sh +6 -0
+	a2
+	a3
+	b1
+	b2
+	b3
+	EOF

For any here-docs without variables, use <<-\EOF, note the backslash.

ACK

Thanks.
- Emily

#592 lore raw

PATCH v3 6/6 hook: allow out-of-repo 'git hook' invocations

In reply to Emily Shaffer

Since hooks can now be supplied via the config, and a config can be present without a gitdir via the global and system configs, we can start to allow 'git hook run' to occur without a gitdir. This enables us to do things like run sendemail-validate hooks when running 'git send-email' from a nongit directory.

It still doesn't make sense to look for hooks in the hookdir in nongit repos, though, as there is no hookdir.

---

Notes:
For hookdir hooks, do we want to run them in nongit dir when core.hooksPath is set? For example, if someone set core.hooksPath in their global config and then ran 'git hook run sendemail-validate' in a nongit dir?

 git.c           |  2 +-
 hook.c          |  2 +-
 t/t1800-hook.sh | 20 +++++++++++++++-----
 3 files changed, 17 insertions(+), 7 deletions(-)
git.c +1 -1
index 540909c391..39988ee3b0 100644
@@ -538,7 +538,7 @@ static struct cmd_struct commands[] = {
 	{ "grep", cmd_grep, RUN_SETUP_GENTLY },
 	{ "hash-object", cmd_hash_object },
 	{ "help", cmd_help },
-	{ "hook", cmd_hook, RUN_SETUP },
+	{ "hook", cmd_hook, RUN_SETUP_GENTLY },
 	{ "index-pack", cmd_index_pack, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "init", cmd_init_db },
 	{ "init-db", cmd_init_db },
hook.c +1 -1
index 581d87cbbd..2e08156546 100644
@@ -218,7 +218,7 @@ struct list_head *list_hooks_gently(const char *hookname)
 
 	/* Add the hook from the hookdir. The placeholder makes it easier to
 	 * allocate work in pick_next_hook. */
-	if (find_hook_gently(hookname))
+	if (have_git_dir() && find_hook_gently(hookname))
 		append_or_move_hook(hook_head, NULL);
 
 	return hook_head;
t/t1800-hook.sh +15 -5
index ef2432f53a..a7e45c0d16 100755
@@ -118,15 +118,25 @@ test_expect_success 'git hook run -- pass arguments' '
 	test_cmp expect actual
 '
 
-test_expect_success 'git hook run -- out-of-repo runs excluded' '
-	write_script .git/hooks/test-hook <<-EOF &&
-	echo Test hook
-	EOF
+test_expect_success 'git hook run: out-of-repo runs execute global hooks' '
+	test_config_global hook.global-hook.event test-hook --add &&
+	test_config_global hook.global-hook.command "echo no repo no problems" --add &&
+
+	echo "global-hook" >expect &&
+	nongit git hook list test-hook >actual &&
+	test_cmp expect actual &&
+
+	echo "no repo no problems" >expect &&
 
-	nongit test_must_fail git hook run test-hook
+	nongit git hook run test-hook 2>actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'git -c core.hooksPath=<PATH> hook run' '
+	write_script .git/hooks/test-hook <<-EOF &&
+	echo Test hook
+	EOF
+
 	mkdir my-hooks &&
 	write_script my-hooks/test-hook <<-\EOF &&
 	echo Hook ran $1 >>actual
Signature · 1 line
2.33.0.rc2.250.ged5fa647cd-goog
#593 lore raw

Re: [PATCH v3 6/6] hook: allow out-of-repo 'git hook' invocations

Quoted 5 lines · Emily Shaffer

Since hooks can now be supplied via the config, and a config can be present without a gitdir via the global and system configs, we can start to allow 'git hook run' to occur without a gitdir. This enables us to do things like run sendemail-validate hooks when running 'git send-email' from a nongit directory.

Sensible goal. Perhaps we should note in an earlier commit when config-based hooks are introduced something like:

Even though we've added config-based hooks, they currently only work if we can find a .git directory, even though certain commands such as "git send-email" (or only that command?) can be run outside of a git directory. A subsequent commit will address that edge-case.

Quoted 62 lines

[...]
Notes:
For hookdir hooks, do we want to run them in nongit dir when core.hooksPath is set? For example, if someone set core.hooksPath in their global config and then ran 'git hook run sendemail-validate' in a nongit dir?
[...]

 git.c           |  2 +-
 hook.c          |  2 +-
 t/t1800-hook.sh | 20 +++++++++++++++-----
 3 files changed, 17 insertions(+), 7 deletions(-)
git.c +1 -1
index 540909c391..39988ee3b0 100644
@@ -538,7 +538,7 @@ static struct cmd_struct commands[] = {
 	{ "grep", cmd_grep, RUN_SETUP_GENTLY },
 	{ "hash-object", cmd_hash_object },
 	{ "help", cmd_help },
-	{ "hook", cmd_hook, RUN_SETUP },
+	{ "hook", cmd_hook, RUN_SETUP_GENTLY },
 	{ "index-pack", cmd_index_pack, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "init", cmd_init_db },
 	{ "init-db", cmd_init_db },
hook.c +1 -1
index 581d87cbbd..2e08156546 100644
@@ -218,7 +218,7 @@ struct list_head *list_hooks_gently(const char *hookname)
 
 	/* Add the hook from the hookdir. The placeholder makes it easier to
 	 * allocate work in pick_next_hook. */
-	if (find_hook_gently(hookname))
+	if (have_git_dir() && find_hook_gently(hookname))
 		append_or_move_hook(hook_head, NULL);
 
 	return hook_head;
t/t1800-hook.sh +11 -5
index ef2432f53a..a7e45c0d16 100755
@@ -118,15 +118,25 @@ test_expect_success 'git hook run -- pass arguments' '
 	test_cmp expect actual
 '
 
-test_expect_success 'git hook run -- out-of-repo runs excluded' '
-	write_script .git/hooks/test-hook <<-EOF &&
-	echo Test hook
-	EOF
+test_expect_success 'git hook run: out-of-repo runs execute global hooks' '
+	test_config_global hook.global-hook.event test-hook --add &&
+	test_config_global hook.global-hook.command "echo no repo no problems" --add &&
+
+	echo "global-hook" >expect &&
+	nongit git hook list test-hook >actual &&
+	test_cmp expect actual &&
+
+	echo "no repo no problems" >expect &&
 
-	nongit test_must_fail git hook run test-hook
+	nongit git hook run test-hook 2>actual &&
+	test_cmp expect actual

t/t1800-hook.sh test_expect_success 'git hook run -- pass arguments' ' · line 142

Quoted 10 lines
t/t1800-hook.sh +4 -0
 '
 
 test_expect_success 'git -c core.hooksPath=<PATH> hook run' '
+	write_script .git/hooks/test-hook <<-EOF &&
+	echo Test hook
+	EOF
+
 	mkdir my-hooks &&
 	write_script my-hooks/test-hook <<-\EOF &&
 	echo Hook ran $1 >>actual

If the only user of this is git-send-email, let's have tests for this in t/t9001-send-email.sh. That should also address your "Notes" above, i.e. let's just test it with core.hooksPath and see what the interaction looks like.

#594 lore raw

RE: [PATCH v3 6/6] hook: allow out-of-repo 'git hook' invocations

Quoted 9 lines · Ævar Arnfjörð Bjarmason
Quoted 5 lines · Emily Shaffer

Since hooks can now be supplied via the config, and a config can be present without a gitdir via the global and system configs, we can start to allow 'git hook run' to occur without a gitdir. This enables us to do things like run sendemail-validate hooks when running 'git send-email' from a nongit directory.

Sensible goal. Perhaps we should note in an earlier commit when config-based hooks are introduced something like:

To clarify the requirements here, if running without a gitdir (and thus without a repository?) how will front-ends know what to supply? Will this just be "some shell script" that runs?

Quoted 5 lines

Even though we've added config-based hooks, they currently only work if we can find a .git directory, even though certain commands such as "git send-email" (or only that command?) can be run outside of a git directory. A subsequent commit will address that edge-case.

So we cannot assume anything about the repository, correct? Similar to running git version but not git status?

Quoted 5 lines
Quoted 5 lines

[...]
Notes:
For hookdir hooks, do we want to run them in nongit dir when core.hooksPath is set? For example, if someone set core.hooksPath in their global config and then ran 'git hook run sendemail-validate' in a nongit dir?

So this is complete consent to run outside of git? I wonder whether there needs to be an attribute associated with the hook that enables this edge capability. That way we can validate whether the hook should be run or not (from front-end scripts).

#595 lore raw

Re: [PATCH v3 6/6] hook: allow out-of-repo 'git hook' invocations

Quoted 9 lines · Randall S. Becker
Quoted 8 lines · Ævar Arnfjörð Bjarmason
Quoted 5 lines · Emily Shaffer

Since hooks can now be supplied via the config, and a config can be present without a gitdir via the global and system configs, we can start to allow 'git hook run' to occur without a gitdir. This enables us to do things like run sendemail-validate hooks when running 'git send-email' from a nongit directory.

Quoted 5 lines
Quoted 1 line

Sensible goal. Perhaps we should note in an earlier commit when config-based hooks are introduced something like:

To clarify the requirements here, if running without a gitdir (and thus without a repository?) how will front-ends know what to supply? Will this just be "some shell script" that runs?

Emily's also aiming to have "git hook run" running "some shell script", but in this case we're talking about any git program that runs hooks, but doesn't require a repo. I think the only one is git-send-email's sendemail-validate hook.

Quoted 7 lines
Quoted 5 lines

Even though we've added config-based hooks, they currently only work if we can find a .git directory, even though certain commands such as "git send-email" (or only that command?) can be run outside of a git directory. A subsequent commit will address that edge-case.

So we cannot assume anything about the repository, correct? Similar to running git version but not git status?

Right, with RUN_SETUP_GENTLY we may not have a repo at all, so like commands such as "git bundle", "git config", "git ls-remote" etc.

Quoted 10 lines
Quoted 5 lines
Quoted 5 lines

[...]
Notes:
For hookdir hooks, do we want to run them in nongit dir when core.hooksPath is set? For example, if someone set core.hooksPath in their global config and then ran 'git hook run sendemail-validate' in a nongit dir?

So this is complete consent to run outside of git? I wonder whether there needs to be an attribute associated with the hook that enables this edge capability. That way we can validate whether the hook should be run or not (from front-end scripts).

If I understand you correctly you're pointing out that anyone with a sendemail-validate script could previously have assumed a repo, but not anymore. So e.g. someone who always has other config the hook requires when they run it in repository might unexpectedly have that hook fail if they naïvely set the config for that sendemail-validate hook via "git config --global".

I think that's a good point, and one I hadn't really considered. I think it's probably best to just document this edge case for the one (or few?) hooks that have this caveat than to not support it. I.e. it seems useful to have a sendemail-validate without a repo (just to parse/validate the about-to-be-sent-email), that we needed a repo before was really only an emergent effect.

Also, I think it's not new with this config-based hook mechanism, but something that I introduced in back when I added core.hooksPath (but I don't think I realized I was adding this edge case at the time).

#596 lore raw

Re: [PATCH v3 6/6] hook: allow out-of-repo 'git hook' invocations

In reply to Ævar Arnfjörð Bjarmason
Quoted 10 lines · Ævar Arnfjörð Bjarmason
Quoted 5 lines · Emily Shaffer

Since hooks can now be supplied via the config, and a config can be present without a gitdir via the global and system configs, we can start to allow 'git hook run' to occur without a gitdir. This enables us to do things like run sendemail-validate hooks when running 'git send-email' from a nongit directory.

Quoted 7 lines

Sensible goal. Perhaps we should note in an earlier commit when config-based hooks are introduced something like:

Even though we've added config-based hooks, they currently only work if we can find a .git directory, even though certain commands such as "git send-email" (or only that command?) can be run outside of a git directory. A subsequent commit will address that edge-case.

Hmmm. I personally do not like "a subsequent commit adds this" in commit messages. But I'm having trouble expressing why :) To me, "I know about me and everything that came before me" is fine and "I wish in the future x would happen" is fine, for commit messages, but "x will happen in the future" feels a little wrong.

Anyway, I think it is a little distracting to include that message in the earlier commit, since the "doesn't work outside of a repo" aspect of hooks is not changing at all.

Quoted 65 lines
Quoted 65 lines

[...]
Notes:
For hookdir hooks, do we want to run them in nongit dir when core.hooksPath is set? For example, if someone set core.hooksPath in their global config and then ran 'git hook run sendemail-validate' in a nongit dir?
[...]

 git.c           |  2 +-
 hook.c          |  2 +-
 t/t1800-hook.sh | 20 +++++++++++++++-----
 3 files changed, 17 insertions(+), 7 deletions(-)
git.c +1 -1
index 540909c391..39988ee3b0 100644
@@ -538,7 +538,7 @@ static struct cmd_struct commands[] = {
 	{ "grep", cmd_grep, RUN_SETUP_GENTLY },
 	{ "hash-object", cmd_hash_object },
 	{ "help", cmd_help },
-	{ "hook", cmd_hook, RUN_SETUP },
+	{ "hook", cmd_hook, RUN_SETUP_GENTLY },
 	{ "index-pack", cmd_index_pack, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "init", cmd_init_db },
 	{ "init-db", cmd_init_db },
hook.c +1 -1
index 581d87cbbd..2e08156546 100644
@@ -218,7 +218,7 @@ struct list_head *list_hooks_gently(const char *hookname)
 
 	/* Add the hook from the hookdir. The placeholder makes it easier to
 	 * allocate work in pick_next_hook. */
-	if (find_hook_gently(hookname))
+	if (have_git_dir() && find_hook_gently(hookname))
 		append_or_move_hook(hook_head, NULL);
 
 	return hook_head;
t/t1800-hook.sh +11 -5
index ef2432f53a..a7e45c0d16 100755
@@ -118,15 +118,25 @@ test_expect_success 'git hook run -- pass arguments' '
 	test_cmp expect actual
 '
 
-test_expect_success 'git hook run -- out-of-repo runs excluded' '
-	write_script .git/hooks/test-hook <<-EOF &&
-	echo Test hook
-	EOF
+test_expect_success 'git hook run: out-of-repo runs execute global hooks' '
+	test_config_global hook.global-hook.event test-hook --add &&
+	test_config_global hook.global-hook.command "echo no repo no problems" --add &&
+
+	echo "global-hook" >expect &&
+	nongit git hook list test-hook >actual &&
+	test_cmp expect actual &&
+
+	echo "no repo no problems" >expect &&
 
-	nongit test_must_fail git hook run test-hook
+	nongit git hook run test-hook 2>actual &&
+	test_cmp expect actual
 '
 
Quoted 13 lines

t/t1800-hook.sh test_expect_success 'git hook run -- pass arguments' ' · line 142

Quoted 8 lines
t/t1800-hook.sh +4 -0
 test_expect_success 'git -c core.hooksPath=<PATH> hook run' '
+	write_script .git/hooks/test-hook <<-EOF &&
+	echo Test hook
+	EOF
+
 	mkdir my-hooks &&
 	write_script my-hooks/test-hook <<-\EOF &&
 	echo Hook ran $1 >>actual

If the only user of this is git-send-email, let's have tests for this in t/t9001-send-email.sh. That should also address your "Notes" above, i.e. let's just test it with core.hooksPath and see what the interaction looks like.

Yeah, I think it is probably fine to do allllll the git-send-email fixups in this one commit, actually. I think your point is correct that right now the only command which can benefit is git-send-email. It might also be reasonable to drop this change from this series, but I won't bother doing so unless it looks like this one commit is holding up the rest of the series.

I could see, though, a later world where we might want something like a "pre-clone" or "post-clone"(-pre-checkout) hook, now that we don't need to have a .gitdir in order to run any hooks. Will make a very brief "later it might be nice if..." addition to the commit message.

- Emily

#597 lore raw

Re: [PATCH v3 0/6] config-based hooks restarted

In reply to Emily Shaffer
Quoted 7 lines · Emily Shaffer

This is the config-based hooks topic rebased onto v4 of Ævar's branch[1]. There is a happy CI build of it on GitHub[2]. [...]
Right now I'm trying to focus on this series first and foremost, hence sending two rerolls based on the same version of Ævar's base restart. I'll try to perform a code review on Ævar's latest tomorrow. [...]

First thank for the review on the base topic at
https://lore.kernel.org/git/[email protected]/ &
https://lore.kernel.org/git/[email protected]/ and related.

I'll re-roll it soon pending your feedback on comments I left on this series that pertain to it. I.e. there's some things that I think are better just fixed up in the base topic (from trivial things like whitespace changes, to some behavior changes), makes for easier reading rather than going back & forth between the two.

I've read this whole series through, as promised in
https://lore.kernel.org/git/[email protected]/:

[I'll discuss my opinions on the new and revised config schema in another mail, just commenting on the patch here in terms of its stated goals].

I.e. here's some general comments, numbered for ease of reference:

0)

I think this is in much much better shape vis-as-vis the simplified config schema that's now being proposed re our discussion starting around https://lore.kernel.org/git/[email protected]/

I.e. the main complexity of the "skip" mechanism is gone, and also the conflation of hook names with hook commands (the "rm -rf /" as a <name-or-cmd> discussed in the above).

So before going any further, I'll just say that I wouldn't object much to this design going in as-is. What I'm about to mention here below is much closer to bikeshedding in my mind than "this is really to complex to go in-tree", which was my opinion of the config schema before.

1)

On the current config design: First, before going into any detail on that, I think whatever anyone's opinion is on that that the design-focused patches as they stand could really use more a more extended discussion of the design.

I.e. talk about the previously considered config schema, why it evolved into its current form. The trade-offs involved, and why the patch proposed to implement the schema it's implementing over another earlier or alternate design.

I.e. https://lore.kernel.org/git/[email protected]
is two very short paragraphs. We won't be able to summarize all our month-long discussion on the config design in one commit message, but I think at least discussing it somewhat / linking to relevant on-list discussions would make future source spelunking easier.

2)

So that out of the way, a comment on the current config design, which should be read in the context of what I noted in #0. I.e. I'm *much* happier with this version.

That being said I'm still not convinced that the simple 1=1 mapping of "hook.<name>.command" and its "value" should be followed by the 1=many mapping of "hook.<name>.event" and its "value".

I.e. we're making the trade-off of saving the user from typing out or specifying:

[hook "my-pre-commit"]
command = ~/hooks/pre-commit-or-push
event = pre-commit
[hook "my-pre-push"]
command = ~/hooks/pre-commit-or-push
event = pre-push

And instead being able to do:

[hook "my-pre-commit-or-push"]
command = ~/hooks/pre-commit-or-push
event = pre-commit
event = pre-push

So for the very common case, saving two config lines. "Two" because as we discussed[1] as there's currently no GIT_HOOK_TYPE env var. So this form will work pretty much only for that case.

I.e. unlike with .git/hook/<name> the hook run via config can't determine what <hook-type> it's being run at, so as it stands this is only useful for those hooks listed in githooks(5) where someone would want to do the exact same thing for one or more <hook-name> names. You can't use it as a general routing mechanism for any hook type as it stands.

I *think* that's only these two, perhaps "update" and "pre-receive", with the hook seeing if it consume stdin/has arguments to disambiguate the two.

But even with a GIT_HOOK_TYPE passed the trade-off, as discussed in [1], and downthread in [2], is that by having it 1=many we're closing the door on any future hook.<name>.<whatever>. I.e. config that would change the behavior of that hook, but you'd want to change it in another way for at least one of the N event types.

Well, "closing the door" as in if you'd want that you'd have to split up the section from the "my-pre-commit-or-push" example above to the "my-pre-commit" and "my-pre-push" example.

But again, on the "is the complexity worth it" we're then having to explain to users that they can do it one way if the want no config other than hook.<name>.{command,event*}, but another if they have another key in that namespace.

You've said that you wanted to add something like a GIT_HOOK_TYPE environment variable. Fair enough, and I guess we could add it in a re-roll of this series. I'm mainly commenting on the end-state of *this* series in particular. I.e. I think it leaves the user & implementation with a config schema that still seems to be needlessly complex for the very limited benefits that complexity brings us in what you're able to do with it now.

But some of that goes back to the comments I had on 5/6[3], i.e. I'm willing to be convinced, but I think that the current commit message & added docs aren't really selling the idea of why it's worth it.

3)

As an extension to my comments on 5/6[3], I think this whole notion of "git hook run <name>" as invoked by a user of git is just more confusing the more I think about it.

I.e. 5/6[4] is apparently seeking to implement a way to just make that facility a general way for users to run some command on their system to do whatever, instead of say using /usr/bin/parallel or a shell alias.

But then we also use that command as our own dispatch mechanism for our own known hooks, except mostly not, since we mostly use the C API ourselves directly.

It's particularly confusing that if you say run "git hook run pre-auto-gc" as a user to test your hook you'll have that hook run in the same way that git-gc(1) would run it. So someone developing a hook might think they can use "git hook run" for testing it.

But if you do the same with say "git hook run pre-receive" or anything else that feeds arguments or stdin (e.g. "update", or "pre-receive"), you'll have your hook happily being run by git, but in a way that's not at all how such a hook will be run when it's run by git "for real".

So I wonder if we shouldn't just have the thing die() if you try to run any hook that's in githooks(5) itself, except for sendemail-validate and the p4 hooks, since we need to run those ourselves.

Or have those use an internal-only "git hook--helper", and start out with "git hook" just supporting "git hook list", and then later on have "git hook run" (or perhaps "git hook run-configured"?) be an entry point for this facility of running some arbitrary script that's not a "real" hook.

I don't know, maybe I'm the only one that finds this confusing...

1. https://lore.kernel.org/git/[email protected]/
2. https://lore.kernel.org/git/[email protected]/
3. https://lore.kernel.org/git/[email protected]/
4. https://lore.kernel.org/git/[email protected]/

#598 lore raw

Re: [PATCH v3 0/6] config-based hooks restarted

Quoted 30 lines · Ævar Arnfjörð Bjarmason

On Wed, Aug 18 2021, Emily Shaffer wrote:
0)

I think this is in much much better shape vis-as-vis the simplified config schema that's now being proposed re our discussion starting around https://lore.kernel.org/git/[email protected]/

I.e. the main complexity of the "skip" mechanism is gone, and also the conflation of hook names with hook commands (the "rm -rf /" as a <name-or-cmd> discussed in the above).

So before going any further, I'll just say that I wouldn't object much to this design going in as-is. What I'm about to mention here below is much closer to bikeshedding in my mind than "this is really to complex to go in-tree", which was my opinion of the config schema before.

1)

On the current config design: First, before going into any detail on that, I think whatever anyone's opinion is on that that the design-focused patches as they stand could really use more a more extended discussion of the design.

I.e. talk about the previously considered config schema, why it evolved into its current form. The trade-offs involved, and why the patch proposed to implement the schema it's implementing over another earlier or alternate design.

Quoted 5 lines

I.e. https://lore.kernel.org/git/[email protected]
is two very short paragraphs. We won't be able to summarize all our month-long discussion on the config design in one commit message, but I think at least discussing it somewhat / linking to relevant on-list discussions would make future source spelunking easier.

Hum. I don't think that it's necessary to summarize the whole discussion in the commit message, but I think it's worth it to describe the rationale ("we like this design because it makes x good practice easy and y bug hard") and link out to the discussion for parties who are interested in reading more.

Quoted 43 lines

2)

So that out of the way, a comment on the current config design, which should be read in the context of what I noted in #0. I.e. I'm *much* happier with this version.

That being said I'm still not convinced that the simple 1=1 mapping of "hook.<name>.command" and its "value" should be followed by the 1=many mapping of "hook.<name>.event" and its "value".

I.e. we're making the trade-off of saving the user from typing out or specifying:

[hook "my-pre-commit"]
command = ~/hooks/pre-commit-or-push
event = pre-commit
[hook "my-pre-push"]
command = ~/hooks/pre-commit-or-push
event = pre-push

And instead being able to do:

[hook "my-pre-commit-or-push"]
command = ~/hooks/pre-commit-or-push
event = pre-commit
event = pre-push

So for the very common case, saving two config lines. "Two" because as we discussed[1] as there's currently no GIT_HOOK_TYPE env var. So this form will work pretty much only for that case.

I.e. unlike with .git/hook/<name> the hook run via config can't determine what <hook-type> it's being run at, so as it stands this is only useful for those hooks listed in githooks(5) where someone would want to do the exact same thing for one or more <hook-name> names. You can't use it as a general routing mechanism for any hook type as it stands.

I *think* that's only these two, perhaps "update" and "pre-receive", with the hook seeing if it consume stdin/has arguments to disambiguate the two.

Quoted 5 lines

But even with a GIT_HOOK_TYPE passed the trade-off, as discussed in [1], and downthread in [2], is that by having it 1=many we're closing the door on any future hook.<name>.<whatever>. I.e. config that would change the behavior of that hook, but you'd want to change it in another way for at least one of the N event types.

I'm not really sure that's the case, to be honest. Even with the config scheme as is in this iteration, you can still define it the way you're describing with no problem, and in fact it makes it easier for users to apply some special config to all-but-one invocation.

Let's say for example the "git-secrets" hook, which we do have defined for at least 3 different events at Google today, and a hypothetical "parallelize me" config; for the sake of argument let's presume that this is the far future and we've added a GIT_HOOK_TYPE envvar:

[hook "git-secrets"]
command = /bin/git-secrets
event = pre-commit
event = pre-merge-commit
parallelizable = true

[hook "git-secrets-mutexed"]
command = /bin/git-secrets
event = prepare-commit-msg
event = commit-msg
parallelizable = false

If we don't allow "hook.myhookname.event" to be multiply configurable, then the user gets this really tedious task of defining every single "hook.git-secrets-$hookevent.parallelizable" config.

Quoted 9 lines

Well, "closing the door" as in if you'd want that you'd have to split up the section from the "my-pre-commit-or-push" example above to the "my-pre-commit" and "my-pre-push" example.

But again, on the "is the complexity worth it" we're then having to explain to users that they can do it one way if the want no config other than hook.<name>.{command,event*}, but another if they have another key in that namespace.

I am not so worried about "this will be hard to explain, so we should not do it" - I think we can make the documentation useful with enough effort and expertise. (And yes, I'm feeling optimistic because I have an actual technical writer taking a look at the manpage right now.)

Quoted 8 lines

You've said that you wanted to add something like a GIT_HOOK_TYPE environment variable. Fair enough, and I guess we could add it in a re-roll of this series. I'm mainly commenting on the end-state of *this* series in particular. I.e. I think it leaves the user & implementation with a config schema that still seems to be needlessly complex for the very limited benefits that complexity brings us in what you're able to do with it now.

I think that limiting ourselves in the way you're describing will make it more difficult to bring additional benefits later. It is certainly possible and valid to write your configuration the way you are describing, without changing this schema.

Quoted 4 lines

But some of that goes back to the comments I had on 5/6[3], i.e. I'm willing to be convinced, but I think that the current commit message & added docs aren't really selling the idea of why it's worth it.

Ok. I think your point is not "the schema is still wrong" as much as it is "the documentation could be much better", and I agree.

Quoted 20 lines

3)

As an extension to my comments on 5/6[3], I think this whole notion of "git hook run <name>" as invoked by a user of git is just more confusing the more I think about it.

I.e. 5/6[4] is apparently seeking to implement a way to just make that facility a general way for users to run some command on their system to do whatever, instead of say using /usr/bin/parallel or a shell alias.

But then we also use that command as our own dispatch mechanism for our own known hooks, except mostly not, since we mostly use the C API ourselves directly.

It's particularly confusing that if you say run "git hook run pre-auto-gc" as a user to test your hook you'll have that hook run in the same way that git-gc(1) would run it. So someone developing a hook might think they can use "git hook run" for testing it.

Quoted 8 lines

But if you do the same with say "git hook run pre-receive" or anything else that feeds arguments or stdin (e.g. "update", or "pre-receive"), you'll have your hook happily being run by git, but in a way that's not at all how such a hook will be run when it's run by git "for real".

So I wonder if we shouldn't just have the thing die() if you try to run any hook that's in githooks(5) itself, except for sendemail-validate and the p4 hooks, since we need to run those ourselves.

I think this again falls to a documentation issue. I would love to have a tighter loop when developing a hook that takes weird mid-merge arguments or whatever; making it easier to test the hook in isolation with known inputs sounds like a good thing to me.

Personally, "I ran 'git hook run pre-receive' without being in the middle of a receive operation and it didn't behave like it was in the middle of a receive operation!" doesn't sound all that surprising to me.

Quoted 7 lines

Or have those use an internal-only "git hook--helper", and start out with "git hook" just supporting "git hook list", and then later on have "git hook run" (or perhaps "git hook run-configured"?) be an entry point for this facility of running some arbitrary script that's not a "real" hook.

Quoted 6 lines

I don't know, maybe I'm the only one that finds this confusing...

1. https://lore.kernel.org/git/[email protected]/
2. https://lore.kernel.org/git/[email protected]/
3. https://lore.kernel.org/git/[email protected]/
4. https://lore.kernel.org/git/[email protected]/

Besides "make the docs more obvious", I don't think there is anything in this mail that I want to act on. I am very happy with the config schema as it is, as well as the behavior of 'git hook run'.

As I mentioned here and somewhere else in the review, I am in the process of getting feedback on the manpage from a tech writer this week, so do not expect a reroll from me until at least next week. I saw your reroll today and I'll try and look at it (or at least the interdiff) tomorrow or Monday.

- Emily

#599 lore raw

v4 cover config-based hooks restarted

In reply to Emily Shaffer

This is a re-roll of Emily's es/config-based-hooks topic that goes on top of the v5 of my ab/config-based-hooks-base, and which doesn't have errors under SANITIZE=leak.

Emily: Sorry, I've got no intention to steal this one too, hopefully you can get around to your own re-roll.

But per my
https://lore.kernel.org/git/[email protected]/ the lack of this re-roll is is currently blocking the pick-up of my re-rolled v5 of ab/config-based-hooks-base at [1], as well as causing a failure in "seen" when combined with my ab/sanitize-leak-ci (and hn/reftable, but that's another issue...).

Junio: So hopefully you can pick up the v5[1] of the base topic now & this preliminary v4 of es/config-based-hooks.

The range-diff below is against Emily's 30ffe98601e, i.e. her v3 at [2].

This version is based on Emily's preliminary cf1f8e34a34 (nasamuffin/config-based-hooks-restart), which appeared to be her August 31 rebasing addressing of many outstanding points in the v3 series.

My own changes on top of that were twofold: Adjustments to changes in the base topic (many done to make the overall diff/changes here smaller), and memory leak fixes to get this to pass under SANITIZE=leak, there's various other minor but not-notable changes here and there, see the range-diff.

1. https://lore.kernel.org/git/[email protected]/
2. https://lore.kernel.org/git/[email protected]/

Emily Shaffer (5):
hook: run a list of hooks instead
hook: allow parallel hook execution
hook: introduce "git hook list"
hook: include hooks from the config
hook: allow out-of-repo 'git hook' invocations

 Documentation/config.txt      |   2 +
 Documentation/config/hook.txt |  22 +++
 Documentation/git-hook.txt    | 157 ++++++++++++++++++-
 builtin/am.c                  |   4 +-
 builtin/checkout.c            |   2 +-
 builtin/clone.c               |   2 +-
 builtin/hook.c                |  71 ++++++++-
 builtin/merge.c               |   2 +-
 builtin/rebase.c              |   2 +-
 builtin/receive-pack.c        |   9 +-
 builtin/worktree.c            |   2 +-
 commit.c                      |   2 +-
 git.c                         |   2 +-
 hook.c                        | 277 +++++++++++++++++++++++++++++-----
 hook.h                        |  45 +++++-
 read-cache.c                  |   2 +-
 refs.c                        |   2 +-
 reset.c                       |   3 +-
 sequencer.c                   |   4 +-
 t/t1800-hook.sh               | 194 +++++++++++++++++++++++-
 transport.c                   |   2 +-
 21 files changed, 734 insertions(+), 74 deletions(-)
 create mode 100644 Documentation/config/hook.txt

Range-diff against v3:
1: 6d6400329cd ! 1: 2f0cac14965 hook: run a list of hooks instead

    @@ Commit message
         executable for a single hook event.
     
         Signed-off-by: Emily Shaffer <[email protected]>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <[email protected]>
     
      ## builtin/hook.c ##
     @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
    @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
      	const char *hook_name;
     -	const char *hook_path;
     +	struct list_head *hooks;
    -+
      	struct option run_options[] = {
      		OPT_BOOL(0, "ignore-missing", &ignore_missing,
      			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
     @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
    - 	git_config(git_default_config, NULL);
    - 
    + 	 * run_hooks() instead...
    + 	 */
      	hook_name = argv[0];
     -	if (ignore_missing)
    ++	hooks = list_hooks(hook_name);
    ++	if (list_empty(hooks)) {
    ++		clear_hook_list(hooks);
    ++
    + 		/* ... act like a plain run_hooks() under --ignore-missing */
     -		return run_hooks_oneshot(hook_name, &opt);
     -	hook_path = find_hook(hook_name);
     -	if (!hook_path) {
    -+	hooks = list_hooks(hook_name);
    -+	if (list_empty(hooks)) {
    -+		/* ... act like run_hooks_oneshot() under --ignore-missing */
     +		if (ignore_missing)
     +			return 0;
      		error("cannot find a hook named %s", hook_name);
    @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
     
      ## hook.c ##
     @@
    - #include "hook-list.h"
    + #include "run-command.h"
      #include "config.h"
      
     +static void free_hook(struct hook *ptr)
     +{
    -+	if (ptr)
    -+		free(ptr->feed_pipe_cb_data);
    ++	if (!ptr)
    ++		return;
    ++
    ++	free(ptr->feed_pipe_cb_data);
     +	free(ptr);
     +}
     +
    @@ hook.c
     +	struct list_head *pos, *tmp;
     +	list_for_each_safe(pos, tmp, head)
     +		remove_hook(pos);
    ++	free(head);
     +}
     +
    - static int known_hook(const char *name)
    + const char *find_hook(const char *name)
      {
    - 	const char **p;
    + 	static struct strbuf path = STRBUF_INIT;
     @@ hook.c: const char *find_hook(const char *name)
      
      int hook_exists(const char *name)
      {
     -	return !!find_hook(name);
    -+	return !list_empty(list_hooks(name));
    ++	struct list_head *hooks;
    ++	int exists;
    ++
    ++	hooks = list_hooks(name);
    ++	exists = !list_empty(hooks);
    ++	clear_hook_list(hooks);
    ++
    ++	return exists;
     +}
     +
     +struct list_head *list_hooks(const char *hookname)
    @@ hook.c: static int notify_hook_finished(int result,
      }
      
     -int run_hooks(const char *hook_name, const char *hook_path,
    --	      struct run_hooks_opt *options)
     +int run_hooks(const char *hook_name, struct list_head *hooks,
    -+		    struct run_hooks_opt *options)
    + 	      struct run_hooks_opt *options)
      {
     -	struct strbuf abs_path = STRBUF_INIT;
     -	struct hook my_hook = {
    @@ hook.c: int run_hooks(const char *hook_name, const char *hook_path,
     -		my_hook.hook_path = abs_path.buf;
     -	}
     -	cb_data.run_me = &my_hook;
    -+
     +	cb_data.head = hooks;
     +	cb_data.run_me = list_first_entry(hooks, struct hook, list);
      
    @@ hook.c: int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *optio
      
     -	ret = run_hooks(hook_name, hook_path, options);
     +	ret = run_hooks(hook_name, hooks, options);
    -+
    + 
      cleanup:
      	run_hooks_opt_clear(options);
    -+	clear_hook_list(hooks);
    - 	return ret;
    - }
     
      ## hook.h ##
     @@
    @@ hook.h
      #include "run-command.h"
     +#include "list.h"
      
    - /*
    -  * Returns the path to the hook file, or NULL if the hook is missing
    -@@ hook.h: const char *find_hook(const char *name);
    - int hook_exists(const char *hookname);
    - 
      struct hook {
     +	struct list_head list;
      	/* The path to the hook */
      	const char *hook_path;
      
    -@@ hook.h: struct hook {
    - 	void *feed_pipe_cb_data;
    - };
    - 
    -+/*
    -+ * Provides a linked list of 'struct hook' detailing commands which should run
    -+ * in response to the 'hookname' event, in execution order.
    -+ */
    -+struct list_head *list_hooks(const char *hookname);
    -+
    - struct run_hooks_opt
    - {
    - 	/* Environment vars to be set for each hook */
     @@ hook.h: struct hook_cb_data {
      	/* rc reflects the cumulative failure state */
      	int rc;
    @@ hook.h: struct hook_cb_data {
      	struct hook *run_me;
      	struct run_hooks_opt *options;
      	int *invoked_hook;
    +@@ hook.h: struct hook_cb_data {
    + const char *find_hook(const char *name);
    + 
    + /**
    +- * A boolean version of find_hook()
    ++ * Provides a linked list of 'struct hook' detailing commands which should run
    ++ * in response to the 'hookname' event, in execution order.
    ++ */
    ++struct list_head *list_hooks(const char *hookname);
    ++
    ++/**
    ++ * A boolean version of list_hooks()
    +  */
    + int hook_exists(const char *hookname);
    + 
     @@ hook.h: void run_hooks_opt_clear(struct run_hooks_opt *o);
    + 
    + /**
    +  * Takes an already resolved hook found via find_hook() and runs
    +- * it. Does not call run_hooks_opt_clear() for you.
    ++ * it. Does not call run_hooks_opt_clear() for you, but does call
    ++ * clear_hook_list().
       *
       * See run_hooks_oneshot() for the simpler one-shot API.
       */
     -int run_hooks(const char *hookname, const char *hook_path,
    --	      struct run_hooks_opt *options);
     +int run_hooks(const char *hookname, struct list_head *hooks,
    -+		    struct run_hooks_opt *options);
    - 
    - /**
    -  * Calls find_hook() on your "hook_name" and runs the hooks (if any)
    -@@ hook.h: int run_hooks(const char *hookname, const char *hook_path,
    -  */
    - int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options);
    + 	      struct run_hooks_opt *options);
      
    -+/* Empties the list at 'head', calling 'free_hook()' on each entry */
    ++/**
    ++ * Empties the list at 'head', calling 'free_hook()' on each
    ++ * entry. Called implicitly by run_hooks() (and run_hooks_oneshot()).
    ++ */
     +void clear_hook_list(struct list_head *head);
     +
    - #endif
    + /**
    +  * Calls find_hook() on your "hook_name" and runs the hooks (if any)
    +  * with run_hooks().

2: dfb995ce4d4 ! 2: b03e70c805e hook: allow parallel hook execution
@@ Commit message

+ ## Documentation/config.txt ##
+@@ Documentation/config.txt: include::config/guitool.txt[]
+
+ include::config/help.txt[]
+
++include::config/hook.txt[]
++
+ include::config/http.txt[]
+
+ include::config/i18n.txt[]
+
## Documentation/config/hook.txt (new) ##
@@
+hook.jobs::
@@ Documentation/git-hook.txt: OPTIONS
+--jobs::
+ Only valid for run.
++
-+Specify how many hooks to run simultaneously. If this flag is not specified, use -+the value of the hook.jobs config. If the config is not specified, use the -+number of CPUs on the current system. Some hooks may be ineligible for -+parallelization: for example, 'commit-msg' intends hooks modify the commit -+message body and cannot be parallelized.
++Specify how many hooks to run simultaneously. If this flag is not specified, ++uses the value of the hook.jobs config, see linkgit:git-config[1]. If the ++config is not specified, uses the number of CPUs on the current system. Some ++hooks may be ineligible for parallelization: for example, 'commit-msg' intends ++hooks modify the commit message body and cannot be parallelized.
+
+CONFIGURATION
+-------------
@@ hook.c: static int notify_hook_finished(int result,
+}
+
int run_hooks(const char *hook_name, struct list_head *hooks,
- struct run_hooks_opt *options)
+ struct run_hooks_opt *options)
{
@@ hook.c: int run_hooks(const char *hook_name, struct list_head *hooks,
.options = options,
@@ hook.c: int run_hooks(const char *hook_name, struct list_head *hooks,

if (!options)
BUG("a struct run_hooks_opt must be provided to run_hooks");
-
--
+@@ hook.c: int run_hooks(const char *hook_name, struct list_head *hooks,
cb_data.head = hooks;
cb_data.run_me = list_first_entry(hooks, struct hook, list);

@@ hook.h: struct run_hooks_opt
+ */
+ int jobs;
+
- /* Resolve and run the "absolute_path(hook)" instead of
+ /*
+ * Resolve and run the "absolute_path(hook)" instead of
* "hook". Used for "git worktree" hooks
- */
@@ hook.h: struct run_hooks_opt
int *invoked_hook;
};

-#define RUN_HOOKS_OPT_INIT { \
-- .env = STRVEC_INIT, \
-- .args = STRVEC_INIT, \
--}
--
- /*
- * To specify a 'struct string_list', set 'run_hooks_opt.feed_pipe_ctx' to the
- * string_list and set 'run_hooks_opt.feed_pipe' to 'pipe_from_string_list()'. -@@ hook.h: struct hook_cb_data {
- int *invoked_hook;
- };
-
+#define RUN_HOOKS_OPT_INIT_SERIAL { \
+ .jobs = 1, \
+ .env = STRVEC_INIT, \
@@ hook.h: struct hook_cb_data {
+
+#define RUN_HOOKS_OPT_INIT_PARALLEL { \
+ .jobs = 0, \
-+ .env = STRVEC_INIT, \
-+ .args = STRVEC_INIT, \
-+}
-+
- void run_hooks_opt_clear(struct run_hooks_opt *o);
-
- /**
+ .env = STRVEC_INIT, \
+ .args = STRVEC_INIT, \
+ }

## read-cache.c ##
@@ read-cache.c: static int do_write_locked_index(struct index_state *istate, struct lock_file *l
3: c8a04306e90 ! 3: 3e647b8dba7 hook: introduce "git hook list"

    @@ Commit message
         hooks were configured and whether or not they will run.
     
         Signed-off-by: Emily Shaffer <[email protected]>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <[email protected]>
     
      ## Documentation/git-hook.txt ##
     @@ Documentation/git-hook.txt: SYNOPSIS
    @@ builtin/hook.c: static const char * const builtin_hook_run_usage[] = {
     +
     +static int list(int argc, const char **argv, const char *prefix)
     +{
    -+	struct list_head *head, *pos;
    ++	struct list_head *hooks;
    ++	struct list_head *pos;
     +	const char *hookname = NULL;
    -+	struct strbuf hookdir_annotation = STRBUF_INIT;
    -+
     +	struct option list_options[] = {
     +		OPT_END(),
     +	};
    ++	int ret = 0;
     +
     +	argc = parse_options(argc, argv, prefix, list_options,
     +			     builtin_hook_list_usage, 0);
     +
    -+	if (argc < 1)
    ++	/*
    ++	 * The only unnamed argument provided should be the hook-name; if we add
    ++	 * arguments later they probably should be caught by parse_options.
    ++	 */
    ++	if (argc != 1)
     +		usage_msg_opt(_("You must specify a hook event name to list."),
     +			      builtin_hook_list_usage, list_options);
     +
     +	hookname = argv[0];
     +
    -+	head = hook_list(hookname);
    ++	hooks = list_hooks(hookname);
     +
    -+	if (list_empty(head))
    -+		return 1;
    ++	if (list_empty(hooks)) {
    ++		ret = 1;
    ++		goto cleanup;
    ++	}
     +
    -+	list_for_each(pos, head) {
    ++	list_for_each(pos, hooks) {
     +		struct hook *item = list_entry(pos, struct hook, list);
     +		item = list_entry(pos, struct hook, list);
     +		if (item)
     +			printf("%s\n", item->hook_path);
     +	}
     +
    -+	clear_hook_list(head);
    -+	strbuf_release(&hookdir_annotation);
    ++cleanup:
    ++	clear_hook_list(hooks);
     +
    -+	return 0;
    ++	return ret;
     +}
      static int run(int argc, const char **argv, const char *prefix)
      {
    @@ builtin/hook.c: int cmd_hook(int argc, const char **argv, const char *prefix)
      		return run(argc, argv, prefix);
      
     
    - ## hook.c ##
    -@@ hook.c: struct list_head *list_hooks(const char *hookname)
    - {
    - 	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
    + ## t/t1800-hook.sh ##
    +@@ t/t1800-hook.sh: test_expect_success 'git hook usage' '
    + 	test_expect_code 129 git hook run &&
    + 	test_expect_code 129 git hook run -h &&
    + 	test_expect_code 129 git hook run --unknown 2>err &&
    ++	test_expect_code 129 git hook list &&
    ++	test_expect_code 129 git hook list -h &&
    + 	grep "unknown option" err
    + '
      
    -+
    - 	INIT_LIST_HEAD(hook_head);
    +@@ t/t1800-hook.sh: test_expect_success 'git hook run -- pass arguments' '
    + 	test_cmp expect actual
    + '
      
    - 	if (!hookname)
    -@@ hook.c: struct list_head *list_hooks(const char *hookname)
    - 
    - 	if (have_git_dir()) {
    - 		const char *hook_path = find_hook(hookname);
    --
    --		/* Add the hook from the hookdir */
    - 		if (hook_path) {
    - 			struct hook *to_add = xmalloc(sizeof(*to_add));
    - 			to_add->hook_path = hook_path;
    ++test_expect_success 'git hook list: does-not-exist hook' '
    ++	test_expect_code 1 git hook list does-not-exist
    ++'
    ++
    ++test_expect_success 'git hook list: existing hook' '
    ++	cat >expect <<-\EOF &&
    ++	.git/hooks/test-hook
    ++	EOF
    ++	git hook list test-hook >actual &&
    ++	test_cmp expect actual
    ++'
    ++
    + test_expect_success 'git hook run -- out-of-repo runs excluded' '
    + 	write_script .git/hooks/test-hook <<-EOF &&
    + 	echo Test hook
4:  af14116d0fa < -:  ----------- hook: allow running non-native hooks
5:  2bbb179962e ! 4:  d0f5b30fb27 hook: include hooks from the config
    @@ Metadata
      ## Commit message ##
         hook: include hooks from the config
     
    -    Teach the hook.[hc] library to parse configs to populare the list of
    +    Teach the hook.[hc] library to parse configs to populate the list of
         hooks to run for a given event.
     
         Multiple commands can be specified for a given hook by providing
         multiple "hook.<friendly-name>.command = <path-to-hook>" and
    -    "hook.<friendly-name>.event = <hook-event>" lines. Hooks will be run in
    -    config order of the "hook.<name>.event" lines.
    +    "hook.<friendly-name>.event = <hook-event>" lines. Hooks will be started
    +    in config order of the "hook.<name>.event" lines (but may run in
    +    parallel).
     
         For example:
     
    -      $ git config --list | grep ^hook
    +      $ git config --get-regexp "^hook\."
           hook.bar.command=~/bar.sh
           hook.bar.event=pre-commit
     
    -      $ git hook run
    -      # Runs ~/bar.sh
    -      # Runs .git/hooks/pre-commit
    +      # Will run ~/bar.sh, then .git/hooks/pre-commit
    +      $ git hook run pre-commit
     
         Signed-off-by: Emily Shaffer <[email protected]>
     
    @@ Documentation/config/hook.txt
      	hook execution. If unspecified, defaults to the number of processors on
     
      ## Documentation/git-hook.txt ##
    -@@ Documentation/git-hook.txt: Git is unlikely to use for a native hook later on. For example, Git is much less
    - likely to create a `mytool-validate-commit` hook than it is to create a
    - `validate-commit` hook.
    +@@ Documentation/git-hook.txt: This command is an interface to git hooks (see linkgit:githooks[5]).
    + Currently it only provides a convenience wrapper for running hooks for
    + use by git itself. In the future it might gain other functionality.
      
    -+This command parses the default configuration files for pairs of configs like
    ++It's possible to use this command to refer to hooks which are not native to Git,
    ++for example if a wrapper around Git wishes to expose hooks into its own
    ++operation in a way which is already familiar to Git users. However, wrappers
    ++invoking such hooks should be careful to name their hook events something which
    ++Git is unlikely to use for a native hook later on. For example, Git is much less
    ++likely to create a `mytool-validate-commit` hook than it is to create a
    ++`validate-commit` hook.
    ++
    ++This command parses the default configuration files for sets of configs like
     +so:
     +
     +  [hook "linter"]
     +    event = pre-commit
    -+    command = ~/bin/linter --c
    ++    command = ~/bin/linter --cpp20
     +
    -+In this example, `[hook "linter"]` represents one script - `~/bin/linter --c` -
    -+which can be shared by many repos, and even by many hook events, if appropriate.
    ++In this example, `[hook "linter"]` represents one script - `~/bin/linter
    ++--cpp20` - which can be shared by many repos, and even by many hook events, if
    ++appropriate.
    ++
    ++To add an unrelated hook which runs on a different event, for example a
    ++spell-checker for your commit messages, you would write a configuration like so:
    ++
    ++  [hook "linter"]
    ++    event = pre-commit
    ++    command = ~/bin/linter --cpp20
    ++  [hook "spellcheck"]
    ++    event = commit-msg
    ++    command = ~/bin/spellchecker
    ++
    ++With this config, when you run 'git commit', first `~/bin/linter --cpp20` will
    ++have a chance to check your files to be committed (during the `pre-commit` hook
    ++event`), and then `~/bin/spellchecker` will have a chance to check your commit
    ++message (during the `commit-msg` hook event).
     +
     +Commands are run in the order Git encounters their associated
     +`hook.<name>.event` configs during the configuration parse (see
    @@ Documentation/git-hook.txt: Git is unlikely to use for a native hook later on. F
     +  [hook "linter"]
     +    event = pre-commit
     +    event = pre-push
    -+    command = ~/bin/linter --c
    ++    command = ~/bin/linter --cpp20
     +
    -+With this config, `~/bin/linter --c` would be run by Git before a commit is
    ++With this config, `~/bin/linter --cpp20` would be run by Git before a commit is
     +generated (during `pre-commit`) as well as before a push is performed (during
     +`pre-push`).
     +
    @@ Documentation/git-hook.txt: Git is unlikely to use for a native hook later on. F
     +
     +  [hook "linter"]
     +    event = pre-commit
    -+    command = ~/bin/linter --c
    ++    command = ~/bin/linter --cpp20
     +  [hook "no-leaks"]
     +    event = pre-commit
     +    command = ~/bin/leak-detector
     +
     +With this config, before a commit is generated (during `pre-commit`), Git would
    -+first start `~/bin/linter --c` and second start `~/bin/leak-detector`. It would
    -+evaluate the output of each when deciding whether to proceed with the commit.
    ++first start `~/bin/linter --cpp20` and second start `~/bin/leak-detector`. It
    ++would evaluate the output of each when deciding whether to proceed with the
    ++commit.
     +
     +For a full list of hook events which you can set your `hook.<name>.event` to,
     +and how hooks are invoked during those events, see linkgit:githooks[5].
     +
    ++Git will ignore any `hook.<name>.event` that specifies an event it doesn't
    ++recognize. This is intended so that tools which wrap Git can use the hook
    ++infrastructure to run their own hooks; see <<WRAPPERS>> for more guidance.
    ++
     +In general, when instructions suggest adding a script to
    -+`.git/hooks/<hook-event>`, you can specify it in the config instead by running
    -+`git config --add hook.<some-name>.command <path-to-script> && git config --add
    -+hook.<some-name>.event <hook-event>` - this way you can share the script between
    -+multiple repos. That is, `cp ~/my-script.sh ~/project/.git/hooks/pre-commit`
    -+would become `git config --add hook.my-script.command ~/my-script.sh && git
    -+config --add hook.my-script.event pre-commit`.
    ++`.git/hooks/<hook-event>`, you can specify it in the config instead by running:
    ++
    ++----
    ++git config hook.<some-name>.command <path-to-script>
    ++git config --add hook.<some-name>.event <hook-event>
    ++----
    ++
    ++This way you can share the script between multiple repos. That is, `cp
    ++~/my-script.sh ~/project/.git/hooks/pre-commit` would become:
    ++
    ++----
    ++git config hook.my-script.command ~/my-script.sh
    ++git config --add hook.my-script.event pre-commit
    ++----
     +
      SUBCOMMANDS
      -----------
    @@ Documentation/git-hook.txt: Git is unlikely to use for a native hook later on. F
      +
      Any positional arguments to the hook should be passed after an
      optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The
    +@@ Documentation/git-hook.txt: config is not specified, uses the number of CPUs on the current system. Some
    + hooks may be ineligible for parallelization: for example, 'commit-msg' intends
    + hooks modify the commit message body and cannot be parallelized.
    + 
    ++[[WRAPPERS]]
    ++WRAPPERS
    ++--------
    ++
    ++`git hook run` has been designed to make it easy for tools which wrap Git to
    ++configure and execute hooks using the Git hook infrastructure. It is possible to
    ++provide arguments, environment variables (TODO this is missing from reroll TODO),
    ++and stdin via the command line, as well as specifying parallel or series
    ++execution if the user has provided multiple hooks.
    ++
    ++Assuming your wrapper wants to support a hook named "mywrapper-start-tests", you
    ++can have your users specify their hooks like so:
    ++
    ++  [hook "setup-test-dashboard"]
    ++    event = mywrapper-start-tests
    ++    command = ~/mywrapper/setup-dashboard.py --tap
    ++
    ++Then, in your 'mywrapper' tool, you can invoke any users' configured hooks by
    ++running:
    ++
    ++----
    ++git hook run mywrapper-start-tests \
    ++  # providing something to stdin
    ++  --stdin some-tempfile-123 \
    ++  # setting an env var (TODO THIS IS MISSING TODO)
    ++  --env MYWRAPPER_EXECUTION_MODE=foo \
    ++  # execute hooks in serial
    ++  --jobs 1 \
    ++  # plus some arguments of your own...
    ++  -- \
    ++  --testname bar \
    ++  baz
    ++----
    ++
    ++Take care to name your wrapper's hook events in a way which is unlikely to
    ++overlap with Git's native hooks (see linkgit:githooks[5]) - a hook event named
    ++`mywrappertool-validate-commit` is much less likely to be added to native Git
    ++than a hook event named `validate-commit`. If Git begins to use a hook event
    ++named the same thing as your wrapper hook, it may invoke your users' hooks in
    ++unintended and unsupported ways.
    ++
    + CONFIGURATION
    + -------------
    + include::config/hook.txt[]
     
      ## builtin/hook.c ##
     @@ builtin/hook.c: static int list(int argc, const char **argv, const char *prefix)
    @@ builtin/hook.c: static int list(int argc, const char **argv, const char *prefix)
     +						  : _("hook from hookdir"));
      	}
      
    - 	clear_hook_list(head);
    + cleanup:
     
      ## hook.c ##
     @@ hook.c: static void free_hook(struct hook *ptr)
    + 	if (!ptr)
    + 		return;
    + 
    ++	free(ptr->name);
    + 	free(ptr->feed_pipe_cb_data);
      	free(ptr);
      }
      
    @@ hook.c: static void free_hook(struct hook *ptr)
     +
     +	if (!to_add) {
     +		/* adding a new hook, not moving an old one */
    -+		to_add = xmalloc(sizeof(*to_add));
    -+		to_add->name = name;
    -+		to_add->feed_pipe_cb_data = NULL;
    ++		to_add = xcalloc(1, sizeof(*to_add));
    ++		to_add->name = xstrdup_or_null(name);
     +	}
     +
     +	list_add_tail(&to_add->list, head);
    @@ hook.c: static void free_hook(struct hook *ptr)
      {
      	struct hook *hook_to_remove = list_entry(to_remove, struct hook, list);
     @@ hook.c: int hook_exists(const char *name)
    + 	return exists;
    + }
      
    - struct hook_config_cb
    - {
    --	struct strbuf *hook_key;
    ++struct hook_config_cb
    ++{
     +	const char *hook_event;
    - 	struct list_head *list;
    - };
    - 
    ++	struct list_head *list;
    ++};
    ++
     +/*
     + * Callback for git_config which adds configured hooks to a hook list.  Hooks
     + * can be configured by specifying both hook.<friend-name>.command = <path> and
    @@ hook.c: int hook_exists(const char *name)
     +	 */
     +	strbuf_add(&subsection_cpy, subsection, subsection_len);
     +
    -+	append_or_move_hook(data->list, strbuf_detach(&subsection_cpy, NULL));
    -+
    ++	append_or_move_hook(data->list, subsection_cpy.buf);
    ++	strbuf_release(&subsection_cpy);
     +
     +	return 0;
     +}
     +
      struct list_head *list_hooks(const char *hookname)
    - {
    - 	if (!known_hook(hookname))
    -@@ hook.c: struct list_head *list_hooks(const char *hookname)
    - struct list_head *list_hooks_gently(const char *hookname)
      {
      	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
     +	struct hook_config_cb cb_data = {
    @@ hook.c: struct list_head *list_hooks(const char *hookname)
      		BUG("null hookname was provided to hook_list()!");
      
     -	if (have_git_dir()) {
    --		const char *hook_path = find_hook_gently(hookname);
    +-		const char *hook_path = find_hook(hookname);
    ++	/* Add the hooks from the config, e.g. hook.myhook.event = pre-commit */
    ++	git_config(hook_config_lookup, &cb_data);
    + 
    +-		/* Add the hook from the hookdir */
     -		if (hook_path) {
     -			struct hook *to_add = xmalloc(sizeof(*to_add));
     -			to_add->hook_path = hook_path;
    @@ hook.c: struct list_head *list_hooks(const char *hookname)
     -			list_add_tail(&to_add->list, hook_head);
     -		}
     -	}
    -+	/* Add the hooks from the config, e.g. hook.myhook.event = pre-commit */
    -+	git_config(hook_config_lookup, &cb_data);
    -+
     +	/* Add the hook from the hookdir. The placeholder makes it easier to
     +	 * allocate work in pick_next_hook. */
    -+	if (find_hook_gently(hookname))
    ++	if (find_hook(hookname))
     +		append_or_move_hook(hook_head, NULL);
      
      	return hook_head;
    @@ hook.c: static int pick_next_hook(struct child_process *cp,
      	cp->trace2_hook_name = hook_cb->hook_name;
      	cp->dir = hook_cb->options->dir;
      
    -+	/* to enable oneliners, let config-specified hooks run in shell.
    -+	 * config-specified hooks have a name. */
    ++	/*
    ++	 * to enable oneliners, let config-specified hooks run in shell.
    ++	 * config-specified hooks have a name.
    ++	 */
     +	cp->use_shell = !!run_me->name;
     +
      	/* add command */
    @@ hook.c: static int pick_next_hook(struct child_process *cp,
     +		}
     +
     +		strvec_push(&cp->args, command);
    ++		free(command);
    ++		strbuf_release(&cmd_key);
     +	} else {
     +		/* ...from hookdir. */
     +		const char *hook_path = NULL;
     +		/*
    -+		 *
     +		 * At this point we are already running, so don't validate
    -+		 * whether the hook name is known or not.
    ++		 * whether the hook name is known or not. Validation was
    ++		 * performed earlier in list_hooks().
     +		 */
    -+		hook_path = find_hook_gently(hook_cb->hook_name);
    ++		hook_path = find_hook(hook_cb->hook_name);
     +		if (!hook_path)
     +			BUG("hookdir hook in hook list but no hookdir hook present in filesystem");
     +
    @@ hook.c: static int notify_start_failure(struct strbuf *out,
      }
     
      ## hook.h ##
    -@@ hook.h: int hook_exists(const char *hookname);
    +@@
      
      struct hook {
      	struct list_head list;
    @@ hook.h: int hook_exists(const char *hookname);
     +	 * The friendly name of the hook. NULL indicates the hook is from the
     +	 * hookdir.
     +	 */
    -+	const char *name;
    ++	char *name;
      
      	/*
      	 * Use this to keep state for your feed_pipe_fn if you are using
     
      ## t/t1800-hook.sh ##
     @@
    - #!/bin/bash
    + #!/bin/sh
      
     -test_description='git-hook command'
     +test_description='git-hook command and config-managed multihooks'
    @@ t/t1800-hook.sh
      . ./test-lib.sh
      
     +setup_hooks () {
    ++	test_config hook.ghi.command "/path/ghi"
     +	test_config hook.ghi.event pre-commit --add
    -+	test_config hook.ghi.command "/path/ghi" --add
    ++	test_config hook.ghi.event test-hook --add
    ++	test_config_global hook.def.command "/path/def"
     +	test_config_global hook.def.event pre-commit --add
    -+	test_config_global hook.def.command "/path/def" --add
     +}
     +
     +setup_hookdir () {
    @@ t/t1800-hook.sh
      	test_expect_code 129 git hook run -h &&
     +	test_expect_code 129 git hook list -h &&
      	test_expect_code 129 git hook run --unknown 2>err &&
    - 	grep "unknown option" err
    - '
    + 	test_expect_code 129 git hook list &&
    + 	test_expect_code 129 git hook list -h &&
    +@@ t/t1800-hook.sh: test_expect_success 'git hook list: does-not-exist hook' '
    + 
    + test_expect_success 'git hook list: existing hook' '
    + 	cat >expect <<-\EOF &&
    +-	.git/hooks/test-hook
    ++	hook from hookdir
    + 	EOF
    + 	git hook list test-hook >actual &&
    + 	test_cmp expect actual
     @@ t/t1800-hook.sh: test_expect_success 'stdin to hooks' '
      	test_cmp expect actual
      '
    @@ t/t1800-hook.sh: test_expect_success 'stdin to hooks' '
     +test_expect_success 'git hook list orders by config order' '
     +	setup_hooks &&
     +
    -+	cat >expected <<-EOF &&
    ++	cat >expected <<-\EOF &&
     +	def
     +	ghi
     +	EOF
    @@ t/t1800-hook.sh: test_expect_success 'stdin to hooks' '
     +	# configuring it locally.
     +	test_config hook.def.event "pre-commit" --add &&
     +
    -+	cat >expected <<-EOF &&
    ++	cat >expected <<-\EOF &&
     +	ghi
     +	def
     +	EOF
    @@ t/t1800-hook.sh: test_expect_success 'stdin to hooks' '
     +	test_cmp expected actual
     +'
     +
    ++test_expect_success 'hook can be configured for multiple events' '
    ++	setup_hooks &&
    ++
    ++	# 'ghi' should be included in both 'pre-commit' and 'test-hook'
    ++	git hook list pre-commit >actual &&
    ++	grep "ghi" actual &&
    ++	git hook list test-hook >actual &&
    ++	grep "ghi" actual
    ++'
    ++
     +test_expect_success 'git hook list shows hooks from the hookdir' '
     +	setup_hookdir &&
     +
    -+	cat >expected <<-EOF &&
    ++	cat >expected <<-\EOF &&
     +	hook from hookdir
     +	EOF
     +
    @@ t/t1800-hook.sh: test_expect_success 'stdin to hooks' '
     +'
     +
     +test_expect_success 'inline hook definitions resolve paths' '
    -+	write_script sample-hook.sh <<-EOF &&
    ++	write_script sample-hook.sh <<-\EOF &&
     +	echo \"Sample Hook\"
     +	EOF
     +
    @@ t/t1800-hook.sh: test_expect_success 'stdin to hooks' '
     +	test_config hook.stdin-b.event "test-hook" --add &&
     +	test_config hook.stdin-b.command "xargs -P1 -I% echo b%" --add &&
     +
    -+	cat >input <<-EOF &&
    ++	cat >input <<-\EOF &&
     +	1
     +	2
     +	3
     +	EOF
     +
    -+	cat >expected <<-EOF &&
    ++	cat >expected <<-\EOF &&
     +	a1
     +	a2
     +	a3
    @@ t/t1800-hook.sh: test_expect_success 'stdin to hooks' '
     +	echo 3
     +	EOF
     +
    -+	cat >expected <<-EOF &&
    ++	cat >expected <<-\EOF &&
     +	1
     +	2
     +	3
    @@ t/t1800-hook.sh: test_expect_success 'stdin to hooks' '
     +
     +	rm -rf .git/hooks
     +'
    ++
    ++test_expect_success 'rejects hooks with no commands configured' '
    ++	test_config hook.broken.event "test-hook" &&
    ++
    ++	echo broken >expected &&
    ++	git hook list test-hook >actual &&
    ++	test_cmp expected actual &&
    ++	test_must_fail git hook run test-hook
    ++'
    ++
      test_done
6:  30ffe98601e ! 5:  5d5e9726fd8 hook: allow out-of-repo 'git hook' invocations
    @@ git.c: static struct cmd_struct commands[] = {
      	{ "init-db", cmd_init_db },
     
      ## hook.c ##
    -@@ hook.c: struct list_head *list_hooks_gently(const char *hookname)
    +@@ hook.c: struct list_head *list_hooks(const char *hookname)
      
      	/* Add the hook from the hookdir. The placeholder makes it easier to
      	 * allocate work in pick_next_hook. */
    --	if (find_hook_gently(hookname))
    -+	if (have_git_dir() && find_hook_gently(hookname))
    +-	if (find_hook(hookname))
    ++	if (have_git_dir() && find_hook(hookname))
      		append_or_move_hook(hook_head, NULL);
      
      	return hook_head;
     
      ## t/t1800-hook.sh ##
    -@@ t/t1800-hook.sh: test_expect_success 'git hook run -- pass arguments' '
    +@@ t/t1800-hook.sh: test_expect_success 'git hook list: existing hook' '
      	test_cmp expect actual
      '
Signature · 1 line
2.33.0.867.g88ec4638586
#600 lore raw

PATCH v4 1/5 hook: run a list of hooks instead

From: Emily Shaffer <[email protected]>

To prepare for multihook support, teach hook.[hc] to take a list of hooks at run_hooks and run_found_hooks. Right now the list is always one entry, but in the future we will allow users to supply more than one executable for a single hook event.

---
 builtin/hook.c |  14 ++++---
 hook.c         | 109 ++++++++++++++++++++++++++++++++++++-------------
 hook.h         |  22 ++++++++--
 3 files changed, 107 insertions(+), 38 deletions(-)
builtin/hook.c +8 -6
index fae69068201..398980523aa 100644
@@ -25,7 +25,7 @@ static int run(int argc, const char **argv, const char *prefix)
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	int ignore_missing = 0;
 	const char *hook_name;
-	const char *hook_path;
+	struct list_head *hooks;
 	struct option run_options[] = {
 		OPT_BOOL(0, "ignore-missing", &ignore_missing,
 			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
@@ -63,16 +63,18 @@ static int run(int argc, const char **argv, const char *prefix)
 	 * run_hooks() instead...
 	 */
 	hook_name = argv[0];
-	if (ignore_missing)
+	hooks = list_hooks(hook_name);
+	if (list_empty(hooks)) {
+		clear_hook_list(hooks);
+
 		/* ... act like a plain run_hooks() under --ignore-missing */
-		return run_hooks_oneshot(hook_name, &opt);
-	hook_path = find_hook(hook_name);
-	if (!hook_path) {
+		if (ignore_missing)
+			return 0;
 		error("cannot find a hook named %s", hook_name);
 		return 1;
 	}
 
-	ret = run_hooks(hook_name, hook_path, &opt);
+	ret = run_hooks(hook_name, hooks, &opt);
 	run_hooks_opt_clear(&opt);
 	return ret;
 usage:
hook.c +80 -29

167 lines not on this page · raw message

hook.h +19 -3
index f6dac75f1cc..49b4c335f86 100644
@@ -3,8 +3,10 @@
 #include "strbuf.h"
 #include "strvec.h"
 #include "run-command.h"
+#include "list.h"
 
 struct hook {
+	struct list_head list;
 	/* The path to the hook */
 	const char *hook_path;
 
@@ -75,6 +77,7 @@ struct hook_cb_data {
 	/* rc reflects the cumulative failure state */
 	int rc;
 	const char *hook_name;
+	struct list_head *head;
 	struct hook *run_me;
 	struct run_hooks_opt *options;
 	int *invoked_hook;
@@ -88,7 +91,13 @@ struct hook_cb_data {
 const char *find_hook(const char *name);
 
 /**
- * A boolean version of find_hook()
+ * Provides a linked list of 'struct hook' detailing commands which should run
+ * in response to the 'hookname' event, in execution order.
+ */
+struct list_head *list_hooks(const char *hookname);
+
+/**
+ * A boolean version of list_hooks()
  */
 int hook_exists(const char *hookname);
 
@@ -99,13 +108,20 @@ void run_hooks_opt_clear(struct run_hooks_opt *o);
 
 /**
  * Takes an already resolved hook found via find_hook() and runs
- * it. Does not call run_hooks_opt_clear() for you.
+ * it. Does not call run_hooks_opt_clear() for you, but does call
+ * clear_hook_list().
  *
  * See run_hooks_oneshot() for the simpler one-shot API.
  */
-int run_hooks(const char *hookname, const char *hook_path,
+int run_hooks(const char *hookname, struct list_head *hooks,
 	      struct run_hooks_opt *options);
 
+/**
+ * Empties the list at 'head', calling 'free_hook()' on each
+ * entry. Called implicitly by run_hooks() (and run_hooks_oneshot()).
+ */
+void clear_hook_list(struct list_head *head);
+
 /**
  * Calls find_hook() on your "hook_name" and runs the hooks (if any)
  * with run_hooks().
Signature · 1 line
2.33.0.867.g88ec4638586
#601 lore raw

PATCH v4 1/1 fixup! hook: run a list of hooks instead

---

And of course just as I sent out <[email protected]> I noticed that there was one last memory leak left, it just wasn't stressed by t1800-hook.sh, but some of the test of the tests, including t0000-basic.sh!

That test coverage should be improved, but in the meantime Junio:
Here's a fixup to the just-sent you can hopefully squash into the 1/5. Sorry about the screwup.

 hook.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
hook.c +3 -1
index dc3033cb4c7..4b4205bce6c 100644
@@ -416,8 +416,10 @@ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 	 * If you need to act on a missing hook, use run_found_hooks()
 	 * instead
 	 */
-	if (list_empty(hooks))
+	if (list_empty(hooks)) {
+		clear_hook_list(hooks);
 		goto cleanup;
+	}
 
 	ret = run_hooks(hook_name, hooks, options);
 
Signature · 1 line
2.33.0.867.g88ec4638586
#602 lore raw

PATCH v4 2/5 hook: allow parallel hook execution

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

In many cases, there's no reason not to allow hooks to execute in parallel, if more than one was provided. hook.c already calls run_processes_parallel(), so all we need to do is allow the job count we hand to run_processes_parallel() to be greater than 1.

If users have specified no alternative, we can use the processor count from online_cpus() to run an efficient number of tasks at once. However, users can also override this number if they want by configuring 'hook.jobs'.

To avoid looking up 'hook.jobs' in cases where we don't end up with any hooks to run anyways, teach the hook runner commands to notice if .jobs==0 and do a config or online_cpus() lookup if so, when we already know we have jobs to run.

Serial execution can still be performed by setting .jobs == 1.

---
 Documentation/config.txt      |  2 ++
 Documentation/config/hook.txt |  4 ++++
 Documentation/git-hook.txt    | 17 ++++++++++++++++-
 builtin/am.c                  |  4 ++--
 builtin/checkout.c            |  2 +-
 builtin/clone.c               |  2 +-
 builtin/hook.c                |  4 +++-
 builtin/merge.c               |  2 +-
 builtin/rebase.c              |  2 +-
 builtin/receive-pack.c        |  9 +++++----
 builtin/worktree.c            |  2 +-
 commit.c                      |  2 +-
 hook.c                        | 35 ++++++++++++++++++++++++++++++++---
 hook.h                        | 16 +++++++++++++++-
 read-cache.c                  |  2 +-
 refs.c                        |  2 +-
 reset.c                       |  3 ++-
 sequencer.c                   |  4 ++--
 transport.c                   |  2 +-
 19 files changed, 92 insertions(+), 24 deletions(-)
 create mode 100644 Documentation/config/hook.txt
Documentation/config.txt +2 -0
index 0c0e6b859f1..6fb218f649d 100644
@@ -391,6 +391,8 @@ include::config/guitool.txt[]
 
 include::config/help.txt[]
 
+include::config/hook.txt[]
+
 include::config/http.txt[]
 
 include::config/i18n.txt[]
Documentation/config/hook.txt new file +4 -0
new file mode 100644
index 00000000000..96d3d6572c1
@@ -0,0 +1,4 @@
+hook.jobs::
+	Specifies how many hooks can be run simultaneously during parallelized
+	hook execution. If unspecified, defaults to the number of processors on
+	the current system.
Documentation/git-hook.txt +16 -1
index fa68c1f3912..79e82479ec6 100644
@@ -8,7 +8,8 @@ git-hook - run git hooks
 SYNOPSIS
 --------
 [verse]
-'git hook' run [--to-stdin=<path>] [--ignore-missing] <hook-name> [-- <hook-args>]
+'git hook' run [--to-stdin=<path>] [--ignore-missing] [(-j|--jobs) <n>]
+	<hook-name> [-- <hook-args>]
 
 DESCRIPTION
 -----------
@@ -42,6 +43,20 @@ OPTIONS
 	tools that want to do a blind one-shot run of a hook that may
 	or may not be present.
 
+-j::
+--jobs::
+	Only valid for `run`.
++
+Specify how many hooks to run simultaneously. If this flag is not specified,
+uses the value of the `hook.jobs` config, see linkgit:git-config[1]. If the
+config is not specified, uses the number of CPUs on the current system. Some
+hooks may be ineligible for parallelization: for example, 'commit-msg' intends
+hooks modify the commit message body and cannot be parallelized.
+
+CONFIGURATION
+-------------
+include::config/hook.txt[]
+
 SEE ALSO
 --------
 linkgit:githooks[5]
builtin/am.c +2 -2
index 9e3d4d9ab44..c7ffc7eafc5 100644
@@ -446,7 +446,7 @@ static void am_destroy(const struct am_state *state)
 static int run_applypatch_msg_hook(struct am_state *state)
 {
 	int ret;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
 
 	assert(state->msg);
 	strvec_push(&opt.args, am_path(state, "final-commit"));
@@ -467,7 +467,7 @@ static int run_applypatch_msg_hook(struct am_state *state)
  */
 static int run_post_rewrite_hook(const struct am_state *state)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 
 	strvec_push(&opt.args, "rebase");
 	opt.path_to_stdin = am_path(state, "rewritten");
builtin/checkout.c +1 -1
index 863b02a7d7c..6b99d31c6ba 100644
@@ -107,7 +107,7 @@ struct branch_info {
 static int post_checkout_hook(struct commit *old_commit, struct commit *new_commit,
 			      int changed)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
 
 	/* "new_commit" can be NULL when checking out from the index before
 	   a commit exists. */
builtin/clone.c +1 -1
index 27fc05ee511..986c3b1932a 100644
@@ -776,7 +776,7 @@ static int checkout(int submodule_progress)
 	struct tree *tree;
 	struct tree_desc t;
 	int err = 0;
-	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_SERIAL;
 
 	if (option_no_checkout)
 		return 0;
builtin/hook.c +3 -1
index 398980523aa..9b6272cfd3b 100644
@@ -22,7 +22,7 @@ static const char * const builtin_hook_run_usage[] = {
 static int run(int argc, const char **argv, const char *prefix)
 {
 	int i;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
 	int ignore_missing = 0;
 	const char *hook_name;
 	struct list_head *hooks;
@@ -31,6 +31,8 @@ static int run(int argc, const char **argv, const char *prefix)
 			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
 		OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"),
 			   N_("file to read into hooks' stdin")),
+		OPT_INTEGER('j', "jobs", &opt.jobs,
+			    N_("run up to <n> hooks simultaneously")),
 		OPT_END(),
 	};
 	int ret;
builtin/merge.c +1 -1
index f215f264cc8..c01eb535d6b 100644
@@ -448,7 +448,7 @@ static void finish(struct commit *head_commit,
 		   const struct object_id *new_head, const char *msg)
 {
 	struct strbuf reflog_message = STRBUF_INIT;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 	const struct object_id *head = &head_commit->object.oid;
 
 	if (!msg)
builtin/rebase.c +1 -1
index ee68a1df492..a4bbb9abb35 100644
@@ -1314,7 +1314,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	char *squash_onto_name = NULL;
 	int reschedule_failed_exec = -1;
 	int allow_preemptive_ff = 1;
-	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 	struct option builtin_rebase_options[] = {
 		OPT_STRING(0, "onto", &options.onto_name,
 			   N_("revision"),
builtin/receive-pack.c +5 -4
index ebec6f3bb10..7460124b743 100644
@@ -909,7 +909,7 @@ static int run_receive_hook(struct command *commands,
 			    int skip_broken,
 			    const struct string_list *push_options)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 	struct receive_hook_feed_context ctx;
 	struct command *iter = commands;
 
@@ -948,7 +948,7 @@ static int run_receive_hook(struct command *commands,
 
 static int run_update_hook(struct command *cmd)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 
 	strvec_pushl(&opt.args,
 		     cmd->ref_name,
@@ -1432,7 +1432,8 @@ static const char *push_to_checkout(unsigned char *hash,
 				    struct strvec *env,
 				    const char *work_tree)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
+
 	opt.invoked_hook = invoked_hook;
 
 	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
@@ -1628,7 +1629,7 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 static void run_update_post_hook(struct command *commands)
 {
 	struct command *cmd;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 
 	for (cmd = commands; cmd; cmd = cmd->next) {
 		if (cmd->error_string || cmd->did_not_exist)
builtin/worktree.c +1 -1
index 330867c19bf..30905067906 100644
@@ -382,7 +382,7 @@ static int add_worktree(const char *path, const char *refname,
 	 * is_junk is cleared, but do return appropriate code when hook fails.
 	 */
 	if (!ret && opts->checkout) {
-		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
 
 		strvec_pushl(&opt.env, "GIT_DIR", "GIT_WORK_TREE", NULL);
 		strvec_pushl(&opt.args,
commit.c +1 -1
index 842e47beae2..a38bd047524 100644
@@ -1700,7 +1700,7 @@ int run_commit_hook(int editor_is_used, const char *index_file,
 		    int *invoked_hook,
 		    const char *name, ...)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
 	va_list args;
 	const char *arg;
 
hook.c +32 -3
index 2b2c16a9095..600030c59ec 100644
@@ -208,6 +208,28 @@ static int notify_hook_finished(int result,
 	return 0;
 }
 
+/*
+ * Determines how many jobs to use after we know we want to parallelize. First
+ * priority is the config 'hook.jobs' and second priority is the number of CPUs.
+ */
+static int configured_hook_jobs(void)
+{
+	/*
+	 * The config and the CPU count probably won't change during the process
+	 * lifetime, so cache the result in case we invoke multiple hooks during
+	 * one process.
+	 */
+	static int jobs = 0;
+	if (jobs)
+		return jobs;
+
+	if (git_config_get_int("hook.jobs", &jobs))
+		/* if the config isn't set, fall back to CPU count. */
+		jobs = online_cpus();
+
+	return jobs;
+}
+
 int run_hooks(const char *hook_name, struct list_head *hooks,
 	      struct run_hooks_opt *options)
 {
@@ -217,7 +239,6 @@ int run_hooks(const char *hook_name, struct list_head *hooks,
 		.options = options,
 		.invoked_hook = options->invoked_hook,
 	};
-	int jobs = 1;
 
 	if (!options)
 		BUG("a struct run_hooks_opt must be provided to run_hooks");
@@ -225,7 +246,11 @@ int run_hooks(const char *hook_name, struct list_head *hooks,
 	cb_data.head = hooks;
 	cb_data.run_me = list_first_entry(hooks, struct hook, list);
 
-	run_processes_parallel_tr2(jobs,
+	/* INIT_PARALLEL sets jobs to 0, so go look up how many to use. */
+	if (!options->jobs)
+		options->jobs = configured_hook_jobs();
+
+	run_processes_parallel_tr2(options->jobs,
 				   pick_next_hook,
 				   notify_start_failure,
 				   options->feed_pipe,
@@ -244,7 +269,11 @@ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 {
 	struct list_head *hooks;
 	int ret = 0;
-	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT;
+	/*
+	 * Turn on parallelism by default. Hooks which don't want it should
+	 * specify their options accordingly.
+	 */
+	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT_PARALLEL;
 
 	if (!options)
 		options = &hook_opt_scratch;
hook.h +15 -1
index 49b4c335f86..fe16ab35028 100644
@@ -25,6 +25,13 @@ struct run_hooks_opt
 	/* Args to be passed to each hook */
 	struct strvec args;
 
+	/*
+	 * Number of threads to parallelize across. Set to 0 to use the
+	 * 'hook.jobs' config or, if that config is unset, the number of cores
+	 * on the system.
+	 */
+	int jobs;
+
 	/*
 	 * Resolve and run the "absolute_path(hook)" instead of
 	 * "hook". Used for "git worktree" hooks
@@ -68,7 +75,14 @@ struct run_hooks_opt
 	int *invoked_hook;
 };
 
-#define RUN_HOOKS_OPT_INIT { \
+#define RUN_HOOKS_OPT_INIT_SERIAL { \
+	.jobs = 1, \
+	.env = STRVEC_INIT, \
+	.args = STRVEC_INIT, \
+}
+
+#define RUN_HOOKS_OPT_INIT_PARALLEL { \
+	.jobs = 0, \
 	.env = STRVEC_INIT, \
 	.args = STRVEC_INIT, \
 }
read-cache.c +1 -1
index 875f6c1dea5..98e9fb8b04e 100644
@@ -3069,7 +3069,7 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 {
 	int ret;
 	int was_full = !istate->sparse_index;
-	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 
 	ret = convert_to_sparse(istate);
 
refs.c +1 -1
index 73d4a939267..5543b8cdaba 100644
@@ -2062,7 +2062,7 @@ int ref_update_reject_duplicates(struct string_list *refnames,
 static int run_transaction_hook(struct ref_transaction *transaction,
 				const char *state)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
 	int ret = 0, i;
 
reset.c +2 -1
index 1237ced8a58..63a9c513409 100644
@@ -127,7 +127,8 @@ int reset_head(struct repository *r, struct object_id *oid, const char *action,
 					    reflog_head);
 	}
 	if (run_hook) {
-		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
+
 		strvec_pushl(&opt.args,
 			     oid_to_hex(orig ? orig : null_oid()),
 			     oid_to_hex(oid),
sequencer.c +2 -2
index db8044ab47d..979cd01c303 100644
@@ -1148,7 +1148,7 @@ int update_head_with_reflog(const struct commit *old_head,
 static int run_rewrite_hook(const struct object_id *oldoid,
 			    const struct object_id *newoid)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 	struct strbuf tmp = STRBUF_INIT;
 	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 	int code;
@@ -4522,7 +4522,7 @@ static int pick_commits(struct repository *r,
 		if (!stat(rebase_path_rewritten_list(), &st) &&
 				st.st_size > 0) {
 			struct child_process notes_cp = CHILD_PROCESS_INIT;
-			struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+			struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 
 			notes_cp.in = open(rebase_path_rewritten_list(), O_RDONLY);
 			notes_cp.git_cmd = 1;
transport.c +1 -1
index 4ca8fc0391d..33da71a108b 100644
@@ -1204,7 +1204,7 @@ static int run_pre_push_hook(struct transport *transport,
 			     struct ref *remote_refs)
 {
 	int ret = 0;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 	struct ref *r;
 	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
 
Signature · 1 line
2.33.0.867.g88ec4638586
#603 lore raw

PATCH v4 3/5 hook: introduce "git hook list"

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

If more than one hook will be run, it may be useful to see a list of which hooks should be run. At very least, it will be useful for us to test the semantics of multihooks ourselves.

For now, only list the hooks which will run in the order they will run in; later, it might be useful to include more information like where the hooks were configured and whether or not they will run.

---
 Documentation/git-hook.txt |  5 ++++
 builtin/hook.c             | 52 ++++++++++++++++++++++++++++++++++++++
 t/t1800-hook.sh            | 14 ++++++++++
 3 files changed, 71 insertions(+)
Documentation/git-hook.txt +5 -0
index 79e82479ec6..88588b38143 100644
@@ -10,6 +10,7 @@ SYNOPSIS
 [verse]
 'git hook' run [--to-stdin=<path>] [--ignore-missing] [(-j|--jobs) <n>]
 	<hook-name> [-- <hook-args>]
+'git hook' list <hook-name>
 
 DESCRIPTION
 -----------
@@ -30,6 +31,10 @@ optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The
 arguments (if any) differ by hook name, see linkgit:githooks[5] for
 what those are.
 
+list::
+	Print a list of hooks which will be run on `<hook-name>` event. If no
+	hooks are configured for that event, print nothing and return 1.
+
 OPTIONS
 -------
 
builtin/hook.c +52 -0

72 lines not on this page · raw message

t/t1800-hook.sh +14 -0
index 6431b19e392..7a1dae4e95e 100755
@@ -9,6 +9,8 @@ test_expect_success 'git hook usage' '
 	test_expect_code 129 git hook run &&
 	test_expect_code 129 git hook run -h &&
 	test_expect_code 129 git hook run --unknown 2>err &&
+	test_expect_code 129 git hook list &&
+	test_expect_code 129 git hook list -h &&
 	grep "unknown option" err
 '
 
@@ -97,6 +99,18 @@ test_expect_success 'git hook run -- pass arguments' '
 	test_cmp expect actual
 '
 
+test_expect_success 'git hook list: does-not-exist hook' '
+	test_expect_code 1 git hook list does-not-exist
+'
+
+test_expect_success 'git hook list: existing hook' '
+	cat >expect <<-\EOF &&
+	.git/hooks/test-hook
+	EOF
+	git hook list test-hook >actual &&
+	test_cmp expect actual
+'
+
 test_expect_success 'git hook run -- out-of-repo runs excluded' '
 	write_script .git/hooks/test-hook <<-EOF &&
 	echo Test hook
Signature · 1 line
2.33.0.867.g88ec4638586
#604 lore raw

PATCH v4 4/5 hook: include hooks from the config

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Teach the hook.[hc] library to parse configs to populate the list of hooks to run for a given event.

Multiple commands can be specified for a given hook by providing multiple "hook.<friendly-name>.command = <path-to-hook>" and "hook.<friendly-name>.event = <hook-event>" lines. Hooks will be started in config order of the "hook.<name>.event" lines (but may run in parallel).

For example:

  $ git config --get-regexp "^hook\."
  hook.bar.command=~/bar.sh
  hook.bar.event=pre-commit

# Will run ~/bar.sh, then .git/hooks/pre-commit

  $ git hook run pre-commit
---
 Documentation/config/hook.txt |  18 ++++
 Documentation/git-hook.txt    | 135 +++++++++++++++++++++++++++-
 builtin/hook.c                |   3 +-
 hook.c                        | 161 +++++++++++++++++++++++++++++----
 hook.h                        |   7 +-
 t/t1800-hook.sh               | 164 +++++++++++++++++++++++++++++++++-
 6 files changed, 465 insertions(+), 23 deletions(-)
Documentation/config/hook.txt +18 -0
index 96d3d6572c1..c3947563280 100644
@@ -1,3 +1,21 @@
+hook.<name>.command::
+	A command to execute whenever `hook.<name>` is invoked. `<name>` should
+	be a unique "friendly" name which you can use to identify this hook
+	command. (You can specify when to invoke this command with
+	`hook.<name>.event`.) The value can be an executable on your device or a
+	oneliner for your shell. If more than one value is specified for the
+	same `<name>`, the last value parsed will be the only command executed.
+	See linkgit:git-hook[1].
+
+hook.<name>.event::
+	The hook events which should invoke `hook.<name>`. `<name>` should be a
+	unique "friendly" name which you can use to identify this hook. The
+	value should be the name of a hook event, like "pre-commit" or "update".
+	(See linkgit:githooks[5] for a complete list of hooks Git knows about.)
+	On the specified event, the associated `hook.<name>.command` will be
+	executed. More than one event can be specified if you wish for
+	`hook.<name>` to execute on multiple events. See linkgit:git-hook[1].
+
 hook.jobs::
 	Specifies how many hooks can be run simultaneously during parallelized
 	hook execution. If unspecified, defaults to the number of processors on
Documentation/git-hook.txt +133 -2

151 lines not on this page · raw message

builtin/hook.c +2 -1
index 1e6b15d565a..bb8fdde6bad 100644
@@ -61,7 +61,8 @@ static int list(int argc, const char **argv, const char *prefix)
 		struct hook *item = list_entry(pos, struct hook, list);
 		item = list_entry(pos, struct hook, list);
 		if (item)
-			printf("%s\n", item->hook_path);
+			printf("%s\n", item->name ? item->name
+						  : _("hook from hookdir"));
 	}
 
 cleanup:
hook.c +145 -16

200 lines not on this page · raw message

hook.h +5 -2
index fe16ab35028..4b728991089 100644
@@ -7,8 +7,11 @@
 
 struct hook {
 	struct list_head list;
-	/* The path to the hook */
-	const char *hook_path;
+	/*
+	 * The friendly name of the hook. NULL indicates the hook is from the
+	 * hookdir.
+	 */
+	char *name;
 
 	/*
 	 * Use this to keep state for your feed_pipe_fn if you are using
t/t1800-hook.sh +162 -2

186 lines not on this page · raw message

Signature · 1 line
2.33.0.867.g88ec4638586
#605 lore raw

PATCH v4 5/5 hook: allow out-of-repo 'git hook' invocations

In reply to Ævar Arnfjörð Bjarmason

From: Emily Shaffer <[email protected]>

Since hooks can now be supplied via the config, and a config can be present without a gitdir via the global and system configs, we can start to allow 'git hook run' to occur without a gitdir. This enables us to do things like run sendemail-validate hooks when running 'git send-email' from a nongit directory.

It still doesn't make sense to look for hooks in the hookdir in nongit repos, though, as there is no hookdir.

---
 git.c           |  2 +-
 hook.c          |  2 +-
 t/t1800-hook.sh | 20 +++++++++++++++-----
 3 files changed, 17 insertions(+), 7 deletions(-)
git.c +1 -1
index 540909c391f..39988ee3b02 100644
@@ -538,7 +538,7 @@ static struct cmd_struct commands[] = {
 	{ "grep", cmd_grep, RUN_SETUP_GENTLY },
 	{ "hash-object", cmd_hash_object },
 	{ "help", cmd_help },
-	{ "hook", cmd_hook, RUN_SETUP },
+	{ "hook", cmd_hook, RUN_SETUP_GENTLY },
 	{ "index-pack", cmd_index_pack, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "init", cmd_init_db },
 	{ "init-db", cmd_init_db },
hook.c +1 -1
index b825fa7c7ae..dc3033cb4c7 100644
@@ -181,7 +181,7 @@ struct list_head *list_hooks(const char *hookname)
 
 	/* Add the hook from the hookdir. The placeholder makes it easier to
 	 * allocate work in pick_next_hook. */
-	if (find_hook(hookname))
+	if (have_git_dir() && find_hook(hookname))
 		append_or_move_hook(hook_head, NULL);
 
 	return hook_head;
t/t1800-hook.sh +15 -5
index 68e7ff7de7e..6b6ba30e88e 100755
@@ -128,15 +128,25 @@ test_expect_success 'git hook list: existing hook' '
 	test_cmp expect actual
 '
 
-test_expect_success 'git hook run -- out-of-repo runs excluded' '
-	write_script .git/hooks/test-hook <<-EOF &&
-	echo Test hook
-	EOF
+test_expect_success 'git hook run: out-of-repo runs execute global hooks' '
+	test_config_global hook.global-hook.event test-hook --add &&
+	test_config_global hook.global-hook.command "echo no repo no problems" --add &&
+
+	echo "global-hook" >expect &&
+	nongit git hook list test-hook >actual &&
+	test_cmp expect actual &&
+
+	echo "no repo no problems" >expect &&
 
-	nongit test_must_fail git hook run test-hook
+	nongit git hook run test-hook 2>actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'git -c core.hooksPath=<PATH> hook run' '
+	write_script .git/hooks/test-hook <<-EOF &&
+	echo Test hook
+	EOF
+
 	mkdir my-hooks &&
 	write_script my-hooks/test-hook <<-\EOF &&
 	echo Hook ran $1 >>actual
Signature · 1 line
2.33.0.867.g88ec4638586
#606 lore raw

Re: [PATCH v4 00/36] Run hooks via "git run hook" & hook library

In reply to Ævar Arnfjörð Bjarmason
Quoted 2 lines · Ævar Arnfjörð Bjarmason

This is a v4 re-roll of the "Base for "config-based-hooks" topic.

I performed the rebase of my series on top of this one, and did not find very much objectionable there, but do not consider this email a thorough review of the interdiff. I expect to have time for that later in the week.

Quoted 10 lines

Other updates:

* In the base topic the s/Signed-off-by/Reviewed-by/g from René change that Junio applied locally has been folded in.

* Almost all the callers were just "one-shot" callers, I introduced a new run_hooks_oneshot() function for those, which gets rid of the verbosity around memory management, see e.g. the "builtin/gc.c" in the range-diff below. That run_hooks_oneshot() can also take a NULL set of options.

I am not so wild about this, to be honest, only because after my patch 2, it is hard to understand whether or not a hook is running in parallel or in series. It is extra code-reader overhead to know that run_hooks_oneshot() assumes that a hook can be parallelized, and I think that is unfortunate.

But that is a pretty minor complaint, and I do like the lower complexity, and I agree the defaults are "good enough" for most hook events. So I guess I'm asking for ideas on how to make it more obvious what this oneshot call does in terms of parallelism.

Thanks, and I hope to get to a fuller review of the interdiff soon.

- Emily

#607 lore raw

Re: [PATCH v4 00/36] Run hooks via "git run hook" & hook library

In reply to Ævar Arnfjörð Bjarmason
Quoted 7 lines · Ævar Arnfjörð Bjarmason

This is a v4 re-roll of the "Base for "config-based-hooks" topic.
* Almost all the callers were just "one-shot" callers, I introduced a new run_hooks_oneshot() function for those, which gets rid of the verbosity around memory management, see e.g. the "builtin/gc.c" in the range-diff below. That run_hooks_oneshot() can also take a NULL set of options.

I discussed this elswhere, but just to make doubly sure that comment doesn't get lost, here's a link to my mail:
https://lore.kernel.org/git/YR2jLdYQA5CVzX5h%40google.com

Quoted 8 lines

Range-diff against v3:

 1:  27c94247f87 =  1:  81fe1ed90d5 Makefile: mark "check" target as .PHONY
 2:  6e164edb0b0 !  2:  0f749530777 Makefile: stop hardcoding {command,config}-list.h
 3:  ddae86802e2 !  3:  644b31fe281 Makefile: remove an out-of-date comment
 4:  58c37e4f06e =  4:  89c4d44b0c3 hook.[ch]: move find_hook() to this new library
 5:  0cf7e078ef4 =  5:  3514e0c0251 hook.c: add a hook_exists() wrapper and use it in bugreport.c
 -:  ----------- >  6:  d5ef40f77dc hook.c users: use "hook_exists()" insted of "find_hook()"
 6:  f343fc7ae66 !  7:  4cfd72722c1 hook-list.h: add a generated list of hooks, like config-list.h

Since these 6 were broken out and then brought back into this series, I'll review them individually downthread.

Quoted 8 lines

7: cf4b06bfdf8 ! 8: 7cb4a4cb69e hook: add 'run' subcommand

    @@ builtin/hook.c (new)
     +#include "strbuf.h"
     +#include "strvec.h"
     +
    ++#define BUILTIN_HOOK_RUN_USAGE \
    ++	N_("git hook run <hook-name> [-- <hook-args>]")
    ++

Nice, now we avoid string duplication here...

Quoted 2 lines
     +static const char * const builtin_hook_usage[] = {
    -+	N_("git hook <command> [...]"),

...and there is no point including this vague thing that has a more specific description right after. Ok.

Quoted 4 lines
    -+	N_("git hook run <hook-name> [-- <hook-args>]"),
    ++	BUILTIN_HOOK_RUN_USAGE,
     +	NULL
     +};

[...]

Quoted 19 lines
    @@ builtin/hook.c (new)
     +{
     +	int i;
     +	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	int rc = 0;
     +	const char *hook_name;
     +	const char *hook_path;
    -+
     +	struct option run_options[] = {
     +		OPT_END(),
     +	};
    ++	int ret;
     +
     +	argc = parse_options(argc, argv, prefix, run_options,
     +			     builtin_hook_run_usage,
    -+			     PARSE_OPT_KEEP_UNKNOWN | PARSE_OPT_KEEP_DASHDASH);
    -+
    -+	if (argc > 1) {
    -+		if (strcmp(argv[1], "--") &&
Quoted 8 lines
    -+		    strcmp(argv[1], "--end-of-options"))
    -+			/* Having a -- for "run" is mandatory */
    -+			usage_with_options(builtin_hook_usage, run_options);
    -+		/* Add our arguments, start after -- */
    -+		for (i = 2 ; i < argc; i++)
    -+			strvec_push(&opt.args, argv[i]);
    -+	}
    ++			     PARSE_OPT_KEEP_DASHDASH);

Nice - this code is being broken up and moved later.

Quoted 10 lines
     +
    -+	/* Need to take into account core.hooksPath */
    -+	git_config(git_default_config, NULL);
    ++	if (!argc)
    ++		goto usage;
     +
     +	/*
    -+	 * We are not using run_hooks() because we'd like to detect
    -+	 * missing hooks. Let's find it ourselves and call
    -+	 * run_found_hooks() instead.

This comment disappears entirely. I am not too terribly upset about it, since this behavior changes in my series anyway, but it seems like it might have been an accident?

Quoted 43 lines
    ++	 * Having a -- for "run" when providing <hook-args> is
    ++	 * mandatory.
     +	 */
    ++	if (argc > 1 && strcmp(argv[1], "--") &&
    ++	    strcmp(argv[1], "--end-of-options"))
    ++		goto usage;
    ++
    ++	/* Add our arguments, start after -- */
    ++	for (i = 2 ; i < argc; i++)
    ++		strvec_push(&opt.args, argv[i]);
    ++
    ++	/* Need to take into account core.hooksPath */
    ++	git_config(git_default_config, NULL);
    ++
     +	hook_name = argv[0];
     +	hook_path = find_hook(hook_name);
     +	if (!hook_path) {
     +		error("cannot find a hook named %s", hook_name);
     +		return 1;
     +	}
    -+	rc = run_found_hooks(hook_name, hook_path, &opt);
     +
    ++	ret = run_hooks(hook_name, hook_path, &opt);
     +	run_hooks_opt_clear(&opt);
    -+
    -+	return rc;
    ++	return ret;
    ++usage:
    ++	usage_with_options(builtin_hook_run_usage, run_options);
     +}
     +
     +int cmd_hook(int argc, const char **argv, const char *prefix)
    @@ builtin/hook.c (new)
     +	struct option builtin_hook_options[] = {
     +		OPT_END(),
     +	};
    ++
     +	argc = parse_options(argc, argv, NULL, builtin_hook_options,
     +			     builtin_hook_usage, PARSE_OPT_STOP_AT_NON_OPTION);
     +	if (!argc)
    -+		usage_with_options(builtin_hook_usage, builtin_hook_options);
    ++		goto usage;
     +
Quoted 8 lines
     +	if (!strcmp(argv[0], "run"))
     +		return run(argc, argv, prefix);
    -+	else
    -+		usage_with_options(builtin_hook_usage, builtin_hook_options);
    ++
    ++usage:
    ++	usage_with_options(builtin_hook_usage, builtin_hook_options);
     +}

The goto pattern seems readable enough elsewhere, though.

Quoted 8 lines
      ## command-list.txt ##
    @@ hook.c: int hook_exists(const char *name)
     +	struct hook_cb_data *hook_cb = pp_cb;
     +	struct hook *run_me = hook_cb->run_me;
     +
    ++	if (!run_me)
    ++		return 0;
    ++

Now we protect ourselves from the repeated calls to pick_next_hook(). Good.

Quoted 6 lines
     +	cp->no_stdin = 1;
     +	cp->env = hook_cb->options->env.v;
     +	cp->stdout_to_stderr = 1;
    @@ hook.c: int hook_exists(const char *name)
     +	/* Provide context for errors if necessary */
     +	*pp_task_cb = run_me;
Quoted 8 lines
     +
    ++	/*
    ++	 * This pick_next_hook() will be called again, we're only
    ++	 * running one hook, so indicate that no more work will be
    ++	 * done.
    ++	 */
    ++	hook_cb->run_me = NULL;
    ++

And clearly explaining the caller's behavior here. Sure.

Quoted 9 lines
     +	return 1;
     +}
     +
    @@ hook.c: int hook_exists(const char *name)
     +
     +	hook_cb->rc |= result;
     +
    -+	return 1;
    ++	return 0;

And finally, report "everything is fine" on task finished. Thanks.

Quoted 6 lines

+}
+
-+int run_found_hooks(const char *hook_name, const char *hook_path,
-+ struct run_hooks_opt *options)
++int run_hooks(const char *hook_name, const char *hook_path,
++ struct run_hooks_opt *options)

Ok. With the rename, we only have "verbose way to call it" and "speedy way to call it". No more "run_found_hooks()". Thanks, I think I like this better - I found "run_found_hooks()" to be a little ambiguous.

Quoted 16 lines
    @@ hook.c: int hook_exists(const char *name)
     +				   hook_name);
     +
     +	return cb_data.rc;
    -+}
    -+
    -+int run_hooks(const char *hook_name, struct run_hooks_opt *options)
    -+{
    -+	const char *hook_path;
    -+	int ret;
    -+	if (!options)
    -+		BUG("a struct run_hooks_opt must be provided to run_hooks");
    -+
    -+	hook_path = find_hook(hook_name);
    -+
    -+	/*
Quoted 8 lines
    -+	 * If you need to act on a missing hook, use run_found_hooks()
    -+	 * instead
    -+	 */
    -+	if (!hook_path)
    -+		return 0;
    -+
    -+	ret = run_found_hooks(hook_name, hook_path, options);
    -+	return ret;

Ok - so we wait for another patch to introduce run_hooks_oneshot. Fine.

Quoted 7 lines
     +}
     
      ## hook.h ##
    @@ hook.h: const char *find_hook(const char *name);
     +
     +	/* Args to be passed to each hook */
     +	struct strvec args;
Quoted 8 lines
    -+
    -+	/*
    -+	 * Number of threads to parallelize across, currently a stub,
    -+	 * we use the parallel API for future-proofing, but we always
    -+	 * have one hook of a given name, so this is always an
    -+	 * implicit 1 for now.
    -+	 */
    -+	int jobs;

Thanks. I think this makes more sense than the weird "futureproofing" state it was in before.

Quoted 4 lines
      ## t/t1800-hook.sh (new) ##
    @@ t/t1800-hook.sh (new)
     +
     +test_expect_success 'git hook usage' '
Quoted 8 lines
     +	test_expect_code 129 git hook &&
    -+	test_expect_code 129 git hook -h &&
    -+	test_expect_code 129 git hook run -h
    ++	test_expect_code 129 git hook run &&
    ++	test_expect_code 129 git hook run -h &&
    ++	test_expect_code 129 git hook run --unknown 2>err &&
    ++	grep "unknown option" err
     +'

Nice. I was able to use this test myself when I added 'git hook list', too.

The changes to this patch look good to me.

Quoted 5 lines

8: 7209f73f281 ! 9: 2b8500aa675 gc: use hook library for pre-auto-gc hook
@@ Metadata
## Commit message ##
gc: use hook library for pre-auto-gc hook

Quoted 10 lines

- Using the hook.h library instead of the run-command.h library to run
- pre-auto-gc means that those hooks can be set up in config files, as
- well as in the hookdir. pre-auto-gc is called only from builtin/gc.c.
+ Move the pre-auto-gc hook away from run-command.h to and over to the
+ new hook.h library.
+
+ To do this introduce a simple run_hooks_oneshot() wrapper, we'll be
+ using it extensively for these simple cases of wanting to run a single
+ hook under a given name, and having it free the memory we allocate for
+ us.

Cool, so we have an excuse for introducing the oneshot wrapper right away. Thanks.

Quoted 14 lines
    -@@ builtin/gc.c: static void add_repack_incremental_option(void)
    - 
    - static int need_to_gc(void)
    - {
    -+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
    -+
    - 	/*
    - 	 * Setting gc.auto to 0 or negative can disable the
    - 	 * automatic gc.
     @@ builtin/gc.c: static int need_to_gc(void)
      	else
      		return 0;
      
     -	if (run_hook_le(NULL, "pre-auto-gc", NULL))
Quoted 8 lines
    -+	if (run_hooks("pre-auto-gc", &hook_opt)) {
    -+		run_hooks_opt_clear(&hook_opt);
    ++	if (run_hooks_oneshot("pre-auto-gc", NULL))
      		return 0;
    -+	}
    -+	run_hooks_opt_clear(&hook_opt);
      	return 1;
      }

Nice - the callsite looks much tidier now.

Quoted 7 lines
    +
    + ## hook.c ##
    +@@ hook.c: int run_hooks(const char *hook_name, const char *hook_path,
      
    + 	return cb_data.rc;
    + }
    ++
Quoted 8 lines
    ++int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
    ++{
    ++	const char *hook_path;
    ++	int ret;
    ++	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT;
    ++
    ++	if (!options)
    ++		options = &hook_opt_scratch;

Ah, it still takes an options (optionally, ha ha) because run_hooks() would require the caller to provide the hook list/hook path. Ok. Seems fine, and I like that the '_oneshot' naming change makes it less surprising that run_hooks_opt_clear() will get called for you.

Quoted 23 lines
    ++
    ++	hook_path = find_hook(hook_name);
    ++	if (!hook_path) {
    ++		ret = 0;
    ++		goto cleanup;
    ++	}
    ++
    ++	ret = run_hooks(hook_name, hook_path, options);
    ++cleanup:
    ++	run_hooks_opt_clear(options);
    ++	return ret;
    ++}
    +
    + ## hook.h ##
    +@@ hook.h: void run_hooks_opt_clear(struct run_hooks_opt *o);
    + /**
    +  * Takes an already resolved hook found via find_hook() and runs
    +  * it. Does not call run_hooks_opt_clear() for you.
    ++ *
    ++ * See run_hooks_oneshot() for the simpler one-shot API.
    +  */
    + int run_hooks(const char *hookname, const char *hook_path,
    + 	      struct run_hooks_opt *options);
Quoted 8 lines
    ++
    ++/**
    ++ * Calls find_hook() on your "hook_name" and runs the hooks (if any)
    ++ * with run_hooks().
    ++ *
    ++ * If "options" is provided calls run_hooks_opt_clear() on it for
    ++ * you. If "options" is NULL a scratch one will be provided for you
    ++ * before calling run_hooks().

"A scratch one will be provided for you" doesn't sound quite right - it's not exposed to the caller at all, but the comment sounds like you're handing the caller this scratch struct. Maybe it's better to say "the default options from RUN_HOOKS_OPT_INIT will be used"?

As a bonus, if you directly reference the initter macro, then it will automatically become clear to folks reading the documentation what the expected parallelism is on their hook, as the parallelism (_SERIES or _PARALLEL) is included on the name of the macro in my change later.

Quoted 13 lines
    ++ */
    ++int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options);
    ++
    + #endif

9: e9a1e7cf61e ! 10: 3ee55d2c10f rebase: teach pre-rebase to use hook.h
@@ Metadata
Author: Emily Shaffer <[email protected]>

## Commit message ##
- rebase: teach pre-rebase to use hook.h
+ rebase: convert pre-rebase to use hook.h

Quoted 9 lines

Move the pre-rebase hook away from run-command.h to and over to the new hook.h library.
@@ builtin/rebase.c: int cmd_rebase(int argc, const char **argv, const char *prefix
if (!ok_to_skip_pre_rebase &&
- run_hook_le(NULL, "pre-rebase", options.upstream_arg,
- argc ? argv[0] : NULL, NULL))
-+ run_hooks("pre-rebase", &hook_opt)) {
-+ run_hooks_opt_clear(&hook_opt);
++ run_hooks_oneshot("pre-rebase", &hook_opt))

Ok, it just uses the oneshot call instead. Looks good.

Quoted 3 lines
10:  1d087269303 ! 11:  050f20d14f0 am: convert applypatch hooks to use config
11:  32eec5dc2f0 ! 12:  ac875d284da hooks: convert 'post-checkout' hook to hook library
12:  e9fa3f67593 ! 13:  69763bc2255 merge: use config-based hooks for post-merge hook

The above all just use the oneshot instead, and have some minor tweaks to commit messages to lose references to the config. Looks good.

Quoted 11 lines

13: 12347d901bb ! 14: 2ca1ca1b8e4 git hook run: add an --ignore-missing flag

    @@ Documentation/git-hook.txt: optional `--` (or `--end-of-options`, see linkgit:gi
      linkgit:githooks[5]
     
      ## builtin/hook.c ##
    +@@
    + #include "strvec.h"
    + 
    + #define BUILTIN_HOOK_RUN_USAGE \
    +-	N_("git hook run <hook-name> [-- <hook-args>]")
    ++	N_("git hook run [--ignore-missing] <hook-name> [-- <hook-args>]")

Ok. We update the usage in the macro here instead. Nice.

Quoted 12 lines
     @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
    - 	/*
    - 	 * We are not using run_hooks() because we'd like to detect
    - 	 * missing hooks. Let's find it ourselves and call
    --	 * run_found_hooks() instead.
    -+	 * run_found_hooks() instead...
    - 	 */
    + 	git_config(git_default_config, NULL);
    + 
      	hook_name = argv[0];
    ++	if (ignore_missing)
    ++		return run_hooks_oneshot(hook_name, &opt);
Quoted 8 lines
      	hook_path = find_hook(hook_name);
      	if (!hook_path) {
    -+		/* ... act like run_hooks() under --ignore-missing */
    -+		if (ignore_missing)
    -+			return 0;
      		error("cannot find a hook named %s", hook_name);
    - 		return 1;
    - 	}

Hm. I actually think this was clearer before. I *think* that is because you and I have different opinions on the clarity of multiple returns :)

But I think it is a little confusing to say "ok, we will call an entirely different entry point in case of this one thing which we will have to check anyways".

Quoted 11 lines

14: 71d209b4077 ! 15: 5b66b04bec7 send-email: use 'git hook run' for 'sendemail-validate'

    @@ git-send-email.perl: sub validate_patch {
      	if ($repo) {
     +		my $hook_name = 'sendemail-validate';
      		my $hooks_path = $repo->command_oneline('rev-parse', '--git-path', 'hooks');
    --		my $validate_hook = catfile($hooks_path,
    + 		require File::Spec;
    +-		my $validate_hook = File::Spec->catfile($hooks_path,
     -					    'sendemail-validate');
    -+		my $validate_hook = catfile($hooks_path, $hook_name);
    ++		my $validate_hook = File::Spec->catfile($hooks_path, $hook_name);

Ok, we are adapting to some other change around File::Spec that happened to git-send-email.perl elsewhere. The change to this patch looks good to me.

Quoted 16 lines
      		my $hook_error;
      		if (-x $validate_hook) {
    - 			my $target = abs_path($fn);
    + 			require Cwd;
     @@ git-send-email.perl: sub validate_patch {
      			chdir($repo->wc_path() or $repo->repo_path())
      				or die("chdir: $!");
15:  246a82b55b2 = 16:  14a37a43db2 git-p4: use 'git hook' to run hooks
16:  e3f8482d803 ! 17:  ad5d0e0e7de commit: use hook.h to execute hooks

@@ Metadata
Author: Emily Shaffer <[email protected]>

## Commit message ##
- commit: use hook.h to execute hooks
+ commit: convert {pre-commit,prepare-commit-msg} hook to hook.h

Quoted 12 lines

- Teach run_commit_hook() to call hook.h instead of run-command.h. This
- covers 'pre-commit', 'commit-msg', and
- 'prepare-commit-msg'.
-
- Additionally, ask the hook library - not run-command - whether any
- hooks will be run, as it's possible hooks may exist in the config but
- not the hookdir.
-
- Because all but 'post-commit' hooks are expected to make some state
- change, force all but 'post-commit' hook to run in series. 'post-commit'
- "is meant primarily for notification, and cannot affect the outcome of
- git commit," so it is fine to run in parallel.

I am a little bummed that the patch reordering effectively removes the justification for parallel/serial execution for each hook from 'git blame'. Oh well.

Quoted 7 lines

+ Move these hooks hook away from run-command.h to and over to the new
+ hook.h library.

## commit.c ##

Otherwise the main change here is dealing with other improvements earlier in the series, like 'hook_exists()' and 'run_hooks_oneshot()'. Looks fine.

Quoted 8 lines

17: 6ed61071c5e ! 18: 3d3a33e2674 read-cache: convert post-index-change hook to use config
@@ Metadata
Author: Emily Shaffer <[email protected]>

## Commit message ##
- read-cache: convert post-index-change hook to use config
+ read-cache: convert post-index-change to use hook.h

Quoted 5 lines

- By using hook.h instead of run-command.h to run, post-index-change hooks
- can now be specified in the config in addition to the hookdir.
- post-index-change is not run anywhere besides in read-cache.c.
+ Move the post-index-change hook away from run-command.h to and over to
+ the new hook.h library.

I do not think it is necessary to drop the mention about 'post-index-change' being run anywhere else, but that's just a nit, not worth fixing :)

Quoted 11 lines
     
         This removes the last direct user of run_hook_ve(), so we can make the
         function static now. It'll be removed entirely soon.
    @@ read-cache.c: static int do_write_locked_index(struct index_state *istate, struc
     +		     istate->updated_workdir ? "1" : "0",
     +		     istate->updated_skipworktree ? "1" : "0",
     +		     NULL);
    -+	run_hooks("post-index-change", &hook_opt);
    -+	run_hooks_opt_clear(&hook_opt);
    ++	run_hooks_oneshot("post-index-change", &hook_opt);
     +

Mechanical change to using run_hooks_oneshot(). OK.

Quoted 7 lines
      	istate->updated_workdir = 0;
      	istate->updated_skipworktree = 0;

18: e4ef3f4548a ! 19: 893f8666301 receive-pack: convert push-to-checkout hook to hook.h
@@ Metadata
## Commit message ##
receive-pack: convert push-to-checkout hook to hook.h

Quoted 5 lines

- By using hook.h instead of run-command.h to invoke push-to-checkout,
- hooks can now be specified in the config as well as in the hookdir.
- push-to-checkout is not called anywhere but in builtin/receive-pack.c.
+ Move the push-to-checkout hook away from run-command.h to and over to
+ the new hook.h library.

You made a comment in the commit before about the final instance of run_hook_ve; do you want to make a similar one here about run_hook_le?

The range-diff here is mechanical so looks fine to me.

Quoted 9 lines
19:  e3dda367ec9 = 20:  070433deba5 run-command: remove old run_hook_{le,ve}() hook API
20:  477d75bf579 = 21:  1028e0c1667 run-command: allow stdin for run_processes_parallel
21:  b7c0ee9719a ! 22:  639e59e9ed0 hook: support passing stdin to hooks

@@ Documentation/git-hook.txt: what those are.

## builtin/hook.c ##
@@
+ #include "strvec.h"

Quoted 16 lines

- static const char * const builtin_hook_usage[] = {
- N_("git hook <command> [...]"),
-- N_("git hook run <hook-name> [-- <hook-args>]"),
-+ N_("git hook run [<args>] <hook-name> [-- <hook-args>]"),
- NULL
- };
-
- static const char * const builtin_hook_run_usage[] = {
- N_("git hook run <hook-name> [-- <hook-args>]"),
-+ N_("git hook run [--to-stdin=<path>] <hook-name> [-- <hook-args>]"),
- NULL
- };
+ #define BUILTIN_HOOK_RUN_USAGE \
+- N_("git hook run [--ignore-missing] <hook-name> [-- <hook-args>]")
++ N_("git hook run [--ignore-missing] [--to-stdin=<path>] <hook-name> [-- <hook-args>]")

Just coping with the macro instead of the duplicated strings. Ok.

The range-diff looks fine to me.

Quoted 26 lines

22: 4035069a98c ! 23: 7d1925cca48 am: convert 'post-rewrite' hook to hook.h

    @@ builtin/am.c: static int run_applypatch_msg_hook(struct am_state *state)
      {
     -	struct child_process cp = CHILD_PROCESS_INIT;
     -	const char *hook = find_hook("post-rewrite");
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    - 	int ret;
    - 
    +-	int ret;
    +-
     -	if (!hook)
     -		return 0;
     -
    @@ builtin/am.c: static int run_applypatch_msg_hook(struct am_state *state)
     -	cp.in = xopen(am_path(state, "rewritten"), O_RDONLY);
     -	cp.stdout_to_stderr = 1;
     -	cp.trace2_hook_name = "post-rewrite";
    -+	strvec_push(&opt.args, "rebase");
    -+	opt.path_to_stdin = am_path(state, "rewritten");
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
      
     -	ret = run_command(&cp);
    -+	ret = run_hooks("post-rewrite", &opt);
    ++	strvec_push(&opt.args, "rebase");
    ++	opt.path_to_stdin = am_path(state, "rewritten");
Quoted 8 lines
     -	close(cp.in);
    -+	run_hooks_opt_clear(&opt);
    - 	return ret;
    +-	return ret;
    ++	return run_hooks_oneshot("post-rewrite", &opt);
      }
      
    + /**

Man, the range-diff did a terrible job of summarizing this :)

The patch itself looks great, though, with the switch to using run_hooks_oneshot. Thanks.

Quoted 14 lines

24: da46c859c1c ! 25: 05d1085f7eb hook: provide stdin by string_list or callback
@@ hook.c: static int pick_next_hook(struct child_process *cp,
} else {
cp->no_stdin = 1;
}
-@@ hook.c: int run_found_hooks(const char *hook_name, const char *hook_path,
- run_processes_parallel_tr2(options->jobs,
+@@ hook.c: int run_hooks(const char *hook_name, const char *hook_path,
+ run_processes_parallel_tr2(jobs,
pick_next_hook, notify_start_failure,
- NULL,
@@ hook.c: int run_found_hooks(const char *hook_name, const char *hook_path,
notify_hook_finished,

Quoted 8 lines

&cb_data,
"hook",
-@@ hook.c: int run_hooks(const char *hook_name, struct run_hooks_opt *options) +@@ hook.c: int run_hooks(const char *hook_name, const char *hook_path, +
+ if (options->absolute_path)
+ strbuf_release(&abs_path);
++ free(my_hook.feed_pipe_cb_data);

Nice catch. Thanks.

Quoted 33 lines
25:  7343be28ef4 ! 26:  4b7175af2e5 hook: convert 'post-rewrite' hook in sequencer.c to hook.h
26:  85bf13a0835 ! 27:  3f24e056410 transport: convert pre-push hook to use config
    @@ Metadata
     Author: Emily Shaffer <[email protected]>
     
      ## Commit message ##
    -    transport: convert pre-push hook to use config
    +    transport: convert pre-push hook to hook.h
     
    -    By using the hook.h:run_hooks API, pre-push hooks can be specified in
    -    the config as well as in the hookdir.
    +    Move the pre-push hook away from run-command.h to and over to the new
    +    hook.h library.
     
         Signed-off-by: Emily Shaffer <[email protected]>
         Signed-off-by: Ævar Arnfjörð Bjarmason <[email protected]>
    @@ transport.c: static void die_with_unpushed_submodules(struct string_list *needs_
     -	int ret = 0, x;
     +	int ret = 0;
     +	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct strbuf tmp = STRBUF_INIT;
      	struct ref *r;
     -	struct child_process proc = CHILD_PROCESS_INIT;
     -	struct strbuf buf;
    @@ transport.c: static void die_with_unpushed_submodules(struct string_list *needs_
     -		finish_command(&proc);
     -		return -1;
     -	}
    --
    --	sigchain_push(SIGPIPE, SIG_IGN);
    -+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
    ++	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
Quoted 8 lines
    +-	sigchain_push(SIGPIPE, SIG_IGN);
    +-
     -	strbuf_init(&buf, 256);
     +	strvec_push(&opt.args, transport->remote->name);
     +	strvec_push(&opt.args, transport->url);
      
      	for (r = remote_refs; r; r = r->next) {
    ++		struct strbuf buf = STRBUF_INIT;

Ah. It is not being freed because its lifetime is being managed by to_stdin instead. Ok, cool!

Quoted 26 lines
    ++
      		if (!r->peer_ref) continue;
    -@@ transport.c: static int run_pre_push_hook(struct transport *transport,
    + 		if (r->status == REF_STATUS_REJECT_NONFASTFORWARD) continue;
    + 		if (r->status == REF_STATUS_REJECT_STALE) continue;
      		if (r->status == REF_STATUS_REJECT_REMOTE_UPDATED) continue;
      		if (r->status == REF_STATUS_UPTODATE) continue;
      
     -		strbuf_reset(&buf);
     -		strbuf_addf( &buf, "%s %s %s %s\n",
    -+		strbuf_reset(&tmp);
    -+		strbuf_addf(&tmp, "%s %s %s %s",
    ++		strbuf_addf(&buf, "%s %s %s %s",
      			 r->peer_ref->name, oid_to_hex(&r->new_oid),
      			 r->name, oid_to_hex(&r->old_oid));
     -
    @@ transport.c: static int run_pre_push_hook(struct transport *transport,
     -				ret = -1;
     -			break;
     -		}
    -+		string_list_append(&to_stdin, tmp.buf);
    ++		string_list_append(&to_stdin, strbuf_detach(&buf, NULL));
      	}
      
     -	strbuf_release(&buf);
    @@ transport.c: static int run_pre_push_hook(struct transport *transport,
Quoted 8 lines
     -	x = finish_command(&proc);
     -	if (!ret)
     -		ret = x;
    -+	ret = run_hooks("pre-push", &opt);
    -+	run_hooks_opt_clear(&opt);
    -+	strbuf_release(&tmp);
    ++	ret = run_hooks_oneshot("pre-push", &opt);
    ++	to_stdin.strdup_strings = 1;

And this is a typical change to the oneshot function. Cool.

Quoted 5 lines
     +	string_list_clear(&to_stdin, 0);
      
      	return ret;
 -:  ----------- > 28:  ecf75f33233 hook tests: test for exact "pre-push" hook input
 -:  ----------- > 29:  2c961be94b4 hook tests: use a modern style for "pre-push" tests

I'll take a look at these separately.

Quoted 33 lines

27: 331014bad17 ! 30: 1ce456f9d9d reference-transaction: use hook.h to run hooks

    @@ refs.c: int ref_update_reject_duplicates(struct string_list *refnames,
      				const char *state)
      {
     -	struct child_process proc = CHILD_PROCESS_INIT;
    - 	struct strbuf buf = STRBUF_INIT;
    +-	struct strbuf buf = STRBUF_INIT;
     -	const char *hook;
     +	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
    ++	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
      	int ret = 0, i;
    -+	char o[GIT_MAX_HEXSZ + 1], n[GIT_MAX_HEXSZ + 1];
      
     -	hook = find_hook("reference-transaction");
     -	if (!hook)
    -+	if (!hook_exists("reference-transaction"))
    - 		return ret;
    - 
    +-		return ret;
    +-
     -	strvec_pushl(&proc.args, hook, state, NULL);
     -	proc.in = -1;
     -	proc.stdout_to_stderr = 1;
    @@ refs.c: int ref_update_reject_duplicates(struct string_list *refnames,
     -
     -	ret = start_command(&proc);
     -	if (ret)
    --		return ret;
    --
    ++	if (!hook_exists("reference-transaction"))
    + 		return ret;
    + 
Quoted 8 lines
     -	sigchain_push(SIGPIPE, SIG_IGN);
     +	strvec_push(&opt.args, state);
      
      	for (i = 0; i < transaction->nr; i++) {
      		struct ref_update *update = transaction->updates[i];
    -+		oid_to_hex_r(o, &update->old_oid);
    -+		oid_to_hex_r(n, &update->new_oid);
    ++		struct strbuf buf = STRBUF_INIT;

Ah, this is doing the same thing as with 'pre-push'. Cool.

Quoted 25 lines
      
    - 		strbuf_reset(&buf);
    +-		strbuf_reset(&buf);
     -		strbuf_addf(&buf, "%s %s %s\n",
    --			    oid_to_hex(&update->old_oid),
    --			    oid_to_hex(&update->new_oid),
    --			    update->refname);
    ++		strbuf_addf(&buf, "%s %s %s",
    + 			    oid_to_hex(&update->old_oid),
    + 			    oid_to_hex(&update->new_oid),
    + 			    update->refname);
     -
     -		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
     -			if (errno != EPIPE)
     -				ret = -1;
     -			break;
     -		}
    -+		strbuf_addf(&buf, "%s %s %s", o, n, update->refname);
    -+		string_list_append(&to_stdin, buf.buf);
    ++		string_list_append(&to_stdin, strbuf_detach(&buf, NULL));
      	}
      
     -	close(proc.in);
     -	sigchain_pop(SIGPIPE);
    +-	strbuf_release(&buf);
Quoted 8 lines
     +	opt.feed_pipe = pipe_from_string_list;
     +	opt.feed_pipe_ctx = &to_stdin;
     +
    -+	ret = run_hooks("reference-transaction", &opt);
    -+	run_hooks_opt_clear(&opt);
    - 	strbuf_release(&buf);
    ++	ret = run_hooks_oneshot("reference-transaction", &opt);
    ++	to_stdin.strdup_strings = 1;

And turning on strdup_strings here means we will free them when we call string_list_clear. Ok.

Quoted 12 lines
     +	string_list_clear(&to_stdin, 0);
      
     -	ret |= finish_command(&proc);
32:  db70b59b3bd ! 35:  ceef2f3e804 receive-pack: convert receive hooks to hook.h
    @@ builtin/receive-pack.c: static void hook_output_to_sideband(struct strbuf *outpu
     +{
     +	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
     +	struct receive_hook_feed_context ctx;
    -+	int rc;
     +	struct command *iter = commands;
     +
     +	/* if there are no valid commands, don't invoke the hook at all. */
Quoted 8 lines
    @@ builtin/receive-pack.c: static void hook_output_to_sideband(struct strbuf *outpu
     +	if (!iter)
     +		return 0;
     +
    -+	/* pre-receive hooks should run in series as the hook updates refs */
    -+	if (!strcmp(hook_name, "pre-receive"))
    -+		opt.jobs = 1;
    -+

Hm, interesting. I'll note this as a case to watch for when I add the .jobs option back in.

Thanks. I'll take a look at the newly added/copied over patches next.

- Emily