Maintenance I: Command, gc and commit-graph tasks

From: Derrick Stolee <[email protected]>

A user may want to only run certain maintenance tasks in a certain order. Add the --task=<task> option, which allows a user to specify an ordered list of tasks to run. These cannot be run multiple times, however.

Here is where our array of maintenance_task pointers becomes critical. We can sort the array of pointers based on the task order, but we do not want to move the struct data itself in order to preserve the hashmap references. We use the hashmap to match the --task=<task> arguments into the task struct data.

Keep in mind that the 'enabled' member of the maintenance_task struct is a placeholder for a future 'maintenance.<task>.enabled' config option. Thus, we use the 'enabled' member to specify which tasks are run when the user does not specify any --task=<task> arguments. The 'enabled' member should be ignored if --task=<task> appears.

---
 Documentation/git-maintenance.txt |  8 +++-
 builtin/gc.c                      | 61 +++++++++++++++++++++++++++++--
 t/t7900-maintenance.sh            | 27 ++++++++++++++
 3 files changed, 92 insertions(+), 4 deletions(-)

index 35b0be7d40..5911213c9c 100644
@@ -30,7 +30,9 @@ SUBCOMMANDS
 -----------
 
 run::
-	Run one or more maintenance tasks.
+	Run one or more maintenance tasks. If one or more `--task=<task>`
+	options are specified, then those tasks are run in the provided
+	order. Otherwise, only the `gc` task is run.
 
 TASKS
 -----

@@ -73,6 +75,10 @@ OPTIONS
 --quiet::
 	Do not report progress or other information over `stderr`.
 
+--task=<task>::
+	If this option is specified one or more times, then only run the
+	specified tasks in the specified order.
+
 GIT
 ---
 Part of the linkgit:git[1] suite

index 384294d111..bcc7131818 100755
@@ -23,4 +23,31 @@ test_expect_success 'run [--auto|--quiet]' '
 	test_subcommand git gc --no-quiet <run-no-quiet.txt
 '
 
+test_expect_success 'run --task=<task>' '
+	GIT_TRACE2_EVENT="$(pwd)/run-commit-graph.txt" \
+		git maintenance run --task=commit-graph 2>/dev/null &&
+	GIT_TRACE2_EVENT="$(pwd)/run-gc.txt" \
+		git maintenance run --task=gc 2>/dev/null &&
+	GIT_TRACE2_EVENT="$(pwd)/run-commit-graph.txt" \
+		git maintenance run --task=commit-graph 2>/dev/null &&
+	GIT_TRACE2_EVENT="$(pwd)/run-both.txt" \
+		git maintenance run --task=commit-graph --task=gc 2>/dev/null &&
+	test_subcommand ! git gc --quiet <run-commit-graph.txt &&
+	test_subcommand git gc --quiet <run-gc.txt &&
+	test_subcommand git gc --quiet <run-both.txt &&
+	test_subcommand git commit-graph write --split --reachable --no-progress <run-commit-graph.txt &&
+	test_subcommand ! git commit-graph write --split --reachable --no-progress <run-gc.txt &&
+	test_subcommand git commit-graph write --split --reachable --no-progress <run-both.txt
+'
+
+test_expect_success 'run --task=bogus' '
+	test_must_fail git maintenance run --task=bogus 2>err &&
+	test_i18ngrep "is not a valid task" err
+'
+
+test_expect_success 'run --task duplicate' '
+	test_must_fail git maintenance run --task=gc --task=gc 2>err &&
+	test_i18ngrep "cannot be selected multiple times" err
+'
+
 test_done

gitgitgadget

From: Derrick Stolee <[email protected]>

Currently, a normal run of "git maintenance run" will only run the 'gc' task, as it is the only one enabled. This is mostly for backwards- compatible reasons since "git maintenance run --auto" commands replaced previous "git gc --auto" commands after some Git processes. Users could manually run specific maintenance tasks by calling "git maintenance run --task=<task>" directly.

Allow users to customize which steps are run automatically using config. The 'maintenance.<task>.enabled' option then can turn on these other tasks (or turn off the 'gc' task).

---
 Documentation/config.txt             |  2 ++
 Documentation/config/maintenance.txt |  4 ++++
 Documentation/git-maintenance.txt    |  8 +++++---
 builtin/gc.c                         | 19 +++++++++++++++++++
 t/t7900-maintenance.sh               |  8 ++++++++
 5 files changed, 38 insertions(+), 3 deletions(-)
 create mode 100644 Documentation/config/maintenance.txt

index ef0768b91a..2783b825f9 100644
@@ -396,6 +396,8 @@ include::config/mailinfo.txt[]
 
 include::config/mailmap.txt[]
 
+include::config/maintenance.txt[]
+
 include::config/man.txt[]
 
 include::config/merge.txt[]

new file mode 100644
index 0000000000..370cbfb42f
@@ -0,0 +1,4 @@
+maintenance.<task>.enabled::
+	This boolean config option controls whether the maintenance task
+	with name `<task>` is run when no `--task` option is specified.
+	By default, only `maintenance.gc.enabled` is true.

index 5911213c9c..e1a2a8902c 100644
@@ -30,9 +30,11 @@ SUBCOMMANDS
 -----------
 
 run::
-	Run one or more maintenance tasks. If one or more `--task=<task>`
-	options are specified, then those tasks are run in the provided
-	order. Otherwise, only the `gc` task is run.
+	Run one or more maintenance tasks. If one or more `--task` options
+	are specified, then those tasks are run in that order. Otherwise,
+	the tasks are determined by which `maintenance.<task>.enabled`
+	config options are true. By default, only `maintenance.gc.enabled`
+	is true.
 
 TASKS
 -----

index 1b7d502b87..dc1d260858 100644
@@ -873,6 +873,24 @@ static int maintenance_run(struct maintenance_opts *opts)
 	return result;
 }
 
+static void initialize_task_config(void)
+{
+	int i;
+	struct strbuf config_name = STRBUF_INIT;
+	for (i = 0; i < TASK__COUNT; i++) {
+		int config_value;
+
+		strbuf_setlen(&config_name, 0);
+		strbuf_addf(&config_name, "maintenance.%s.enabled",
+			    tasks[i].name);
+
+		if (!git_config_get_bool(config_name.buf, &config_value))
+			tasks[i].enabled = config_value;
+	}
+
+	strbuf_release(&config_name);
+}
+
 static int task_option_parse(const struct option *opt,
 			     const char *arg, int unset)
 {

@@ -925,6 +943,7 @@ int cmd_maintenance(int argc, const char **argv, const char *prefix)
 				   builtin_maintenance_options);
 
 	opts.quiet = !isatty(2);
+	initialize_task_config();
 
 	argc = parse_options(argc, argv, prefix,
 			     builtin_maintenance_options,

index bcc7131818..cdf0bf2e60 100755
@@ -23,6 +23,14 @@ test_expect_success 'run [--auto|--quiet]' '
 	test_subcommand git gc --no-quiet <run-no-quiet.txt
 '
 
+test_expect_success 'maintenance.<task>.enabled' '
+	git config maintenance.gc.enabled false &&
+	git config maintenance.commit-graph.enabled true &&
+	GIT_TRACE2_EVENT="$(pwd)/run-config.txt" git maintenance run 2>err &&
+	test_subcommand ! git gc --quiet <run-config.txt &&
+	test_subcommand git commit-graph write --split --reachable --no-progress <run-config.txt
+'
+
 test_expect_success 'run --task=<task>' '
 	GIT_TRACE2_EVENT="$(pwd)/run-commit-graph.txt" \
 		git maintenance run --task=commit-graph 2>/dev/null &&

gitgitgadget

From: Derrick Stolee <[email protected]>

The first new task in the 'git maintenance' builtin is the 'commit-graph' job. It is based on the sequence of events in the 'commit-graph' job in Scalar [1]. This sequence is as follows:

1. git commit-graph write --reachable --split
2. git commit-graph verify --shallow
3. If the verify succeeds, stop.
4. Delete the commit-graph-chain file.
5. git commit-graph write --reachable --split

By writing an incremental commit-graph file using the "--split" option we minimize the disruption from this operation. The default behavior is to merge layers until the new "top" layer is less than half the size of the layer below. This provides quick writes most of the time, with the longer writes following a power law distribution.

Most importantly, concurrent Git processes only look at the commit-graph-chain file for a very short amount of time, so they will verly likely not be holding a handle to the file when we try to replace it. (This only matters on Windows.)

If a concurrent process reads the old commit-graph-chain file, but our job expires some of the .graph files before they can be read, then those processes will see a warning message (but not fail). This could be avoided by a future update to use the --expire-time argument when writing the commit-graph.

By using 'git commit-graph verify --shallow' we can ensure that the file we just wrote is valid. This is an extra safety precaution that is faster than our 'write' subcommand. In the rare situation that the newest layer of the commit-graph is corrupt, we can "fix" the corruption by deleting the commit-graph-chain file and rewrite the full commit-graph as a new one-layer commit graph. This does not completely prevent _that_ file from being corrupt, but it does recompute the commit-graph by parsing commits from the object database. In our use of this step in Scalar and VFS for Git, we have only seen this issue arise because our microsoft/git fork reverted 43d3561 ("commit-graph write: don't die if the existing graph is corrupt" 2019-03-25) for a while to keep commit-graph writes very fast. We dropped the revert when updating to v2.23.0. The verify still has potential for catching corrupt data across the layer boundary: if the new file has commit X with parent Y in an old file but the commit ID for Y in the old file had a bitswap, then we will notice that in the 'verify' command.

[1] https://github.com/microsoft/scalar/blob/master/Scalar.Common/Maintenance/CommitGraphStep.cs

---
 Documentation/git-maintenance.txt | 18 +++++++++
 builtin/gc.c                      | 63 +++++++++++++++++++++++++++++++
 commit-graph.c                    |  8 ++--
 commit-graph.h                    |  1 +
 t/t7900-maintenance.sh            |  2 +
 5 files changed, 88 insertions(+), 4 deletions(-)

index 089fa4cedc..35b0be7d40 100644
@@ -35,6 +35,24 @@ run::
 TASKS
 -----
 
+commit-graph::
+	The `commit-graph` job updates the `commit-graph` files incrementally,
+	then verifies that the written data is correct. If the new layer has an
+	issue, then the chain file is removed and the `commit-graph` is
+	rewritten from scratch.
++
+The verification only checks the top layer of the `commit-graph` chain.
+If the incremental write merged the new commits with at least one
+existing layer, then there is potential for on-disk corruption being
+carried forward into the new file. This will be noticed and the new
+commit-graph file will be clean as Git reparses the commit data from
+the object database.
++
+The incremental write is safe to run alongside concurrent Git processes
+since it will not expire `.graph` files that were in the previous
+`commit-graph-chain` file. They will be deleted by a later run based on
+the expiration delay.
+
 gc::
 	Cleanup unnecessary files and optimize the local repository. "GC"
 	stands for "garbage collection," but this task performs many

index 1af68c297d..9705d237e4 100644
@@ -172,7 +172,7 @@ static char *get_split_graph_filename(struct object_directory *odb,
 		       oid_hex);
 }
 
-static char *get_chain_filename(struct object_directory *odb)
+char *get_commit_graph_chain_filename(struct object_directory *odb)
 {
 	return xstrfmt("%s/info/commit-graphs/commit-graph-chain", odb->path);
 }

@@ -521,7 +521,7 @@ static struct commit_graph *load_commit_graph_chain(struct repository *r,
 	struct stat st;
 	struct object_id *oids;
 	int i = 0, valid = 1, count;
-	char *chain_name = get_chain_filename(odb);
+	char *chain_name = get_commit_graph_chain_filename(odb);
 	FILE *fp;
 	int stat_res;
 

@@ -1619,7 +1619,7 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx)
 	}
 
 	if (ctx->split) {
-		char *lock_name = get_chain_filename(ctx->odb);
+		char *lock_name = get_commit_graph_chain_filename(ctx->odb);
 
 		hold_lock_file_for_update_mode(&lk, lock_name,
 					       LOCK_DIE_ON_ERROR, 0444);

@@ -1996,7 +1996,7 @@ static void expire_commit_graphs(struct write_commit_graph_context *ctx)
 	if (ctx->split_opts && ctx->split_opts->expire_time)
 		expire_time = ctx->split_opts->expire_time;
 	if (!ctx->split) {
-		char *chain_file_name = get_chain_filename(ctx->odb);
+		char *chain_file_name = get_commit_graph_chain_filename(ctx->odb);
 		unlink(chain_file_name);
 		free(chain_file_name);
 		ctx->num_commit_graphs_after = 0;

index 28f89cdf3e..3c202748c3 100644
@@ -25,6 +25,7 @@ struct commit;
 struct bloom_filter_settings;
 
 char *get_commit_graph_filename(struct object_directory *odb);
+char *get_commit_graph_chain_filename(struct object_directory *odb);
 int open_commit_graph(const char *graph_file, int *fd, struct stat *st);
 
 /*

index 7b63b4ec0c..384294d111 100755
@@ -4,6 +4,8 @@ test_description='git maintenance builtin'
 
 . ./test-lib.sh
 
+GIT_TEST_COMMIT_GRAPH=0
+
 test_expect_success 'help text' '
 	test_expect_code 129 git maintenance -h 2>err &&
 	test_i18ngrep "usage: git maintenance run" err

gitgitgadget

From: Derrick Stolee <[email protected]>

In anticipation of implementing multiple maintenance tasks inside the 'maintenance' builtin, use a list of structs to describe the work to be done.

The struct maintenance_task stores the name of the task (as given by a future command-line argument) along with a function pointer to its implementation and a boolean for whether the step is enabled.

A list these structs are initialized with the full list of implemented tasks along with a default order. For now, this list only contains the "gc" task. This task is also the only task enabled by default.

The run subcommand will return a nonzero exit code if any task fails. However, it will attempt all tasks in its loop before returning with the failure. Also each failed task will send an error message.

---
 builtin/gc.c | 38 +++++++++++++++++++++++++++++++++++++-
 1 file changed, 37 insertions(+), 1 deletion(-)

index a060ba8424..150dce4301 100644
@@ -728,9 +728,45 @@ static int maintenance_task_gc(struct maintenance_opts *opts)
 	return run_command(&child);
 }
 
+typedef int maintenance_task_fn(struct maintenance_opts *opts);
+
+struct maintenance_task {
+	const char *name;
+	maintenance_task_fn *fn;
+	unsigned enabled:1;
+};
+
+enum maintenance_task_label {
+	TASK_GC,
+
+	/* Leave as final value */
+	TASK__COUNT
+};
+
+static struct maintenance_task tasks[] = {
+	[TASK_GC] = {
+		"gc",
+		maintenance_task_gc,
+		1,
+	},
+};
+
 static int maintenance_run(struct maintenance_opts *opts)
 {
-	return maintenance_task_gc(opts);
+	int i;
+	int result = 0;
+
+	for (i = 0; i < TASK__COUNT; i++) {
+		if (!tasks[i].enabled)
+			continue;
+
+		if (tasks[i].fn(opts)) {
+			error(_("task '%s' failed"), tasks[i].name);
+			result = 1;
+		}
+	}
+
+	return result;
 }
 
 int cmd_maintenance(int argc, const char **argv, const char *prefix)

gitgitgadget

From: Derrick Stolee <[email protected]>

Instead of writing a new commit-graph in every 'git maintenance run --auto' process (when maintenance.commit-graph.enalbed is configured to be true), only write when there are "enough" commits not in a commit-graph file.

This count is controlled by the maintenance.commit-graph.auto config option.

To compute the count, use a depth-first search starting at each ref, and leaving markers using the PARENT1 flag. If this count reaches the limit, then terminate early and start the task. Otherwise, this operation will peel every ref and parse the commit it points to. If these are all in the commit-graph, then this is typically a very fast operation. Users with many refs might feel a slow-down, and hence could consider updating their limit to be very small. A negative value will force the step to run every time.

---
 Documentation/config/maintenance.txt | 10 ++++
 builtin/gc.c                         | 76 ++++++++++++++++++++++++++++
 object.h                             |  1 +
 3 files changed, 87 insertions(+)

index 370cbfb42f..9bd69b9df3 100644
@@ -2,3 +2,13 @@ maintenance.<task>.enabled::
 	This boolean config option controls whether the maintenance task
 	with name `<task>` is run when no `--task` option is specified.
 	By default, only `maintenance.gc.enabled` is true.
+
+maintenance.commit-graph.auto::
+	This integer config option controls how often the `commit-graph` task
+	should be run as part of `git maintenance run --auto`. If zero, then
+	the `commit-graph` task will not run with the `--auto` option. A
+	negative value will force the task to run every time. Otherwise, a
+	positive value implies the command should run when the number of
+	reachable commits that are not in the commit-graph file is at least
+	the value of `maintenance.commit-graph.auto`. The default value is
+	100.

index 96a2105859..30d4e0f6a0 100644
@@ -74,6 +74,7 @@ struct object_array {
  * list-objects-filter.c:                                      21
  * builtin/fsck.c:           0--3
  * builtin/index-pack.c:                                     2021
+ * builtin/maintenance.c:                           16
  * builtin/pack-objects.c:                                   20
  * builtin/reflog.c:                   10--12
  * builtin/show-branch.c:    0-------------------------------------------26

gitgitgadget

From: Derrick Stolee <[email protected]>

---
 builtin/gc.c | 2 ++
 1 file changed, 2 insertions(+)

index 20d85f6f4e..8162bca974 100644
@@ -953,10 +953,12 @@ static int maintenance_run(struct maintenance_opts *opts)
 		     !tasks[i].auto_condition()))
 			continue;
 
+		trace2_region_enter("maintenance", tasks[i].name, r);
 		if (tasks[i].fn(opts)) {
 			error(_("task '%s' failed"), tasks[i].name);
 			result = 1;
 		}
+		trace2_region_leave("maintenance", tasks[i].name, r);
 	}
 
 	rollback_lock_file(&lk);

gitgitgadget

From: Derrick Stolee <[email protected]>

Maintenance activities are commonly used as steps in larger scripts. Providing a '--quiet' option allows those scripts to be less noisy when run on a terminal window. Turn this mode on by default when stderr is not a terminal.

Pipe the option to the 'git gc' child process.

---
 Documentation/git-maintenance.txt |  3 +++
 builtin/gc.c                      |  9 +++++++++
 t/t7900-maintenance.sh            | 15 ++++++++++-----
 3 files changed, 22 insertions(+), 5 deletions(-)

index 34cd2b4417..089fa4cedc 100644
@@ -52,6 +52,9 @@ OPTIONS
 	in the `gc.auto` config setting, or when the number of pack-files
 	exceeds the `gc.autoPackLimit` config setting.
 
+--quiet::
+	Do not report progress or other information over `stderr`.
+
 GIT
 ---
 Part of the linkgit:git[1] suite

index e4f0ce1c86..a060ba8424 100644
@@ -707,6 +707,7 @@ static const char * const builtin_maintenance_usage[] = {
 
 struct maintenance_opts {
 	int auto_flag;
+	int quiet;
 };
 
 static int maintenance_task_gc(struct maintenance_opts *opts)

@@ -718,6 +719,10 @@ static int maintenance_task_gc(struct maintenance_opts *opts)
 
 	if (opts->auto_flag)
 		strvec_push(&child.args, "--auto");
+	if (opts->quiet)
+		strvec_push(&child.args, "--quiet");
+	else
+		strvec_push(&child.args, "--no-quiet");
 
 	close_object_store(the_repository->objects);
 	return run_command(&child);

@@ -734,6 +739,8 @@ int cmd_maintenance(int argc, const char **argv, const char *prefix)
 	static struct option builtin_maintenance_options[] = {
 		OPT_BOOL(0, "auto", &opts.auto_flag,
 			 N_("run tasks based on the state of the repository")),
+		OPT_BOOL(0, "quiet", &opts.quiet,
+			 N_("do not report progress or other information over stderr")),
 		OPT_END()
 	};
 

@@ -743,6 +750,8 @@ int cmd_maintenance(int argc, const char **argv, const char *prefix)
 		usage_with_options(builtin_maintenance_usage,
 				   builtin_maintenance_options);
 
+	opts.quiet = !isatty(2);
+
 	argc = parse_options(argc, argv, prefix,
 			     builtin_maintenance_options,
 			     builtin_maintenance_usage,

index c4b9b4a6fe..7b63b4ec0c 100755
@@ -9,11 +9,16 @@ test_expect_success 'help text' '
 	test_i18ngrep "usage: git maintenance run" err
 '
 
-test_expect_success 'run [--auto]' '
-	GIT_TRACE2_EVENT="$(pwd)/run-no-auto.txt" git maintenance run &&
-	GIT_TRACE2_EVENT="$(pwd)/run-auto.txt" git maintenance run --auto &&
-	test_subcommand git gc <run-no-auto.txt &&
-	test_subcommand git gc --auto <run-auto.txt
+test_expect_success 'run [--auto|--quiet]' '
+	GIT_TRACE2_EVENT="$(pwd)/run-no-auto.txt" \
+		git maintenance run 2>/dev/null &&
+	GIT_TRACE2_EVENT="$(pwd)/run-auto.txt" \
+		git maintenance run --auto 2>/dev/null &&
+	GIT_TRACE2_EVENT="$(pwd)/run-no-quiet.txt" \
+		git maintenance run --no-quiet 2>/dev/null &&
+	test_subcommand git gc --quiet <run-no-auto.txt &&
+	test_subcommand git gc --auto --quiet <run-auto.txt &&
+	test_subcommand git gc --no-quiet <run-no-quiet.txt
 '
 
 test_done

gitgitgadget

This series is based on jk/strvec.

This patch series contains 11patches that were going to be part of v4 of ds/maintenance [1], but the discussion has gotten really long. To help focus the conversation, I'm splitting out the portions that create and test the 'maintenance' builtin from the additional tasks (prefetch, loose-objects, incremental-repack) that can be brought in later.

[1]
https://lore.kernel.org/git/[email protected]/

As mentioned before, git gc already plays the role of maintaining Git repositories. It has accumulated several smaller pieces in its long history, including:

1. Repacking all reachable objects into one pack-file (and deleting unreachable objects).
2. Packing refs.
3. Expiring reflogs.
4. Clearing rerere logs.
5. Updating the commit-graph file.
6. Pruning worktrees.

While expiring reflogs, clearing rererelogs, and deleting unreachable objects are suitable under the guise of "garbage collection", packing refs and updating the commit-graph file are not as obviously fitting. Further, these operations are "all or nothing" in that they rewrite almost all repository data, which does not perform well at extremely large scales. These operations can also be disruptive to foreground Git commands when git gc --auto triggers during routine use.

This series does not intend to change what git gc does, but instead create new choices for automatic maintenance activities, of which git gc remains the only one enabled by default.

The new maintenance tasks are:

* 'commit-graph' : write and verify a single layer of an incremental commit-graph.
* 'loose-objects' : prune packed loose objects, then create a new pack from a batch of loose objects.
* 'pack-files' : expire redundant packs from the multi-pack-index, then repack using the multi-pack-index's incremental repack strategy.
* 'prefetch' : fetch from each remote, storing the refs in 'refs/prefetch/ /'.

The only included tasks are the 'gc' and 'commit-graph' tasks. The rest will follow in a follow-up series. Including the 'commit-graph' task here allows us to build and test features like config settings and the --task= command-line argument.

These tasks are all disabled by default, but can be enabled with config options or run explicitly using "git maintenance run --task=". There are additional config options to allow customizing the conditions for which the tasks run during the '--auto' option.

Because 'gc' is implemented as a maintenance task, the most dramatic change
of this series is to convert the 'git gc --auto' calls into 'git maintenance run --auto' calls at the end of some Git commands. By default, the only change is that 'git gc --auto' will be run below an additional 'git maintenance' process.

The 'git maintenance' builtin has a 'run' subcommand so it can be extended later with subcommands that manage background maintenance, such as 'start' or 'stop'. These are not the subject of this series, as it is important to focus on the maintenance activities themselves.

UPDATES since v3 of [1]
=======================

* The biggest change here is the use of "test_subcommand", based on Jonathan Nieder's approach. This requires having the exact command-line figured out, which now requires spelling out all --no- [quiet%7Cprogress] options. I also added a bunch of "2>/dev/null" checks because of the isatty(2) calls. Without that, the behavior will change depending on whether the test is run with -x/-v or without.

* The option parsing has changed to use a local struct and pass that struct to the helper methods. This is instead of having a global singleton.

Thanks, -Stolee

Here is the range-diff from the v3 of [1].

1: 12fe73bb72 ! 1: 2b9deb6d6a maintenance: create basic maintenance runner

    @@ Commit message
         'gc' builtin entirely at some point, leaving 'git gc' as an alias for
         some specific arguments to 'git maintenance run'.

    +    Create a new test_subcommand helper that allows us to test if a certain
    +    subcommand was run. It requires storing the GIT_TRACE2_EVENT logs in a
    +    file. A negation mode is available that will be used in later tests.
    +
    +    Helped-by: Jonathan Nieder <[email protected]>
         Signed-off-by: Derrick Stolee <[email protected]>

      ## .gitignore ##
    @@ builtin/gc.c: int cmd_gc(int argc, const char **argv, const char *prefix)
     +    NULL
     +};
     +
    -+static struct maintenance_opts {
    ++struct maintenance_opts {
     +    int auto_flag;
    -+} opts;
    ++};
     +
    -+static int maintenance_task_gc(void)
    ++static int maintenance_task_gc(struct maintenance_opts *opts)
     +{
     +    struct child_process child = CHILD_PROCESS_INIT;
     +
     +    child.git_cmd = 1;
     +    strvec_push(&child.args, "gc");
     +
    -+    if (opts.auto_flag)
    ++    if (opts->auto_flag)
     +        strvec_push(&child.args, "--auto");
     +
     +    close_object_store(the_repository->objects);
     +    return run_command(&child);
     +}
     +
    -+static int maintenance_run(void)
    ++static int maintenance_run(struct maintenance_opts *opts)
     +{
    -+    return maintenance_task_gc();
    ++    return maintenance_task_gc(opts);
     +}
     +
     +int cmd_maintenance(int argc, const char **argv, const char *prefix)
     +{
    ++    static struct maintenance_opts opts;
     +    static struct option builtin_maintenance_options[] = {
     +        OPT_BOOL(0, "auto", &opts.auto_flag,
     +             N_("run tasks based on the state of the repository")),
    @@ builtin/gc.c: int cmd_gc(int argc, const char **argv, const char *prefix)
     +
     +    if (argc == 1) {
     +        if (!strcmp(argv[0], "run"))
    -+            return maintenance_run();
    ++            return maintenance_run(&opts);
     +    }
     +
     +    usage_with_options(builtin_maintenance_usage,
    @@ t/t7900-maintenance.sh (new)
     +test_expect_success 'run [--auto]' '
     +    GIT_TRACE2_EVENT="$(pwd)/run-no-auto.txt" git maintenance run &&
     +    GIT_TRACE2_EVENT="$(pwd)/run-auto.txt" git maintenance run --auto &&
    -+    grep ",\"gc\"]" run-no-auto.txt  &&
    -+    grep ",\"gc\",\"--auto\"]" run-auto.txt
    ++    test_subcommand git gc <run-no-auto.txt &&
    ++    test_subcommand git gc --auto <run-auto.txt
     +'
     +
     +test_done
    +
    + ## t/test-lib-functions.sh ##
    +@@ t/test-lib-functions.sh: test_path_is_hidden () {
    +     case "$("$SYSTEMROOT"/system32/attrib "$1")" in *H*?:*) return 0;; esac
    +     return 1
    + }
    ++
    ++# Check that the given command was invoked as part of the
    ++# trace2-format trace on stdin.
    ++#
    ++#    test_subcommand [!] <command> <args>... < <trace>
    ++#
    ++# For example, to look for an invocation of "git upload-pack
    ++# /path/to/repo"
    ++#
    ++#    GIT_TRACE2_EVENT=event.log git fetch ... &&
    ++#    test_subcommand git upload-pack "$PATH" <event.log
    ++#
    ++# If the first parameter passed is !, this instead checks that
    ++# the given command was not called.
    ++#
    ++test_subcommand () {
    ++    local negate=
    ++    if test "$1" = "!"
    ++    then
    ++        negate=t
    ++        shift
    ++    fi
    ++
    ++    local expr=$(printf '"%s",' "$@")
    ++    expr="${expr%,}"
    ++
    ++    if test -n "$negate"
    ++    then
    ++        ! grep "\[$expr\]"
    ++    else
    ++        grep "\[$expr\]"
    ++    fi
    ++}
 2:  6e533e43d7 !  2:  d5faef26af maintenance: add --quiet option
    @@ Documentation/git-maintenance.txt: OPTIONS
      ## builtin/gc.c ##
     @@ builtin/gc.c: static const char * const builtin_maintenance_usage[] = {

    - static struct maintenance_opts {
    + struct maintenance_opts {
          int auto_flag;
     +    int quiet;
    - } opts;
    + };

    - static int maintenance_task_gc(void)
    -@@ builtin/gc.c: static int maintenance_task_gc(void)
    + static int maintenance_task_gc(struct maintenance_opts *opts)
    +@@ builtin/gc.c: static int maintenance_task_gc(struct maintenance_opts *opts)

    -     if (opts.auto_flag)
    +     if (opts->auto_flag)
              strvec_push(&child.args, "--auto");
    -+    if (opts.quiet)
    ++    if (opts->quiet)
     +        strvec_push(&child.args, "--quiet");
    ++    else
    ++        strvec_push(&child.args, "--no-quiet");

          close_object_store(the_repository->objects);
          return run_command(&child);
    @@ t/t7900-maintenance.sh: test_expect_success 'help text' '

     -test_expect_success 'run [--auto]' '
     -    GIT_TRACE2_EVENT="$(pwd)/run-no-auto.txt" git maintenance run &&
    +-    GIT_TRACE2_EVENT="$(pwd)/run-auto.txt" git maintenance run --auto &&
    +-    test_subcommand git gc <run-no-auto.txt &&
    +-    test_subcommand git gc --auto <run-auto.txt
     +test_expect_success 'run [--auto|--quiet]' '
    -+    GIT_TRACE2_EVENT="$(pwd)/run-no-auto.txt" git maintenance run --no-quiet &&
    -     GIT_TRACE2_EVENT="$(pwd)/run-auto.txt" git maintenance run --auto &&
    -+    GIT_TRACE2_EVENT="$(pwd)/run-quiet.txt" git maintenance run --quiet &&
    -     grep ",\"gc\"]" run-no-auto.txt  &&
    --    grep ",\"gc\",\"--auto\"]" run-auto.txt
    -+    grep ",\"gc\",\"--auto\"" run-auto.txt &&
    -+    grep ",\"gc\",\"--quiet\"" run-quiet.txt
    ++    GIT_TRACE2_EVENT="$(pwd)/run-no-auto.txt" \
    ++        git maintenance run 2>/dev/null &&
    ++    GIT_TRACE2_EVENT="$(pwd)/run-auto.txt" \
    ++        git maintenance run --auto 2>/dev/null &&
    ++    GIT_TRACE2_EVENT="$(pwd)/run-no-quiet.txt" \
    ++        git maintenance run --no-quiet 2>/dev/null &&
    ++    test_subcommand git gc --quiet <run-no-auto.txt &&
    ++    test_subcommand git gc --auto --quiet <run-auto.txt &&
    ++    test_subcommand git gc --no-quiet <run-no-quiet.txt
      '

      test_done

3: c4674fc211 ! 3: 233811310b maintenance: replace run_auto_gc()
@@ run-command.c: int run_processes_parallel_tr2(int n, get_next_task_fn get_next_t
- strvec_pushl(&argv_gc_auto, "gc", "--auto", NULL);
- if (quiet)
- strvec_push(&argv_gc_auto, "--quiet");
-- status = run_command_v_opt(argv_gc_auto.items, RUN_GIT_CMD);
+- status = run_command_v_opt(argv_gc_auto.v, RUN_GIT_CMD);
- strvec_clear(&argv_gc_auto);
- return status;
+ maint.git_cmd = 1;
4: b9332c1318 ! 4: 7efa23abc8 maintenance: initialize task array
@@ Commit message

## builtin/gc.c ##
-@@ builtin/gc.c: static int maintenance_task_gc(void)
+@@ builtin/gc.c: static int maintenance_task_gc(struct maintenance_opts *opts)
return run_command(&child);
}

-+typedef int maintenance_task_fn(void);
++typedef int maintenance_task_fn(struct maintenance_opts *opts);
+
+struct maintenance_task {
+ const char *name;
@@ builtin/gc.c: static int maintenance_task_gc(void)
+ },
+};
+
- static int maintenance_run(void)
+ static int maintenance_run(struct maintenance_opts *opts) {
-- return maintenance_task_gc();
+- return maintenance_task_gc(opts);
+ int i;
+ int result = 0;
+
@@ builtin/gc.c: static int maintenance_task_gc(void)
+ if (!tasks[i].enabled)
+ continue;
+
-+ if (tasks[i].fn()) {
++ if (tasks[i].fn(opts)) {
+ error(_("task '%s' failed"), tasks[i].name);
+ result = 1;
+ }
5: a4d9836bed ! 5: 902b742032 maintenance: add commit-graph task
@@ Documentation/git-maintenance.txt: run::
stands for "garbage collection," but this task performs many

## builtin/gc.c ##
-@@ builtin/gc.c: static struct maintenance_opts { +@@ builtin/gc.c: struct maintenance_opts {
int quiet;
- } opts;
+ };

-+static int run_write_commit_graph(void)
++static int run_write_commit_graph(struct maintenance_opts *opts)
+{
+ struct child_process child = CHILD_PROCESS_INIT;
+
@@ builtin/gc.c: static struct maintenance_opts {
+ strvec_pushl(&child.args, "commit-graph", "write",
+ "--split", "--reachable", NULL);
+
-+ if (opts.quiet)
++ if (opts->quiet)
+ strvec_push(&child.args, "--no-progress");
+
+ return !!run_command(&child);
+}
+
-+static int run_verify_commit_graph(void)
++static int run_verify_commit_graph(struct maintenance_opts *opts)
+{
+ struct child_process child = CHILD_PROCESS_INIT;
+
@@ builtin/gc.c: static struct maintenance_opts {
+ strvec_pushl(&child.args, "commit-graph", "verify",
+ "--shallow", NULL);
+
-+ if (opts.quiet)
++ if (opts->quiet)
+ strvec_push(&child.args, "--no-progress");
+
+ return !!run_command(&child);
+}
+
-+static int maintenance_task_commit_graph(void)
++static int maintenance_task_commit_graph(struct maintenance_opts *opts)
+{
+ struct repository *r = the_repository;
+ char *chain_path;
+
+ close_object_store(r->objects);
-+ if (run_write_commit_graph()) {
++ if (run_write_commit_graph(opts)) {
+ error(_("failed to write commit-graph"));
+ return 1;
+ }
+
-+ if (!run_verify_commit_graph())
++ if (!run_verify_commit_graph(opts))
+ return 0;
+
+ warning(_("commit-graph verify caught error, rewriting"));
@@ builtin/gc.c: static struct maintenance_opts {
+ }
+ free(chain_path);
+
-+ if (!run_write_commit_graph())
++ if (!run_write_commit_graph(opts))
+ return 0;
+
+ error(_("failed to rewrite commit-graph"));
+ return 1;
+}
+
- static int maintenance_task_gc(void)
+ static int maintenance_task_gc(struct maintenance_opts *opts) {
struct child_process child = CHILD_PROCESS_INIT;
@@ builtin/gc.c: struct maintenance_task {
@@ builtin/gc.c: static struct maintenance_task tasks[] = {
+ },
};

- static int maintenance_run(void)
+ static int maintenance_run(struct maintenance_opts *opts)

## commit-graph.c ##
@@ commit-graph.c: static char *get_split_graph_filename(struct object_directory *odb,
6: dafb0d9bbc ! 6: dddbcc4f3d maintenance: add --task option

    @@ Commit message
         Signed-off-by: Derrick Stolee <[email protected]>

      ## Documentation/git-maintenance.txt ##
    +@@ Documentation/git-maintenance.txt: SUBCOMMANDS
    + -----------
    + 
    + run::
    +-    Run one or more maintenance tasks.
    ++    Run one or more maintenance tasks. If one or more `--task=<task>`
    ++    options are specified, then those tasks are run in the provided
    ++    order. Otherwise, only the `gc` task is run.
    + 
    + TASKS
    + -----
     @@ Documentation/git-maintenance.txt: OPTIONS
      --quiet::
          Do not report progress or other information over `stderr`.
    @@ Documentation/git-maintenance.txt: OPTIONS
      Part of the linkgit:git[1] suite

      ## builtin/gc.c ##
    -@@ builtin/gc.c: static const char * const builtin_maintenance_usage[] = {
    - static struct maintenance_opts {
    -     int auto_flag;
    -     int quiet;
    -+    int tasks_selected;
    - } opts;
    - 
    - static int run_write_commit_graph(void)
    -@@ builtin/gc.c: typedef int maintenance_task_fn(void);
    +@@ builtin/gc.c: typedef int maintenance_task_fn(struct maintenance_opts *opts);
      struct maintenance_task {
          const char *name;
          maintenance_task_fn *fn;
    @@ builtin/gc.c: static struct maintenance_task tasks[] = {
     +    return b->selected_order - a->selected_order;
     +}
     +
    - static int maintenance_run(void)
    + static int maintenance_run(struct maintenance_opts *opts)
      {
    -     int i;
    +-    int i;
    ++    int i, found_selected = 0;
          int result = 0;

    -+    if (opts.tasks_selected)
    ++    for (i = 0; !found_selected && i < TASK__COUNT; i++)
    ++        found_selected = tasks[i].selected;
    ++
    ++    if (found_selected)
     +        QSORT(tasks, TASK__COUNT, compare_tasks_by_selection);
     +
          for (i = 0; i < TASK__COUNT; i++) {
     -        if (!tasks[i].enabled)
    -+        if (opts.tasks_selected && !tasks[i].selected)
    ++        if (found_selected && !tasks[i].selected)
     +            continue;
     +
    -+        if (!opts.tasks_selected && !tasks[i].enabled)
    ++        if (!found_selected && !tasks[i].enabled)
                  continue;

    -         if (tasks[i].fn()) {
    -@@ builtin/gc.c: static int maintenance_run(void)
    +         if (tasks[i].fn(opts)) {
    +@@ builtin/gc.c: static int maintenance_run(struct maintenance_opts *opts)
          return result;
      }

     +static int task_option_parse(const struct option *opt,
     +                 const char *arg, int unset)
     +{
    -+    int i;
    ++    int i, num_selected = 0;
     +    struct maintenance_task *task = NULL;
     +
     +    BUG_ON_OPT_NEG(unset);
     +
    -+    opts.tasks_selected++;
    -+
     +    for (i = 0; i < TASK__COUNT; i++) {
    ++        num_selected += tasks[i].selected;
     +        if (!strcasecmp(tasks[i].name, arg)) {
     +            task = &tasks[i];
    -+            break;
     +        }
     +    }
     +
    @@ builtin/gc.c: static int maintenance_run(void)
     +    }
     +
     +    task->selected = 1;
    -+    task->selected_order = opts.tasks_selected;
    ++    task->selected_order = num_selected + 1;
     +
     +    return 0;
     +}
     +
      int cmd_maintenance(int argc, const char **argv, const char *prefix)
      {
    -     static struct option builtin_maintenance_options[] = {
    +     static struct maintenance_opts opts;
     @@ builtin/gc.c: int cmd_maintenance(int argc, const char **argv, const char *prefix)
                   N_("run tasks based on the state of the repository")),
              OPT_BOOL(0, "quiet", &opts.quiet,
    @@ builtin/gc.c: int cmd_maintenance(int argc, const char **argv, const char *prefi

      ## t/t7900-maintenance.sh ##
     @@ t/t7900-maintenance.sh: test_expect_success 'run [--auto|--quiet]' '
    -     grep ",\"gc\",\"--quiet\"" run-quiet.txt
    +     test_subcommand git gc --no-quiet <run-no-quiet.txt
      '

     +test_expect_success 'run --task=<task>' '
    -+    GIT_TRACE2_EVENT="$(pwd)/run-commit-graph.txt" git maintenance run --task=commit-graph &&
    -+    GIT_TRACE2_EVENT="$(pwd)/run-gc.txt" git maintenance run --task=gc &&
    -+    GIT_TRACE2_EVENT="$(pwd)/run-commit-graph.txt" git maintenance run --task=commit-graph &&
    -+    GIT_TRACE2_EVENT="$(pwd)/run-both.txt" git maintenance run --task=commit-graph --task=gc &&
    -+    ! grep ",\"gc\"" run-commit-graph.txt  &&
    -+    grep ",\"gc\"" run-gc.txt  &&
    -+    grep ",\"gc\"" run-both.txt  &&
    -+    grep ",\"commit-graph\",\"write\"" run-commit-graph.txt  &&
    -+    ! grep ",\"commit-graph\",\"write\"" run-gc.txt  &&
    -+    grep ",\"commit-graph\",\"write\"" run-both.txt
    ++    GIT_TRACE2_EVENT="$(pwd)/run-commit-graph.txt" \
    ++        git maintenance run --task=commit-graph 2>/dev/null &&
    ++    GIT_TRACE2_EVENT="$(pwd)/run-gc.txt" \
    ++        git maintenance run --task=gc 2>/dev/null &&
    ++    GIT_TRACE2_EVENT="$(pwd)/run-commit-graph.txt" \
    ++        git maintenance run --task=commit-graph 2>/dev/null &&
    ++    GIT_TRACE2_EVENT="$(pwd)/run-both.txt" \
    ++        git maintenance run --task=commit-graph --task=gc 2>/dev/null &&
    ++    test_subcommand ! git gc --quiet <run-commit-graph.txt &&
    ++    test_subcommand git gc --quiet <run-gc.txt &&
    ++    test_subcommand git gc --quiet <run-both.txt &&
    ++    test_subcommand git commit-graph write --split --reachable --no-progress <run-commit-graph.txt &&
    ++    test_subcommand ! git commit-graph write --split --reachable --no-progress <run-gc.txt &&
    ++    test_subcommand git commit-graph write --split --reachable --no-progress <run-both.txt
     +'
     +
     +test_expect_success 'run --task=bogus' '

7: 1b00524da3 ! 7: 79af39be13 maintenance: take a lock on the objects directory
@@ Commit message

## builtin/gc.c ##
-@@ builtin/gc.c: static int maintenance_run(void)
+@@ builtin/gc.c: static int maintenance_run(struct maintenance_opts *opts)
{
- int i;
+ int i, found_selected = 0;
int result = 0;
+ struct lock_file lk;
+ struct repository *r = the_repository;
@@ builtin/gc.c: static int maintenance_run(void)
+ * recursive process stack. Do not report an error in
+ * that case.
+ */
-+ if (!opts.auto_flag && !opts.quiet)
++ if (!opts->auto_flag && !opts->quiet)
+ error(_("lock file '%s' exists, skipping maintenance"),
+ lock_path);
+ free(lock_path);
@@ builtin/gc.c: static int maintenance_run(void)
+ }
+ free(lock_path);

- if (opts.tasks_selected)
- QSORT(tasks, TASK__COUNT, compare_tasks_by_selection);
-@@ builtin/gc.c: static int maintenance_run(void)
+ for (i = 0; !found_selected && i < TASK__COUNT; i++)
+ found_selected = tasks[i].selected;
+@@ builtin/gc.c: static int maintenance_run(struct maintenance_opts *opts)
}
}

 8:  0e94e04dcd <  -:  ---------- fetch: optionally allow disabling FETCH_HEAD update
 9:  9e38ade15c <  -:  ---------- maintenance: add prefetch task
10:  0128fdfd1a <  -:  ---------- maintenance: add loose-objects task
11:  c2baf6e119 <  -:  ---------- midx: enable core.multiPackIndex by default
12:  00f47c4848 <  -:  ---------- maintenance: add incremental-repack task
13:  ef2a231956 <  -:  ---------- maintenance: auto-size incremental-repack batch
14:  99840c4b8f !  8:  69bfc6a4b2 maintenance: create maintenance.<task>.enabled config
    @@ Documentation/git-maintenance.txt: SUBCOMMANDS
      -----------

      run::
    --    Run one or more maintenance tasks.
    +-    Run one or more maintenance tasks. If one or more `--task=<task>`
    +-    options are specified, then those tasks are run in the provided
    +-    order. Otherwise, only the `gc` task is run.
     +    Run one or more maintenance tasks. If one or more `--task` options
     +    are specified, then those tasks are run in that order. Otherwise,
     +    the tasks are determined by which `maintenance.<task>.enabled`
    @@ Documentation/git-maintenance.txt: SUBCOMMANDS
      -----

      ## builtin/gc.c ##
    -@@ builtin/gc.c: static int maintenance_run(void)
    +@@ builtin/gc.c: static int maintenance_run(struct maintenance_opts *opts)
          return result;
      }

    @@ builtin/gc.c: int cmd_maintenance(int argc, const char **argv, const char *prefi

      ## t/t7900-maintenance.sh ##
     @@ t/t7900-maintenance.sh: test_expect_success 'run [--auto|--quiet]' '
    -     grep ",\"gc\",\"--quiet\"" run-quiet.txt
    +     test_subcommand git gc --no-quiet <run-no-quiet.txt
      '

     +test_expect_success 'maintenance.<task>.enabled' '
     +    git config maintenance.gc.enabled false &&
     +    git config maintenance.commit-graph.enabled true &&
    -+    git config maintenance.loose-objects.enabled true &&
    -+    GIT_TRACE2_EVENT="$(pwd)/run-config.txt" git maintenance run &&
    -+    ! grep ",\"fetch\"" run-config.txt &&
    -+    ! grep ",\"gc\"" run-config.txt &&
    -+    ! grep ",\"multi-pack-index\"" run-config.txt &&
    -+    grep ",\"commit-graph\"" run-config.txt &&
    -+    grep ",\"prune-packed\"" run-config.txt
    ++    GIT_TRACE2_EVENT="$(pwd)/run-config.txt" git maintenance run 2>err &&
    ++    test_subcommand ! git gc --quiet <run-config.txt &&
    ++    test_subcommand git commit-graph write --split --reachable --no-progress <run-config.txt
     +'
     +
      test_expect_success 'run --task=<task>' '
    -     GIT_TRACE2_EVENT="$(pwd)/run-commit-graph.txt" git maintenance run --task=commit-graph &&
    -     GIT_TRACE2_EVENT="$(pwd)/run-gc.txt" git maintenance run --task=gc &&
    +     GIT_TRACE2_EVENT="$(pwd)/run-commit-graph.txt" \
    +         git maintenance run --task=commit-graph 2>/dev/null &&

15: a087c63572 ! 9: df21bbb000 maintenance: use pointers to check --auto
@@ Commit message

## builtin/gc.c ##
-@@ builtin/gc.c: static int maintenance_task_incremental_repack(void) +@@ builtin/gc.c: static int maintenance_task_gc(struct maintenance_opts *opts)

- typedef int maintenance_task_fn(void);
+ typedef int maintenance_task_fn(struct maintenance_opts *opts);

+/*
+ * An auto condition function returns 1 if the task should run
@@ builtin/gc.c: static struct maintenance_task tasks[] = {
1,
},
[TASK_COMMIT_GRAPH] = {
-@@ builtin/gc.c: static int maintenance_run(void)
- if (!opts.tasks_selected && !tasks[i].enabled)
+@@ builtin/gc.c: static int maintenance_run(struct maintenance_opts *opts)
+ if (!found_selected && !tasks[i].enabled)
continue;

-+ if (opts.auto_flag &&
++ if (opts->auto_flag &&
+ (!tasks[i].auto_condition ||
+ !tasks[i].auto_condition()))
+ continue;
+
- if (tasks[i].fn()) {
+ if (tasks[i].fn(opts)) {
error(_("task '%s' failed"), tasks[i].name);
result = 1;
@@ builtin/gc.c: static void initialize_task_config(void)
@@ t/t5514-fetch-multiple.sh: test_expect_success 'git fetch --multiple (two remote

## t/t7900-maintenance.sh ##
@@ t/t7900-maintenance.sh: test_expect_success 'run [--auto|--quiet]' '
- GIT_TRACE2_EVENT="$(pwd)/run-auto.txt" git maintenance run --auto &&
- GIT_TRACE2_EVENT="$(pwd)/run-quiet.txt" git maintenance run --quiet &&
- grep ",\"gc\"]" run-no-auto.txt &&
-- grep ",\"gc\",\"--auto\"" run-auto.txt &&
-+ ! grep ",\"gc\",\"--auto\"" run-auto.txt &&
- grep ",\"gc\",\"--quiet\"" run-quiet.txt
+ GIT_TRACE2_EVENT="$(pwd)/run-no-quiet.txt" \
+ git maintenance run --no-quiet 2>/dev/null &&
+ test_subcommand git gc --quiet <run-no-auto.txt &&
+- test_subcommand git gc --auto --quiet <run-auto.txt &&
++ test_subcommand ! git gc --auto --quiet <run-auto.txt &&
+ test_subcommand git gc --no-quiet <run-no-quiet.txt
'

16: ef3a854508 ! 10: e67e259aef maintenance: add auto condition for commit-graph task

    @@ Documentation/config/maintenance.txt: maintenance.<task>.enabled::

      ## builtin/gc.c ##
     @@
    + #include "blob.h"
    + #include "tree.h"
      #include "promisor-remote.h"
    - #include "remote.h"
    - #include "midx.h"
     +#include "refs.h"

      #define FAILED_RUN "failed to run %s"

    -@@ builtin/gc.c: static struct maintenance_opts {
    -     int tasks_selected;
    - } opts;
    +@@ builtin/gc.c: struct maintenance_opts {
    +     int quiet;
    + };

     +/* Remember to update object flag allocation in object.h */
     +#define PARENT1        (1u<<16)
    @@ builtin/gc.c: static struct maintenance_opts {
     +    return result;
     +}
     +
    - static int run_write_commit_graph(void)
    + static int run_write_commit_graph(struct maintenance_opts *opts)
      {
          struct child_process child = CHILD_PROCESS_INIT;
     @@ builtin/gc.c: static struct maintenance_task tasks[] = {
17:  6ac3a58f2f <  -:  ---------- maintenance: create auto condition for loose-objects
18:  801b262d1c <  -:  ---------- maintenance: add incremental-repack auto condition
19:  9b4cef7635 <  -:  ---------- midx: use start_delayed_progress()
20:  39eb83ad1e ! 11:  a5d1914846 maintenance: add trace2 regions for task execution

@@ Commit message

## builtin/gc.c ##
-@@ builtin/gc.c: static int maintenance_run(void)
+@@ builtin/gc.c: static int maintenance_run(struct maintenance_opts *opts)
!tasks[i].auto_condition()))
continue;

+ trace2_region_enter("maintenance", tasks[i].name, r);
- if (tasks[i].fn()) {
+ if (tasks[i].fn(opts)) {
error(_("task '%s' failed"), tasks[i].name); result = 1;
}

Derrick Stolee (11):
maintenance: create basic maintenance runner
maintenance: add --quiet option
maintenance: replace run_auto_gc()
maintenance: initialize task array
maintenance: add commit-graph task
maintenance: add --task option
maintenance: take a lock on the objects directory
maintenance: create maintenance.<task>.enabled config
maintenance: use pointers to check --auto
maintenance: add auto condition for commit-graph task
maintenance: add trace2 regions for task execution

 .gitignore                           |   1 +
 Documentation/config.txt             |   2 +
 Documentation/config/maintenance.txt |  14 ++
 Documentation/fetch-options.txt      |   6 +-
 Documentation/git-clone.txt          |   6 +-
 Documentation/git-maintenance.txt    |  86 +++++++
 builtin.h                            |   1 +
 builtin/am.c                         |   2 +-
 builtin/commit.c                     |   2 +-
 builtin/fetch.c                      |   6 +-
 builtin/gc.c                         | 353 +++++++++++++++++++++++++++
 builtin/merge.c                      |   2 +-
 builtin/rebase.c                     |   4 +-
 commit-graph.c                       |   8 +-
 commit-graph.h                       |   1 +
 git.c                                |   1 +
 object.h                             |   1 +
 run-command.c                        |  16 +-
 run-command.h                        |   2 +-
 t/t5510-fetch.sh                     |   2 +-
 t/t5514-fetch-multiple.sh            |   2 +-
 t/t7900-maintenance.sh               |  61 +++++
 t/test-lib-functions.sh              |  33 +++
 23 files changed, 584 insertions(+), 28 deletions(-)
 create mode 100644 Documentation/config/maintenance.txt
 create mode 100644 Documentation/git-maintenance.txt
 create mode 100755 t/t7900-maintenance.sh

base-commit: d70a9eb611a9d242c1d26847d223b8677609305b
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-695%2Fderrickstolee%2Fmaintenance%2Fbuiltin-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-695/derrickstolee/maintenance/builtin-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/695

gitgitgadget

From: Derrick Stolee <[email protected]>

The 'gc' builtin is our current entrypoint for automatically maintaining a repository. This one tool does many operations, such as repacking the repository, packing refs, and rewriting the commit-graph file. The name implies it performs "garbage collection" which means several different things, and some users may not want to use this operation that rewrites the entire object database.

Create a new 'maintenance' builtin that will become a more general- purpose command. To start, it will only support the 'run' subcommand, but will later expand to add subcommands for scheduling maintenance in the background.

For now, the 'maintenance' builtin is a thin shim over the 'gc' builtin. In fact, the only option is the '--auto' toggle, which is handed directly to the 'gc' builtin. The current change is isolated to this simple operation to prevent more interesting logic from being lost in all of the boilerplate of adding a new builtin.

Use existing builtin/gc.c file because we want to share code between the two builtins. It is possible that we will have 'maintenance' replace the 'gc' builtin entirely at some point, leaving 'git gc' as an alias for some specific arguments to 'git maintenance run'.

Create a new test_subcommand helper that allows us to test if a certain subcommand was run. It requires storing the GIT_TRACE2_EVENT logs in a file. A negation mode is available that will be used in later tests.

---
 .gitignore                        |  1 +
 Documentation/git-maintenance.txt | 57 +++++++++++++++++++++++++++++++
 builtin.h                         |  1 +
 builtin/gc.c                      | 57 +++++++++++++++++++++++++++++++
 git.c                             |  1 +
 t/t7900-maintenance.sh            | 19 +++++++++++
 t/test-lib-functions.sh           | 33 ++++++++++++++++++
 7 files changed, 169 insertions(+)
 create mode 100644 Documentation/git-maintenance.txt
 create mode 100755 t/t7900-maintenance.sh

index ee509a2ad2..a5808fa30d 100644
@@ -90,6 +90,7 @@
 /git-ls-tree
 /git-mailinfo
 /git-mailsplit
+/git-maintenance
 /git-merge
 /git-merge-base
 /git-merge-index

index a5ae15bfe5..17c1c0ce49 100644
@@ -167,6 +167,7 @@ int cmd_ls_tree(int argc, const char **argv, const char *prefix);
 int cmd_ls_remote(int argc, const char **argv, const char *prefix);
 int cmd_mailinfo(int argc, const char **argv, const char *prefix);
 int cmd_mailsplit(int argc, const char **argv, const char *prefix);
+int cmd_maintenance(int argc, const char **argv, const char *prefix);
 int cmd_merge(int argc, const char **argv, const char *prefix);
 int cmd_merge_base(int argc, const char **argv, const char *prefix);
 int cmd_merge_index(int argc, const char **argv, const char *prefix);

index 8bd1d7551d..24f250d29a 100644
@@ -529,6 +529,7 @@ static struct cmd_struct commands[] = {
 	{ "ls-tree", cmd_ls_tree, RUN_SETUP },
 	{ "mailinfo", cmd_mailinfo, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "mailsplit", cmd_mailsplit, NO_PARSEOPT },
+	{ "maintenance", cmd_maintenance, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "merge", cmd_merge, RUN_SETUP | NEED_WORK_TREE },
 	{ "merge-base", cmd_merge_base, RUN_SETUP },
 	{ "merge-file", cmd_merge_file, RUN_SETUP_GENTLY },

new file mode 100755
index 0000000000..c4b9b4a6fe
@@ -0,0 +1,19 @@
+#!/bin/sh
+
+test_description='git maintenance builtin'
+
+. ./test-lib.sh
+
+test_expect_success 'help text' '
+	test_expect_code 129 git maintenance -h 2>err &&
+	test_i18ngrep "usage: git maintenance run" err
+'
+
+test_expect_success 'run [--auto]' '
+	GIT_TRACE2_EVENT="$(pwd)/run-no-auto.txt" git maintenance run &&
+	GIT_TRACE2_EVENT="$(pwd)/run-auto.txt" git maintenance run --auto &&
+	test_subcommand git gc <run-no-auto.txt &&
+	test_subcommand git gc --auto <run-auto.txt
+'
+
+test_done

index 3103be8a32..0adf2b85f8 100644
@@ -1561,3 +1561,36 @@ test_path_is_hidden () {
 	case "$("$SYSTEMROOT"/system32/attrib "$1")" in *H*?:*) return 0;; esac
 	return 1
 }
+
+# Check that the given command was invoked as part of the
+# trace2-format trace on stdin.
+#
+#	test_subcommand [!] <command> <args>... < <trace>
+#
+# For example, to look for an invocation of "git upload-pack
+# /path/to/repo"
+#
+#	GIT_TRACE2_EVENT=event.log git fetch ... &&
+#	test_subcommand git upload-pack "$PATH" <event.log
+#
+# If the first parameter passed is !, this instead checks that
+# the given command was not called.
+#
+test_subcommand () {
+	local negate=
+	if test "$1" = "!"
+	then
+		negate=t
+		shift
+	fi
+
+	local expr=$(printf '"%s",' "$@")
+	expr="${expr%,}"
+
+	if test -n "$negate"
+	then
+		! grep "\[$expr\]"
+	else
+		grep "\[$expr\]"
+	fi
+}

gitgitgadget

From: Derrick Stolee <[email protected]>

The run_auto_gc() method is used in several places to trigger a check for repo maintenance after some Git commands, such as 'git commit' or 'git fetch'.

To allow for extra customization of this maintenance activity, replace the 'git gc --auto [--quiet]' call with one to 'git maintenance run --auto [--quiet]'. As we extend the maintenance builtin with other steps, users will be able to select different maintenance activities.

Rename run_auto_gc() to run_auto_maintenance() to be clearer what is happening on this call, and to expose all callers in the current diff. Rewrite the method to use a struct child_process to simplify the calls slightly.

Since 'git fetch' already allows disabling the 'git gc --auto' subprocess, add an equivalent option with a different name to be more descriptive of the new behavior: '--[no-]maintenance'. Update the documentation to include these options at the same time.

---
 Documentation/fetch-options.txt |  6 ++++--
 Documentation/git-clone.txt     |  6 +++---
 builtin/am.c                    |  2 +-
 builtin/commit.c                |  2 +-
 builtin/fetch.c                 |  6 ++++--
 builtin/merge.c                 |  2 +-
 builtin/rebase.c                |  4 ++--
 run-command.c                   | 16 +++++++---------
 run-command.h                   |  2 +-
 t/t5510-fetch.sh                |  2 +-
 10 files changed, 25 insertions(+), 23 deletions(-)

index 6e2a160a47..495bc8ab5a 100644
@@ -86,9 +86,11 @@ ifndef::git-pull[]
 	Allow several <repository> and <group> arguments to be
 	specified. No <refspec>s may be specified.
 
+--[no-]auto-maintenance::
 --[no-]auto-gc::
-	Run `git gc --auto` at the end to perform garbage collection
-	if needed. This is enabled by default.
+	Run `git maintenance run --auto` at the end to perform automatic
+	repository maintenance if needed. (`--[no-]auto-gc` is a synonym.)
+	This is enabled by default.
 
 --[no-]write-commit-graph::
 	Write a commit-graph after fetching. This overrides the config

index c898310099..097e6a86c5 100644
@@ -78,9 +78,9 @@ repository using this option and then delete branches (or use any
 other Git command that makes any existing commit unreferenced) in the
 source repository, some objects may become unreferenced (or dangling).
 These objects may be removed by normal Git operations (such as `git commit`)
-which automatically call `git gc --auto`. (See linkgit:git-gc[1].)
-If these objects are removed and were referenced by the cloned repository,
-then the cloned repository will become corrupt.
+which automatically call `git maintenance run --auto`. (See
+linkgit:git-maintenance[1].) If these objects are removed and were referenced
+by the cloned repository, then the cloned repository will become corrupt.
 +
 Note that running `git repack` without the `--local` option in a repository
 cloned with `--shared` will copy objects from the source repository into a pack

index dd4e6c2d9b..68e9d17379 100644
@@ -1795,7 +1795,7 @@ static void am_run(struct am_state *state, int resume)
 	if (!state->rebasing) {
 		am_destroy(state);
 		close_object_store(the_repository->objects);
-		run_auto_gc(state->quiet);
+		run_auto_maintenance(state->quiet);
 	}
 }
 

index 69ac78d5e5..f9b0a0c05d 100644
@@ -1702,7 +1702,7 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 	git_test_write_commit_graph_or_die();
 
 	repo_rerere(the_repository, 0);
-	run_auto_gc(quiet);
+	run_auto_maintenance(quiet);
 	run_commit_hook(use_editor, get_index_file(), "post-commit", NULL);
 	if (amend && !no_post_rewrite) {
 		commit_post_rewrite(the_repository, current_head, &oid);

index c49f0e9752..c8b9366d3c 100644
@@ -196,8 +196,10 @@ static struct option builtin_fetch_options[] = {
 	OPT_STRING_LIST(0, "negotiation-tip", &negotiation_tip, N_("revision"),
 			N_("report that we have only objects reachable from this object")),
 	OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options),
+	OPT_BOOL(0, "auto-maintenance", &enable_auto_gc,
+		 N_("run 'maintenance --auto' after fetching")),
 	OPT_BOOL(0, "auto-gc", &enable_auto_gc,
-		 N_("run 'gc --auto' after fetching")),
+		 N_("run 'maintenance --auto' after fetching")),
 	OPT_BOOL(0, "show-forced-updates", &fetch_show_forced_updates,
 		 N_("check for forced-updates on all updated branches")),
 	OPT_BOOL(0, "write-commit-graph", &fetch_write_commit_graph,

@@ -1882,7 +1884,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
 	close_object_store(the_repository->objects);
 
 	if (enable_auto_gc)
-		run_auto_gc(verbosity < 0);
+		run_auto_maintenance(verbosity < 0);
 
 	return result;
 }

index 7da707bf55..c068e73037 100644
@@ -457,7 +457,7 @@ static void finish(struct commit *head_commit,
 			 * user should see them.
 			 */
 			close_object_store(the_repository->objects);
-			run_auto_gc(verbosity < 0);
+			run_auto_maintenance(verbosity < 0);
 		}
 	}
 	if (new_head && show_diffstat) {

index dadb52fa92..9ffba9009d 100644
@@ -728,10 +728,10 @@ static int finish_rebase(struct rebase_options *opts)
 	apply_autostash(state_dir_path("autostash", opts));
 	close_object_store(the_repository->objects);
 	/*
-	 * We ignore errors in 'gc --auto', since the
+	 * We ignore errors in 'git maintenance run --auto', since the
 	 * user should see them.
 	 */
-	run_auto_gc(!(opts->flags & (REBASE_NO_QUIET|REBASE_VERBOSE)));
+	run_auto_maintenance(!(opts->flags & (REBASE_NO_QUIET|REBASE_VERBOSE)));
 	if (opts->type == REBASE_MERGE) {
 		struct replay_opts replay = REPLAY_OPTS_INIT;
 

index cc9c3296ba..2ee59acdc8 100644
@@ -1866,15 +1866,13 @@ int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 	return result;
 }
 
-int run_auto_gc(int quiet)
+int run_auto_maintenance(int quiet)
 {
-	struct strvec argv_gc_auto = STRVEC_INIT;
-	int status;
+	struct child_process maint = CHILD_PROCESS_INIT;
 
-	strvec_pushl(&argv_gc_auto, "gc", "--auto", NULL);
-	if (quiet)
-		strvec_push(&argv_gc_auto, "--quiet");
-	status = run_command_v_opt(argv_gc_auto.v, RUN_GIT_CMD);
-	strvec_clear(&argv_gc_auto);
-	return status;
+	maint.git_cmd = 1;
+	strvec_pushl(&maint.args, "maintenance", "run", "--auto", NULL);
+	strvec_push(&maint.args, quiet ? "--quiet" : "--no-quiet");
+
+	return run_command(&maint);
 }

index 8b9bfaef16..6472b38bde 100644
@@ -221,7 +221,7 @@ int run_hook_ve(const char *const *env, const char *name, va_list args);
 /*
  * Trigger an auto-gc
  */
-int run_auto_gc(int quiet);
+int run_auto_maintenance(int quiet);
 
 #define RUN_COMMAND_NO_STDIN 1
 #define RUN_GIT_CMD	     2	/*If this is to be git sub-command */

index a66dbe0bde..9850ecde5d 100755
@@ -919,7 +919,7 @@ test_expect_success 'fetching with auto-gc does not lock up' '
 		git config fetch.unpackLimit 1 &&
 		git config gc.autoPackLimit 1 &&
 		git config gc.autoDetach false &&
-		GIT_ASK_YESNO="$D/askyesno" git fetch >fetch.out 2>&1 &&
+		GIT_ASK_YESNO="$D/askyesno" git fetch --verbose >fetch.out 2>&1 &&
 		test_i18ngrep "Auto packing the repository" fetch.out &&
 		! grep "Should I try again" fetch.out
 	)

gitgitgadget

From: Derrick Stolee <[email protected]>

Performing maintenance on a Git repository involves writing data to the .git directory, which is not safe to do with multiple writers attempting the same operation. Ensure that only one 'git maintenance' process is running at a time by holding a file-based lock. Simply the presence of the .git/maintenance.lock file will prevent future maintenance. This lock is never committed, since it does not represent meaningful data. Instead, it is only a placeholder.

If the lock file already exists, then fail silently. This will become very important later when we implement the 'fetch' task, as this is our stop-gap from creating a recursive process loop between 'git fetch' and 'git maintenance run'.

---
 builtin/gc.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

index 3d50ab7ac9..1b7d502b87 100644
@@ -830,6 +830,25 @@ static int maintenance_run(struct maintenance_opts *opts)
 {
 	int i, found_selected = 0;
 	int result = 0;
+	struct lock_file lk;
+	struct repository *r = the_repository;
+	char *lock_path = xstrfmt("%s/maintenance", r->objects->odb->path);
+
+	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0) {
+		/*
+		 * Another maintenance command is running.
+		 *
+		 * If --auto was provided, then it is likely due to a
+		 * recursive process stack. Do not report an error in
+		 * that case.
+		 */
+		if (!opts->auto_flag && !opts->quiet)
+			error(_("lock file '%s' exists, skipping maintenance"),
+			      lock_path);
+		free(lock_path);
+		return 0;
+	}
+	free(lock_path);
 
 	for (i = 0; !found_selected && i < TASK__COUNT; i++)
 		found_selected = tasks[i].selected;

@@ -850,6 +869,7 @@ static int maintenance_run(struct maintenance_opts *opts)
 		}
 	}
 
+	rollback_lock_file(&lk);
 	return result;
 }
 

gitgitgadget

From: Derrick Stolee <[email protected]>

The 'git maintenance run' command has an '--auto' option. This is used by other Git commands such as 'git commit' or 'git fetch' to check if maintenance should be run after adding data to the repository.

Previously, this --auto option was only used to add the argument to the 'git gc' command as part of the 'gc' task. We will be expanding the other tasks to perform a check to see if they should do work as part of the --auto flag, when they are enabled by config.

First, update the 'gc' task to perform the auto check inside the maintenance process. This prevents running an extra 'git gc --auto' command when not needed. It also shows a model for other tasks.

Second, use the 'auto_condition' function pointer as a signal for whether we enable the maintenance task under '--auto'. For instance, we do not want to enable the 'fetch' task in '--auto' mode, so that function pointer will remain NULL.

Now that we are not automatically calling 'git gc', a test in t5514-fetch-multiple.sh must be changed to watch for 'git maintenance' instead.

We continue to pass the '--auto' option to the 'git gc' command when necessary, because of the gc.autoDetach config option changes behavior. Likely, we will want to absorb the daemonizing behavior implied by gc.autoDetach as a maintenance.autoDetach config option.

---
 builtin/gc.c              | 16 ++++++++++++++++
 t/t5514-fetch-multiple.sh |  2 +-
 t/t7900-maintenance.sh    |  2 +-
 3 files changed, 18 insertions(+), 2 deletions(-)

index dc1d260858..2b3e45eb71 100644
@@ -788,9 +788,17 @@ static int maintenance_task_gc(struct maintenance_opts *opts)
 
 typedef int maintenance_task_fn(struct maintenance_opts *opts);
 
+/*
+ * An auto condition function returns 1 if the task should run
+ * and 0 if the task should NOT run. See needs_to_gc() for an
+ * example.
+ */
+typedef int maintenance_auto_fn(void);
+
 struct maintenance_task {
 	const char *name;
 	maintenance_task_fn *fn;
+	maintenance_auto_fn *auto_condition;
 	unsigned enabled:1,
 		 selected:1;
 	int selected_order;

@@ -808,6 +816,7 @@ static struct maintenance_task tasks[] = {
 	[TASK_GC] = {
 		"gc",
 		maintenance_task_gc,
+		need_to_gc,
 		1,
 	},
 	[TASK_COMMIT_GRAPH] = {

@@ -863,6 +872,11 @@ static int maintenance_run(struct maintenance_opts *opts)
 		if (!found_selected && !tasks[i].enabled)
 			continue;
 
+		if (opts->auto_flag &&
+		    (!tasks[i].auto_condition ||
+		     !tasks[i].auto_condition()))
+			continue;
+
 		if (tasks[i].fn(opts)) {
 			error(_("task '%s' failed"), tasks[i].name);
 			result = 1;

@@ -877,6 +891,8 @@ static void initialize_task_config(void)
 {
 	int i;
 	struct strbuf config_name = STRBUF_INIT;
+	gc_config();
+
 	for (i = 0; i < TASK__COUNT; i++) {
 		int config_value;
 

index de8e2f1531..bd202ec6f3 100755
@@ -108,7 +108,7 @@ test_expect_success 'git fetch --multiple (two remotes)' '
 	 GIT_TRACE=1 git fetch --multiple one two 2>trace &&
 	 git branch -r > output &&
 	 test_cmp ../expect output &&
-	 grep "built-in: git gc" trace >gc &&
+	 grep "built-in: git maintenance" trace >gc &&
 	 test_line_count = 1 gc
 	)
 '

index cdf0bf2e60..406dc7c303 100755
@@ -19,7 +19,7 @@ test_expect_success 'run [--auto|--quiet]' '
 	GIT_TRACE2_EVENT="$(pwd)/run-no-quiet.txt" \
 		git maintenance run --no-quiet 2>/dev/null &&
 	test_subcommand git gc --quiet <run-no-auto.txt &&
-	test_subcommand git gc --auto --quiet <run-auto.txt &&
+	test_subcommand ! git gc --auto --quiet <run-auto.txt &&
 	test_subcommand git gc --no-quiet <run-no-quiet.txt
 '
 

gitgitgadget

This is my understanding. We focus on the data we just wrote to see if anything went wrong at a lower level (i.e. filesystem or RAM corruption during the write process) instead of the data "at rest".

Right. And we want to keep the amount of work to be very small in most cases, amortizing the cost of the big merge operations across many much smaller runs. Perhaps a later change could introduce an option to drop the '--shallow' option and check the entire chain, for users willing to pay that price.

Back to the point of your comments: I'm not sure this second paragraph is required at all in the documentation. The first paragraph already says:

...then verifies that the written data is correct.

This "written data" _is_ the top layer of the chain. There is probably no reason to dig deeper into _why_ we do this in this user-facing documentation.

So, I propose just deleting this paragraph. What do you think?

Thanks for keeping a close eye on documentation changes! I updated my local branch to include the '+' problem from your other reply.

Thanks,
-Stolee

Looks reasonable.

[...]

	if (argc != 1)
		usage ...

	...

Do we have preference? I think it is more common in our codebase to have these non-static but I do not think we've chosen which is more preferable with technical reasoning behind it. As the top-level function in any callchain, cmd_foo() does not have multiple instances running at the same time, or it does not have to be prepared to be called twice, so they can afford to be static, but is there any upside for them to be static?

Okay, so that's one upside, albeit a small one.

Code size, execution time:

- benefit of static: can rely on automatic zero-initialization of pages instead of spending cycles and text size on explicit zero-initialization

- benefit of local: avoids wasting address space in bss when the function isn't called

Neither of those seems important enough to care about. :)

Maintainability:

- benefit of static: address is determined at compile time, so can build using C89 compilers that require struct initializers to be constant (but Git already cannot be built with such compilers)

- benefit of local: less likely to accidentally move the static var into a function that needs to be reentrant

- benefit of local: allows readers and reviewers to build the habit of seeing non-const "static" declarations within a function as the unusual case. When a "static" is declared in a function body, this means we are caching something or have some other performance reason to sacrifice reentrancy

It's mostly that last aspect (saving readers from having to think about it) that motivates my suggestion above.

Older Git code used static more because it was needed when building using C89 compilers.

Thanks,
Jonathan

As I mentioned in the earlier version, I disagree with this.

Both good.

For this and the next comment, I prefer (for now) to keep all of the deep details of the 'gc' task in the git-gc.txt documentation. However, I should use "linkgit:git-gc[1]" to point users to that for reference.

Eventually, the maintenance builtin might replace the gc builtin, but only after all of its behavior has been extracted into smaller tasks. Those tasks would be documented in detail here, allowing us to make the blurb for the 'gc' task be "Run the 'prune-reflog', 'pack-refs', ..., and 'full-repack' tasks."

I remember Junio mentioning something about how 'gc' interprets '--auto' as a "quick" mode, but like you I cannot find any reference to that in the docs. Since it is not documented there and none of the other tasks I am implementing here use that interpretation, I'll plan to leave this portion as-is.

We need to close our file handles on the pack-files (and commit-graph and multi-pack-index) or else the GC subcommand cannot delete the files on Windows.

I see my problem here. The builtin_maintenance_options is static (copied from another builtin, I'm sure) which is why the 'opts' struct needs to be static or else this doesn't compile.

Sure. Were you thinking of anything more specific than

	die(_("invalid subcommand: %s"), argv[0]);

? Of course, running "git maintenance barf" would result in

fatal: invalid subcommand: barf

with an exit code of 128 instead of 129 like the usage string does, so I'm not sure of the best way to make a custom "usage" error.

Thanks,
-Stolee

[snip quite a bit]

Yeah, that makes lots of sense. Thanks!

Martin

This series is based on jk/strvec.

This patch series contains 11patches that were going to be part of v4 of ds/maintenance [1], but the discussion has gotten really long. To help focus the conversation, I'm splitting out the portions that create and test the 'maintenance' builtin from the additional tasks (prefetch, loose-objects, incremental-repack) that can be brought in later.

[1]
https://lore.kernel.org/git/[email protected]/

As mentioned before, git gc already plays the role of maintaining Git repositories. It has accumulated several smaller pieces in its long history, including:

1. Repacking all reachable objects into one pack-file (and deleting unreachable objects).
2. Packing refs.
3. Expiring reflogs.
4. Clearing rerere logs.
5. Updating the commit-graph file.
6. Pruning worktrees.

While expiring reflogs, clearing rererelogs, and deleting unreachable objects are suitable under the guise of "garbage collection", packing refs and updating the commit-graph file are not as obviously fitting. Further, these operations are "all or nothing" in that they rewrite almost all repository data, which does not perform well at extremely large scales. These operations can also be disruptive to foreground Git commands when git gc --auto triggers during routine use.

This series does not intend to change what git gc does, but instead create new choices for automatic maintenance activities, of which git gc remains the only one enabled by default.

The new maintenance tasks are:

* 'commit-graph' : write and verify a single layer of an incremental commit-graph.
* 'loose-objects' : prune packed loose objects, then create a new pack from a batch of loose objects.
* 'pack-files' : expire redundant packs from the multi-pack-index, then repack using the multi-pack-index's incremental repack strategy.
* 'prefetch' : fetch from each remote, storing the refs in 'refs/prefetch/ /'.

The only included tasks are the 'gc' and 'commit-graph' tasks. The rest will follow in a follow-up series. Including the 'commit-graph' task here allows us to build and test features like config settings and the --task= command-line argument.

These tasks are all disabled by default, but can be enabled with config options or run explicitly using "git maintenance run --task=". There are additional config options to allow customizing the conditions for which the tasks run during the '--auto' option.

Because 'gc' is implemented as a maintenance task, the most dramatic change
of this series is to convert the 'git gc --auto' calls into 'git maintenance run --auto' calls at the end of some Git commands. By default, the only change is that 'git gc --auto' will be run below an additional 'git maintenance' process.

The 'git maintenance' builtin has a 'run' subcommand so it can be extended later with subcommands that manage background maintenance, such as 'start' or 'stop'. These are not the subject of this series, as it is important to focus on the maintenance activities themselves.

Updates since v1 (of this series)
=================================

* Documentation fixes.

* The builtin code had some slight tweaks in PATCH 1.

UPDATES since v3 of [1]
=======================

* The biggest change here is the use of "test_subcommand", based on Jonathan Nieder's approach. This requires having the exact command-line figured out, which now requires spelling out all --no- [quiet%7Cprogress] options. I also added a bunch of "2>/dev/null" checks because of the isatty(2) calls. Without that, the behavior will change depending on whether the test is run with -x/-v or without.

* The option parsing has changed to use a local struct and pass that struct to the helper methods. This is instead of having a global singleton.

Thanks, -Stolee

Derrick Stolee (11):
maintenance: create basic maintenance runner
maintenance: add --quiet option
maintenance: replace run_auto_gc()
maintenance: initialize task array
maintenance: add commit-graph task
maintenance: add --task option
maintenance: take a lock on the objects directory
maintenance: create maintenance.<task>.enabled config
maintenance: use pointers to check --auto
maintenance: add auto condition for commit-graph task
maintenance: add trace2 regions for task execution

 .gitignore                           |   1 +
 Documentation/config.txt             |   2 +
 Documentation/config/maintenance.txt |  14 ++
 Documentation/fetch-options.txt      |   6 +-
 Documentation/git-clone.txt          |   6 +-
 Documentation/git-maintenance.txt    |  79 ++++++
 builtin.h                            |   1 +
 builtin/am.c                         |   2 +-
 builtin/commit.c                     |   2 +-
 builtin/fetch.c                      |   6 +-
 builtin/gc.c                         | 354 +++++++++++++++++++++++++++
 builtin/merge.c                      |   2 +-
 builtin/rebase.c                     |   4 +-
 commit-graph.c                       |   8 +-
 commit-graph.h                       |   1 +
 git.c                                |   1 +
 object.h                             |   1 +
 run-command.c                        |  16 +-
 run-command.h                        |   2 +-
 t/t5510-fetch.sh                     |   2 +-
 t/t5514-fetch-multiple.sh            |   2 +-
 t/t7900-maintenance.sh               |  63 +++++
 t/test-lib-functions.sh              |  33 +++
 23 files changed, 580 insertions(+), 28 deletions(-)
 create mode 100644 Documentation/config/maintenance.txt
 create mode 100644 Documentation/git-maintenance.txt
 create mode 100755 t/t7900-maintenance.sh

base-commit: d70a9eb611a9d242c1d26847d223b8677609305b
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-695%2Fderrickstolee%2Fmaintenance%2Fbuiltin-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-695/derrickstolee/maintenance/builtin-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/695

Range-diff vs v1:

1: 2b9deb6d6a ! 1: e09e4a4a87 maintenance: create basic maintenance runner

     @@ Documentation/git-maintenance.txt (new)
      +-----------
      +Run tasks to optimize Git repository data, speeding up other Git commands
      +and reducing storage requirements for the repository.
     -++
     ++
      +Git commands that add repository data, such as `git add` or `git fetch`,
      +are optimized for a responsive user experience. These commands do not take
      +time to optimize the Git data, since such optimizations scale with the full
      +size of the repository while these user commands each perform a relatively
      +small action.
     -++
     ++
      +The `git maintenance` command provides flexibility for how to optimize the
      +Git repository.
      +
     @@ Documentation/git-maintenance.txt (new)
      +-----
      +
      +gc::
     -+	Cleanup unnecessary files and optimize the local repository. "GC"
     ++	Clean up unnecessary files and optimize the local repository. "GC"
      +	stands for "garbage collection," but this task performs many
     -+	smaller tasks. This task can be rather expensive for large
     -+	repositories, as it repacks all Git objects into a single pack-file.
     -+	It can also be disruptive in some situations, as it deletes stale
     -+	data.
     ++	smaller tasks. This task can be expensive for large repositories,
     ++	as it repacks all Git objects into a single pack-file. It can also
     ++	be disruptive in some situations, as it deletes stale data. See
     ++	linkgit:git-gc[1] for more details on garbage collection in Git.
      +
      +OPTIONS
      +-------
     @@ builtin/gc.c: int cmd_gc(int argc, const char **argv, const char *prefix)
      +
      +int cmd_maintenance(int argc, const char **argv, const char *prefix)
      +{
     -+	static struct maintenance_opts opts;
     -+	static struct option builtin_maintenance_options[] = {
     ++	struct maintenance_opts opts;
     ++	struct option builtin_maintenance_options[] = {
      +		OPT_BOOL(0, "auto", &opts.auto_flag,
      +			 N_("run tasks based on the state of the repository")),
      +		OPT_END()
     @@ builtin/gc.c: int cmd_gc(int argc, const char **argv, const char *prefix)
      +			     builtin_maintenance_usage,
      +			     PARSE_OPT_KEEP_UNKNOWN);
      +
     -+	if (argc == 1) {
     -+		if (!strcmp(argv[0], "run"))
     -+			return maintenance_run(&opts);
     -+	}
     ++	if (argc != 1)
     ++		usage_with_options(builtin_maintenance_usage,
     ++				   builtin_maintenance_options);
     ++
     ++	if (!strcmp(argv[0], "run"))
     ++		return maintenance_run(&opts);
      +
     -+	usage_with_options(builtin_maintenance_usage,
     -+			   builtin_maintenance_options);
     ++	die(_("invalid subcommand: %s"), argv[0]);
      +}
      
       ## git.c ##
     @@ t/t7900-maintenance.sh (new)
      +
      +test_expect_success 'help text' '
      +	test_expect_code 129 git maintenance -h 2>err &&
     -+	test_i18ngrep "usage: git maintenance run" err
     ++	test_i18ngrep "usage: git maintenance run" err &&
     ++	test_expect_code 128 git maintenance barf 2>err &&
     ++	test_i18ngrep "invalid subcommand: barf" err
      +'
      +
      +test_expect_success 'run [--auto]' '

2: d5faef26af ! 2: adae48d235 maintenance: add --quiet option
@@ builtin/gc.c: static int maintenance_task_gc(struct maintenance_opts *opts)
close_object_store(the_repository->objects);
return run_command(&child);
@@ builtin/gc.c: int cmd_maintenance(int argc, const char **argv, const char *prefix)
- static struct option builtin_maintenance_options[] = {
+ struct option builtin_maintenance_options[] = {
OPT_BOOL(0, "auto", &opts.auto_flag,
N_("run tasks based on the state of the repository")),
+ OPT_BOOL(0, "quiet", &opts.quiet,
@@ builtin/gc.c: int cmd_maintenance(int argc, const char **argv, const char *prefi

## t/t7900-maintenance.sh ##
@@ t/t7900-maintenance.sh: test_expect_success 'help text' '
- test_i18ngrep "usage: git maintenance run" err
+ test_i18ngrep "invalid subcommand: barf" err
'

-test_expect_success 'run [--auto]' '

  3:  233811310b =  3:  91741a0cfc maintenance: replace run_auto_gc()
  4:  7efa23abc8 =  4:  1db3b96280 maintenance: initialize task array
  5:  902b742032 !  5:  50b457fd57 maintenance: add commit-graph task

@@ Documentation/git-maintenance.txt: run::
+ issue, then the chain file is removed and the commit-graph is
+ rewritten from scratch.
++
-+The verification only checks the top layer of the commit-graph chain. -+If the incremental write merged the new commits with at least one -+existing layer, then there is potential for on-disk corruption being -+carried forward into the new file. This will be noticed and the new -+commit-graph file will be clean as Git reparses the commit data from -+the object database.
-++
+The incremental write is safe to run alongside concurrent Git processes +since it will not expire .graph files that were in the previous +commit-graph-chain file. They will be deleted by a later run based on +the expiration delay.
+
gc::
- Cleanup unnecessary files and optimize the local repository. "GC"
+ Clean up unnecessary files and optimize the local repository. "GC"
stands for "garbage collection," but this task performs many

## builtin/gc.c ##
@@ t/t7900-maintenance.sh: test_description='git maintenance builtin'
+
test_expect_success 'help text' '
test_expect_code 129 git maintenance -h 2>err &&
- test_i18ngrep "usage: git maintenance run" err
+ test_i18ngrep "usage: git maintenance run" err &&
6: dddbcc4f3d ! 6: 85268bd53e maintenance: add --task option
@@ builtin/gc.c: static int maintenance_run(struct maintenance_opts *opts)
+
int cmd_maintenance(int argc, const char **argv, const char *prefix) {
- static struct maintenance_opts opts;
+ struct maintenance_opts opts;
@@ builtin/gc.c: int cmd_maintenance(int argc, const char **argv, const char *prefix)
N_("run tasks based on the state of the repository")),
OPT_BOOL(0, "quiet", &opts.quiet,

  7:  79af39be13 =  7:  6f86cfaa94 maintenance: take a lock on the objects directory
  8:  69bfc6a4b2 =  8:  5c0f9d69d1 maintenance: create maintenance.<task>.enabled config
  9:  df21bbb000 =  9:  68bf5bef4b maintenance: use pointers to check --auto
 10:  e67e259aef = 10:  fc097c389a maintenance: add auto condition for commit-graph task
 11:  a5d1914846 = 11:  46fbe161aa maintenance: add trace2 regions for task execution

gitgitgadget

From: Derrick Stolee <[email protected]>

Maintenance activities are commonly used as steps in larger scripts. Providing a '--quiet' option allows those scripts to be less noisy when run on a terminal window. Turn this mode on by default when stderr is not a terminal.

Pipe the option to the 'git gc' child process.

---
 Documentation/git-maintenance.txt |  3 +++
 builtin/gc.c                      |  9 +++++++++
 t/t7900-maintenance.sh            | 15 ++++++++++-----
 3 files changed, 22 insertions(+), 5 deletions(-)

index ff47fb3641..04fa0fe329 100644
@@ -52,6 +52,9 @@ OPTIONS
 	in the `gc.auto` config setting, or when the number of pack-files
 	exceeds the `gc.autoPackLimit` config setting.
 
+--quiet::
+	Do not report progress or other information over `stderr`.
+
 GIT
 ---
 Part of the linkgit:git[1] suite

index ce91c754ee..b75ddb780b 100644
@@ -707,6 +707,7 @@ static const char * const builtin_maintenance_usage[] = {
 
 struct maintenance_opts {
 	int auto_flag;
+	int quiet;
 };
 
 static int maintenance_task_gc(struct maintenance_opts *opts)

@@ -718,6 +719,10 @@ static int maintenance_task_gc(struct maintenance_opts *opts)
 
 	if (opts->auto_flag)
 		strvec_push(&child.args, "--auto");
+	if (opts->quiet)
+		strvec_push(&child.args, "--quiet");
+	else
+		strvec_push(&child.args, "--no-quiet");
 
 	close_object_store(the_repository->objects);
 	return run_command(&child);

@@ -734,6 +739,8 @@ int cmd_maintenance(int argc, const char **argv, const char *prefix)
 	struct option builtin_maintenance_options[] = {
 		OPT_BOOL(0, "auto", &opts.auto_flag,
 			 N_("run tasks based on the state of the repository")),
+		OPT_BOOL(0, "quiet", &opts.quiet,
+			 N_("do not report progress or other information over stderr")),
 		OPT_END()
 	};
 

@@ -743,6 +750,8 @@ int cmd_maintenance(int argc, const char **argv, const char *prefix)
 		usage_with_options(builtin_maintenance_usage,
 				   builtin_maintenance_options);
 
+	opts.quiet = !isatty(2);
+
 	argc = parse_options(argc, argv, prefix,
 			     builtin_maintenance_options,
 			     builtin_maintenance_usage,

index 14aa691f19..47d512464c 100755
@@ -11,11 +11,16 @@ test_expect_success 'help text' '
 	test_i18ngrep "invalid subcommand: barf" err
 '
 
-test_expect_success 'run [--auto]' '
-	GIT_TRACE2_EVENT="$(pwd)/run-no-auto.txt" git maintenance run &&
-	GIT_TRACE2_EVENT="$(pwd)/run-auto.txt" git maintenance run --auto &&
-	test_subcommand git gc <run-no-auto.txt &&
-	test_subcommand git gc --auto <run-auto.txt
+test_expect_success 'run [--auto|--quiet]' '
+	GIT_TRACE2_EVENT="$(pwd)/run-no-auto.txt" \
+		git maintenance run 2>/dev/null &&
+	GIT_TRACE2_EVENT="$(pwd)/run-auto.txt" \
+		git maintenance run --auto 2>/dev/null &&
+	GIT_TRACE2_EVENT="$(pwd)/run-no-quiet.txt" \
+		git maintenance run --no-quiet 2>/dev/null &&
+	test_subcommand git gc --quiet <run-no-auto.txt &&
+	test_subcommand git gc --auto --quiet <run-auto.txt &&
+	test_subcommand git gc --no-quiet <run-no-quiet.txt
 '
 
 test_done

gitgitgadget

From: Derrick Stolee <[email protected]>

The 'gc' builtin is our current entrypoint for automatically maintaining a repository. This one tool does many operations, such as repacking the repository, packing refs, and rewriting the commit-graph file. The name implies it performs "garbage collection" which means several different things, and some users may not want to use this operation that rewrites the entire object database.

Create a new 'maintenance' builtin that will become a more general- purpose command. To start, it will only support the 'run' subcommand, but will later expand to add subcommands for scheduling maintenance in the background.

For now, the 'maintenance' builtin is a thin shim over the 'gc' builtin. In fact, the only option is the '--auto' toggle, which is handed directly to the 'gc' builtin. The current change is isolated to this simple operation to prevent more interesting logic from being lost in all of the boilerplate of adding a new builtin.

Use existing builtin/gc.c file because we want to share code between the two builtins. It is possible that we will have 'maintenance' replace the 'gc' builtin entirely at some point, leaving 'git gc' as an alias for some specific arguments to 'git maintenance run'.

Create a new test_subcommand helper that allows us to test if a certain subcommand was run. It requires storing the GIT_TRACE2_EVENT logs in a file. A negation mode is available that will be used in later tests.

---
 .gitignore                        |  1 +
 Documentation/git-maintenance.txt | 57 ++++++++++++++++++++++++++++++
 builtin.h                         |  1 +
 builtin/gc.c                      | 58 +++++++++++++++++++++++++++++++
 git.c                             |  1 +
 t/t7900-maintenance.sh            | 21 +++++++++++
 t/test-lib-functions.sh           | 33 ++++++++++++++++++
 7 files changed, 172 insertions(+)
 create mode 100644 Documentation/git-maintenance.txt
 create mode 100755 t/t7900-maintenance.sh

index ee509a2ad2..a5808fa30d 100644
@@ -90,6 +90,7 @@
 /git-ls-tree
 /git-mailinfo
 /git-mailsplit
+/git-maintenance
 /git-merge
 /git-merge-base
 /git-merge-index

index a5ae15bfe5..17c1c0ce49 100644
@@ -167,6 +167,7 @@ int cmd_ls_tree(int argc, const char **argv, const char *prefix);
 int cmd_ls_remote(int argc, const char **argv, const char *prefix);
 int cmd_mailinfo(int argc, const char **argv, const char *prefix);
 int cmd_mailsplit(int argc, const char **argv, const char *prefix);
+int cmd_maintenance(int argc, const char **argv, const char *prefix);
 int cmd_merge(int argc, const char **argv, const char *prefix);
 int cmd_merge_base(int argc, const char **argv, const char *prefix);
 int cmd_merge_index(int argc, const char **argv, const char *prefix);

index 8bd1d7551d..24f250d29a 100644
@@ -529,6 +529,7 @@ static struct cmd_struct commands[] = {
 	{ "ls-tree", cmd_ls_tree, RUN_SETUP },
 	{ "mailinfo", cmd_mailinfo, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "mailsplit", cmd_mailsplit, NO_PARSEOPT },
+	{ "maintenance", cmd_maintenance, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "merge", cmd_merge, RUN_SETUP | NEED_WORK_TREE },
 	{ "merge-base", cmd_merge_base, RUN_SETUP },
 	{ "merge-file", cmd_merge_file, RUN_SETUP_GENTLY },

new file mode 100755
index 0000000000..14aa691f19
@@ -0,0 +1,21 @@
+#!/bin/sh
+
+test_description='git maintenance builtin'
+
+. ./test-lib.sh
+
+test_expect_success 'help text' '
+	test_expect_code 129 git maintenance -h 2>err &&
+	test_i18ngrep "usage: git maintenance run" err &&
+	test_expect_code 128 git maintenance barf 2>err &&
+	test_i18ngrep "invalid subcommand: barf" err
+'
+
+test_expect_success 'run [--auto]' '
+	GIT_TRACE2_EVENT="$(pwd)/run-no-auto.txt" git maintenance run &&
+	GIT_TRACE2_EVENT="$(pwd)/run-auto.txt" git maintenance run --auto &&
+	test_subcommand git gc <run-no-auto.txt &&
+	test_subcommand git gc --auto <run-auto.txt
+'
+
+test_done

index 3103be8a32..0adf2b85f8 100644
@@ -1561,3 +1561,36 @@ test_path_is_hidden () {
 	case "$("$SYSTEMROOT"/system32/attrib "$1")" in *H*?:*) return 0;; esac
 	return 1
 }
+
+# Check that the given command was invoked as part of the
+# trace2-format trace on stdin.
+#
+#	test_subcommand [!] <command> <args>... < <trace>
+#
+# For example, to look for an invocation of "git upload-pack
+# /path/to/repo"
+#
+#	GIT_TRACE2_EVENT=event.log git fetch ... &&
+#	test_subcommand git upload-pack "$PATH" <event.log
+#
+# If the first parameter passed is !, this instead checks that
+# the given command was not called.
+#
+test_subcommand () {
+	local negate=
+	if test "$1" = "!"
+	then
+		negate=t
+		shift
+	fi
+
+	local expr=$(printf '"%s",' "$@")
+	expr="${expr%,}"
+
+	if test -n "$negate"
+	then
+		! grep "\[$expr\]"
+	else
+		grep "\[$expr\]"
+	fi
+}

gitgitgadget

From: Derrick Stolee <[email protected]>

The run_auto_gc() method is used in several places to trigger a check for repo maintenance after some Git commands, such as 'git commit' or 'git fetch'.

To allow for extra customization of this maintenance activity, replace the 'git gc --auto [--quiet]' call with one to 'git maintenance run --auto [--quiet]'. As we extend the maintenance builtin with other steps, users will be able to select different maintenance activities.

Rename run_auto_gc() to run_auto_maintenance() to be clearer what is happening on this call, and to expose all callers in the current diff. Rewrite the method to use a struct child_process to simplify the calls slightly.

Since 'git fetch' already allows disabling the 'git gc --auto' subprocess, add an equivalent option with a different name to be more descriptive of the new behavior: '--[no-]maintenance'. Update the documentation to include these options at the same time.

---
 Documentation/fetch-options.txt |  6 ++++--
 Documentation/git-clone.txt     |  6 +++---
 builtin/am.c                    |  2 +-
 builtin/commit.c                |  2 +-
 builtin/fetch.c                 |  6 ++++--
 builtin/merge.c                 |  2 +-
 builtin/rebase.c                |  4 ++--
 run-command.c                   | 16 +++++++---------
 run-command.h                   |  2 +-
 t/t5510-fetch.sh                |  2 +-
 10 files changed, 25 insertions(+), 23 deletions(-)

index 6e2a160a47..495bc8ab5a 100644
@@ -86,9 +86,11 @@ ifndef::git-pull[]
 	Allow several <repository> and <group> arguments to be
 	specified. No <refspec>s may be specified.
 
+--[no-]auto-maintenance::
 --[no-]auto-gc::
-	Run `git gc --auto` at the end to perform garbage collection
-	if needed. This is enabled by default.
+	Run `git maintenance run --auto` at the end to perform automatic
+	repository maintenance if needed. (`--[no-]auto-gc` is a synonym.)
+	This is enabled by default.
 
 --[no-]write-commit-graph::
 	Write a commit-graph after fetching. This overrides the config

index c898310099..097e6a86c5 100644
@@ -78,9 +78,9 @@ repository using this option and then delete branches (or use any
 other Git command that makes any existing commit unreferenced) in the
 source repository, some objects may become unreferenced (or dangling).
 These objects may be removed by normal Git operations (such as `git commit`)
-which automatically call `git gc --auto`. (See linkgit:git-gc[1].)
-If these objects are removed and were referenced by the cloned repository,
-then the cloned repository will become corrupt.
+which automatically call `git maintenance run --auto`. (See
+linkgit:git-maintenance[1].) If these objects are removed and were referenced
+by the cloned repository, then the cloned repository will become corrupt.
 +
 Note that running `git repack` without the `--local` option in a repository
 cloned with `--shared` will copy objects from the source repository into a pack

index dd4e6c2d9b..68e9d17379 100644
@@ -1795,7 +1795,7 @@ static void am_run(struct am_state *state, int resume)
 	if (!state->rebasing) {
 		am_destroy(state);
 		close_object_store(the_repository->objects);
-		run_auto_gc(state->quiet);
+		run_auto_maintenance(state->quiet);
 	}
 }
 

index 69ac78d5e5..f9b0a0c05d 100644
@@ -1702,7 +1702,7 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 	git_test_write_commit_graph_or_die();
 
 	repo_rerere(the_repository, 0);
-	run_auto_gc(quiet);
+	run_auto_maintenance(quiet);
 	run_commit_hook(use_editor, get_index_file(), "post-commit", NULL);
 	if (amend && !no_post_rewrite) {
 		commit_post_rewrite(the_repository, current_head, &oid);

index c49f0e9752..c8b9366d3c 100644
@@ -196,8 +196,10 @@ static struct option builtin_fetch_options[] = {
 	OPT_STRING_LIST(0, "negotiation-tip", &negotiation_tip, N_("revision"),
 			N_("report that we have only objects reachable from this object")),
 	OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options),
+	OPT_BOOL(0, "auto-maintenance", &enable_auto_gc,
+		 N_("run 'maintenance --auto' after fetching")),
 	OPT_BOOL(0, "auto-gc", &enable_auto_gc,
-		 N_("run 'gc --auto' after fetching")),
+		 N_("run 'maintenance --auto' after fetching")),
 	OPT_BOOL(0, "show-forced-updates", &fetch_show_forced_updates,
 		 N_("check for forced-updates on all updated branches")),
 	OPT_BOOL(0, "write-commit-graph", &fetch_write_commit_graph,

@@ -1882,7 +1884,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
 	close_object_store(the_repository->objects);
 
 	if (enable_auto_gc)
-		run_auto_gc(verbosity < 0);
+		run_auto_maintenance(verbosity < 0);
 
 	return result;
 }

index 7da707bf55..c068e73037 100644
@@ -457,7 +457,7 @@ static void finish(struct commit *head_commit,
 			 * user should see them.
 			 */
 			close_object_store(the_repository->objects);
-			run_auto_gc(verbosity < 0);
+			run_auto_maintenance(verbosity < 0);
 		}
 	}
 	if (new_head && show_diffstat) {

index dadb52fa92..9ffba9009d 100644
@@ -728,10 +728,10 @@ static int finish_rebase(struct rebase_options *opts)
 	apply_autostash(state_dir_path("autostash", opts));
 	close_object_store(the_repository->objects);
 	/*
-	 * We ignore errors in 'gc --auto', since the
+	 * We ignore errors in 'git maintenance run --auto', since the
 	 * user should see them.
 	 */
-	run_auto_gc(!(opts->flags & (REBASE_NO_QUIET|REBASE_VERBOSE)));
+	run_auto_maintenance(!(opts->flags & (REBASE_NO_QUIET|REBASE_VERBOSE)));
 	if (opts->type == REBASE_MERGE) {
 		struct replay_opts replay = REPLAY_OPTS_INIT;
 

index cc9c3296ba..2ee59acdc8 100644
@@ -1866,15 +1866,13 @@ int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 	return result;
 }
 
-int run_auto_gc(int quiet)
+int run_auto_maintenance(int quiet)
 {
-	struct strvec argv_gc_auto = STRVEC_INIT;
-	int status;
+	struct child_process maint = CHILD_PROCESS_INIT;
 
-	strvec_pushl(&argv_gc_auto, "gc", "--auto", NULL);
-	if (quiet)
-		strvec_push(&argv_gc_auto, "--quiet");
-	status = run_command_v_opt(argv_gc_auto.v, RUN_GIT_CMD);
-	strvec_clear(&argv_gc_auto);
-	return status;
+	maint.git_cmd = 1;
+	strvec_pushl(&maint.args, "maintenance", "run", "--auto", NULL);
+	strvec_push(&maint.args, quiet ? "--quiet" : "--no-quiet");
+
+	return run_command(&maint);
 }

index 8b9bfaef16..6472b38bde 100644
@@ -221,7 +221,7 @@ int run_hook_ve(const char *const *env, const char *name, va_list args);
 /*
  * Trigger an auto-gc
  */
-int run_auto_gc(int quiet);
+int run_auto_maintenance(int quiet);
 
 #define RUN_COMMAND_NO_STDIN 1
 #define RUN_GIT_CMD	     2	/*If this is to be git sub-command */

index a66dbe0bde..9850ecde5d 100755
@@ -919,7 +919,7 @@ test_expect_success 'fetching with auto-gc does not lock up' '
 		git config fetch.unpackLimit 1 &&
 		git config gc.autoPackLimit 1 &&
 		git config gc.autoDetach false &&
-		GIT_ASK_YESNO="$D/askyesno" git fetch >fetch.out 2>&1 &&
+		GIT_ASK_YESNO="$D/askyesno" git fetch --verbose >fetch.out 2>&1 &&
 		test_i18ngrep "Auto packing the repository" fetch.out &&
 		! grep "Should I try again" fetch.out
 	)

gitgitgadget

From: Derrick Stolee <[email protected]>

In anticipation of implementing multiple maintenance tasks inside the 'maintenance' builtin, use a list of structs to describe the work to be done.

The struct maintenance_task stores the name of the task (as given by a future command-line argument) along with a function pointer to its implementation and a boolean for whether the step is enabled.

A list these structs are initialized with the full list of implemented tasks along with a default order. For now, this list only contains the "gc" task. This task is also the only task enabled by default.

The run subcommand will return a nonzero exit code if any task fails. However, it will attempt all tasks in its loop before returning with the failure. Also each failed task will send an error message.

---
 builtin/gc.c | 38 +++++++++++++++++++++++++++++++++++++-
 1 file changed, 37 insertions(+), 1 deletion(-)

index b75ddb780b..946d871d54 100644
@@ -728,9 +728,45 @@ static int maintenance_task_gc(struct maintenance_opts *opts)
 	return run_command(&child);
 }
 
+typedef int maintenance_task_fn(struct maintenance_opts *opts);
+
+struct maintenance_task {
+	const char *name;
+	maintenance_task_fn *fn;
+	unsigned enabled:1;
+};
+
+enum maintenance_task_label {
+	TASK_GC,
+
+	/* Leave as final value */
+	TASK__COUNT
+};
+
+static struct maintenance_task tasks[] = {
+	[TASK_GC] = {
+		"gc",
+		maintenance_task_gc,
+		1,
+	},
+};
+
 static int maintenance_run(struct maintenance_opts *opts)
 {
-	return maintenance_task_gc(opts);
+	int i;
+	int result = 0;
+
+	for (i = 0; i < TASK__COUNT; i++) {
+		if (!tasks[i].enabled)
+			continue;
+
+		if (tasks[i].fn(opts)) {
+			error(_("task '%s' failed"), tasks[i].name);
+			result = 1;
+		}
+	}
+
+	return result;
 }
 
 int cmd_maintenance(int argc, const char **argv, const char *prefix)

gitgitgadget

From: Derrick Stolee <[email protected]>

A user may want to only run certain maintenance tasks in a certain order. Add the --task=<task> option, which allows a user to specify an ordered list of tasks to run. These cannot be run multiple times, however.

Here is where our array of maintenance_task pointers becomes critical. We can sort the array of pointers based on the task order, but we do not want to move the struct data itself in order to preserve the hashmap references. We use the hashmap to match the --task=<task> arguments into the task struct data.

Keep in mind that the 'enabled' member of the maintenance_task struct is a placeholder for a future 'maintenance.<task>.enabled' config option. Thus, we use the 'enabled' member to specify which tasks are run when the user does not specify any --task=<task> arguments. The 'enabled' member should be ignored if --task=<task> appears.

---
 Documentation/git-maintenance.txt |  8 +++-
 builtin/gc.c                      | 61 +++++++++++++++++++++++++++++--
 t/t7900-maintenance.sh            | 27 ++++++++++++++
 3 files changed, 92 insertions(+), 4 deletions(-)

index c816fa1dcd..cf59eb258c 100644
@@ -30,7 +30,9 @@ SUBCOMMANDS
 -----------
 
 run::
-	Run one or more maintenance tasks.
+	Run one or more maintenance tasks. If one or more `--task=<task>`
+	options are specified, then those tasks are run in the provided
+	order. Otherwise, only the `gc` task is run.
 
 TASKS
 -----

@@ -66,6 +68,10 @@ OPTIONS
 --quiet::
 	Do not report progress or other information over `stderr`.
 
+--task=<task>::
+	If this option is specified one or more times, then only run the
+	specified tasks in the specified order.
+
 GIT
 ---
 Part of the linkgit:git[1] suite

index c0c4e6846e..792765aff7 100755
@@ -25,4 +25,31 @@ test_expect_success 'run [--auto|--quiet]' '
 	test_subcommand git gc --no-quiet <run-no-quiet.txt
 '
 
+test_expect_success 'run --task=<task>' '
+	GIT_TRACE2_EVENT="$(pwd)/run-commit-graph.txt" \
+		git maintenance run --task=commit-graph 2>/dev/null &&
+	GIT_TRACE2_EVENT="$(pwd)/run-gc.txt" \
+		git maintenance run --task=gc 2>/dev/null &&
+	GIT_TRACE2_EVENT="$(pwd)/run-commit-graph.txt" \
+		git maintenance run --task=commit-graph 2>/dev/null &&
+	GIT_TRACE2_EVENT="$(pwd)/run-both.txt" \
+		git maintenance run --task=commit-graph --task=gc 2>/dev/null &&
+	test_subcommand ! git gc --quiet <run-commit-graph.txt &&
+	test_subcommand git gc --quiet <run-gc.txt &&
+	test_subcommand git gc --quiet <run-both.txt &&
+	test_subcommand git commit-graph write --split --reachable --no-progress <run-commit-graph.txt &&
+	test_subcommand ! git commit-graph write --split --reachable --no-progress <run-gc.txt &&
+	test_subcommand git commit-graph write --split --reachable --no-progress <run-both.txt
+'
+
+test_expect_success 'run --task=bogus' '
+	test_must_fail git maintenance run --task=bogus 2>err &&
+	test_i18ngrep "is not a valid task" err
+'
+
+test_expect_success 'run --task duplicate' '
+	test_must_fail git maintenance run --task=gc --task=gc 2>err &&
+	test_i18ngrep "cannot be selected multiple times" err
+'
+
 test_done

gitgitgadget

From: Derrick Stolee <[email protected]>

The first new task in the 'git maintenance' builtin is the 'commit-graph' job. It is based on the sequence of events in the 'commit-graph' job in Scalar [1]. This sequence is as follows:

1. git commit-graph write --reachable --split
2. git commit-graph verify --shallow
3. If the verify succeeds, stop.
4. Delete the commit-graph-chain file.
5. git commit-graph write --reachable --split

By writing an incremental commit-graph file using the "--split" option we minimize the disruption from this operation. The default behavior is to merge layers until the new "top" layer is less than half the size of the layer below. This provides quick writes most of the time, with the longer writes following a power law distribution.

Most importantly, concurrent Git processes only look at the commit-graph-chain file for a very short amount of time, so they will verly likely not be holding a handle to the file when we try to replace it. (This only matters on Windows.)

If a concurrent process reads the old commit-graph-chain file, but our job expires some of the .graph files before they can be read, then those processes will see a warning message (but not fail). This could be avoided by a future update to use the --expire-time argument when writing the commit-graph.

By using 'git commit-graph verify --shallow' we can ensure that the file we just wrote is valid. This is an extra safety precaution that is faster than our 'write' subcommand. In the rare situation that the newest layer of the commit-graph is corrupt, we can "fix" the corruption by deleting the commit-graph-chain file and rewrite the full commit-graph as a new one-layer commit graph. This does not completely prevent _that_ file from being corrupt, but it does recompute the commit-graph by parsing commits from the object database. In our use of this step in Scalar and VFS for Git, we have only seen this issue arise because our microsoft/git fork reverted 43d3561 ("commit-graph write: don't die if the existing graph is corrupt" 2019-03-25) for a while to keep commit-graph writes very fast. We dropped the revert when updating to v2.23.0. The verify still has potential for catching corrupt data across the layer boundary: if the new file has commit X with parent Y in an old file but the commit ID for Y in the old file had a bitswap, then we will notice that in the 'verify' command.

[1] https://github.com/microsoft/scalar/blob/master/Scalar.Common/Maintenance/CommitGraphStep.cs

---
 Documentation/git-maintenance.txt | 11 ++++++
 builtin/gc.c                      | 63 +++++++++++++++++++++++++++++++
 commit-graph.c                    |  8 ++--
 commit-graph.h                    |  1 +
 t/t7900-maintenance.sh            |  2 +
 5 files changed, 81 insertions(+), 4 deletions(-)

index 04fa0fe329..c816fa1dcd 100644
@@ -35,6 +35,17 @@ run::
 TASKS
 -----
 
+commit-graph::
+	The `commit-graph` job updates the `commit-graph` files incrementally,
+	then verifies that the written data is correct. If the new layer has an
+	issue, then the chain file is removed and the `commit-graph` is
+	rewritten from scratch.
++
+The incremental write is safe to run alongside concurrent Git processes
+since it will not expire `.graph` files that were in the previous
+`commit-graph-chain` file. They will be deleted by a later run based on
+the expiration delay.
+
 gc::
 	Clean up unnecessary files and optimize the local repository. "GC"
 	stands for "garbage collection," but this task performs many

index 1af68c297d..9705d237e4 100644
@@ -172,7 +172,7 @@ static char *get_split_graph_filename(struct object_directory *odb,
 		       oid_hex);
 }
 
-static char *get_chain_filename(struct object_directory *odb)
+char *get_commit_graph_chain_filename(struct object_directory *odb)
 {
 	return xstrfmt("%s/info/commit-graphs/commit-graph-chain", odb->path);
 }

@@ -521,7 +521,7 @@ static struct commit_graph *load_commit_graph_chain(struct repository *r,
 	struct stat st;
 	struct object_id *oids;
 	int i = 0, valid = 1, count;
-	char *chain_name = get_chain_filename(odb);
+	char *chain_name = get_commit_graph_chain_filename(odb);
 	FILE *fp;
 	int stat_res;
 

@@ -1619,7 +1619,7 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx)
 	}
 
 	if (ctx->split) {
-		char *lock_name = get_chain_filename(ctx->odb);
+		char *lock_name = get_commit_graph_chain_filename(ctx->odb);
 
 		hold_lock_file_for_update_mode(&lk, lock_name,
 					       LOCK_DIE_ON_ERROR, 0444);

@@ -1996,7 +1996,7 @@ static void expire_commit_graphs(struct write_commit_graph_context *ctx)
 	if (ctx->split_opts && ctx->split_opts->expire_time)
 		expire_time = ctx->split_opts->expire_time;
 	if (!ctx->split) {
-		char *chain_file_name = get_chain_filename(ctx->odb);
+		char *chain_file_name = get_commit_graph_chain_filename(ctx->odb);
 		unlink(chain_file_name);
 		free(chain_file_name);
 		ctx->num_commit_graphs_after = 0;

index 28f89cdf3e..3c202748c3 100644
@@ -25,6 +25,7 @@ struct commit;
 struct bloom_filter_settings;
 
 char *get_commit_graph_filename(struct object_directory *odb);
+char *get_commit_graph_chain_filename(struct object_directory *odb);
 int open_commit_graph(const char *graph_file, int *fd, struct stat *st);
 
 /*

index 47d512464c..c0c4e6846e 100755
@@ -4,6 +4,8 @@ test_description='git maintenance builtin'
 
 . ./test-lib.sh
 
+GIT_TEST_COMMIT_GRAPH=0
+
 test_expect_success 'help text' '
 	test_expect_code 129 git maintenance -h 2>err &&
 	test_i18ngrep "usage: git maintenance run" err &&

gitgitgadget

From: Derrick Stolee <[email protected]>

Currently, a normal run of "git maintenance run" will only run the 'gc' task, as it is the only one enabled. This is mostly for backwards- compatible reasons since "git maintenance run --auto" commands replaced previous "git gc --auto" commands after some Git processes. Users could manually run specific maintenance tasks by calling "git maintenance run --task=<task>" directly.

Allow users to customize which steps are run automatically using config. The 'maintenance.<task>.enabled' option then can turn on these other tasks (or turn off the 'gc' task).

---
 Documentation/config.txt             |  2 ++
 Documentation/config/maintenance.txt |  4 ++++
 Documentation/git-maintenance.txt    |  8 +++++---
 builtin/gc.c                         | 19 +++++++++++++++++++
 t/t7900-maintenance.sh               |  8 ++++++++
 5 files changed, 38 insertions(+), 3 deletions(-)
 create mode 100644 Documentation/config/maintenance.txt

index ef0768b91a..2783b825f9 100644
@@ -396,6 +396,8 @@ include::config/mailinfo.txt[]
 
 include::config/mailmap.txt[]
 
+include::config/maintenance.txt[]
+
 include::config/man.txt[]
 
 include::config/merge.txt[]

new file mode 100644
index 0000000000..370cbfb42f
@@ -0,0 +1,4 @@
+maintenance.<task>.enabled::
+	This boolean config option controls whether the maintenance task
+	with name `<task>` is run when no `--task` option is specified.
+	By default, only `maintenance.gc.enabled` is true.

index cf59eb258c..9af08c644f 100644
@@ -30,9 +30,11 @@ SUBCOMMANDS
 -----------
 
 run::
-	Run one or more maintenance tasks. If one or more `--task=<task>`
-	options are specified, then those tasks are run in the provided
-	order. Otherwise, only the `gc` task is run.
+	Run one or more maintenance tasks. If one or more `--task` options
+	are specified, then those tasks are run in that order. Otherwise,
+	the tasks are determined by which `maintenance.<task>.enabled`
+	config options are true. By default, only `maintenance.gc.enabled`
+	is true.
 
 TASKS
 -----

index a47bf64623..e66326ede9 100644
@@ -873,6 +873,24 @@ static int maintenance_run(struct maintenance_opts *opts)
 	return result;
 }
 
+static void initialize_task_config(void)
+{
+	int i;
+	struct strbuf config_name = STRBUF_INIT;
+	for (i = 0; i < TASK__COUNT; i++) {
+		int config_value;
+
+		strbuf_setlen(&config_name, 0);
+		strbuf_addf(&config_name, "maintenance.%s.enabled",
+			    tasks[i].name);
+
+		if (!git_config_get_bool(config_name.buf, &config_value))
+			tasks[i].enabled = config_value;
+	}
+
+	strbuf_release(&config_name);
+}
+
 static int task_option_parse(const struct option *opt,
 			     const char *arg, int unset)
 {

@@ -925,6 +943,7 @@ int cmd_maintenance(int argc, const char **argv, const char *prefix)
 				   builtin_maintenance_options);
 
 	opts.quiet = !isatty(2);
+	initialize_task_config();
 
 	argc = parse_options(argc, argv, prefix,
 			     builtin_maintenance_options,

index 792765aff7..290abb7832 100755
@@ -25,6 +25,14 @@ test_expect_success 'run [--auto|--quiet]' '
 	test_subcommand git gc --no-quiet <run-no-quiet.txt
 '
 
+test_expect_success 'maintenance.<task>.enabled' '
+	git config maintenance.gc.enabled false &&
+	git config maintenance.commit-graph.enabled true &&
+	GIT_TRACE2_EVENT="$(pwd)/run-config.txt" git maintenance run 2>err &&
+	test_subcommand ! git gc --quiet <run-config.txt &&
+	test_subcommand git commit-graph write --split --reachable --no-progress <run-config.txt
+'
+
 test_expect_success 'run --task=<task>' '
 	GIT_TRACE2_EVENT="$(pwd)/run-commit-graph.txt" \
 		git maintenance run --task=commit-graph 2>/dev/null &&

gitgitgadget

From: Derrick Stolee <[email protected]>

Instead of writing a new commit-graph in every 'git maintenance run --auto' process (when maintenance.commit-graph.enalbed is configured to be true), only write when there are "enough" commits not in a commit-graph file.

This count is controlled by the maintenance.commit-graph.auto config option.

To compute the count, use a depth-first search starting at each ref, and leaving markers using the PARENT1 flag. If this count reaches the limit, then terminate early and start the task. Otherwise, this operation will peel every ref and parse the commit it points to. If these are all in the commit-graph, then this is typically a very fast operation. Users with many refs might feel a slow-down, and hence could consider updating their limit to be very small. A negative value will force the step to run every time.

---
 Documentation/config/maintenance.txt | 10 ++++
 builtin/gc.c                         | 76 ++++++++++++++++++++++++++++
 object.h                             |  1 +
 3 files changed, 87 insertions(+)

index 370cbfb42f..9bd69b9df3 100644
@@ -2,3 +2,13 @@ maintenance.<task>.enabled::
 	This boolean config option controls whether the maintenance task
 	with name `<task>` is run when no `--task` option is specified.
 	By default, only `maintenance.gc.enabled` is true.
+
+maintenance.commit-graph.auto::
+	This integer config option controls how often the `commit-graph` task
+	should be run as part of `git maintenance run --auto`. If zero, then
+	the `commit-graph` task will not run with the `--auto` option. A
+	negative value will force the task to run every time. Otherwise, a
+	positive value implies the command should run when the number of
+	reachable commits that are not in the commit-graph file is at least
+	the value of `maintenance.commit-graph.auto`. The default value is
+	100.

index 96a2105859..30d4e0f6a0 100644
@@ -74,6 +74,7 @@ struct object_array {
  * list-objects-filter.c:                                      21
  * builtin/fsck.c:           0--3
  * builtin/index-pack.c:                                     2021
+ * builtin/maintenance.c:                           16
  * builtin/pack-objects.c:                                   20
  * builtin/reflog.c:                   10--12
  * builtin/show-branch.c:    0-------------------------------------------26

gitgitgadget

From: Derrick Stolee <[email protected]>

---
 builtin/gc.c | 2 ++
 1 file changed, 2 insertions(+)

index 73e485bfc0..3fdb08655c 100644
@@ -953,10 +953,12 @@ static int maintenance_run(struct maintenance_opts *opts)
 		     !tasks[i].auto_condition()))
 			continue;
 
+		trace2_region_enter("maintenance", tasks[i].name, r);
 		if (tasks[i].fn(opts)) {
 			error(_("task '%s' failed"), tasks[i].name);
 			result = 1;
 		}
+		trace2_region_leave("maintenance", tasks[i].name, r);
 	}
 
 	rollback_lock_file(&lk);

gitgitgadget

From: Derrick Stolee <[email protected]>

The 'git maintenance run' command has an '--auto' option. This is used by other Git commands such as 'git commit' or 'git fetch' to check if maintenance should be run after adding data to the repository.

Previously, this --auto option was only used to add the argument to the 'git gc' command as part of the 'gc' task. We will be expanding the other tasks to perform a check to see if they should do work as part of the --auto flag, when they are enabled by config.

First, update the 'gc' task to perform the auto check inside the maintenance process. This prevents running an extra 'git gc --auto' command when not needed. It also shows a model for other tasks.

Second, use the 'auto_condition' function pointer as a signal for whether we enable the maintenance task under '--auto'. For instance, we do not want to enable the 'fetch' task in '--auto' mode, so that function pointer will remain NULL.

Now that we are not automatically calling 'git gc', a test in t5514-fetch-multiple.sh must be changed to watch for 'git maintenance' instead.

We continue to pass the '--auto' option to the 'git gc' command when necessary, because of the gc.autoDetach config option changes behavior. Likely, we will want to absorb the daemonizing behavior implied by gc.autoDetach as a maintenance.autoDetach config option.

---
 builtin/gc.c              | 16 ++++++++++++++++
 t/t5514-fetch-multiple.sh |  2 +-
 t/t7900-maintenance.sh    |  2 +-
 3 files changed, 18 insertions(+), 2 deletions(-)

index e66326ede9..c54c3070b8 100644
@@ -788,9 +788,17 @@ static int maintenance_task_gc(struct maintenance_opts *opts)
 
 typedef int maintenance_task_fn(struct maintenance_opts *opts);
 
+/*
+ * An auto condition function returns 1 if the task should run
+ * and 0 if the task should NOT run. See needs_to_gc() for an
+ * example.
+ */
+typedef int maintenance_auto_fn(void);
+
 struct maintenance_task {
 	const char *name;
 	maintenance_task_fn *fn;
+	maintenance_auto_fn *auto_condition;
 	unsigned enabled:1,
 		 selected:1;
 	int selected_order;

@@ -808,6 +816,7 @@ static struct maintenance_task tasks[] = {
 	[TASK_GC] = {
 		"gc",
 		maintenance_task_gc,
+		need_to_gc,
 		1,
 	},
 	[TASK_COMMIT_GRAPH] = {

@@ -863,6 +872,11 @@ static int maintenance_run(struct maintenance_opts *opts)
 		if (!found_selected && !tasks[i].enabled)
 			continue;
 
+		if (opts->auto_flag &&
+		    (!tasks[i].auto_condition ||
+		     !tasks[i].auto_condition()))
+			continue;
+
 		if (tasks[i].fn(opts)) {
 			error(_("task '%s' failed"), tasks[i].name);
 			result = 1;

@@ -877,6 +891,8 @@ static void initialize_task_config(void)
 {
 	int i;
 	struct strbuf config_name = STRBUF_INIT;
+	gc_config();
+
 	for (i = 0; i < TASK__COUNT; i++) {
 		int config_value;
 

index de8e2f1531..bd202ec6f3 100755
@@ -108,7 +108,7 @@ test_expect_success 'git fetch --multiple (two remotes)' '
 	 GIT_TRACE=1 git fetch --multiple one two 2>trace &&
 	 git branch -r > output &&
 	 test_cmp ../expect output &&
-	 grep "built-in: git gc" trace >gc &&
+	 grep "built-in: git maintenance" trace >gc &&
 	 test_line_count = 1 gc
 	)
 '

index 290abb7832..4f6a04ddb1 100755
@@ -21,7 +21,7 @@ test_expect_success 'run [--auto|--quiet]' '
 	GIT_TRACE2_EVENT="$(pwd)/run-no-quiet.txt" \
 		git maintenance run --no-quiet 2>/dev/null &&
 	test_subcommand git gc --quiet <run-no-auto.txt &&
-	test_subcommand git gc --auto --quiet <run-auto.txt &&
+	test_subcommand ! git gc --auto --quiet <run-auto.txt &&
 	test_subcommand git gc --no-quiet <run-no-quiet.txt
 '
 

gitgitgadget

From: Derrick Stolee <[email protected]>

Performing maintenance on a Git repository involves writing data to the .git directory, which is not safe to do with multiple writers attempting the same operation. Ensure that only one 'git maintenance' process is running at a time by holding a file-based lock. Simply the presence of the .git/maintenance.lock file will prevent future maintenance. This lock is never committed, since it does not represent meaningful data. Instead, it is only a placeholder.

If the lock file already exists, then fail silently. This will become very important later when we implement the 'fetch' task, as this is our stop-gap from creating a recursive process loop between 'git fetch' and 'git maintenance run'.

---
 builtin/gc.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

index 0231d2f8c1..a47bf64623 100644
@@ -830,6 +830,25 @@ static int maintenance_run(struct maintenance_opts *opts)
 {
 	int i, found_selected = 0;
 	int result = 0;
+	struct lock_file lk;
+	struct repository *r = the_repository;
+	char *lock_path = xstrfmt("%s/maintenance", r->objects->odb->path);
+
+	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0) {
+		/*
+		 * Another maintenance command is running.
+		 *
+		 * If --auto was provided, then it is likely due to a
+		 * recursive process stack. Do not report an error in
+		 * that case.
+		 */
+		if (!opts->auto_flag && !opts->quiet)
+			error(_("lock file '%s' exists, skipping maintenance"),
+			      lock_path);
+		free(lock_path);
+		return 0;
+	}
+	free(lock_path);
 
 	for (i = 0; !found_selected && i < TASK__COUNT; i++)
 		found_selected = tasks[i].selected;

@@ -850,6 +869,7 @@ static int maintenance_run(struct maintenance_opts *opts)
 		}
 	}
 
+	rollback_lock_file(&lk);
 	return result;
 }
 

gitgitgadget

As jc/no-update-fetch-head topic seems to be getting popular for some reason, let's do this:

- recreate jc/no-update-fetch-head topic with patch [1/9] of the maintenance-2 series directly on top of 'master';

- fork ds/maintenance-part-1 topic off of jc/no-update-fetch-head and queue these 11 patches;

- fork ds/maintenance-part-2 topic off of ds/maintenance-part-1 and queue its 8 patches [2/9 - 9/9].

Thanks.

Better to say "will print an error message", I think.

Other than that, patches 1-4 look good to me.

I'm concerned about having this extra precaution, because it is never tested (neither here or in a future patch). When you saw this issue arise, was there ever an instance in which a valid set of commit graph files turned invalid after this maintenance step? (It seems from your description that the set was invalid to begin with, so the maintenance step did not fix it but also did not make it worse. And it does not make it worse, that seems not too bad to me.)

Other than that, this patch looks good to me.

Maybe "skip all maintenance steps silently"?

s/enalbed/enabled/

[snip]

Thanks. Overall this looks like a good patch set. As you can see from my other emails the only issues I have remaining are minor.

It is good to get rid of redundancies.

I had the same reaction as "(to me)" above during an earlier review round, IIRC. This definitely deserves documentation.

Sounds good. I will update my branches before their next versions.

Thanks,
-Stolee

The cases we've seen this happen were root-caused to hardware problems (disk or RAM), and the invalid data was present immediately after the "git commit-graph write" command. Since implementing the "verify" step after each write, we have not had any user reports of problems with these files.

Are you suggesting one of these options?

1. Remove this validation and rewrite entirely.

2. Remove the rewrite and only delete the known-bad data.

3. Insert a way to cause the verify to return failure mid-process, so that this function can be covered by tests.

Thanks,
-Stolee

It's hard to see from the patch-context, but there is a section titled "TASKS" that lists the 'gc' and 'commit-graph' tasks from the earlier patches. Would a reference to that section be helpful?

See the TASKS section for the list of available tasks.

Will do. Thanks.

-Stolee

Sorry, it is silent when "--auto" is specified. The commit message needs to reflect this properly. This could easily be downgraded to a warning.

Thanks,
-Stolee

For some reason I thought that there might be issues using SEEN (1u<<0) but trying it locally does not seem to be a problem. I think I've just been bit by how revision.c uses it a bit too often. The use here is independent enough to not cause problems.

I should have done this right in the first place. Thanks for catching it.

-Stolee

Ah, I missed that. What you have now is fine then, but a reference might be helpful if this man page ever grows.

I was suggesting 1, although 2 and 3 would assuage my concerns also (2 less so, but just deleting the file is relatively simple and easier to reason about). I don't think Git in general defends much against hardware problems, but if an issue is noticeable in some hardware (as is in this case, at least in the past) I can see why we would want to defend against it. My concern is that if we do defend against it, but leave it untested, several versions of Git later we might find that we need this defense but it does not work.

This series is based on jc/no-update-fetch-head.

This patch series contains 11patches that were going to be part of v4 of ds/maintenance [1], but the discussion has gotten really long. To help focus the conversation, I'm splitting out the portions that create and test the 'maintenance' builtin from the additional tasks (prefetch, loose-objects, incremental-repack) that can be brought in later.

[1]
https://lore.kernel.org/git/[email protected]/

As mentioned before, git gc already plays the role of maintaining Git repositories. It has accumulated several smaller pieces in its long history, including:

1. Repacking all reachable objects into one pack-file (and deleting unreachable objects).
2. Packing refs.
3. Expiring reflogs.
4. Clearing rerere logs.
5. Updating the commit-graph file.
6. Pruning worktrees.

While expiring reflogs, clearing rererelogs, and deleting unreachable objects are suitable under the guise of "garbage collection", packing refs and updating the commit-graph file are not as obviously fitting. Further, these operations are "all or nothing" in that they rewrite almost all repository data, which does not perform well at extremely large scales. These operations can also be disruptive to foreground Git commands when git gc --auto triggers during routine use.

This series does not intend to change what git gc does, but instead create new choices for automatic maintenance activities, of which git gc remains the only one enabled by default.

The new maintenance tasks are:

* 'commit-graph' : write and verify a single layer of an incremental commit-graph.
* 'loose-objects' : prune packed loose objects, then create a new pack from a batch of loose objects.
* 'pack-files' : expire redundant packs from the multi-pack-index, then repack using the multi-pack-index's incremental repack strategy.
* 'prefetch' : fetch from each remote, storing the refs in 'refs/prefetch/ /'.

The only included tasks are the 'gc' and 'commit-graph' tasks. The rest will follow in a follow-up series. Including the 'commit-graph' task here allows us to build and test features like config settings and the --task= command-line argument.

These tasks are all disabled by default, but can be enabled with config options or run explicitly using "git maintenance run --task=". There are additional config options to allow customizing the conditions for which the tasks run during the '--auto' option.

Because 'gc' is implemented as a maintenance task, the most dramatic change
of this series is to convert the 'git gc --auto' calls into 'git maintenance run --auto' calls at the end of some Git commands. By default, the only change is that 'git gc --auto' will be run below an additional 'git maintenance' process.

The 'git maintenance' builtin has a 'run' subcommand so it can be extended later with subcommands that manage background maintenance, such as 'start' or 'stop'. These are not the subject of this series, as it is important to focus on the maintenance activities themselves.

Updates since v2
================

* Based on jc/no-update-fetch-head instead of jk/strvec.

* I realized that the other maintenance subcommands should not accept the options for the 'run' subcommand, so I reorganized the option parsing into that subcommand. This makes the range-diff noisier than it would have been otherwise.

* While updating the parsing, I also updated the usage strings.

* The "verify, then delete and rewrite on failure" logic is removed. I'll consider bringing this back with a way to test the behavior in a later patch series.

* Other feedback from Jonathan Tan is applied.

Updates since v1 (of this series)
=================================

* Documentation fixes.

* The builtin code had some slight tweaks in PATCH 1.

UPDATES since v3 of [1]
=======================

* The biggest change here is the use of "test_subcommand", based on Jonathan Nieder's approach. This requires having the exact command-line figured out, which now requires spelling out all --no- [quiet%7Cprogress] options. I also added a bunch of "2>/dev/null" checks because of the isatty(2) calls. Without that, the behavior will change depending on whether the test is run with -x/-v or without.

* The option parsing has changed to use a local struct and pass that struct to the helper methods. This is instead of having a global singleton.

Thanks, -Stolee

Derrick Stolee (11):
maintenance: create basic maintenance runner
maintenance: add --quiet option
maintenance: replace run_auto_gc()
maintenance: initialize task array
maintenance: add commit-graph task
maintenance: add --task option
maintenance: take a lock on the objects directory
maintenance: create maintenance.<task>.enabled config
maintenance: use pointers to check --auto
maintenance: add auto condition for commit-graph task
maintenance: add trace2 regions for task execution

 .gitignore                           |   1 +
 Documentation/config.txt             |   2 +
 Documentation/config/maintenance.txt |  16 ++
 Documentation/fetch-options.txt      |   6 +-
 Documentation/git-clone.txt          |   6 +-
 Documentation/git-maintenance.txt    |  79 +++++++
 builtin.h                            |   1 +
 builtin/am.c                         |   2 +-
 builtin/commit.c                     |   2 +-
 builtin/fetch.c                      |   6 +-
 builtin/gc.c                         | 336 +++++++++++++++++++++++++++
 builtin/merge.c                      |   2 +-
 builtin/rebase.c                     |   4 +-
 command-list.txt                     |   1 +
 commit-graph.c                       |   8 +-
 commit-graph.h                       |   1 +
 git.c                                |   1 +
 object.h                             |   1 +
 run-command.c                        |  16 +-
 run-command.h                        |   2 +-
 t/t5510-fetch.sh                     |   2 +-
 t/t5514-fetch-multiple.sh            |   2 +-
 t/t7900-maintenance.sh               |  63 +++++
 t/test-lib-functions.sh              |  33 +++
 24 files changed, 565 insertions(+), 28 deletions(-)
 create mode 100644 Documentation/config/maintenance.txt
 create mode 100644 Documentation/git-maintenance.txt
 create mode 100755 t/t7900-maintenance.sh

base-commit: 887952b8c680626f4721cb5fa57704478801aca4
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-695%2Fderrickstolee%2Fmaintenance%2Fbuiltin-v3
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-695/derrickstolee/maintenance/builtin-v3
Pull-Request: https://github.com/gitgitgadget/git/pull/695

Range-diff vs v2:

  1:  e09e4a4a87 !  1:  aa961af387 maintenance: create basic maintenance runner
     @@ builtin/gc.c: int cmd_gc(int argc, const char **argv, const char *prefix)
       	return 0;
       }
      +
     -+static const char * const builtin_maintenance_usage[] = {
     -+	N_("git maintenance run [<options>]"),
     ++static const char * const builtin_maintenance_run_usage[] = {
     ++	N_("git maintenance run [--auto]"),
      +	NULL
      +};
      +
     -+struct maintenance_opts {
     ++struct maintenance_run_opts {
      +	int auto_flag;
      +};
      +
     -+static int maintenance_task_gc(struct maintenance_opts *opts)
     ++static int maintenance_task_gc(struct maintenance_run_opts *opts)
      +{
      +	struct child_process child = CHILD_PROCESS_INIT;
      +
     @@ builtin/gc.c: int cmd_gc(int argc, const char **argv, const char *prefix)
      +	return run_command(&child);
      +}
      +
     -+static int maintenance_run(struct maintenance_opts *opts)
     ++static int maintenance_run(int argc, const char **argv, const char *prefix)
      +{
     -+	return maintenance_task_gc(opts);
     -+}
     -+
     -+int cmd_maintenance(int argc, const char **argv, const char *prefix)
     -+{
     -+	struct maintenance_opts opts;
     -+	struct option builtin_maintenance_options[] = {
     ++	struct maintenance_run_opts opts;
     ++	struct option builtin_maintenance_run_options[] = {
      +		OPT_BOOL(0, "auto", &opts.auto_flag,
      +			 N_("run tasks based on the state of the repository")),
      +		OPT_END()
      +	};
     -+
      +	memset(&opts, 0, sizeof(opts));
      +
     -+	if (argc == 2 && !strcmp(argv[1], "-h"))
     -+		usage_with_options(builtin_maintenance_usage,
     -+				   builtin_maintenance_options);
     -+
      +	argc = parse_options(argc, argv, prefix,
     -+			     builtin_maintenance_options,
     -+			     builtin_maintenance_usage,
     -+			     PARSE_OPT_KEEP_UNKNOWN);
     ++			     builtin_maintenance_run_options,
     ++			     builtin_maintenance_run_usage,
     ++			     PARSE_OPT_STOP_AT_NON_OPTION);
     ++
     ++	if (argc != 0)
     ++		usage_with_options(builtin_maintenance_run_usage,
     ++				   builtin_maintenance_run_options);
     ++	return maintenance_task_gc(&opts);
     ++}
     ++
     ++static const char builtin_maintenance_usage[] = N_("git maintenance run [<options>]");
      +
     -+	if (argc != 1)
     -+		usage_with_options(builtin_maintenance_usage,
     -+				   builtin_maintenance_options);
     ++int cmd_maintenance(int argc, const char **argv, const char *prefix)
     ++{
     ++	if (argc == 2 && !strcmp(argv[1], "-h"))
     ++		usage(builtin_maintenance_usage);
      +
     -+	if (!strcmp(argv[0], "run"))
     -+		return maintenance_run(&opts);
     ++	if (!strcmp(argv[1], "run"))
     ++		return maintenance_run(argc - 1, argv + 1, prefix);
      +
     -+	die(_("invalid subcommand: %s"), argv[0]);
     ++	die(_("invalid subcommand: %s"), argv[1]);
      +}

+ ## command-list.txt ##
+@@ command-list.txt: git-ls-remote plumbinginterrogators
+ git-ls-tree plumbinginterrogators
+ git-mailinfo purehelpers
+ git-mailsplit purehelpers
++git-maintenance mainporcelain
+ git-merge mainporcelain history
+ git-merge-base plumbinginterrogators
+ git-merge-file plumbingmanipulators
+
## git.c ##
@@ git.c: static struct cmd_struct commands[] = {
{ "ls-tree", cmd_ls_tree, RUN_SETUP },
2: adae48d235 ! 2: 5386d8a628 maintenance: add --quiet option
@@ Documentation/git-maintenance.txt: OPTIONS
Part of the linkgit:git[1] suite

## builtin/gc.c ##
-@@ builtin/gc.c: static const char * const builtin_maintenance_usage[] = { +@@ builtin/gc.c: int cmd_gc(int argc, const char **argv, const char *prefix)
+ }

- struct maintenance_opts {
+ static const char * const builtin_maintenance_run_usage[] = {
+- N_("git maintenance run [--auto]"),
++ N_("git maintenance run [--auto] [--[no-]quiet]"),
+ NULL
+ };
+
+ struct maintenance_run_opts {
int auto_flag;
+ int quiet;
};

- static int maintenance_task_gc(struct maintenance_opts *opts)
-@@ builtin/gc.c: static int maintenance_task_gc(struct maintenance_opts *opts)
+ static int maintenance_task_gc(struct maintenance_run_opts *opts) +@@ builtin/gc.c: static int maintenance_task_gc(struct maintenance_run_opts *opts)

if (opts->auto_flag)
strvec_push(&child.args, "--auto");
@@ builtin/gc.c: static int maintenance_task_gc(struct maintenance_opts *opts)

close_object_store(the_repository->objects);
return run_command(&child);
-@@ builtin/gc.c: int cmd_maintenance(int argc, const char **argv, const char *prefix)
- struct option builtin_maintenance_options[] = {
+@@ builtin/gc.c: static int maintenance_run(int argc, const char **argv, const char *prefix)
+ struct option builtin_maintenance_run_options[] = {
OPT_BOOL(0, "auto", &opts.auto_flag,
N_("run tasks based on the state of the repository")),
+ OPT_BOOL(0, "quiet", &opts.quiet,
+ N_("do not report progress or other information over stderr")),
OPT_END()
};
-
-@@ builtin/gc.c: int cmd_maintenance(int argc, const char **argv, const char *prefix)
- usage_with_options(builtin_maintenance_usage,
- builtin_maintenance_options);
+ memset(&opts, 0, sizeof(opts));

+ opts.quiet = !isatty(2);
+
argc = parse_options(argc, argv, prefix,
- builtin_maintenance_options,
- builtin_maintenance_usage,
+ builtin_maintenance_run_options,
+ builtin_maintenance_run_usage,

## t/t7900-maintenance.sh ##
@@ t/t7900-maintenance.sh: test_expect_success 'help text' '

  3:  91741a0cfc =  3:  e28b332df4 maintenance: replace run_auto_gc()
  4:  1db3b96280 !  4:  82326c1c38 maintenance: initialize task array

@@ Commit message

The run subcommand will return a nonzero exit code if any task fails. However, it will attempt all tasks in its loop before returning with the
- failure. Also each failed task will send an error message.
+ failure. Also each failed task will print an error message.

## builtin/gc.c ##
-@@ builtin/gc.c: static int maintenance_task_gc(struct maintenance_opts *opts) +@@ builtin/gc.c: static int maintenance_task_gc(struct maintenance_run_opts *opts)
return run_command(&child);
}

-+typedef int maintenance_task_fn(struct maintenance_opts *opts);
++typedef int maintenance_task_fn(struct maintenance_run_opts *opts);
+
+struct maintenance_task {
+ const char *name;
@@ builtin/gc.c: static int maintenance_task_gc(struct maintenance_opts *opts)
+ },
+};
+
- static int maintenance_run(struct maintenance_opts *opts)
- {
-- return maintenance_task_gc(opts);
++static int maintenance_run_tasks(struct maintenance_run_opts *opts)
++{
+ int i;
+ int result = 0;
+
@@ builtin/gc.c: static int maintenance_task_gc(struct maintenance_opts *opts)
+ }
+
+ return result;
++}
++
+ static int maintenance_run(int argc, const char **argv, const char *prefix)
+ {
+ struct maintenance_run_opts opts;
+@@ builtin/gc.c: static int maintenance_run(int argc, const char **argv, const char *prefix)
+ if (argc != 0)
+ usage_with_options(builtin_maintenance_run_usage,
+ builtin_maintenance_run_options);
+- return maintenance_task_gc(&opts);
++ return maintenance_run_tasks(&opts);
}

- int cmd_maintenance(int argc, const char **argv, const char *prefix)
+ static const char builtin_maintenance_usage[] = N_("git maintenance run [<options>]");
5: 50b457fd57 ! 5: 06984a42bf maintenance: add commit-graph task

     @@ Commit message
          maintenance: add commit-graph task
      
          The first new task in the 'git maintenance' builtin is the
     -    'commit-graph' job. It is based on the sequence of events in the
     -    'commit-graph' job in Scalar [1]. This sequence is as follows:
     +    'commit-graph' task. This updates the commit-graph file
     +    incrementally with the command
      
     -    1. git commit-graph write --reachable --split
     -    2. git commit-graph verify --shallow
     -    3. If the verify succeeds, stop.
     -    4. Delete the commit-graph-chain file.
     -    5. git commit-graph write --reachable --split
     +            git commit-graph write --reachable --split
      
          By writing an incremental commit-graph file using the "--split"
          option we minimize the disruption from this operation. The default
     @@ Commit message
          This could be avoided by a future update to use the --expire-time
          argument when writing the commit-graph.
      
     -    By using 'git commit-graph verify --shallow' we can ensure that
     -    the file we just wrote is valid. This is an extra safety precaution
     -    that is faster than our 'write' subcommand. In the rare situation
     -    that the newest layer of the commit-graph is corrupt, we can "fix"
     -    the corruption by deleting the commit-graph-chain file and rewrite
     -    the full commit-graph as a new one-layer commit graph. This does
     -    not completely prevent _that_ file from being corrupt, but it does
     -    recompute the commit-graph by parsing commits from the object
     -    database. In our use of this step in Scalar and VFS for Git, we
     -    have only seen this issue arise because our microsoft/git fork
     -    reverted 43d3561 ("commit-graph write: don't die if the existing
     -    graph is corrupt" 2019-03-25) for a while to keep commit-graph
     -    writes very fast. We dropped the revert when updating to v2.23.0.
     -    The verify still has potential for catching corrupt data across
     -    the layer boundary: if the new file has commit X with parent Y
     -    in an old file but the commit ID for Y in the old file had a
     -    bitswap, then we will notice that in the 'verify' command.
     -
     -    [1] https://github.com/microsoft/scalar/blob/master/Scalar.Common/Maintenance/CommitGraphStep.cs
     -
          Signed-off-by: Derrick Stolee <[email protected]>
      
       ## Documentation/git-maintenance.txt ##
     @@ Documentation/git-maintenance.txt: run::
       
      +commit-graph::
      +	The `commit-graph` job updates the `commit-graph` files incrementally,
     -+	then verifies that the written data is correct. If the new layer has an
     -+	issue, then the chain file is removed and the `commit-graph` is
     -+	rewritten from scratch.
     -++
     -+The incremental write is safe to run alongside concurrent Git processes
     -+since it will not expire `.graph` files that were in the previous
     -+`commit-graph-chain` file. They will be deleted by a later run based on
     -+the expiration delay.
     ++	then verifies that the written data is correct. The incremental
     ++	write is safe to run alongside concurrent Git processes since it
     ++	will not expire `.graph` files that were in the previous
     ++	`commit-graph-chain` file. They will be deleted by a later run based
     ++	on the expiration delay.
      +
       gc::
       	Clean up unnecessary files and optimize the local repository. "GC"
       	stands for "garbage collection," but this task performs many
      
       ## builtin/gc.c ##
     -@@ builtin/gc.c: struct maintenance_opts {
     +@@ builtin/gc.c: struct maintenance_run_opts {
       	int quiet;
       };
       
     -+static int run_write_commit_graph(struct maintenance_opts *opts)
     ++static int run_write_commit_graph(struct maintenance_run_opts *opts)
      +{
      +	struct child_process child = CHILD_PROCESS_INIT;
      +
     @@ builtin/gc.c: struct maintenance_opts {
      +	return !!run_command(&child);
      +}
      +
     -+static int run_verify_commit_graph(struct maintenance_opts *opts)
     ++static int maintenance_task_commit_graph(struct maintenance_run_opts *opts)
      +{
     -+	struct child_process child = CHILD_PROCESS_INIT;
     -+
     -+	child.git_cmd = 1;
     -+	strvec_pushl(&child.args, "commit-graph", "verify",
     -+		     "--shallow", NULL);
     -+
     -+	if (opts->quiet)
     -+		strvec_push(&child.args, "--no-progress");
     -+
     -+	return !!run_command(&child);
     -+}
     -+
     -+static int maintenance_task_commit_graph(struct maintenance_opts *opts)
     -+{
     -+	struct repository *r = the_repository;
     -+	char *chain_path;
     -+
     -+	close_object_store(r->objects);
     ++	close_object_store(the_repository->objects);
      +	if (run_write_commit_graph(opts)) {
      +		error(_("failed to write commit-graph"));
      +		return 1;
      +	}
      +
     -+	if (!run_verify_commit_graph(opts))
     -+		return 0;
     -+
     -+	warning(_("commit-graph verify caught error, rewriting"));
     -+
     -+	chain_path = get_commit_graph_chain_filename(r->objects->odb);
     -+	if (unlink(chain_path)) {
     -+		UNLEAK(chain_path);
     -+		die(_("failed to remove commit-graph at %s"), chain_path);
     -+	}
     -+	free(chain_path);
     -+
     -+	if (!run_write_commit_graph(opts))
     -+		return 0;
     -+
     -+	error(_("failed to rewrite commit-graph"));
     -+	return 1;
     ++	return 0;
      +}
      +
     - static int maintenance_task_gc(struct maintenance_opts *opts)
     + static int maintenance_task_gc(struct maintenance_run_opts *opts)
       {
       	struct child_process child = CHILD_PROCESS_INIT;
      @@ builtin/gc.c: struct maintenance_task {
     @@ builtin/gc.c: static struct maintenance_task tasks[] = {
      +	},
       };
       
     - static int maintenance_run(struct maintenance_opts *opts)
     + static int maintenance_run_tasks(struct maintenance_run_opts *opts)
      
       ## commit-graph.c ##
      @@ commit-graph.c: static char *get_split_graph_filename(struct object_directory *odb,
     @@ commit-graph.c: static void expire_commit_graphs(struct write_commit_graph_conte
       		ctx->num_commit_graphs_after = 0;
      
       ## commit-graph.h ##
     -@@ commit-graph.h: struct commit;
     - struct bloom_filter_settings;
     +@@ commit-graph.h: struct raw_object_store;
     + struct string_list;
       
       char *get_commit_graph_filename(struct object_directory *odb);
      +char *get_commit_graph_chain_filename(struct object_directory *odb);
  6:  85268bd53e !  6:  69298aee24 maintenance: add --task option
     @@ Documentation/git-maintenance.txt: OPTIONS
       
      +--task=<task>::
      +	If this option is specified one or more times, then only run the
     -+	specified tasks in the specified order.
     ++	specified tasks in the specified order. See the 'TASKS' section
     ++	for the list of accepted `<task>` values.
      +
       GIT
       ---
       Part of the linkgit:git[1] suite
      
       ## builtin/gc.c ##
     -@@ builtin/gc.c: typedef int maintenance_task_fn(struct maintenance_opts *opts);
     - struct maintenance_task {
     +@@ builtin/gc.c: int cmd_gc(int argc, const char **argv, const char *prefix)
     + }
     + 
     + static const char * const builtin_maintenance_run_usage[] = {
     +-	N_("git maintenance run [--auto] [--[no-]quiet]"),
     ++	N_("git maintenance run [--auto] [--[no-]quiet] [--task=<task>]"),
     + 	NULL
     + };
     + 
     +@@ builtin/gc.c: struct maintenance_task {
       	const char *name;
       	maintenance_task_fn *fn;
     --	unsigned enabled:1;
     -+	unsigned enabled:1,
     -+		 selected:1;
     + 	unsigned enabled:1;
     ++
     ++	/* -1 if not selected. */
      +	int selected_order;
       };
       
     @@ builtin/gc.c: static struct maintenance_task tasks[] = {
      +	return b->selected_order - a->selected_order;
      +}
      +
     - static int maintenance_run(struct maintenance_opts *opts)
     + static int maintenance_run_tasks(struct maintenance_run_opts *opts)
       {
      -	int i;
      +	int i, found_selected = 0;
       	int result = 0;
       
      +	for (i = 0; !found_selected && i < TASK__COUNT; i++)
     -+		found_selected = tasks[i].selected;
     ++		found_selected = tasks[i].selected_order >= 0;
      +
      +	if (found_selected)
      +		QSORT(tasks, TASK__COUNT, compare_tasks_by_selection);
      +
       	for (i = 0; i < TASK__COUNT; i++) {
      -		if (!tasks[i].enabled)
     -+		if (found_selected && !tasks[i].selected)
     ++		if (found_selected && tasks[i].selected_order < 0)
      +			continue;
      +
      +		if (!found_selected && !tasks[i].enabled)
       			continue;
       
       		if (tasks[i].fn(opts)) {
     -@@ builtin/gc.c: static int maintenance_run(struct maintenance_opts *opts)
     +@@ builtin/gc.c: static int maintenance_run_tasks(struct maintenance_run_opts *opts)
       	return result;
       }
       
     @@ builtin/gc.c: static int maintenance_run(struct maintenance_opts *opts)
      +	BUG_ON_OPT_NEG(unset);
      +
      +	for (i = 0; i < TASK__COUNT; i++) {
     -+		num_selected += tasks[i].selected;
     ++		if (tasks[i].selected_order >= 0)
     ++			num_selected++;
      +		if (!strcasecmp(tasks[i].name, arg)) {
      +			task = &tasks[i];
      +		}
     @@ builtin/gc.c: static int maintenance_run(struct maintenance_opts *opts)
      +		return 1;
      +	}
      +
     -+	if (task->selected) {
     ++	if (task->selected_order >= 0) {
      +		error(_("task '%s' cannot be selected multiple times"), arg);
      +		return 1;
      +	}
      +
     -+	task->selected = 1;
      +	task->selected_order = num_selected + 1;
      +
      +	return 0;
      +}
      +
     - int cmd_maintenance(int argc, const char **argv, const char *prefix)
     + static int maintenance_run(int argc, const char **argv, const char *prefix)
       {
     - 	struct maintenance_opts opts;
     -@@ builtin/gc.c: int cmd_maintenance(int argc, const char **argv, const char *prefix)
     ++	int i;
     + 	struct maintenance_run_opts opts;
     + 	struct option builtin_maintenance_run_options[] = {
     + 		OPT_BOOL(0, "auto", &opts.auto_flag,
       			 N_("run tasks based on the state of the repository")),
       		OPT_BOOL(0, "quiet", &opts.quiet,
       			 N_("do not report progress or other information over stderr")),
     @@ builtin/gc.c: int cmd_maintenance(int argc, const char **argv, const char *prefi
      +			PARSE_OPT_NONEG, task_option_parse),
       		OPT_END()
       	};
     + 	memset(&opts, 0, sizeof(opts));
     + 
     + 	opts.quiet = !isatty(2);
       
     ++	for (i = 0; i < TASK__COUNT; i++)
     ++		tasks[i].selected_order = -1;
     ++
     + 	argc = parse_options(argc, argv, prefix,
     + 			     builtin_maintenance_run_options,
     + 			     builtin_maintenance_run_usage,
      
       ## t/t7900-maintenance.sh ##
      @@ t/t7900-maintenance.sh: test_expect_success 'run [--auto|--quiet]' '

7: 6f86cfaa94 ! 7: 3d513acdd8 maintenance: take a lock on the objects directory
@@ Commit message
lock is never committed, since it does not represent meaningful data. Instead, it is only a placeholder.

- If the lock file already exists, then fail silently. This will become
- very important later when we implement the 'fetch' task, as this is our
- stop-gap from creating a recursive process loop between 'git fetch' and
- 'git maintenance run'.
+ If the lock file already exists, then fail with a warning. If '--auto'
+ is specified, then instead no warning is shown and no tasks are attempted.
+ This will become very important later when we implement the 'prefetch'
+ task, as this is our stop-gap from creating a recursive process loop
+ between 'git fetch' and 'git maintenance run --auto'.

## builtin/gc.c ##
-@@ builtin/gc.c: static int maintenance_run(struct maintenance_opts *opts) +@@ builtin/gc.c: static int maintenance_run_tasks(struct maintenance_run_opts *opts)
{
int i, found_selected = 0;
int result = 0;
@@ builtin/gc.c: static int maintenance_run(struct maintenance_opts *opts)
+ * that case.
+ */
+ if (!opts->auto_flag && !opts->quiet)
-+ error(_("lock file '%s' exists, skipping maintenance"),
-+ lock_path);
++ warning(_("lock file '%s' exists, skipping maintenance"),
++ lock_path);
+ free(lock_path);
+ return 0;
+ }
+ free(lock_path);

for (i = 0; !found_selected && i < TASK__COUNT; i++)
- found_selected = tasks[i].selected;
-@@ builtin/gc.c: static int maintenance_run(struct maintenance_opts *opts)
+ found_selected = tasks[i].selected_order >= 0;
+@@ builtin/gc.c: static int maintenance_run_tasks(struct maintenance_run_opts *opts)
}
}

8: 5c0f9d69d1 ! 8: 712f5f2d8e maintenance: create maintenance.<task>.enabled config
@@ Documentation/config/maintenance.txt (new)
@@
+maintenance.<task>.enabled::
+ This boolean config option controls whether the maintenance task
-+ with name <task> is run when no --task option is specified.
-+ By default, only maintenance.gc.enabled is true.
++ with name <task> is run when no --task option is specified to
++ git maintenance run. These config values are ignored if a
++ --task option exists. By default, only maintenance.gc.enabled
++ is true.

## Documentation/git-maintenance.txt ##
@@ Documentation/git-maintenance.txt: SUBCOMMANDS
@@ Documentation/git-maintenance.txt: SUBCOMMANDS

TASKS
-----
+@@ Documentation/git-maintenance.txt: OPTIONS +
+ --task=<task>::
+ If this option is specified one or more times, then only run the
+- specified tasks in the specified order. See the 'TASKS' section
+- for the list of accepted <task> values.
++ specified tasks in the specified order. If no --task=<task>
++ arguments are specified, then only the tasks with
++ maintenance.<task>.enabled configured as true are considered.
++ See the 'TASKS' section for the list of accepted <task> values.
+
+ GIT
+ ---

## builtin/gc.c ##
-@@ builtin/gc.c: static int maintenance_run(struct maintenance_opts *opts) +@@ builtin/gc.c: static int maintenance_run_tasks(struct maintenance_run_opts *opts)
return result;
}

@@ builtin/gc.c: static int maintenance_run(struct maintenance_opts *opts)
static int task_option_parse(const struct option *opt,
const char *arg, int unset)
{
-@@ builtin/gc.c: int cmd_maintenance(int argc, const char **argv, const char *prefix)
- builtin_maintenance_options);
+@@ builtin/gc.c: static int maintenance_run(int argc, const char **argv, const char *prefix)
+ memset(&opts, 0, sizeof(opts));

opts.quiet = !isatty(2);
+ initialize_task_config();

- argc = parse_options(argc, argv, prefix,
- builtin_maintenance_options,
+ for (i = 0; i < TASK__COUNT; i++)
+ tasks[i].selected_order = -1;

## t/t7900-maintenance.sh ##
@@ t/t7900-maintenance.sh: test_expect_success 'run [--auto|--quiet]' '
9: 68bf5bef4b ! 9: 69d3b48fd4 maintenance: use pointers to check --auto
@@ Commit message

## builtin/gc.c ##
-@@ builtin/gc.c: static int maintenance_task_gc(struct maintenance_opts *opts) +@@ builtin/gc.c: static int maintenance_task_gc(struct maintenance_run_opts *opts)

- typedef int maintenance_task_fn(struct maintenance_opts *opts);
+ typedef int maintenance_task_fn(struct maintenance_run_opts *opts);

+/*
+ * An auto condition function returns 1 if the task should run
@@ builtin/gc.c: static int maintenance_task_gc(struct maintenance_opts *opts)
const char *name;
maintenance_task_fn *fn;
+ maintenance_auto_fn *auto_condition;
- unsigned enabled:1,
- selected:1;
- int selected_order;
+ unsigned enabled:1;
+
+ /* -1 if not selected. */
@@ builtin/gc.c: static struct maintenance_task tasks[] = {
[TASK_GC] = {
"gc",
@@ builtin/gc.c: static struct maintenance_task tasks[] = {
1,
},
[TASK_COMMIT_GRAPH] = {
-@@ builtin/gc.c: static int maintenance_run(struct maintenance_opts *opts) +@@ builtin/gc.c: static int maintenance_run_tasks(struct maintenance_run_opts *opts)
if (!found_selected && !tasks[i].enabled)
continue;

10: fc097c389a ! 10: 4c3115fe35 maintenance: add auto condition for commit-graph task

     @@ Commit message
      
       ## Documentation/config/maintenance.txt ##
      @@ Documentation/config/maintenance.txt: maintenance.<task>.enabled::
     - 	This boolean config option controls whether the maintenance task
     - 	with name `<task>` is run when no `--task` option is specified.
     - 	By default, only `maintenance.gc.enabled` is true.
     + 	`git maintenance run`. These config values are ignored if a
     + 	`--task` option exists. By default, only `maintenance.gc.enabled`
     + 	is true.
      +
      +maintenance.commit-graph.auto::
      +	This integer config option controls how often the `commit-graph` task
     @@ builtin/gc.c
       
       #define FAILED_RUN "failed to run %s"
       
     -@@ builtin/gc.c: struct maintenance_opts {
     +@@ builtin/gc.c: struct maintenance_run_opts {
       	int quiet;
       };
       
      +/* Remember to update object flag allocation in object.h */
     -+#define PARENT1		(1u<<16)
     ++#define SEEN		(1u<<0)
      +
     -+static int num_commits_not_in_graph = 0;
     -+static int limit_commits_not_in_graph = 100;
     ++struct cg_auto_data {
     ++	int num_not_in_graph;
     ++	int limit;
     ++};
      +
      +static int dfs_on_ref(const char *refname,
      +		      const struct object_id *oid, int flags,
      +		      void *cb_data)
      +{
     ++	struct cg_auto_data *data = (struct cg_auto_data *)cb_data;
      +	int result = 0;
      +	struct object_id peeled;
      +	struct commit_list *stack = NULL;
     @@ builtin/gc.c: struct maintenance_opts {
      +		for (parent = commit->parents; parent; parent = parent->next) {
      +			if (parse_commit(parent->item) ||
      +			    commit_graph_position(parent->item) != COMMIT_NOT_FROM_GRAPH ||
     -+			    parent->item->object.flags & PARENT1)
     ++			    parent->item->object.flags & SEEN)
      +				continue;
      +
     -+			parent->item->object.flags |= PARENT1;
     -+			num_commits_not_in_graph++;
     ++			parent->item->object.flags |= SEEN;
     ++			data->num_not_in_graph++;
      +
     -+			if (num_commits_not_in_graph >= limit_commits_not_in_graph) {
     ++			if (data->num_not_in_graph >= data->limit) {
      +				result = 1;
      +				break;
      +			}
     @@ builtin/gc.c: struct maintenance_opts {
      +static int should_write_commit_graph(void)
      +{
      +	int result;
     ++	struct cg_auto_data data;
      +
     ++	data.num_not_in_graph = 0;
     ++	data.limit = 100;
      +	git_config_get_int("maintenance.commit-graph.auto",
     -+			   &limit_commits_not_in_graph);
     ++			   &data.limit);
      +
     -+	if (!limit_commits_not_in_graph)
     ++	if (!data.limit)
      +		return 0;
     -+	if (limit_commits_not_in_graph < 0)
     ++	if (data.limit < 0)
      +		return 1;
      +
     -+	result = for_each_ref(dfs_on_ref, NULL);
     ++	result = for_each_ref(dfs_on_ref, &data);
      +
     -+	clear_commit_marks_all(PARENT1);
     ++	clear_commit_marks_all(SEEN);
      +
      +	return result;
      +}
      +
     - static int run_write_commit_graph(struct maintenance_opts *opts)
     + static int run_write_commit_graph(struct maintenance_run_opts *opts)
       {
       	struct child_process child = CHILD_PROCESS_INIT;
      @@ builtin/gc.c: static struct maintenance_task tasks[] = {
     @@ builtin/gc.c: static struct maintenance_task tasks[] = {
      
       ## object.h ##
      @@ object.h: struct object_array {
     +  * sha1-name.c:                                              20
        * list-objects-filter.c:                                      21
        * builtin/fsck.c:           0--3
     ++ * builtin/gc.c:             0
        * builtin/index-pack.c:                                     2021
     -+ * builtin/maintenance.c:                           16
        * builtin/pack-objects.c:                                   20
        * builtin/reflog.c:                   10--12
     -  * builtin/show-branch.c:    0-------------------------------------------26

11: 46fbe161aa ! 11: 652a8eac57 maintenance: add trace2 regions for task execution
@@ Commit message

## builtin/gc.c ##
-@@ builtin/gc.c: static int maintenance_run(struct maintenance_opts *opts) +@@ builtin/gc.c: static int maintenance_run_tasks(struct maintenance_run_opts *opts)
!tasks[i].auto_condition()))
continue;

gitgitgadget

From: Derrick Stolee <[email protected]>

The 'gc' builtin is our current entrypoint for automatically maintaining a repository. This one tool does many operations, such as repacking the repository, packing refs, and rewriting the commit-graph file. The name implies it performs "garbage collection" which means several different things, and some users may not want to use this operation that rewrites the entire object database.

Create a new 'maintenance' builtin that will become a more general- purpose command. To start, it will only support the 'run' subcommand, but will later expand to add subcommands for scheduling maintenance in the background.

For now, the 'maintenance' builtin is a thin shim over the 'gc' builtin. In fact, the only option is the '--auto' toggle, which is handed directly to the 'gc' builtin. The current change is isolated to this simple operation to prevent more interesting logic from being lost in all of the boilerplate of adding a new builtin.

Use existing builtin/gc.c file because we want to share code between the two builtins. It is possible that we will have 'maintenance' replace the 'gc' builtin entirely at some point, leaving 'git gc' as an alias for some specific arguments to 'git maintenance run'.

Create a new test_subcommand helper that allows us to test if a certain subcommand was run. It requires storing the GIT_TRACE2_EVENT logs in a file. A negation mode is available that will be used in later tests.

---
 .gitignore                        |  1 +
 Documentation/git-maintenance.txt | 57 +++++++++++++++++++++++++++++++
 builtin.h                         |  1 +
 builtin/gc.c                      | 57 +++++++++++++++++++++++++++++++
 command-list.txt                  |  1 +
 git.c                             |  1 +
 t/t7900-maintenance.sh            | 21 ++++++++++++
 t/test-lib-functions.sh           | 33 ++++++++++++++++++
 8 files changed, 172 insertions(+)
 create mode 100644 Documentation/git-maintenance.txt
 create mode 100755 t/t7900-maintenance.sh

index ee509a2ad2..a5808fa30d 100644
@@ -90,6 +90,7 @@
 /git-ls-tree
 /git-mailinfo
 /git-mailsplit
+/git-maintenance
 /git-merge
 /git-merge-base
 /git-merge-index

index a5ae15bfe5..17c1c0ce49 100644
@@ -167,6 +167,7 @@ int cmd_ls_tree(int argc, const char **argv, const char *prefix);
 int cmd_ls_remote(int argc, const char **argv, const char *prefix);
 int cmd_mailinfo(int argc, const char **argv, const char *prefix);
 int cmd_mailsplit(int argc, const char **argv, const char *prefix);
+int cmd_maintenance(int argc, const char **argv, const char *prefix);
 int cmd_merge(int argc, const char **argv, const char *prefix);
 int cmd_merge_base(int argc, const char **argv, const char *prefix);
 int cmd_merge_index(int argc, const char **argv, const char *prefix);

index e5901f2213..0e3204e7d1 100644
@@ -117,6 +117,7 @@ git-ls-remote                           plumbinginterrogators
 git-ls-tree                             plumbinginterrogators
 git-mailinfo                            purehelpers
 git-mailsplit                           purehelpers
+git-maintenance                         mainporcelain
 git-merge                               mainporcelain           history
 git-merge-base                          plumbinginterrogators
 git-merge-file                          plumbingmanipulators

index 8bd1d7551d..24f250d29a 100644
@@ -529,6 +529,7 @@ static struct cmd_struct commands[] = {
 	{ "ls-tree", cmd_ls_tree, RUN_SETUP },
 	{ "mailinfo", cmd_mailinfo, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "mailsplit", cmd_mailsplit, NO_PARSEOPT },
+	{ "maintenance", cmd_maintenance, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "merge", cmd_merge, RUN_SETUP | NEED_WORK_TREE },
 	{ "merge-base", cmd_merge_base, RUN_SETUP },
 	{ "merge-file", cmd_merge_file, RUN_SETUP_GENTLY },

new file mode 100755
index 0000000000..14aa691f19
@@ -0,0 +1,21 @@
+#!/bin/sh
+
+test_description='git maintenance builtin'
+
+. ./test-lib.sh
+
+test_expect_success 'help text' '
+	test_expect_code 129 git maintenance -h 2>err &&
+	test_i18ngrep "usage: git maintenance run" err &&
+	test_expect_code 128 git maintenance barf 2>err &&
+	test_i18ngrep "invalid subcommand: barf" err
+'
+
+test_expect_success 'run [--auto]' '
+	GIT_TRACE2_EVENT="$(pwd)/run-no-auto.txt" git maintenance run &&
+	GIT_TRACE2_EVENT="$(pwd)/run-auto.txt" git maintenance run --auto &&
+	test_subcommand git gc <run-no-auto.txt &&
+	test_subcommand git gc --auto <run-auto.txt
+'
+
+test_done

index 6a8e194a99..d805e73f45 100644
@@ -1628,3 +1628,36 @@ test_path_is_hidden () {
 	case "$("$SYSTEMROOT"/system32/attrib "$1")" in *H*?:*) return 0;; esac
 	return 1
 }
+
+# Check that the given command was invoked as part of the
+# trace2-format trace on stdin.
+#
+#	test_subcommand [!] <command> <args>... < <trace>
+#
+# For example, to look for an invocation of "git upload-pack
+# /path/to/repo"
+#
+#	GIT_TRACE2_EVENT=event.log git fetch ... &&
+#	test_subcommand git upload-pack "$PATH" <event.log
+#
+# If the first parameter passed is !, this instead checks that
+# the given command was not called.
+#
+test_subcommand () {
+	local negate=
+	if test "$1" = "!"
+	then
+		negate=t
+		shift
+	fi
+
+	local expr=$(printf '"%s",' "$@")
+	expr="${expr%,}"
+
+	if test -n "$negate"
+	then
+		! grep "\[$expr\]"
+	else
+		grep "\[$expr\]"
+	fi
+}

gitgitgadget

From: Derrick Stolee <[email protected]>

The run_auto_gc() method is used in several places to trigger a check for repo maintenance after some Git commands, such as 'git commit' or 'git fetch'.

To allow for extra customization of this maintenance activity, replace the 'git gc --auto [--quiet]' call with one to 'git maintenance run --auto [--quiet]'. As we extend the maintenance builtin with other steps, users will be able to select different maintenance activities.

Rename run_auto_gc() to run_auto_maintenance() to be clearer what is happening on this call, and to expose all callers in the current diff. Rewrite the method to use a struct child_process to simplify the calls slightly.

Since 'git fetch' already allows disabling the 'git gc --auto' subprocess, add an equivalent option with a different name to be more descriptive of the new behavior: '--[no-]maintenance'. Update the documentation to include these options at the same time.

---
 Documentation/fetch-options.txt |  6 ++++--
 Documentation/git-clone.txt     |  6 +++---
 builtin/am.c                    |  2 +-
 builtin/commit.c                |  2 +-
 builtin/fetch.c                 |  6 ++++--
 builtin/merge.c                 |  2 +-
 builtin/rebase.c                |  4 ++--
 run-command.c                   | 16 +++++++---------
 run-command.h                   |  2 +-
 t/t5510-fetch.sh                |  2 +-
 10 files changed, 25 insertions(+), 23 deletions(-)

index 07d06ff445..b65a758661 100644
@@ -95,9 +95,11 @@ ifndef::git-pull[]
 	Allow several <repository> and <group> arguments to be
 	specified. No <refspec>s may be specified.
 
+--[no-]auto-maintenance::
 --[no-]auto-gc::
-	Run `git gc --auto` at the end to perform garbage collection
-	if needed. This is enabled by default.
+	Run `git maintenance run --auto` at the end to perform automatic
+	repository maintenance if needed. (`--[no-]auto-gc` is a synonym.)
+	This is enabled by default.
 
 --[no-]write-commit-graph::
 	Write a commit-graph after fetching. This overrides the config

index c898310099..097e6a86c5 100644
@@ -78,9 +78,9 @@ repository using this option and then delete branches (or use any
 other Git command that makes any existing commit unreferenced) in the
 source repository, some objects may become unreferenced (or dangling).
 These objects may be removed by normal Git operations (such as `git commit`)
-which automatically call `git gc --auto`. (See linkgit:git-gc[1].)
-If these objects are removed and were referenced by the cloned repository,
-then the cloned repository will become corrupt.
+which automatically call `git maintenance run --auto`. (See
+linkgit:git-maintenance[1].) If these objects are removed and were referenced
+by the cloned repository, then the cloned repository will become corrupt.
 +
 Note that running `git repack` without the `--local` option in a repository
 cloned with `--shared` will copy objects from the source repository into a pack

index dd4e6c2d9b..68e9d17379 100644
@@ -1795,7 +1795,7 @@ static void am_run(struct am_state *state, int resume)
 	if (!state->rebasing) {
 		am_destroy(state);
 		close_object_store(the_repository->objects);
-		run_auto_gc(state->quiet);
+		run_auto_maintenance(state->quiet);
 	}
 }
 

index 69ac78d5e5..f9b0a0c05d 100644
@@ -1702,7 +1702,7 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 	git_test_write_commit_graph_or_die();
 
 	repo_rerere(the_repository, 0);
-	run_auto_gc(quiet);
+	run_auto_maintenance(quiet);
 	run_commit_hook(use_editor, get_index_file(), "post-commit", NULL);
 	if (amend && !no_post_rewrite) {
 		commit_post_rewrite(the_repository, current_head, &oid);

index 2eb8d6a5a5..cb38e6f5ec 100644
@@ -199,8 +199,10 @@ static struct option builtin_fetch_options[] = {
 	OPT_STRING_LIST(0, "negotiation-tip", &negotiation_tip, N_("revision"),
 			N_("report that we have only objects reachable from this object")),
 	OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options),
+	OPT_BOOL(0, "auto-maintenance", &enable_auto_gc,
+		 N_("run 'maintenance --auto' after fetching")),
 	OPT_BOOL(0, "auto-gc", &enable_auto_gc,
-		 N_("run 'gc --auto' after fetching")),
+		 N_("run 'maintenance --auto' after fetching")),
 	OPT_BOOL(0, "show-forced-updates", &fetch_show_forced_updates,
 		 N_("check for forced-updates on all updated branches")),
 	OPT_BOOL(0, "write-commit-graph", &fetch_write_commit_graph,