Upstreaming the Scalar command

tl;dr: This series contributes the Scalar command to the Git project. This command provides an opinionated way to create and configure repositories with a focus on very large repositories.

Background
==========

Years ago, Microsoft wanted to move the source code of the Windows operating system to Git. The challenge there was to prove that Git could scale to massive monorepos. The VFS for Git (formerly GVFS) project was born to take up that challenge.

The final solution included a virtual filesystem (with both user-mode and kernel components) and a customized fork of Git for Windows. This solution contained several key concepts, such as only populating a portion of the working directory, demand-fetching blobs, and performing periodic repo maintenance in the background. However, the required kernel drivers made it difficult to port the solution to other platforms.

But it was realized that many of these key concepts were independent of the actual VFS and its projection of the working directory. The Scalar project was created to make that separation, refine the key concepts, and then extract those features into the new Scalar command.

The present
===========

The Scalar project provides a completely functional non-virtual experience for monorepos. But why stop there. The Scalar project was designed to be a self-destructing vehicle to allow those key concepts to be moved into core Git itself for the benefit of all. For example, partial clone, sparse-checkout, and background maintenance have already been upstreamed and removed from Scalar proper. This patch series provides a C-based implementation of the final remaining portions of the Scalar command. This will make it easier for users to experiment with the Scalar command. It will also make it substantially easier to experiment with moving functionality from Scalar into core Git, while maintaining backwards-compatibility for existing Scalar users.

The C-based Scalar has been shipped to Scalar users, and can be tested by any interested reader:
https://github.com/microsoft/git/releases/tag/v2.33.0.vfs.0.0 (it offers a Git for Windows installer, a macOS package and an Ubuntu package).

Opportunities
=============

Apart from providing the Scalar command, this contribution is intended to serve as a basis for further mailing list discussions on moving (some of) these key concepts into the main Git commands.

For example, we previously discussed the idea of a "git big-clone" that does much of what "scalar clone" is doing. This patch series is a step to make such functionality exist in the Git code base while we simmer on what such a "git big-clone" command-line interface would look like.

This is one of many possible ways to do this. Creating a 'git big-clone' could lock Git into backwards compatibility concerns so it is necessary to approach such an endeavor with caution. As a discussion starter, the scalar clone <url> command does roughly this:

1. git clone --sparse --filter=blob:none /src
2. git -C /src sparse-checkout init --cone
3. git -C /src config (many times)
4. git -C /src maintenance start

It is my hope inspire discussions about what parts of Scalar could go into core Git, and where, and in which form. While we wish to maintain backwards-compatibility of Scalar's command-line interface (because it is already in use), by having the Scalar code in the same code base as Git's, it will be much easier to move functionality without having to maintain loose version coupling between independently-versioned Scalar and Git. The tight version-coupling, along with having access to libgit.a also allows the C-based implementation of Scalar to be much smaller than the original .NET version.

For example, we might choose in the future to implement, say, git clone --scale=partial,cone to initialize a partial clone with a cone-sparse checkout, that would not only be totally doable, and not only would we already have precedent and data to prove that this actually makes engineers happy who have to work on ginormous repositories, but we could then also implement it by moving parts of contrib/scalar/ to builtin/ (where contrib/scalar/ would then call the built-ins accordingly rather than hard-coding the defaults itself).

We now also have the opportunity to discuss the merits of Scalar's clone caching, which is not actually part of this patch series because it is a bit coupled with the GVFS parts of microsoft/git for the moment, where clones automatically get registered with a populated alternate repository that is identified by the URL, meaning: subsequent clones of the same repository are vastly faster than the first one because they do not actually download the already-received objects again, they access the cache instead.

Another thing that I could imagine to be discussed at length is the distinction between enlistment and worktree (where the latter is the actual Git worktree and usually lives in the src/ subdirectory of the former). This encourages untracked and ignored files to be placed outside the worktree, making Git's job much easier. This idea, too, might find its way in one way or another into Git proper.

These are just a few concepts in Scalar that do not yet have equivalents in Git. By putting this initial implementation into contrib/, we create a foundation for future discussions of these concepts.

We plan on updating the recommended config settings in scalar register as new Git features are available (such as builtin FSMonitor and sparse-index, when ready). To facilitate upgrading existing Scalar enlistments, their paths are automatically added to the [scalar] section of the global Git config, and the scalar reconfigure --all command will process all of them.

Epilogue
========

Now, to address some questions that I imagine every reader has who made it this far:

* Why not put the Scalar functionality directly into a built-in? Creating a Git builtin requires scrutiny over every aspect of the feature, which is difficult to do while also maintaining the command-line interface contract and expected behavior of the Scalar command (there are existing users, after all). By having the Scalar command in contrib/, we present a simple option for users to have these features in the short term while the Git contributor community decides which bits to absorb into Git built-ins.
* Why implement the Scalar command in the Git codebase? We ported Scalar to the microsoft/git fork for several reasons. First, we realized it was possible now that the core features exist inside Git itself. Second, compiling Scalar directly within a version of Git allows us to remove a version compatibility check from each config option that might or might not apply based on the installed Git version. Finally, this new location has greatly simplified our release process and the installation process for users. We now have ways to install Scalar with microsoft/git via winget, brew, and apt-get. This has been the case since we shipped v2.32.0 to our users, read: this setup has served us well already.
* Why contribute Scalar to the Git project? We are biased, of course, yet we do have evidence that the Scalar command is a helpful tool that offers an simple way to handle huge repositories with ease. By contributing it to the core Git project, we are able to share it with more users, especially some users who do not want to install the microsoft/git fork. We intend to include Scalar as a component in git-for-windows/git, but are contributing it here first. Further, we think there is benefit to the Git developer community as this presents an example of how to set certain defaults that work for large repositories.
* Does this integrate with the built-in FSMonitor yet? No, not yet. I do have a couple of add-on patch series lined up, one of them being the integration with the built-in FSMonitor, which obviously has to wait until the FSMonitor patch series advances further.

Derrick Stolee (4):
scalar: 'register' sets recommended config and starts maintenance
scalar: 'unregister' stops background maintenance
scalar: implement 'scalar list'
scalar: implement the run command

Johannes Schindelin (10):
scalar: create a rudimentary executable
scalar: start documenting the command
scalar: create test infrastructure
scalar: let 'unregister' handle a deleted enlistment directory
gracefully
scalar: implement the clone subcommand
scalar: teach 'clone' to support the --single-branch option
scalar: allow reconfiguring an existing enlistment
scalar: teach 'reconfigure' to optionally handle all registered
enlistments
scalar: implement the version command
scalar: accept -C and -c options before the subcommand

Matthew John Cheetham (1):
scalar: implement the delete command

 Makefile                         |   8 +
 contrib/scalar/.gitignore        |   5 +
 contrib/scalar/Makefile          |  57 +++
 contrib/scalar/scalar.c          | 844 +++++++++++++++++++++++++++++++
 contrib/scalar/scalar.txt        | 152 ++++++
 contrib/scalar/t/Makefile        |  78 +++
 contrib/scalar/t/t9099-scalar.sh |  88 ++++
 7 files changed, 1232 insertions(+)
 create mode 100644 contrib/scalar/.gitignore
 create mode 100644 contrib/scalar/Makefile
 create mode 100644 contrib/scalar/scalar.c
 create mode 100644 contrib/scalar/scalar.txt
 create mode 100644 contrib/scalar/t/Makefile
 create mode 100755 contrib/scalar/t/t9099-scalar.sh

base-commit: ebf3c04b262aa27fbb97f8a0156c2347fecafafb
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1005%2Fdscho%2Fscalar-the-beginning-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1005/dscho/scalar-the-beginning-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1005

gitgitgadget

From: Johannes Schindelin <[email protected]>

The idea of Scalar (https://github.com/microsoft/scalar), and before that, of VFS for Git, has always been to prove that Git _can_ scale, and to upstream whatever strategies have been demonstrated to help.

With this patch, we start the journey from that C# project to move what is left to Git's own contrib/ directory, reimplementing it in pure C, with the intention to facilitate integrating the functionality into core Git all while maintaining backwards-compatibility for existing Scalar users (which will be much easier when both live in the same worktree). It was always to plan to contribute all of the proven strategies back to core Git.

For example, while the virtual filesystem provided by VFS for Git helped the team developing the Windows operating system to move onto Git, while trying to upstream it we realized that it cannot be done: getting the virtual filesystem to work (which we only managed to implement fully on Windows, but not on, say, macOS or Linux), and the required server-side support for the GVFS protocol, made this not quite feasible.

The Scalar project learned from that and tackled the problem with different tactics: instead of pretending to Git that the working directory is fully populated, it _specifically_ teaches Git about partial clone (which is based on VFS for Git's cache server), about sparse checkout (which VFS for Git tried to do transparently, in the file system layer), and regularly runs maintenance tasks to keep the repository in a healthy state.

With partial clone, sparse checkout and git maintenance having been upstreamed, there is little left that scalar.exe does that which git.exe cannot do. One such thing is that scalar clone <url> will automatically set up a partial, sparse clone, and configure known-helpful settings from the start.

So let's bring this convenience into Git's tree.

The idea here is that you can (optionally) build Scalar via

make -C contrib/scalar/Makefile

This will build the scalar executable and put it into the contrib/scalar/ subdirectory.

The slightly awkward addition of the contrib/scalar/* bits to the top-level Makefile are actually really required: we want to link to libgit.a, which means that we will need to use the very same CFLAGS and LDFLAGS as the rest of Git.

An early development version of this patch tried to replicate all the conditional code in contrib/scalar/Makefile (e.g. NO_POLL) just like contrib/svn-fe/Makefile used to do before it was retired. It turned out to be quite the whack-a-mole game: the SHA-1-related flags, the flags enabling/disabling compat/poll/, compat/regex/, compat/win32mmap.c & friends depending on the current platform... To put it mildly: it was a major mess.

Instead, this patch makes minimal changes to the top-level Makefile so that the bits in contrib/scalar/ can be compiled and linked, and adds a contrib/scalar/Makefile that uses the top-level Makefile in a most minimal way to do the actual compiling.

Note: With this commit, we only establish the infrastructure, no Scalar functionality is implemented yet; We will do that incrementally over the next few commits.

---
 Makefile                  |  8 ++++++++
 contrib/scalar/.gitignore |  2 ++
 contrib/scalar/Makefile   | 34 ++++++++++++++++++++++++++++++++++
 contrib/scalar/scalar.c   | 36 ++++++++++++++++++++++++++++++++++++
 4 files changed, 80 insertions(+)
 create mode 100644 contrib/scalar/.gitignore
 create mode 100644 contrib/scalar/Makefile
 create mode 100644 contrib/scalar/scalar.c

index c3565fc0f8f..2d5c822f7a8 100644
@@ -2447,6 +2447,10 @@ endif
 .PHONY: objects
 objects: $(OBJECTS)
 
+SCALAR_SOURCES := contrib/scalar/scalar.c
+SCALAR_OBJECTS := $(SCALAR_SOURCES:c=o)
+OBJECTS += $(SCALAR_OBJECTS)
+
 dep_files := $(foreach f,$(OBJECTS),$(dir $f).depend/$(notdir $f).d)
 dep_dirs := $(addsuffix .depend,$(sort $(dir $(OBJECTS))))
 

@@ -2586,6 +2590,10 @@ $(REMOTE_CURL_PRIMARY): remote-curl.o http.o http-walker.o GIT-LDFLAGS $(GITLIBS
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(CURL_LIBCURL) $(EXPAT_LIBEXPAT) $(LIBS)
 
+contrib/scalar/scalar$X: $(SCALAR_OBJECTS) GIT-LDFLAGS $(GITLIBS)
+	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \
+		$(filter %.o,$^) $(LIBS)
+
 $(LIB_FILE): $(LIB_OBJS)
 	$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^
 

new file mode 100644
index 00000000000..ff3d47e84d0
@@ -0,0 +1,2 @@
+/*.exe
+/scalar

new file mode 100644
index 00000000000..40c03ad10e1
@@ -0,0 +1,34 @@
+QUIET_SUBDIR0  = +$(MAKE) -C # space to separate -C and subdir
+QUIET_SUBDIR1  =
+
+ifneq ($(findstring s,$(MAKEFLAGS)),s)
+ifndef V
+	QUIET_SUBDIR0  = +@subdir=
+	QUIET_SUBDIR1  = ;$(NO_SUBDIR) echo '   ' SUBDIR $$subdir; \
+			 $(MAKE) $(PRINT_DIR) -C $$subdir
+else
+	export V
+endif
+endif
+
+all:
+
+include ../../config.mak.uname
+-include ../../config.mak.autogen
+-include ../../config.mak
+
+TARGETS = scalar$(X) scalar.o
+GITLIBS = ../../common-main.o ../../libgit.a ../../xdiff/lib.a
+
+all: scalar$X
+
+$(GITLIBS):
+	$(QUIET_SUBDIR0)../.. $(QUIET_SUBDIR1) $(subst ../../,,$@)
+
+$(TARGETS): $(GITLIBS) scalar.c
+	$(QUIET_SUBDIR0)../.. $(QUIET_SUBDIR1) $(patsubst %,contrib/scalar/%,$@)
+
+clean:
+	$(RM) $(TARGETS)
+
+.PHONY: all clean FORCE

new file mode 100644
index 00000000000..7cff29e0fcd
@@ -0,0 +1,36 @@
+/*
+ * The Scalar command-line interface.
+ */
+
+#include "cache.h"
+#include "gettext.h"
+#include "parse-options.h"
+
+static struct {
+	const char *name;
+	int (*fn)(int, const char **);
+} builtins[] = {
+	{ NULL, NULL},
+};
+
+int cmd_main(int argc, const char **argv)
+{
+	struct strbuf scalar_usage = STRBUF_INIT;
+	int i;
+
+	if (argc > 1) {
+		argv++;
+		argc--;
+
+		for (i = 0; builtins[i].name; i++)
+			if (!strcmp(builtins[i].name, argv[0]))
+				return !!builtins[i].fn(argc, argv);
+	}
+
+	strbuf_addstr(&scalar_usage,
+		      N_("scalar <command> [<options>]\n\nCommands:\n"));
+	for (i = 0; builtins[i].name; i++)
+		strbuf_addf(&scalar_usage, "\t%s\n", builtins[i].name);
+
+	usage(scalar_usage.buf);
+}

gitgitgadget

From: Johannes Schindelin <[email protected]>

This commit establishes the infrastructure to build the manual page for the scalar command.

---
 contrib/scalar/.gitignore |  3 +++
 contrib/scalar/Makefile   | 14 +++++++++++++-
 contrib/scalar/scalar.txt | 38 ++++++++++++++++++++++++++++++++++++++
 3 files changed, 54 insertions(+), 1 deletion(-)
 create mode 100644 contrib/scalar/scalar.txt

index ff3d47e84d0..00441073f59 100644
@@ -1,2 +1,5 @@
+/*.xml
+/*.1
+/*.html
 /*.exe
 /scalar

index 40c03ad10e1..85c186634e9 100644
@@ -6,6 +6,7 @@ ifndef V
 	QUIET_SUBDIR0  = +@subdir=
 	QUIET_SUBDIR1  = ;$(NO_SUBDIR) echo '   ' SUBDIR $$subdir; \
 			 $(MAKE) $(PRINT_DIR) -C $$subdir
+	QUIET          = @
 else
 	export V
 endif

@@ -30,5 +31,16 @@ $(TARGETS): $(GITLIBS) scalar.c
 
 clean:
 	$(RM) $(TARGETS)
+	$(RM) scalar.1 scalar.html scalar.xml
 
-.PHONY: all clean FORCE
+docs: scalar.html scalar.1
+
+scalar.html: | scalar.1 # prevent them from trying to build `doc.dep` in parallel
+
+scalar.html scalar.1: scalar.txt
+	$(QUIET_SUBDIR0)../../Documentation$(QUIET_SUBDIR1) \
+		MAN_TXT=../contrib/scalar/scalar.txt \
+		../contrib/scalar/$@
+	$(QUIET)test scalar.1 != "$@" || mv ../../Documentation/$@ .
+
+.PHONY: all clean docs FORCE

new file mode 100644
index 00000000000..5f7131861a5
@@ -0,0 +1,38 @@
+scalar(1)
+=========
+
+NAME
+----
+scalar - an opinionated repository management tool
+
+SYNOPSIS
+--------
+[verse]
+scalar <command> [<options>]
+
+DESCRIPTION
+-----------
+
+Scalar is an opinionated repository management tool. By creating new
+repositories or registering existing repositories with Scalar, your Git
+experience will speed up. Scalar sets advanced Git config settings,
+maintains your repositories in the background, and helps reduce data sent
+across the network.
+
+An important Scalar concept is the enlistment: this is the top-level directory
+of the project. It usually contains the subdirectory `src/` which is a Git
+worktree. This encourages the separation between tracked files (inside `src/`)
+and untracked files, such as build artifacts (outside `src/`). When registering
+an existing Git worktree with Scalar whose name is not `src`, the enlistment
+will be identical to the worktree.
+
+The `scalar` command implements various subcommands, and different options
+depending on the subcommand.
+
+SEE ALSO
+--------
+linkgit:git-maintenance[1].
+
+Scalar
+---
+Associated with the linkgit:git[1] suite

gitgitgadget

From: Johannes Schindelin <[email protected]>

To test the Scalar command, create a test script in contrib/scalar/t that is executed as make -C contrib/scalar test. Since Scalar has no meaningful capabilities yet, the only test is rather simple. We will add more tests in subsequent commits that introduce corresponding, new functionality.

Note: this test script is intended to test scalar only lightly, even after all of the functionality is implemented.

A more comprehensive functional (or: integration) test suite can be found at https://github.com/microsoft/scalar; It is used in the workflow
https://github.com/microsoft/git/blob/HEAD/.github/workflows/scalar-functional-tests.yml
in Microsoft's Git fork. This test suite performs end-to-end tests with a real remote repository, and is run as part of the regular CI builds. Since those tests require some functionality supported only by Microsoft's Git fork ("GVFS protocol"), there is no intention to port that fuller test suite to contrib/scalar/.

---
 contrib/scalar/Makefile          | 17 +++++--
 contrib/scalar/t/Makefile        | 78 ++++++++++++++++++++++++++++++++
 contrib/scalar/t/t9099-scalar.sh | 17 +++++++
 3 files changed, 109 insertions(+), 3 deletions(-)
 create mode 100644 contrib/scalar/t/Makefile
 create mode 100755 contrib/scalar/t/t9099-scalar.sh

index 85c186634e9..8620042f281 100644
@@ -3,6 +3,7 @@ QUIET_SUBDIR1  =
 
 ifneq ($(findstring s,$(MAKEFLAGS)),s)
 ifndef V
+	QUIET_GEN      = @echo '   ' GEN $@;
 	QUIET_SUBDIR0  = +@subdir=
 	QUIET_SUBDIR1  = ;$(NO_SUBDIR) echo '   ' SUBDIR $$subdir; \
 			 $(MAKE) $(PRINT_DIR) -C $$subdir

@@ -21,7 +22,7 @@ include ../../config.mak.uname
 TARGETS = scalar$(X) scalar.o
 GITLIBS = ../../common-main.o ../../libgit.a ../../xdiff/lib.a
 
-all: scalar$X
+all: scalar$X ../../bin-wrappers/scalar
 
 $(GITLIBS):
 	$(QUIET_SUBDIR0)../.. $(QUIET_SUBDIR1) $(subst ../../,,$@)

@@ -30,9 +31,19 @@ $(TARGETS): $(GITLIBS) scalar.c
 	$(QUIET_SUBDIR0)../.. $(QUIET_SUBDIR1) $(patsubst %,contrib/scalar/%,$@)
 
 clean:
-	$(RM) $(TARGETS)
+	$(RM) $(TARGETS) ../../bin-wrappers/scalar
 	$(RM) scalar.1 scalar.html scalar.xml
 
+../../bin-wrappers/scalar: ../../wrap-for-bin.sh Makefile
+	@mkdir -p ../../bin-wrappers
+	$(QUIET_GEN)sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \
+	     -e 's|@@BUILD_DIR@@|$(shell cd ../.. && pwd)|' \
+	     -e 's|@@PROG@@|contrib/scalar/scalar$(X)|' < $< > $@ && \
+	chmod +x $@
+
+test: all
+	$(MAKE) -C t
+
 docs: scalar.html scalar.1
 
 scalar.html: | scalar.1 # prevent them from trying to build `doc.dep` in parallel

@@ -43,4 +54,4 @@ scalar.html scalar.1: scalar.txt
 		../contrib/scalar/$@
 	$(QUIET)test scalar.1 != "$@" || mv ../../Documentation/$@ .
 
-.PHONY: all clean docs FORCE
+.PHONY: all clean docs test FORCE

new file mode 100755
index 00000000000..16f2b72b126
@@ -0,0 +1,17 @@
+#!/bin/sh
+
+test_description='test the `scalar` command'
+
+TEST_DIRECTORY=$PWD/../../../t
+export TEST_DIRECTORY
+
+# Make it work with --no-bin-wrappers
+PATH=$PWD/..:$PATH
+
+. ../../../t/test-lib.sh
+
+test_expect_success 'scalar shows a usage' '
+	test_expect_code 129 scalar -h
+'
+
+test_done

gitgitgadget

From: Derrick Stolee <[email protected]>

Let's start implementing the register command. With this commit, recommended settings are configured upon scalar register, and Git's background maintenance is started.

The recommended config settings may very well change in the future. For example, once the built-in FSMonitor is available, we will want to enable it upon scalar register. For that reason, we explicitly support running scalar register in an already-registered enlistment.

---
 contrib/scalar/scalar.c   | 256 ++++++++++++++++++++++++++++++++++++++
 contrib/scalar/scalar.txt |  14 ++-
 2 files changed, 269 insertions(+), 1 deletion(-)

index 7cff29e0fcd..7660327c27b 100644
@@ -5,11 +5,267 @@
 #include "cache.h"
 #include "gettext.h"
 #include "parse-options.h"
+#include "config.h"
+#include "run-command.h"
+
+/*
+ * Remove the deepest subdirectory in the provided path string. Path must not
+ * include a trailing path separator. Returns 1 if parent directory found,
+ * otherwise 0.
+ */
+static int strbuf_parent_directory(struct strbuf *buf)
+{
+	size_t len = buf->len;
+	size_t offset = offset_1st_component(buf->buf);
+	char *path_sep = find_last_dir_sep(buf->buf + offset);
+	strbuf_setlen(buf, path_sep ? path_sep - buf->buf : offset);
+
+	return buf->len < len;
+}
+
+static void setup_enlistment_directory(int argc, const char **argv,
+				       const char * const *usagestr,
+				       const struct option *options,
+				       struct strbuf *enlistment_root)
+{
+	struct strbuf path = STRBUF_INIT;
+	char *root;
+	int enlistment_found = 0;
+
+	if (startup_info->have_repository)
+		BUG("gitdir already set up?!?");
+
+	if (argc > 1)
+		usage_with_options(usagestr, options);
+
+	/* find the worktree, determine its corresponding root */
+	if (argc == 1)
+		strbuf_add_absolute_path(&path, argv[0]);
+	else if (strbuf_getcwd(&path) < 0)
+		die(_("need a working directory"));
+
+	strbuf_trim_trailing_dir_sep(&path);
+	do {
+		const size_t len = path.len;
+
+		/* check if currently in enlistment root with src/ workdir */
+		strbuf_addstr(&path, "/src/.git");
+		if (is_git_directory(path.buf)) {
+			strbuf_strip_suffix(&path, "/.git");
+
+			if (enlistment_root)
+				strbuf_add(enlistment_root, path.buf, len);
+
+			enlistment_found = 1;
+			break;
+		}
+
+		/* reset to original path */
+		strbuf_setlen(&path, len);
+
+		/* check if currently in workdir */
+		strbuf_addstr(&path, "/.git");
+		if (is_git_directory(path.buf)) {
+			strbuf_setlen(&path, len);
+
+			if (enlistment_root) {
+				/*
+				 * If the worktree's directory's name is `src`, the enlistment is the
+				 * parent directory, otherwise it is identical to the worktree.
+				 */
+				root = strip_path_suffix(path.buf, "src");
+				strbuf_addstr(enlistment_root, root ? root : path.buf);
+				free(root);
+			}
+
+			enlistment_found = 1;
+			break;
+		}
+
+		strbuf_setlen(&path, len);
+	} while (strbuf_parent_directory(&path));
+
+	if (!enlistment_found)
+		die(_("could not find enlistment root"));
+
+	if (chdir(path.buf) < 0)
+		die_errno(_("could not switch to '%s'"), path.buf);
+
+	strbuf_release(&path);
+	setup_git_directory();
+}
+
+static int run_git(const char *arg, ...)
+{
+	struct strvec argv = STRVEC_INIT;
+	va_list args;
+	const char *p;
+	int res;
+
+	va_start(args, arg);
+	strvec_push(&argv, arg);
+	while ((p = va_arg(args, const char *)))
+		strvec_push(&argv, p);
+	va_end(args);
+
+	res = run_command_v_opt(argv.v, RUN_GIT_CMD);
+
+	strvec_clear(&argv);
+	return res;
+}
+
+static int set_recommended_config(void)
+{
+	struct {
+		const char *key;
+		const char *value;
+	} config[] = {
+		{ "am.keepCR", "true" },
+		{ "core.FSCache", "true" },
+		{ "core.multiPackIndex", "true" },
+		{ "core.preloadIndex", "true" },
+#ifndef WIN32
+		{ "core.untrackedCache", "true" },
+#else
+		/*
+		 * Unfortunately, Scalar's Functional Tests demonstrated
+		 * that the untracked cache feature is unreliable on Windows
+		 * (which is a bummer because that platform would benefit the
+		 * most from it). For some reason, freshly created files seem
+		 * not to update the directory's `lastModified` time
+		 * immediately, but the untracked cache would need to rely on
+		 * that.
+		 *
+		 * Therefore, with a sad heart, we disable this very useful
+		 * feature on Windows.
+		 */
+		{ "core.untrackedCache", "false" },
+#endif
+		{ "core.bare", "false" },
+		{ "core.logAllRefUpdates", "true" },
+		{ "credential.https://dev.azure.com.useHttpPath", "true" },
+		{ "credential.validate", "false" }, /* GCM4W-only */
+		{ "gc.auto", "0" },
+		{ "gui.GCWarning", "false" },
+		{ "index.threads", "true" },
+		{ "index.version", "4" },
+		{ "merge.stat", "false" },
+		{ "merge.renames", "false" },
+		{ "pack.useBitmaps", "false" },
+		{ "pack.useSparse", "true" },
+		{ "receive.autoGC", "false" },
+		{ "reset.quiet", "true" },
+		{ "feature.manyFiles", "false" },
+		{ "feature.experimental", "false" },
+		{ "fetch.unpackLimit", "1" },
+		{ "fetch.writeCommitGraph", "false" },
+#ifdef WIN32
+		{ "http.sslBackend", "schannel" },
+#endif
+		{ "status.aheadBehind", "false" },
+		{ "commitGraph.generationVersion", "1" },
+		{ "core.autoCRLF", "false" },
+		{ "core.safeCRLF", "false" },
+		{ NULL, NULL },
+	};
+	int i;
+	char *value;
+
+	for (i = 0; config[i].key; i++) {
+		if (git_config_get_string(config[i].key, &value)) {
+			trace2_data_string("scalar", the_repository, config[i].key, "created");
+			if (git_config_set_gently(config[i].key,
+						  config[i].value) < 0)
+				return error(_("could not configure %s=%s"),
+					     config[i].key, config[i].value);
+		} else {
+			trace2_data_string("scalar", the_repository, config[i].key, "exists");
+			free(value);
+		}
+	}
+
+	/*
+	 * The `log.excludeDecoration` setting is special because it allows
+	 * for multiple values.
+	 */
+	if (git_config_get_string("log.excludeDecoration", &value)) {
+		trace2_data_string("scalar", the_repository,
+				   "log.excludeDecoration", "created");
+		if (git_config_set_multivar_gently("log.excludeDecoration",
+						   "refs/prefetch/*",
+						   CONFIG_REGEX_NONE, 0))
+			return error(_("could not configure "
+				       "log.excludeDecoration"));
+	} else {
+		trace2_data_string("scalar", the_repository,
+				   "log.excludeDecoration", "exists");
+		free(value);
+	}
+
+	return 0;
+}
+
+static int start_maintenance(void)
+{
+	return run_git("maintenance", "start", NULL);
+}
+
+static int add_enlistment(void)
+{
+	int res;
+
+	if (!the_repository->worktree)
+		die(_("Scalar enlistments require a worktree"));
+
+	res = run_git("config", "--global", "--get", "--fixed-value",
+		      "scalar.repo", the_repository->worktree, NULL);
+
+	/*
+	 * If the setting is already there, then do nothing.
+	 */
+	if (!res)
+		return 0;
+
+	return run_git("config", "--global", "--add",
+		       "scalar.repo", the_repository->worktree, NULL);
+}
+
+static int register_dir(void)
+{
+	int res = add_enlistment();
+
+	if (!res)
+		res = set_recommended_config();
+
+	if (!res)
+		res = start_maintenance();
+
+	return res;
+}
+
+static int cmd_register(int argc, const char **argv)
+{
+	struct option options[] = {
+		OPT_END(),
+	};
+	const char * const usage[] = {
+		N_("scalar register [<enlistment>]"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, NULL, options,
+			     usage, 0);
+
+	setup_enlistment_directory(argc, argv, usage, options, NULL);
+
+	return register_dir();
+}
 
 static struct {
 	const char *name;
 	int (*fn)(int, const char **);
 } builtins[] = {
+	{ "register", cmd_register },
 	{ NULL, NULL},
 };
 

index 5f7131861a5..41429db7990 100644
@@ -8,7 +8,7 @@ scalar - an opinionated repository management tool
 SYNOPSIS
 --------
 [verse]
-scalar <command> [<options>]
+scalar register [<enlistment>]
 
 DESCRIPTION
 -----------

@@ -29,6 +29,18 @@ will be identical to the worktree.
 The `scalar` command implements various subcommands, and different options
 depending on the subcommand.
 
+COMMANDS
+--------
+
+Register
+~~~~~~~~
+
+register [<enlistment>]::
+    Adds the enlistment's repository to the list of registered repositories
+    and starts background maintenance. If `<enlistment>` is not provided,
+    then the enlistment associated with the current working directory is
+    registered.
+
 SEE ALSO
 --------
 linkgit:git-maintenance[1].

gitgitgadget

From: Derrick Stolee <[email protected]>

Just like scalar register starts the scheduled background maintenance, scalar unregister stops it. Note that we use git maintenance start in scalar register, but we do not use git maintenance stop in scalar unregister: this would stop maintenance for _all_ repositories, not just for the one we want to unregister.

The unregister command also removes the corresponding entry from the [scalar] section in the global Git config.

---
 contrib/scalar/scalar.c   | 50 ++++++++++++++++++++++++++++++++-------
 contrib/scalar/scalar.txt |  8 +++++++
 2 files changed, 50 insertions(+), 8 deletions(-)

index 7660327c27b..ef91a1af38b 100644
@@ -205,12 +205,12 @@ static int set_recommended_config(void)
 	return 0;
 }
 
-static int start_maintenance(void)
+static int toggle_maintenance(int enable)
 {
-	return run_git("maintenance", "start", NULL);
+	return run_git("maintenance", enable ? "start" : "unregister", NULL);
 }
 
-static int add_enlistment(void)
+static int add_or_remove_enlistment(int add)
 {
 	int res;
 

@@ -221,24 +221,39 @@ static int add_enlistment(void)
 		      "scalar.repo", the_repository->worktree, NULL);
 
 	/*
-	 * If the setting is already there, then do nothing.
+	 * If we want to add and the setting is already there, then do nothing.
+	 * If we want to remove and the setting is not there, then do nothing.
 	 */
-	if (!res)
+	if ((add && !res) || (!add && res))
 		return 0;
 
-	return run_git("config", "--global", "--add",
+	return run_git("config", "--global", add ? "--add" : "--unset",
+		       add ? "--no-fixed-value" : "--fixed-value",
 		       "scalar.repo", the_repository->worktree, NULL);
 }
 
 static int register_dir(void)
 {
-	int res = add_enlistment();
+	int res = add_or_remove_enlistment(1);
 
 	if (!res)
 		res = set_recommended_config();
 
 	if (!res)
-		res = start_maintenance();
+		res = toggle_maintenance(1);
+
+	return res;
+}
+
+static int unregister_dir(void)
+{
+	int res = 0;
+
+	if (toggle_maintenance(0) < 0)
+		res = -1;
+
+	if (add_or_remove_enlistment(0) < 0)
+		res = -1;
 
 	return res;
 }

@@ -261,11 +276,30 @@ static int cmd_register(int argc, const char **argv)
 	return register_dir();
 }
 
+static int cmd_unregister(int argc, const char **argv)
+{
+	struct option options[] = {
+		OPT_END(),
+	};
+	const char * const usage[] = {
+		N_("scalar unregister [<enlistment>]"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, NULL, options,
+			     usage, 0);
+
+	setup_enlistment_directory(argc, argv, usage, options, NULL);
+
+	return unregister_dir();
+}
+
 static struct {
 	const char *name;
 	int (*fn)(int, const char **);
 } builtins[] = {
 	{ "register", cmd_register },
+	{ "unregister", cmd_unregister },
 	{ NULL, NULL},
 };
 

index 41429db7990..cddaa969403 100644
@@ -9,6 +9,7 @@ SYNOPSIS
 --------
 [verse]
 scalar register [<enlistment>]
+scalar unregister [<enlistment>]
 
 DESCRIPTION
 -----------

@@ -41,6 +42,13 @@ register [<enlistment>]::
     then the enlistment associated with the current working directory is
     registered.
 
+Unregister
+~~~~~~~~~~
+
+unregister [<enlistment>]::
+    Remove the specified repository from the list of repositories
+    registered with Scalar and stop the scheduled background maintenance.
+
 SEE ALSO
 --------
 linkgit:git-maintenance[1].

gitgitgadget

From: Derrick Stolee <[email protected]>

The produced list simply consists of those repositories registered under the multi-valued scalar.repo config setting in the user's Git config.

---
 contrib/scalar/scalar.c   | 11 +++++++++++
 contrib/scalar/scalar.txt | 12 +++++++++++-
 2 files changed, 22 insertions(+), 1 deletion(-)

index 14e688f44a9..91ceb97e552 100644
@@ -258,6 +258,16 @@ static int unregister_dir(void)
 	return res;
 }
 
+static int cmd_list(int argc, const char **argv)
+{
+	if (argc != 1)
+		die(_("`scalar list` does not take arguments"));
+
+	if (run_git("config", "--global", "--get-all", "scalar.repo", NULL) < 0)
+		return -1;
+	return 0;
+}
+
 static int cmd_register(int argc, const char **argv)
 {
 	struct option options[] = {

@@ -344,6 +354,7 @@ static struct {
 	const char *name;
 	int (*fn)(int, const char **);
 } builtins[] = {
+	{ "list", cmd_list },
 	{ "register", cmd_register },
 	{ "unregister", cmd_unregister },
 	{ NULL, NULL},

index cddaa969403..e1f629fddad 100644
@@ -8,6 +8,7 @@ scalar - an opinionated repository management tool
 SYNOPSIS
 --------
 [verse]
+scalar list
 scalar register [<enlistment>]
 scalar unregister [<enlistment>]
 

@@ -28,11 +29,20 @@ an existing Git worktree with Scalar whose name is not `src`, the enlistment
 will be identical to the worktree.
 
 The `scalar` command implements various subcommands, and different options
-depending on the subcommand.
+depending on the subcommand. With the exception of `list`, all subcommands
+expect to be run in an enlistment.
 
 COMMANDS
 --------
 
+List
+~~~~
+
+list::
+    To see which repositories are currently registered by the service, run
+    `scalar list`. This subcommand does not need to be run inside a Scalar
+    enlistment.
+
 Register
 ~~~~~~~~
 

gitgitgadget

From: Johannes Schindelin <[email protected]>

When a user deleted an enlistment manually, let's be generous and _still_ unregister it.

---
 contrib/scalar/scalar.c          | 46 ++++++++++++++++++++++++++++++++
 contrib/scalar/t/t9099-scalar.sh | 15 +++++++++++
 2 files changed, 61 insertions(+)

index ef91a1af38b..14e688f44a9 100644
@@ -276,6 +276,24 @@ static int cmd_register(int argc, const char **argv)
 	return register_dir();
 }
 
+static int remove_deleted_enlistment(struct strbuf *path)
+{
+	int res = 0;
+	strbuf_realpath_forgiving(path, path->buf, 1);
+
+	if (run_git("config", "--global",
+		    "--unset", "--fixed-value",
+		    "scalar.repo", path->buf, NULL) < 0)
+		res = -1;
+
+	if (run_git("config", "--global",
+		    "--unset", "--fixed-value",
+		    "maintenance.repo", path->buf, NULL) < 0)
+		res = -1;
+
+	return res;
+}
+
 static int cmd_unregister(int argc, const char **argv)
 {
 	struct option options[] = {

@@ -289,6 +307,34 @@ static int cmd_unregister(int argc, const char **argv)
 	argc = parse_options(argc, argv, NULL, options,
 			     usage, 0);
 
+	/*
+	 * Be forgiving when the enlistment or worktree does not even exist any
+	 * longer; This can be the case if a user deleted the worktree by
+	 * mistake and _still_ wants to unregister the thing.
+	 */
+	if (argc == 1) {
+		struct strbuf src_path = STRBUF_INIT, workdir_path = STRBUF_INIT;
+
+		strbuf_addf(&src_path, "%s/src/.git", argv[0]);
+		strbuf_addf(&workdir_path, "%s/.git", argv[0]);
+		if (!is_directory(src_path.buf) && !is_directory(workdir_path.buf)) {
+			/* remove possible matching registrations */
+			int res = -1;
+
+			strbuf_strip_suffix(&src_path, "/.git");
+			res = remove_deleted_enlistment(&src_path) && res;
+
+			strbuf_strip_suffix(&workdir_path, "/.git");
+			res = remove_deleted_enlistment(&workdir_path) && res;
+
+			strbuf_release(&src_path);
+			strbuf_release(&workdir_path);
+			return res;
+		}
+		strbuf_release(&src_path);
+		strbuf_release(&workdir_path);
+	}
+
 	setup_enlistment_directory(argc, argv, usage, options, NULL);
 
 	return unregister_dir();

index 16f2b72b126..ef0e8d680d5 100755
@@ -14,4 +14,19 @@ test_expect_success 'scalar shows a usage' '
 	test_expect_code 129 scalar -h
 '
 
+test_expect_success 'scalar unregister' '
+	git init vanish/src &&
+	scalar register vanish/src &&
+	git config --get --global --fixed-value \
+		maintenance.repo "$(pwd)/vanish/src" &&
+	scalar list >scalar.repos &&
+	grep -F "$(pwd)/vanish/src" scalar.repos &&
+	rm -rf vanish/src/.git &&
+	scalar unregister vanish &&
+	test_must_fail git config --get --global --fixed-value \
+		maintenance.repo "$(pwd)/vanish/src" &&
+	scalar list >scalar.repos &&
+	! grep -F "$(pwd)/vanish/src" scalar.repos
+'
+
 test_done

gitgitgadget

From: Johannes Schindelin <[email protected]>

This implements Scalar's opinionated clone command: it tries to use a partial clone and sets up a sparse checkout by default. In contrast to git clone, scalar clone sets up the worktree in the src/ subdirectory, to encourage a separation between the source files and the build output (which helps Git tremendously because it avoids untracked files that have to be specifically ignored when refreshing the index).

Also, it registers the repository for regular, scheduled maintenance, and configures a flurry of configuration settings based on the experience and experiments of the Microsoft Windows and the Microsoft Office development teams.

Note: We intentionally use a slightly wasteful set_config() function (which does not reuse a single strbuf, for example, though performance _really_ does not matter here) for convenience and readability.

Also note: since the scalar clone command is by far the most commonly called scalar subcommand, we document it at the top of the manual page.

---
 contrib/scalar/scalar.c          | 200 +++++++++++++++++++++++++++++++
 contrib/scalar/scalar.txt        |  35 +++++-
 contrib/scalar/t/t9099-scalar.sh |  32 +++++
 3 files changed, 262 insertions(+), 5 deletions(-)

index 91ceb97e552..13cdfa94d16 100644
@@ -7,6 +7,7 @@
 #include "parse-options.h"
 #include "config.h"
 #include "run-command.h"
+#include "refs.h"
 
 /*
  * Remove the deepest subdirectory in the provided path string. Path must not

@@ -258,6 +259,204 @@ static int unregister_dir(void)
 	return res;
 }
 
+/* printf-style interface, expects `<key>=<value>` argument */
+static int set_config(const char *fmt, ...)
+{
+	struct strbuf buf = STRBUF_INIT;
+	char *value;
+	int res;
+	va_list args;
+
+	va_start(args, fmt);
+	strbuf_vaddf(&buf, fmt, args);
+	va_end(args);
+
+	value = strchr(buf.buf, '=');
+	if (value)
+		*(value++) = '\0';
+	res = git_config_set_gently(buf.buf, value);
+	strbuf_release(&buf);
+
+	return res;
+}
+
+static char *remote_default_branch(const char *url)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+	struct strbuf out = STRBUF_INIT;
+
+	cp.git_cmd = 1;
+	strvec_pushl(&cp.args, "ls-remote", "--symref", url, "HEAD", NULL);
+	strbuf_addstr(&out, "-\n");
+	if (!pipe_command(&cp, NULL, 0, &out, 0, NULL, 0)) {
+		char *ref = out.buf;
+
+		while ((ref = strstr(ref + 1, "\nref: "))) {
+			const char *p;
+			char *head, *branch;
+
+			ref += strlen("\nref: ");
+			head = strstr(ref, "\tHEAD");
+
+			if (!head || memchr(ref, '\n', head - ref))
+				continue;
+
+			if (skip_prefix(ref, "refs/heads/", &p)) {
+				branch = xstrndup(p, head - p);
+				strbuf_release(&out);
+				return branch;
+			}
+
+			error(_("remote HEAD is not a branch: '%.*s'"),
+			      (int)(head - ref), ref);
+			strbuf_release(&out);
+			return NULL;
+		}
+	}
+	warning(_("failed to get default branch name from remote; "
+		  "using local default"));
+	strbuf_reset(&out);
+
+	child_process_init(&cp);
+	cp.git_cmd = 1;
+	strvec_pushl(&cp.args, "symbolic-ref", "--short", "HEAD", NULL);
+	if (!pipe_command(&cp, NULL, 0, &out, 0, NULL, 0)) {
+		strbuf_trim(&out);
+		return strbuf_detach(&out, NULL);
+	}
+
+	strbuf_release(&out);
+	error(_("failed to get default branch name"));
+	return NULL;
+}
+
+static int cmd_clone(int argc, const char **argv)
+{
+	const char *branch = NULL;
+	int full_clone = 0;
+	struct option clone_options[] = {
+		OPT_STRING('b', "branch", &branch, N_("<branch>"),
+			   N_("branch to checkout after clone")),
+		OPT_BOOL(0, "full-clone", &full_clone,
+			 N_("when cloning, create full working directory")),
+		OPT_END(),
+	};
+	const char * const clone_usage[] = {
+		N_("scalar clone [<options>] [--] <repo> [<dir>]"),
+		NULL
+	};
+	const char *url;
+	char *enlistment = NULL, *dir = NULL;
+	struct strbuf buf = STRBUF_INIT;
+	int res;
+
+	argc = parse_options(argc, argv, NULL, clone_options, clone_usage, 0);
+
+	if (argc == 2) {
+		url = argv[0];
+		enlistment = xstrdup(argv[1]);
+	} else if (argc == 1) {
+		url = argv[0];
+
+		strbuf_addstr(&buf, url);
+		/* Strip trailing slashes, if any */
+		while (buf.len > 0 && is_dir_sep(buf.buf[buf.len - 1]))
+			strbuf_setlen(&buf, buf.len - 1);
+		/* Strip suffix `.git`, if any */
+		strbuf_strip_suffix(&buf, ".git");
+
+		enlistment = find_last_dir_sep(buf.buf);
+		if (!enlistment) {
+			die(_("cannot deduce worktree name from '%s'"), url);
+		}
+		enlistment = xstrdup(enlistment + 1);
+	} else {
+		usage_msg_opt(_("You must specify a repository to clone."),
+			      clone_usage, clone_options);
+	}
+
+	if (is_directory(enlistment))
+		die(_("directory '%s' exists already"), enlistment);
+
+	dir = xstrfmt("%s/src", enlistment);
+
+	strbuf_reset(&buf);
+	if (branch)
+		strbuf_addf(&buf, "init.defaultBranch=%s", branch);
+	else {
+		char *b = repo_default_branch_name(the_repository, 1);
+		strbuf_addf(&buf, "init.defaultBranch=%s", b);
+		free(b);
+	}
+
+	if ((res = run_git("-c", buf.buf, "init", "--", dir, NULL)))
+		goto cleanup;
+
+	if (chdir(dir) < 0) {
+		res = error_errno(_("could not switch to '%s'"), dir);
+		goto cleanup;
+	}
+
+	setup_git_directory();
+
+	/* common-main already logs `argv` */
+	trace2_def_repo(the_repository);
+
+	if (!branch && !(branch = remote_default_branch(url))) {
+		res = error(_("failed to get default branch for '%s'"), url);
+		goto cleanup;
+	}
+
+	if (set_config("remote.origin.url=%s", url) ||
+	    set_config("remote.origin.fetch="
+		       "+refs/heads/*:refs/remotes/origin/*") ||
+	    set_config("remote.origin.promisor=true") ||
+	    set_config("remote.origin.partialCloneFilter=blob:none")) {
+		res = error(_("could not configure remote in '%s'"), dir);
+		goto cleanup;
+	}
+
+	if (!full_clone &&
+	    (res = run_git("sparse-checkout", "init", "--cone", NULL)))
+		goto cleanup;
+
+	if (set_recommended_config())
+		return error(_("could not configure '%s'"), dir);
+
+	if ((res = run_git("fetch", "--quiet", "origin", NULL))) {
+		warning(_("partial clone failed; attempting full clone"));
+
+		if (set_config("remote.origin.promisor") ||
+		    set_config("remote.origin.partialCloneFilter")) {
+			res = error(_("could not configure for full clone"));
+			goto cleanup;
+		}
+
+		if ((res = run_git("fetch", "--quiet", "origin", NULL)))
+			goto cleanup;
+	}
+
+	if ((res = set_config("branch.%s.remote=origin", branch)))
+		goto cleanup;
+	if ((res = set_config("branch.%s.merge=refs/heads/%s",
+			      branch, branch)))
+		goto cleanup;
+
+	strbuf_reset(&buf);
+	strbuf_addf(&buf, "origin/%s", branch);
+	res = run_git("checkout", "-f", "-t", buf.buf, NULL);
+	if (res)
+		goto cleanup;
+
+	res = register_dir();
+
+cleanup:
+	free(enlistment);
+	free(dir);
+	strbuf_release(&buf);
+	return res;
+}
+
 static int cmd_list(int argc, const char **argv)
 {
 	if (argc != 1)

@@ -354,6 +553,7 @@ static struct {
 	const char *name;
 	int (*fn)(int, const char **);
 } builtins[] = {
+	{ "clone", cmd_clone },
 	{ "list", cmd_list },
 	{ "register", cmd_register },
 	{ "unregister", cmd_unregister },

index e1f629fddad..90d59f1d79f 100644
@@ -8,6 +8,7 @@ scalar - an opinionated repository management tool
 SYNOPSIS
 --------
 [verse]
+scalar clone [--branch <main-branch>] [--full-clone] <url> [<enlistment>]
 scalar list
 scalar register [<enlistment>]
 scalar unregister [<enlistment>]

@@ -29,19 +30,43 @@ an existing Git worktree with Scalar whose name is not `src`, the enlistment
 will be identical to the worktree.
 
 The `scalar` command implements various subcommands, and different options
-depending on the subcommand. With the exception of `list`, all subcommands
-expect to be run in an enlistment.
+depending on the subcommand. With the exception of `clone` and `list`, all
+subcommands expect to be run in an enlistment.
 
 COMMANDS
 --------
 
+Clone
+~~~~~
+
+clone [<options>] <url> [<enlistment>]::
+    Clones the specified repository, similar to linkgit:git-clone[1]. By
+    default, only commit and tree objects are cloned. Once finished, the
+    worktree is located at `<enlistment>/src`.
++
+The sparse-checkout feature is enabled (except when run with `--full-clone`)
+and the only files present are those in the top-level directory. Use
+`git sparse-checkout set` to expand the set of directories you want to see,
+or `git sparse-checkout disable` to expand to all files (see
+linkgit:git-sparse-checkout[1] for more details). You can explore the
+subdirectories outside your sparse-checkout by using `git ls-tree HEAD`.
+
+-b <name>::
+--branch <name>::
+    Instead of checking out the branch pointed to by the cloned repository's
+    HEAD, check out the `<name>` branch instead.
+
+--[no-]full-clone::
+    A sparse-checkout is initialized by default. This behavior can be turned
+    off via `--full-clone`.
+
 List
 ~~~~
 
 list::
     To see which repositories are currently registered by the service, run
-    `scalar list`. This subcommand does not need to be run inside a Scalar
-    enlistment.
+    `scalar list`. This subcommand, like `clone`, does not need to be run
+    inside a Scalar enlistment.
 
 Register
 ~~~~~~~~

@@ -61,7 +86,7 @@ unregister [<enlistment>]::
 
 SEE ALSO
 --------
-linkgit:git-maintenance[1].
+linkgit:git-clone[1], linkgit:git-maintenance[1].
 
 Scalar
 ---

index ef0e8d680d5..295398f62cc 100755
@@ -10,6 +10,9 @@ PATH=$PWD/..:$PATH
 
 . ../../../t/test-lib.sh
 
+GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab ../cron.txt"
+export GIT_TEST_MAINT_SCHEDULER
+
 test_expect_success 'scalar shows a usage' '
 	test_expect_code 129 scalar -h
 '

@@ -29,4 +32,33 @@ test_expect_success 'scalar unregister' '
 	! grep -F "$(pwd)/vanish/src" scalar.repos
 '
 
+test_expect_success 'set up repository to clone' '
+	test_commit first &&
+	test_commit second &&
+	test_commit third &&
+	git switch -c parallel first &&
+	mkdir -p 1/2 &&
+	test_commit 1/2/3 &&
+	git config uploadPack.allowFilter true &&
+	git config uploadPack.allowAnySHA1InWant true
+'
+
+test_expect_success 'scalar clone' '
+	second=$(git rev-parse --verify second:second.t) &&
+	scalar clone "file://$(pwd)" cloned &&
+	(
+		cd cloned/src &&
+
+		git config --get --global --fixed-value maintenance.repo \
+			"$(pwd)" &&
+
+		test_path_is_missing 1/2 &&
+		test_must_fail git rev-list --missing=print $second &&
+		git rev-list $second &&
+		git cat-file blob $second >actual &&
+		echo "second" >expect &&
+		test_cmp expect actual
+	)
+'
+
 test_done

gitgitgadget

From: Johannes Schindelin <[email protected]>

Just like git clone, the scalar clone command now also offers to restrict the clone to a single branch.

---
 contrib/scalar/scalar.c          |  9 +++++++--
 contrib/scalar/scalar.txt        | 12 +++++++++++-
 contrib/scalar/t/t9099-scalar.sh |  6 +++++-
 3 files changed, 23 insertions(+), 4 deletions(-)

index 13cdfa94d16..908eaa84df1 100644
@@ -333,12 +333,15 @@ static char *remote_default_branch(const char *url)
 static int cmd_clone(int argc, const char **argv)
 {
 	const char *branch = NULL;
-	int full_clone = 0;
+	int full_clone = 0, single_branch = 0;
 	struct option clone_options[] = {
 		OPT_STRING('b', "branch", &branch, N_("<branch>"),
 			   N_("branch to checkout after clone")),
 		OPT_BOOL(0, "full-clone", &full_clone,
 			 N_("when cloning, create full working directory")),
+		OPT_BOOL(0, "single-branch", &single_branch,
+			 N_("only download metadata for the branch that will "
+			    "be checked out")),
 		OPT_END(),
 	};
 	const char * const clone_usage[] = {

@@ -409,7 +412,9 @@ static int cmd_clone(int argc, const char **argv)
 
 	if (set_config("remote.origin.url=%s", url) ||
 	    set_config("remote.origin.fetch="
-		       "+refs/heads/*:refs/remotes/origin/*") ||
+		       "+refs/heads/%s:refs/remotes/origin/%s",
+		       single_branch ? branch : "*",
+		       single_branch ? branch : "*") ||
 	    set_config("remote.origin.promisor=true") ||
 	    set_config("remote.origin.partialCloneFilter=blob:none")) {
 		res = error(_("could not configure remote in '%s'"), dir);

index 90d59f1d79f..bb9411b38cb 100644
@@ -8,7 +8,7 @@ scalar - an opinionated repository management tool
 SYNOPSIS
 --------
 [verse]
-scalar clone [--branch <main-branch>] [--full-clone] <url> [<enlistment>]
+scalar clone [--single-branch] [--branch <main-branch>] [--full-clone] <url> [<enlistment>]
 scalar list
 scalar register [<enlistment>]
 scalar unregister [<enlistment>]

@@ -56,6 +56,16 @@ subdirectories outside your sparse-checkout by using `git ls-tree HEAD`.
     Instead of checking out the branch pointed to by the cloned repository's
     HEAD, check out the `<name>` branch instead.
 
+--[no-]single-branch::
+    Clone only the history leading to the tip of a single branch,
+    either specified by the `--branch` option or the primary
+    branch remote's `HEAD` points at.
++
+Further fetches into the resulting repository will only update the
+remote-tracking branch for the branch this option was used for the initial
+cloning. If the HEAD at the remote did not point at any branch when
+`--single-branch` clone was made, no remote-tracking branch is created.
+
 --[no-]full-clone::
     A sparse-checkout is initialized by default. This behavior can be turned
     off via `--full-clone`.

index 295398f62cc..9a35ab4fde6 100755
@@ -45,13 +45,17 @@ test_expect_success 'set up repository to clone' '
 
 test_expect_success 'scalar clone' '
 	second=$(git rev-parse --verify second:second.t) &&
-	scalar clone "file://$(pwd)" cloned &&
+	scalar clone "file://$(pwd)" cloned --single-branch &&
 	(
 		cd cloned/src &&
 
 		git config --get --global --fixed-value maintenance.repo \
 			"$(pwd)" &&
 
+		git for-each-ref --format="%(refname)" refs/remotes/origin/ >actual &&
+		echo "refs/remotes/origin/parallel" >expect &&
+		test_cmp expect actual &&
+
 		test_path_is_missing 1/2 &&
 		test_must_fail git rev-list --missing=print $second &&
 		git rev-list $second &&

gitgitgadget

From: Derrick Stolee <[email protected]>

Note: this subcommand is provided primarily for backwards-compatibility, for existing Scalar uses. It is mostly just a shim for git maintenance, mapping task names from the way Scalar called them to the way Git calls them.

The reason why those names differ? The background maintenance was first implemented in Scalar, and when it was contributed as a patch series implementing the git maintenance command, reviewers suggested better names, those suggestions were accepted before the patches were integrated into core Git.

---
 contrib/scalar/scalar.c   | 64 +++++++++++++++++++++++++++++++++++++++
 contrib/scalar/scalar.txt | 19 ++++++++++++
 2 files changed, 83 insertions(+)

index 908eaa84df1..d5d38a1afeb 100644
@@ -490,6 +490,69 @@ static int cmd_register(int argc, const char **argv)
 	return register_dir();
 }
 
+static int cmd_run(int argc, const char **argv)
+{
+	struct option options[] = {
+		OPT_END(),
+	};
+	struct {
+		const char *arg, *task;
+	} tasks[] = {
+		{ "config", NULL },
+		{ "commit-graph", "commit-graph" },
+		{ "fetch", "prefetch" },
+		{ "loose-objects", "loose-objects" },
+		{ "pack-files", "incremental-repack" },
+		{ NULL, NULL }
+	};
+	struct strbuf buf = STRBUF_INIT;
+	const char *usagestr[] = { NULL, NULL };
+	int i;
+
+	strbuf_addstr(&buf, N_("scalar run <task> [<enlistment>]\nTasks:\n"));
+	for (i = 0; tasks[i].arg; i++)
+		strbuf_addf(&buf, "\t%s\n", tasks[i].arg);
+	usagestr[0] = buf.buf;
+
+	argc = parse_options(argc, argv, NULL, options,
+			     usagestr, 0);
+
+	if (argc == 0)
+		usage_with_options(usagestr, options);
+
+	if (!strcmp("all", argv[0]))
+		i = -1;
+	else {
+		for (i = 0; tasks[i].arg && strcmp(tasks[i].arg, argv[0]); i++)
+			; /* keep looking for the task */
+
+		if (i > 0 && !tasks[i].arg) {
+			error(_("no such task: '%s'"), argv[0]);
+			usage_with_options(usagestr, options);
+		}
+	}
+
+	argc--;
+	argv++;
+	setup_enlistment_directory(argc, argv, usagestr, options, NULL);
+	strbuf_release(&buf);
+
+	if (i == 0)
+		return register_dir();
+
+	if (i > 0)
+		return run_git("maintenance", "run",
+			       "--task", tasks[i].task, NULL);
+
+	if (register_dir())
+		return -1;
+	for (i = 1; tasks[i].arg; i++)
+		if (run_git("maintenance", "run",
+			    "--task", tasks[i].task, NULL))
+			return -1;
+	return 0;
+}
+
 static int remove_deleted_enlistment(struct strbuf *path)
 {
 	int res = 0;

@@ -562,6 +625,7 @@ static struct {
 	{ "list", cmd_list },
 	{ "register", cmd_register },
 	{ "unregister", cmd_unregister },
+	{ "run", cmd_run },
 	{ NULL, NULL},
 };
 

index bb9411b38cb..9aadaf6323f 100644
@@ -12,6 +12,7 @@ scalar clone [--single-branch] [--branch <main-branch>] [--full-clone] <url> [<e
 scalar list
 scalar register [<enlistment>]
 scalar unregister [<enlistment>]
+scalar run ( all | config | commit-graph | fetch | loose-objects | pack-files ) [<enlistment>]
 
 DESCRIPTION
 -----------

@@ -94,6 +95,24 @@ unregister [<enlistment>]::
     Remove the specified repository from the list of repositories
     registered with Scalar and stop the scheduled background maintenance.
 
+Run
+~~~
+
+scalar run ( all | config | commit-graph | fetch | loose-objects | pack-files ) [<enlistment>]::
+    Run the given maintenance task (or all tasks, if `all` was specified).
+    Except for `all` and `config`, this subcommand simply hands off to
+    linkgit:git-maintenance[1] (mapping `fetch` to `prefetch` and
+    `pack-files` to `incremental-repack`).
++
+These tasks are run automatically as part of the scheduled maintenance,
+as soon as the repository is registered with Scalar. It should therefore
+not be necessary to run this subcommand manually.
++
+The `config` task is specific to Scalar and configures all those
+opinionated default settings that make Git work more efficiently with
+large repositories. As this task is run as part of `scalar clone`
+automatically, explicit invocations of this task are rarely needed.
+
 SEE ALSO
 --------
 linkgit:git-clone[1], linkgit:git-maintenance[1].

gitgitgadget

From: Johannes Schindelin <[email protected]>

This comes in handy during Scalar upgrades, or when config settings were messed up by mistake.

---
 contrib/scalar/scalar.c          | 81 ++++++++++++++++++++------------
 contrib/scalar/scalar.txt        |  8 ++++
 contrib/scalar/t/t9099-scalar.sh |  8 ++++
 3 files changed, 68 insertions(+), 29 deletions(-)

index d5d38a1afeb..4eff3464a13 100644
@@ -115,18 +115,20 @@ static int run_git(const char *arg, ...)
 	return res;
 }
 
-static int set_recommended_config(void)
+static int set_recommended_config(int reconfigure)
 {
 	struct {
 		const char *key;
 		const char *value;
+		int overwrite_on_reconfigure;
 	} config[] = {
-		{ "am.keepCR", "true" },
-		{ "core.FSCache", "true" },
-		{ "core.multiPackIndex", "true" },
-		{ "core.preloadIndex", "true" },
+		/* Required */
+		{ "am.keepCR", "true", 1 },
+		{ "core.FSCache", "true", 1 },
+		{ "core.multiPackIndex", "true", 1 },
+		{ "core.preloadIndex", "true", 1 },
 #ifndef WIN32
-		{ "core.untrackedCache", "true" },
+		{ "core.untrackedCache", "true", 1 },
 #else
 		/*
 		 * Unfortunately, Scalar's Functional Tests demonstrated

@@ -140,29 +142,30 @@ static int set_recommended_config(void)
 		 * Therefore, with a sad heart, we disable this very useful
 		 * feature on Windows.
 		 */
-		{ "core.untrackedCache", "false" },
+		{ "core.untrackedCache", "false", 1 },
 #endif
-		{ "core.bare", "false" },
-		{ "core.logAllRefUpdates", "true" },
-		{ "credential.https://dev.azure.com.useHttpPath", "true" },
-		{ "credential.validate", "false" }, /* GCM4W-only */
-		{ "gc.auto", "0" },
-		{ "gui.GCWarning", "false" },
-		{ "index.threads", "true" },
-		{ "index.version", "4" },
-		{ "merge.stat", "false" },
-		{ "merge.renames", "false" },
-		{ "pack.useBitmaps", "false" },
-		{ "pack.useSparse", "true" },
-		{ "receive.autoGC", "false" },
-		{ "reset.quiet", "true" },
-		{ "feature.manyFiles", "false" },
-		{ "feature.experimental", "false" },
-		{ "fetch.unpackLimit", "1" },
-		{ "fetch.writeCommitGraph", "false" },
+		{ "core.bare", "false", 1 },
+		{ "core.logAllRefUpdates", "true", 1 },
+		{ "credential.https://dev.azure.com.useHttpPath", "true", 1 },
+		{ "credential.validate", "false", 1 }, /* GCM4W-only */
+		{ "gc.auto", "0", 1 },
+		{ "gui.GCWarning", "false", 1 },
+		{ "index.threads", "true", 1 },
+		{ "index.version", "4", 1 },
+		{ "merge.stat", "false", 1 },
+		{ "merge.renames", "false", 1 },
+		{ "pack.useBitmaps", "false", 1 },
+		{ "pack.useSparse", "true", 1 },
+		{ "receive.autoGC", "false", 1 },
+		{ "reset.quiet", "true", 1 },
+		{ "feature.manyFiles", "false", 1 },
+		{ "feature.experimental", "false", 1 },
+		{ "fetch.unpackLimit", "1", 1 },
+		{ "fetch.writeCommitGraph", "false", 1 },
 #ifdef WIN32
-		{ "http.sslBackend", "schannel" },
+		{ "http.sslBackend", "schannel", 1 },
 #endif
+		/* Optional */
 		{ "status.aheadBehind", "false" },
 		{ "commitGraph.generationVersion", "1" },
 		{ "core.autoCRLF", "false" },

@@ -173,7 +176,8 @@ static int set_recommended_config(void)
 	char *value;
 
 	for (i = 0; config[i].key; i++) {
-		if (git_config_get_string(config[i].key, &value)) {
+		if ((reconfigure && config[i].overwrite_on_reconfigure) ||
+		    git_config_get_string(config[i].key, &value)) {
 			trace2_data_string("scalar", the_repository, config[i].key, "created");
 			if (git_config_set_gently(config[i].key,
 						  config[i].value) < 0)

@@ -238,7 +242,7 @@ static int register_dir(void)
 	int res = add_or_remove_enlistment(1);
 
 	if (!res)
-		res = set_recommended_config();
+		res = set_recommended_config(0);
 
 	if (!res)
 		res = toggle_maintenance(1);

@@ -425,7 +429,7 @@ static int cmd_clone(int argc, const char **argv)
 	    (res = run_git("sparse-checkout", "init", "--cone", NULL)))
 		goto cleanup;
 
-	if (set_recommended_config())
+	if (set_recommended_config(0))
 		return error(_("could not configure '%s'"), dir);
 
 	if ((res = run_git("fetch", "--quiet", "origin", NULL))) {

@@ -490,6 +494,24 @@ static int cmd_register(int argc, const char **argv)
 	return register_dir();
 }
 
+static int cmd_reconfigure(int argc, const char **argv)
+{
+	struct option options[] = {
+		OPT_END(),
+	};
+	const char * const usage[] = {
+		N_("scalar reconfigure [<enlistment>]"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, NULL, options,
+			     usage, 0);
+
+	setup_enlistment_directory(argc, argv, usage, options, NULL);
+
+	return set_recommended_config(1);
+}
+
 static int cmd_run(int argc, const char **argv)
 {
 	struct option options[] = {

@@ -626,6 +648,7 @@ static struct {
 	{ "register", cmd_register },
 	{ "unregister", cmd_unregister },
 	{ "run", cmd_run },
+	{ "reconfigure", cmd_reconfigure },
 	{ NULL, NULL},
 };
 

index 9aadaf6323f..227e3542a07 100644
@@ -13,6 +13,7 @@ scalar list
 scalar register [<enlistment>]
 scalar unregister [<enlistment>]
 scalar run ( all | config | commit-graph | fetch | loose-objects | pack-files ) [<enlistment>]
+scalar reconfigure <enlistment>
 
 DESCRIPTION
 -----------

@@ -113,6 +114,13 @@ opinionated default settings that make Git work more efficiently with
 large repositories. As this task is run as part of `scalar clone`
 automatically, explicit invocations of this task are rarely needed.
 
+Reconfigure
+~~~~~~~~~~~
+
+After a Scalar upgrade, or when the configuration of a Scalar enlistment
+was somehow corrupted or changed by mistake, this subcommand allows to
+reconfigure the enlistment.
+
 SEE ALSO
 --------
 linkgit:git-clone[1], linkgit:git-maintenance[1].

index 9a35ab4fde6..e6d74a06ca0 100755
@@ -65,4 +65,12 @@ test_expect_success 'scalar clone' '
 	)
 '
 
+test_expect_success 'scalar reconfigure' '
+	git init one/src &&
+	scalar register one &&
+	git -C one/src config core.preloadIndex false &&
+	scalar reconfigure one &&
+	test true = "$(git -C one/src config core.preloadIndex)"
+'
+
 test_done

gitgitgadget

From: Johannes Schindelin <[email protected]>

After a Scalar upgrade, it can come in really handy if there is an easy way to reconfigure all Scalar enlistments. This new option offers this functionality.

---
 contrib/scalar/scalar.c          | 61 ++++++++++++++++++++++++++++++--
 contrib/scalar/scalar.txt        | 10 ++++--
 contrib/scalar/t/t9099-scalar.sh |  3 ++
 3 files changed, 68 insertions(+), 6 deletions(-)

index 4eff3464a13..1f8778cbb39 100644
@@ -494,22 +494,77 @@ static int cmd_register(int argc, const char **argv)
 	return register_dir();
 }
 
+static int get_scalar_repos(const char *key, const char *value, void *data)
+{
+	struct string_list *list = data;
+
+	if (!strcmp(key, "scalar.repo"))
+		string_list_append(list, value);
+
+	return 0;
+}
+
 static int cmd_reconfigure(int argc, const char **argv)
 {
+	int all = 0;
 	struct option options[] = {
+		OPT_BOOL('a', "all", &all,
+			 N_("reconfigure all registered enlistments")),
 		OPT_END(),
 	};
 	const char * const usage[] = {
-		N_("scalar reconfigure [<enlistment>]"),
+		N_("scalar reconfigure [--all | <enlistment>]"),
 		NULL
 	};
+	struct string_list scalar_repos = STRING_LIST_INIT_DUP;
+	int i, res = 0;
+	struct repository r = { NULL };
+	struct strbuf commondir = STRBUF_INIT, gitdir = STRBUF_INIT;
 
 	argc = parse_options(argc, argv, NULL, options,
 			     usage, 0);
 
-	setup_enlistment_directory(argc, argv, usage, options, NULL);
+	if (!all) {
+		setup_enlistment_directory(argc, argv, usage, options, NULL);
+
+		return set_recommended_config(1);
+	}
+
+	if (argc > 0)
+		usage_msg_opt(_("--all or <enlistment>, but not both"),
+			      usage, options);
+
+	git_config(get_scalar_repos, &scalar_repos);
 
-	return set_recommended_config(1);
+	for (i = 0; i < scalar_repos.nr; i++) {
+		const char *dir = scalar_repos.items[i].string;
+
+		strbuf_reset(&commondir);
+		strbuf_reset(&gitdir);
+
+		if (chdir(dir) < 0) {
+			warning_errno(_("could not switch to '%s'"), dir);
+			res = -1;
+		} else if (discover_git_directory(&commondir, &gitdir) < 0) {
+			warning_errno(_("git repository gone in '%s'"), dir);
+			res = -1;
+		} else {
+			git_config_clear();
+
+			the_repository = &r;
+			r.commondir = commondir.buf;
+			r.gitdir = gitdir.buf;
+
+			if (set_recommended_config(1) < 0)
+				res = -1;
+		}
+	}
+
+	string_list_clear(&scalar_repos, 1);
+	strbuf_release(&commondir);
+	strbuf_release(&gitdir);
+
+	return res;
 }
 
 static int cmd_run(int argc, const char **argv)

index 227e3542a07..2a1a0695b4d 100644
@@ -13,7 +13,7 @@ scalar list
 scalar register [<enlistment>]
 scalar unregister [<enlistment>]
 scalar run ( all | config | commit-graph | fetch | loose-objects | pack-files ) [<enlistment>]
-scalar reconfigure <enlistment>
+scalar reconfigure [ --all | <enlistment> ]
 
 DESCRIPTION
 -----------

@@ -32,8 +32,8 @@ an existing Git worktree with Scalar whose name is not `src`, the enlistment
 will be identical to the worktree.
 
 The `scalar` command implements various subcommands, and different options
-depending on the subcommand. With the exception of `clone` and `list`, all
-subcommands expect to be run in an enlistment.
+depending on the subcommand. With the exception of `clone`, `list` and
+`reconfigure --all`, all subcommands expect to be run in an enlistment.
 
 COMMANDS
 --------

@@ -121,6 +121,10 @@ After a Scalar upgrade, or when the configuration of a Scalar enlistment
 was somehow corrupted or changed by mistake, this subcommand allows to
 reconfigure the enlistment.
 
+With the `--all` option, all enlistments currently registered with Scalar
+will be reconfigured. This option is meant to to be run every time Scalar
+was upgraded.
+
 SEE ALSO
 --------
 linkgit:git-clone[1], linkgit:git-maintenance[1].

index e6d74a06ca0..5fe7fabd0e5 100755
@@ -70,6 +70,9 @@ test_expect_success 'scalar reconfigure' '
 	scalar register one &&
 	git -C one/src config core.preloadIndex false &&
 	scalar reconfigure one &&
+	test true = "$(git -C one/src config core.preloadIndex)" &&
+	git -C one/src config core.preloadIndex false &&
+	scalar reconfigure -a &&
 	test true = "$(git -C one/src config core.preloadIndex)"
 '
 

gitgitgadget

From: Matthew John Cheetham <[email protected]>

Delete an enlistment by first unregistering the repository and then deleting the enlistment directory (usually the directory containing the worktree src/ directory).

On Windows, if the current directory is inside the enlistment's directory, change to the parent of the enlistment directory, to allow us to delete the enlistment (directories used by processes e.g. as current working directories cannot be deleted on Windows).

---
 contrib/scalar/scalar.c          | 55 ++++++++++++++++++++++++++++++++
 contrib/scalar/scalar.txt        |  8 +++++
 contrib/scalar/t/t9099-scalar.sh |  9 ++++++
 3 files changed, 72 insertions(+)

index 1f8778cbb39..c616e91231a 100644
@@ -8,6 +8,7 @@
 #include "config.h"
 #include "run-command.h"
 #include "refs.h"
+#include "dir.h"
 
 /*
  * Remove the deepest subdirectory in the provided path string. Path must not

@@ -334,6 +335,33 @@ static char *remote_default_branch(const char *url)
 	return NULL;
 }
 
+static int delete_enlistment(struct strbuf *enlistment)
+{
+#ifdef WIN32
+	struct strbuf parent = STRBUF_INIT;
+#endif
+
+	if (unregister_dir())
+		die(_("failed to unregister repository"));
+
+#ifdef WIN32
+	/*
+	 * Change the current directory to one outside of the enlistment so
+	 * that we may delete everything underneath it.
+	 */
+	strbuf_addbuf(&parent, enlistment);
+	strbuf_parent_directory(&parent);
+	if (chdir(parent.buf) < 0)
+		die_errno(_("could not switch to '%s'"), parent.buf);
+	strbuf_release(&parent);
+#endif
+
+	if (remove_dir_recursively(enlistment, 0))
+		die(_("failed to delete enlistment directory"));
+
+	return 0;
+}
+
 static int cmd_clone(int argc, const char **argv)
 {
 	const char *branch = NULL;

@@ -694,6 +722,32 @@ static int cmd_unregister(int argc, const char **argv)
 	return unregister_dir();
 }
 
+static int cmd_delete(int argc, const char **argv)
+{
+	struct option options[] = {
+		OPT_END(),
+	};
+	const char * const usage[] = {
+		N_("scalar delete <enlistment>"),
+		NULL
+	};
+	struct strbuf enlistment = STRBUF_INIT;
+	int res = 0;
+
+	argc = parse_options(argc, argv, NULL, options,
+			     usage, 0);
+
+	if (argc != 1)
+		usage_with_options(usage, options);
+
+	setup_enlistment_directory(argc, argv, usage, options, &enlistment);
+
+	res = delete_enlistment(&enlistment);
+	strbuf_release(&enlistment);
+
+	return res;
+}
+
 static struct {
 	const char *name;
 	int (*fn)(int, const char **);

@@ -704,6 +758,7 @@ static struct {
 	{ "unregister", cmd_unregister },
 	{ "run", cmd_run },
 	{ "reconfigure", cmd_reconfigure },
+	{ "delete", cmd_delete },
 	{ NULL, NULL},
 };
 

index 2a1a0695b4d..00923023243 100644
@@ -14,6 +14,7 @@ scalar register [<enlistment>]
 scalar unregister [<enlistment>]
 scalar run ( all | config | commit-graph | fetch | loose-objects | pack-files ) [<enlistment>]
 scalar reconfigure [ --all | <enlistment> ]
+scalar delete <enlistment>
 
 DESCRIPTION
 -----------

@@ -125,6 +126,13 @@ With the `--all` option, all enlistments currently registered with Scalar
 will be reconfigured. This option is meant to to be run every time Scalar
 was upgraded.
 
+Delete
+~~~~~~
+
+delete <enlistment>::
+    This subcommand lets you delete an existing Scalar enlistment from your
+    local file system, unregistering the repository.
+
 SEE ALSO
 --------
 linkgit:git-clone[1], linkgit:git-maintenance[1].

index 5fe7fabd0e5..7e8771d0eff 100755
@@ -76,4 +76,13 @@ test_expect_success 'scalar reconfigure' '
 	test true = "$(git -C one/src config core.preloadIndex)"
 '
 
+test_expect_success 'scalar delete without enlistment shows a usage' '
+	test_expect_code 129 scalar delete
+'
+
+test_expect_success 'scalar delete with enlistment' '
+	scalar delete cloned &&
+	test_path_is_missing cloned
+'
+
 test_done

gitgitgadget

From: Johannes Schindelin <[email protected]>

The git executable has these two very useful options:

-C <directory>:
switch to the specified directory before performing any actions

-c <key>=<value>:
temporarily configure this setting for the duration of the specified scalar subcommand

With this commit, we teach the scalar executable the same trick.

---
 contrib/scalar/scalar.c   | 22 +++++++++++++++++++++-
 contrib/scalar/scalar.txt | 10 ++++++++++
 2 files changed, 31 insertions(+), 1 deletion(-)

index be0a49b0d75..41ba4b2f3b1 100644
@@ -806,6 +806,25 @@ int cmd_main(int argc, const char **argv)
 	struct strbuf scalar_usage = STRBUF_INIT;
 	int i;
 
+	while (argc > 1 && *argv[1] == '-') {
+		if (!strcmp(argv[1], "-C")) {
+			if (argc < 3)
+				die(_("-C requires a <directory>"));
+			if (chdir(argv[2]) < 0)
+				die_errno(_("could not change to '%s'"),
+					  argv[2]);
+			argc -= 2;
+			argv += 2;
+		} else if (!strcmp(argv[1], "-c")) {
+			if (argc < 3)
+				die(_("-c requires a <key>=<value> argument"));
+			git_config_push_parameter(argv[2]);
+			argc -= 2;
+			argv += 2;
+		} else
+			break;
+	}
+
 	if (argc > 1) {
 		argv++;
 		argc--;

@@ -816,7 +835,8 @@ int cmd_main(int argc, const char **argv)
 	}
 
 	strbuf_addstr(&scalar_usage,
-		      N_("scalar <command> [<options>]\n\nCommands:\n"));
+		      N_("scalar [-C <directory>] [-c <key>=<value>] "
+			 "<command> [<options>]\n\nCommands:\n"));
 	for (i = 0; builtins[i].name; i++)
 		strbuf_addf(&scalar_usage, "\t%s\n", builtins[i].name);
 

index 00923023243..6d85640ef42 100644
@@ -36,6 +36,16 @@ The `scalar` command implements various subcommands, and different options
 depending on the subcommand. With the exception of `clone`, `list` and
 `reconfigure --all`, all subcommands expect to be run in an enlistment.
 
+The following options can be specified _before_ the subcommand:
+
+-C <directory>::
+    Before running the subcommand, change the working directory. This
+    option imitates the same option of linkgit:git[1].
+
+-c <key>=<value>::
+    For the duration of running the specified subcommand, configure this
+    setting. This option imitates the same option of linkgit:git[1].
+
 COMMANDS
 --------
 

gitgitgadget

From: Johannes Schindelin <[email protected]>

The .NET version of Scalar has a version command. This was necessary because it was versioned independently of Git.

Since Scalar is now tightly coupled with Git, it does not make sense for them to show different versions. Therefore, it shows the same output as git versions. For backwards-compatibility with the .NET version, scalar version prints to stderr, though (git version prints to stdout instead).

---
 contrib/scalar/scalar.c | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)

index c616e91231a..be0a49b0d75 100644
@@ -9,6 +9,7 @@
 #include "run-command.h"
 #include "refs.h"
 #include "dir.h"
+#include "help.h"
 
 /*
  * Remove the deepest subdirectory in the provided path string. Path must not

@@ -362,6 +363,15 @@ static int delete_enlistment(struct strbuf *enlistment)
 	return 0;
 }
 
+/*
+ * Dummy implementation; Using `get_version_info()` would cause a link error
+ * without this.
+ */
+void load_builtin_commands(const char *prefix, struct cmdnames *cmds)
+{
+	die("not implemented");
+}
+
 static int cmd_clone(int argc, const char **argv)
 {
 	const char *branch = NULL;

@@ -748,6 +758,34 @@ static int cmd_delete(int argc, const char **argv)
 	return res;
 }
 
+static int cmd_version(int argc, const char **argv)
+{
+	int verbose = 0, build_options = 0;
+	struct option options[] = {
+		OPT__VERBOSE(&verbose, N_("include Git version")),
+		OPT_BOOL(0, "build-options", &build_options,
+			 N_("include Git's build options")),
+		OPT_END(),
+	};
+	const char * const usage[] = {
+		N_("scalar verbose [-v | --verbose] [--build-options]"),
+		NULL
+	};
+	struct strbuf buf = STRBUF_INIT;
+
+	argc = parse_options(argc, argv, NULL, options,
+			     usage, 0);
+
+	if (argc != 0)
+		usage_with_options(usage, options);
+
+	get_version_info(&buf, build_options);
+	fprintf(stderr, "%s\n", buf.buf);
+	strbuf_release(&buf);
+
+	return 0;
+}
+
 static struct {
 	const char *name;
 	int (*fn)(int, const char **);

@@ -759,6 +797,7 @@ static struct {
 	{ "run", cmd_run },
 	{ "reconfigure", cmd_reconfigure },
 	{ "delete", cmd_delete },
+	{ "version", cmd_version },
 	{ NULL, NULL},
 };
 

gitgitgadget

I want to give Johannes a big thanks for organizing this RFC. As you can see from the authorship of the patches, this was an amazingly collaborative effort, but Johannes led the way by creating a base that the rest of us could work with, then finally he brought in all of the gritty details to finish the effort.

...

When people have asked me how Scalar fits with the core Git client, I point them to our "Philosophy of Scalar" document [1]. The most concise summary of our goals since starting Scalar has been that Scalar aligns with features already within Git that enable scale. I've said several times that we are constantly making Scalar do less by making Git do more.

[1] https://github.com/microsoft/git/blob/HEAD/contrib/scalar/docs/philosophy.md

Here is an example: when our large, internal customer told us that they required Linux support for Scalar, we looked at what it would take. We could have done the necessary platform-specific things to convince .NET Core to create a long-running process that launched Git maintenance tasks at different intervals, creating a similar mechanism to the Windows and macOS services that did those operations. But we also knew that the existing system was stuck with architectural decisions from VFS for Git that were not actually in service of how Scalar worked. Instead, we decided to build background maintenance into Git itself and had our Linux port of Scalar run "git maintenance start".

Once the Linux port was proven out with Git's background maintenance, we realized that the window where a user actually interacts with Scalar instead of Git is extremely narrow: users run "scalar clone" or "scalar register" and otherwise only run Git commands. The Scalar process does not need to exist outside of that. (There are some other helpers that can be used in a pinch to diagnose and fix problems, but they are rarely used. These commands, such as 'scalar diagnose' can be contributed separately.)

It became clear that for our own needs it would be easier to ship one installer that included the microsoft/git fork and the Scalar CLI, and it would be simple to rewrite the Scalar CLI with all of the Git helper APIs. We organized the code in a way that we thought would be amenable to an upstream contribution (by placing in contrib/ and using Git code style).

The thing about these commands is that they are _opinionated_. We rely on these opinions for important internal users, but we realize that they are not necessarily optimal for all users. Hence, we did not think it wise to push those opinions onto the 'git' executable. Having 'scalar' continue to live as a separate executable made sense to us.

I believe that by contributing Scalar to the full community, that we create opportunities for Git in the future. For one, users and Git distributors can opt into compiling Scalar so it is more available to users who are interested. Another hopeful idea is that maybe this reinvigorates ideas of how to streamline Git clones for large repos without users needing to learn each and every knob to twist to get things working. Since the Scalar CLI is contributed in the full license of the Git project, pieces of it can be adapted into Git proper as needed.

I look forward to hearing your thoughts.

Thanks,
-Stolee

s/versions/version/

As a comment on 01..03/15: I'd really prefer if we stop using this pattern of sub-Makefile, the dependencies are a pain to manage, and we end up copy/pasting large sets of functionality.

That would mean just adding the build of this command to the top-level Makefile behind some "CONTRIB_SCALAR" flag or whatever, but I find that much cleaner than....

Perhaps nobody else wondered this while reading this, but I thought this might be some sparse/worktree magic where cloning into "foo" would have "foo/.git", but the worktree was somehow magically mapped at foo/src/".

But no, it just takes your "scalar clone <url> foo" and translates it to "foo/src", so you'll get a directory at "foo".

FWIW I think the commit message could do without this, that part of the code is obviously not performance sensitive at all. But maybe an explicit note helps anyway...

Users can overwrite this local config value, but this is placed to avoid a global config value from applying specifically within Scalar-created repos.

This one is probably a bit _too_ defensive. It can be removed.

It's an opinionated thing related to supporting monorepo users. It helps us diagnose issues they have by recreating a sequence of events.

But if a user has them on globally, then we don't want them to apply locally (in favor of the settings that we set explicitly).

If we want to do such a tracing change within git_config_set*(), then that would be an appropriate replacement. The biggest reason to include them here is to trace that an existing value already exists, for the case of running 'scalar reconfigure' during an upgrade. That part doesn't make much sense to put into config.c.

Thanks,
-Stolee

Such a change would likely eliminate the ability to not include Scalar when building the Git codebase, which we tried to avoid by keeping it within contrib and have it be compiled via an opt-in flag.

If we want to talk about integrating Scalar into Git in a deeper way, then that is an interesting discussion to have, but it lives at a much higher level than Makefile details.

The questions we are really looking to answer in this RFC are:

1. Will the Git project accept Scalar into its codebase?

2. What is the best place for Scalar to live in the Git codebase?

We erred on the side of keeping Scalar as optional as possible. If the community is more interested in a deeper integration, then that could be an interesting direction.

In my opinion, I think the current tactic is safest. We could always decide on a deeper integration later by moving the code around. It seems harder to do the reverse.

Thanks,
-Stolee

I mean to still have it behind a flag, but to handle it similar to how we handle NO_CURL, EXCLUDED_PROGRAMS and the like, i.e. not requiring parallel maintenance of copy/pasted Makefile logic in contrib/.

To be clear I'm proposing no change at all in term of what happens when you run "make install", just commenting on the implementation details of how we arrange for things to be built and configured before that step.

I realize that this is following some prior art of
e.g. contrib/subtree/Makefile, but IMNSHO that approach is a historical mistake we should be backing out of. There was some recent discussion of this here:
https://lore.kernel.org/git/[email protected]/

E.g. now we have some painful management of the depencency graph between /Makefile and Documentation/Makefile requiring fixes like 56550ea7180 (Makefile: add missing dependencies of 'config-list.h', 2021-04-08), adding yet another Makefile into the mix which (to take one example) depends on doc.dep, which in turn depends on ...; It's all a bunch of needless complexity we can avoid.

Indeed, to be clear I realize I'm entirely punting on the real questions you're interested in. I just gave this an initial cursory skimming for now, I have not formed an informed opinion on your #1, but just a little bit of #2.

My initial reaction to #1 without having looked into it deeply is some combination of "sure, why not?", and that the people/group contributing major scalability work to git.git should be given the benefit of the doubt. Maybe we won't keep "scalar" long-term, or change its UI etc., all of that can be handled in some carefully worded documentation somewhere.

Of course all these suggestions I'm making about Makefile arrangement are rather pointless if there isn't consensus to get past the hurdle of your #1.

I think "deeper integration" is the reverse of what you think it is.

I.e. if I'm patching or maintaining part of the Makefile logic to it's deeper (or perhaps "gnarlier" is the righ word?) integration to need to duplicate that work in two places, or always take into account that some not-built-by-default-but-quite-common command's *.txt docs and *.sh tests live in some unusual place for the purposes of CI, lint, tooling etc.

In other words, it's a question of how much net complexity is being added to the (build) system. That complexity doesn't automatically reduce just because some files live in another directory, sometimes that's an increase in complexity.

Whereas just conditionally adding it to some list in the top-level Makefile (or Documentation/Makefile) is relatively maintenance-free, and to our users / packagers the result should be the same or near enough. It won't matter to them if building the optional thing is another "make" command or just a flag to the existing "make" command.

FWIW, I also found this distracting; it takes the reader's attention away from more important aspects of the patch. (But it alone is not worth a re-roll; it was just a minor hiccup.)

Looks like exciting stuff, you two. I'm behind on review as it is; I still need to get back to Stolee's sparse-index add/rm/mv series, but I'll try to circle back and take a look.

"Johannes Schindelin via GitGitGadget" <[email protected]>

...

I agree that 'git clone' plus maybe some more improvements like '--sparse=cone' to set up cone-mode sparse-checkout would be good. And also the implementation being contributed here is cleaner if we can use 'git clone'.

We are trying to balance a clean upstream implementation with some custom things that we still need in our microsoft/git fork to handle the integration with the GVFS Protocol (i.e. partial clone on Azure Repos). That customization is cleaner to keep here in the scalar code instead of adding an option to 'git clone'. It is difficult to justify code patterns here due to choices we have made in our fork, so I _could_ see a way to replace those custom bits with new, custom flags to 'git clone'. It just requires additional investment during our integration when we incorporate these upstream changes. Naturally, I'm motivated to avoid that extra work.

If your opinion to switch to 'git clone' is a strong one, then I could see us doing that change. I just want you to be aware of the hidden reasons for choices like these.

Thanks,
-Stolee

Hi Junio,

The - is actually needed because of the ref + 1 below, over which you stumbled.

I would use a different adjective, one that is less judgemental in nature, but then, you were talking about your feelings.

It would even work if the current line is shorter, but as you point out:
it is wasteful. And it could be improved to be more readable. I reworked it, and it now looks like this:

	if (!pipe_command(&cp, NULL, 0, &out, 0, NULL, 0)) {
		const char *line = out.buf;

		while (*line) {
			const char *eol = strchrnul(line, '\n'), *p;
			size_t len = eol - line;
			char *branch;

			if (!skip_prefix(line, "ref: ", &p) ||
			    !strip_suffix_mem(line, &len, "\tHEAD")) {
				line = eol + (*eol == '\n');
				continue;
			}

			eol = line + len;
			if (skip_prefix(p, "refs/heads/", &p)) {
				branch = xstrndup(p, eol - p);
				strbuf_release(&out);
				return branch;
			}

			error(_("remote HEAD is not a branch: '%.*s'"),
			      (int)(eol - p), p);
			strbuf_release(&out);
			return NULL;
		}
	}

It now parses the output line by line, looking for the expected prefix and suffix, then verifies the ref name format, and either returns the short branch name or errors out with the message that this is not a branch.

Yes, the reason is that I wanted to err on the side of caution. If the remote repository reports a default branch that is not a default branch at all, I do not want to pretend that things are fine and then run into trouble later when we set up a non-branch as remote-tracking target or something like that.

Right. It is a shame, I agree. And it is one of the things I want to work on, after the Scalar patch series made it into Git.

The reason why I don't want to work on this now is that I expect this effort to result in new options for git clone, new options that need to be designed well, and where I fully expect a long discussion until we reach a consensus how these options should look like, especially since we will need to maintain backwards-compatibility of Scalar's functionality.

Therefore, in the interest to keep the patch series relatively easy to review, I left this in the "for later" pile, for now.

Ciao,
Dscho

Hi Eric,

Since I reworked the remote default branch parsing anyway, I removed this paragraph from the commit message.

Ciao,
Dscho

Hi Eric,

I wanted to convey a temporal order, so I changed it to "every time after Scalar is upgraded". Okay?

Ciao,
Dscho

Hi Eric,

Thank you!
Dscho

Hi Junio,

All true. I wish we had come up with a better way, or with a way to override this via an option.

Unfortunately, we are now bound by the fact that there are already users out there...

I had hoped that the initial blurb of the manual page was sufficient, but you're right, the register subcommand is particular in that it allows to force Scalar to consider the worktree to be identical to the Scalar enlistment. I added this:

diff --git a/contrib/scalar/scalar.txt b/contrib/scalar/scalar.txt index 1593da45eae..568987064b2 100644
--- a/contrib/scalar/scalar.txt
+++ b/contrib/scalar/scalar.txt
@@ -40,6 +40,10 @@ register [<enlistment>]::
and starts background maintenance. If <enlistment> is not provided, then the enlistment associated with the current working directory is registered.
++
+Note: when this subcommand is called in a worktree that is called src/, its +parent directory is considered to be the Scalar enlistment. If the worktree is +_not_ called src/, it itself will be considered to be the Scalar enlistment.

Even with such a central registry, there would still be the question whether the user, by staying with the default, wanted Git (or in this instance, Scalar) to keep using the old default. The intention is unfortunately not clear just from setting the variable.

So I think this is the best we can do.

Ciao,
Dscho

Hi Ævar,

This command will show a generated usage, i.e. a non-static string. It therefore cannot be specified here already. See the strbuf_*() calls populating buf and the usagestr[0] = buf.buf; assignment.

It is true that we often do this, but in this instance it would be
misleading: argc is a counter, not a Boolean.

The style guide specifically allows my preference to leave single-line blocks without curlies.

Ciao,
Johannes

Hi Ævar,

I do not think that this would be a better name, or that renaming this field would do anything except cause more work for me.

To me, it makes things less readable. There is an entire section with the header /* Optional */ below, and I want this list to stay as readable as it is now.

Ciao,
Dscho

I think I understood the intent of the original, but it causes a grammatical hiccup. Your revised version can work, although I might write it this way:

This option is meant to be run each time Scalar is upgraded.

However, perhaps that is too ambiguous and some users may think that the process of upgrading Scalar will automatically run this command, and you'd like to make it clear that it is the user's responsibility. So, perhaps:

Use this option after each Scalar upgrade.

or something.

Oh, there is no disagreement on that point, at least in the short term. I was wondering why "clone" subcommand needs a duplicated logic that should be unnecessary, before realizing that this was not implemented as a wrapper to (possibly updated) "clone", and I agree that starting with a looser coupling like this step does is easier to everybody.

"Kind of shame" is just that I wished we had already prepared "git clone" side to accept customization more easily before its various distinct phases (new repository creation, where a custom logic may want to affect the name and location of it and how "git init" is driven, initial "fetch", where a custom logic may want to affect the fetch refspec and its parameters like depths and cones, and initial "checkout") do their things. If we allowed such plug-in of logic to affect how "git clone" worked already, it would have been possible to do "scalar clone" with much less code. It also would allow us to reorganize the "clone --local" hack in a way that is easier to reason about (I think even in today's code, the way I hooked it up can be seen which is quite messy). It may even help folks who want to extend "git clone" to clone a repository recursively its submodules with project-specific customizations (like which ones to clone by default, etc.).

I suspect that learning from the way "scalar clone" is done on top of "init" + "fetch" + "checkout" in this initial series may help us extend "git clone" later to fill such needs.

Not at all at this moment.

It is mostly that the way "init" + "fetch" + "checkout" was done in this step reminded me of a much longer-term wish I have had for a while.

It is much easier to read and understand how the loop works with above.

Wouldn't we have the same problem when the remote end does not advertise HEAD and we fall back to "local default", though? We'd run into trouble later as we use "local default" that may correspond to a non-branch there as remote-tracking target, or something like that.

Not that I care too deeply in the error case, though. I just felt that this early return was an uneven way to follow the principle to err on the side of caution, as we continue with the local default when the other side fails to tell us what their HEAD points at.

Thanks.

Sorry, if it weren't clear that I was commenting on each step as I read along without peeking later steps. I think I saw it was written somewhere that this was to encourage use of read-only directory that keeps the sources with build artifacts and crufts created outside it (so forests of projects will not have the source directories, each of which has its own .git/, next to each other--- instead we would have shell directories, each with its own src/ and src/.git, next to each other). The additional documentation below is a good thing to have handy when readers learn how to use "register" (or more generally, what an "enlistment" is). As long as the motivation behind that design is given somewhere (not necessarily here) for readers to discover, I am OK with the design.

Thanks.

That argument could be a plausible excuse to deviate from the style if it were

	if (argc == 0)
		do no args case;
	else if (argc == 1)
		do one arg case;
	else if (argc == 2)
		do two args case;
	...

Replacing the first one with "if (!argc)" may make it less readable.

But I do not think the reasoning applies here

	if (argc == 0)
		do a thing that applies only to no args case;

without "else". This is talking about "do we have any argument? Yes or no?" Boolean here.

Actually, the exception goes the other way, no?

We generally want to avoid such an unnecessary braces around a single statement block. But when we have an else clause that has a block with multiple statements (hence braces are required), as an exception, the guide asks you to write braces around the body of the if side for consistency.

When you only have just a couple of lines on the "else {}" side, I do not think it matters too much either way for readability, though. I cannot see the "else" side in the above clause, but IIRC it wasn't just a few lines, was it?

Thanks.

tl;dr: This series contributes the Scalar command to the Git project. This command provides an opinionated way to create and configure repositories with a focus on very large repositories.

Background
==========

Years ago, Microsoft wanted to move the source code of the Windows operating system to Git. The challenge there was to prove that Git could scale to massive monorepos. The VFS for Git (formerly GVFS) project was born to take up that challenge.

The final solution included a virtual filesystem (with both user-mode and kernel components) and a customized fork of Git for Windows. This solution contained several key concepts, such as only populating a portion of the working directory, demand-fetching blobs, and performing periodic repo maintenance in the background. However, the required kernel drivers made it difficult to port the solution to other platforms.

But it was realized that many of these key concepts were independent of the actual VFS and its projection of the working directory. The Scalar project was created to make that separation, refine the key concepts, and then extract those features into the new Scalar command.

The present
===========

The Scalar project provides a completely functional non-virtual experience for monorepos. But why stop there. The Scalar project was designed to be a self-destructing vehicle to allow those key concepts to be moved into core Git itself for the benefit of all. For example, partial clone, sparse-checkout, and background maintenance have already been upstreamed and removed from Scalar proper. This patch series provides a C-based implementation of the final remaining portions of the Scalar command. This will make it easier for users to experiment with the Scalar command. It will also make it substantially easier to experiment with moving functionality from Scalar into core Git, while maintaining backwards-compatibility for existing Scalar users.

The C-based Scalar has been shipped to Scalar users, and can be tested by any interested reader:
https://github.com/microsoft/git/releases/tag/v2.33.0.vfs.0.0 (it offers a Git for Windows installer, a macOS package and an Ubuntu package).

Opportunities
=============

Apart from providing the Scalar command, this contribution is intended to serve as a basis for further mailing list discussions on moving (some of) these key concepts into the main Git commands.

For example, we previously discussed the idea of a "git big-clone" that does much of what "scalar clone" is doing. This patch series is a step to make such functionality exist in the Git code base while we simmer on what such a "git big-clone" command-line interface would look like.

This is one of many possible ways to do this. Creating a 'git big-clone' could lock Git into backwards compatibility concerns so it is necessary to approach such an endeavor with caution. As a discussion starter, the scalar clone <url> command does roughly this:

1. git clone --sparse --filter=blob:none /src
2. git -C /src sparse-checkout init --cone
3. git -C /src config (many times)
4. git -C /src maintenance start

It is my hope inspire discussions about what parts of Scalar could go into core Git, and where, and in which form. While we wish to maintain backwards-compatibility of Scalar's command-line interface (because it is already in use), by having the Scalar code in the same code base as Git's, it will be much easier to move functionality without having to maintain loose version coupling between independently-versioned Scalar and Git. The tight version-coupling, along with having access to libgit.a also allows the C-based implementation of Scalar to be much smaller than the original .NET version.

For example, we might choose in the future to implement, say, git clone --scale=partial,cone to initialize a partial clone with a cone-sparse checkout, that would not only be totally doable, and not only would we already have precedent and data to prove that this actually makes engineers happy who have to work on ginormous repositories, but we could then also implement it by moving parts of contrib/scalar/ to builtin/ (where contrib/scalar/ would then call the built-ins accordingly rather than hard-coding the defaults itself).

We now also have the opportunity to discuss the merits of Scalar's clone caching, which is not actually part of this patch series because it is a bit coupled with the GVFS parts of microsoft/git for the moment, where clones automatically get registered with a populated alternate repository that is identified by the URL, meaning: subsequent clones of the same repository are vastly faster than the first one because they do not actually download the already-received objects again, they access the cache instead.

Another thing that I could imagine to be discussed at length is the distinction between enlistment and worktree (where the latter is the actual Git worktree and usually lives in the src/ subdirectory of the former). This encourages untracked and ignored files to be placed outside the worktree, making Git's job much easier. This idea, too, might find its way in one way or another into Git proper.

These are just a few concepts in Scalar that do not yet have equivalents in Git. By putting this initial implementation into contrib/, we create a foundation for future discussions of these concepts.

We plan on updating the recommended config settings in scalar register as new Git features are available (such as builtin FSMonitor and sparse-index, when ready). To facilitate upgrading existing Scalar enlistments, their paths are automatically added to the [scalar] section of the global Git config, and the scalar reconfigure --all command will process all of them.

Epilogue
========

Now, to address some questions that I imagine every reader has who made it this far:

* Why not put the Scalar functionality directly into a built-in? Creating a Git builtin requires scrutiny over every aspect of the feature, which is difficult to do while also maintaining the command-line interface contract and expected behavior of the Scalar command (there are existing users, after all). By having the Scalar command in contrib/, we present a simple option for users to have these features in the short term while the Git contributor community decides which bits to absorb into Git built-ins.
* Why implement the Scalar command in the Git codebase? We ported Scalar to the microsoft/git fork for several reasons. First, we realized it was possible now that the core features exist inside Git itself. Second, compiling Scalar directly within a version of Git allows us to remove a version compatibility check from each config option that might or might not apply based on the installed Git version. Finally, this new location has greatly simplified our release process and the installation process for users. We now have ways to install Scalar with microsoft/git via winget, brew, and apt-get. This has been the case since we shipped v2.32.0 to our users, read: this setup has served us well already.
* Why contribute Scalar to the Git project? We are biased, of course, yet we do have evidence that the Scalar command is a helpful tool that offers an simple way to handle huge repositories with ease. By contributing it to the core Git project, we are able to share it with more users, especially some users who do not want to install the microsoft/git fork. We intend to include Scalar as a component in git-for-windows/git, but are contributing it here first. Further, we think there is benefit to the Git developer community as this presents an example of how to set certain defaults that work for large repositories.
* Does this integrate with the built-in FSMonitor yet? No, not yet. I do have a couple of add-on patch series lined up, one of them being the integration with the built-in FSMonitor, which obviously has to wait until the FSMonitor patch series advances further.

Changes since v1:

* A couple typos were fixed
* The code parsing the output of ls-remote was made more readable
* The indentation used in scalar.txt now consistently uses tabs
* We no longer hard-code core.bare = false when registering with Scalar

Derrick Stolee (4):
scalar: 'register' sets recommended config and starts maintenance
scalar: 'unregister' stops background maintenance
scalar: implement 'scalar list'
scalar: implement the run command

Johannes Schindelin (10):
scalar: create a rudimentary executable
scalar: start documenting the command
scalar: create test infrastructure
scalar: let 'unregister' handle a deleted enlistment directory
gracefully
scalar: implement the clone subcommand
scalar: teach 'clone' to support the --single-branch option
scalar: allow reconfiguring an existing enlistment
scalar: teach 'reconfigure' to optionally handle all registered
enlistments
scalar: implement the version command
scalar: accept -C and -c options before the subcommand

Matthew John Cheetham (1):
scalar: implement the delete command

 Makefile                         |   8 +
 contrib/scalar/.gitignore        |   5 +
 contrib/scalar/Makefile          |  57 +++
 contrib/scalar/scalar.c          | 844 +++++++++++++++++++++++++++++++
 contrib/scalar/scalar.txt        | 156 ++++++
 contrib/scalar/t/Makefile        |  78 +++
 contrib/scalar/t/t9099-scalar.sh |  88 ++++
 7 files changed, 1236 insertions(+)
 create mode 100644 contrib/scalar/.gitignore
 create mode 100644 contrib/scalar/Makefile
 create mode 100644 contrib/scalar/scalar.c
 create mode 100644 contrib/scalar/scalar.txt
 create mode 100644 contrib/scalar/t/Makefile
 create mode 100755 contrib/scalar/t/t9099-scalar.sh

base-commit: ebf3c04b262aa27fbb97f8a0156c2347fecafafb
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1005%2Fdscho%2Fscalar-the-beginning-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1005/dscho/scalar-the-beginning-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/1005

Range-diff vs v1:

  1:  b8c7d3f8450 =  1:  b8c7d3f8450 scalar: create a rudimentary executable
  2:  4f886575dcf =  2:  4f886575dcf scalar: start documenting the command
  3:  bcfde9bc765 =  3:  bcfde9bc765 scalar: create test infrastructure
  4:  3786f4c597f !  4:  ee3e26a0c4e scalar: 'register' sets recommended config and starts maintenance
     @@ contrib/scalar/scalar.c
      +		 */
      +		{ "core.untrackedCache", "false" },
      +#endif
     -+		{ "core.bare", "false" },
      +		{ "core.logAllRefUpdates", "true" },
      +		{ "credential.https://dev.azure.com.useHttpPath", "true" },
      +		{ "credential.validate", "false" }, /* GCM4W-only */
     @@ contrib/scalar/scalar.txt: will be identical to the worktree.
      +~~~~~~~~
      +
      +register [<enlistment>]::
     -+    Adds the enlistment's repository to the list of registered repositories
     -+    and starts background maintenance. If `<enlistment>` is not provided,
     -+    then the enlistment associated with the current working directory is
     -+    registered.
     ++	Adds the enlistment's repository to the list of registered repositories
     ++	and starts background maintenance. If `<enlistment>` is not provided,
     ++	then the enlistment associated with the current working directory is
     ++	registered.
     +++
     ++Note: when this subcommand is called in a worktree that is called `src/`, its
     ++parent directory is considered to be the Scalar enlistment. If the worktree is
     ++_not_ called `src/`, it itself will be considered to be the Scalar enlistment.
      +
       SEE ALSO
       --------

5: 2a6ac170e6b ! 5: 6142f75875b scalar: 'unregister' stops background maintenance
@@ contrib/scalar/scalar.txt: SYNOPSIS

DESCRIPTION
-----------
-@@ contrib/scalar/scalar.txt: register [<enlistment>]::
- then the enlistment associated with the current working directory is
- registered.
+@@ contrib/scalar/scalar.txt: Note: when this subcommand is called in a worktree that is called src/, its
+ parent directory is considered to be the Scalar enlistment. If the worktree is
+ _not_ called src/, it itself will be considered to be the Scalar enlistment.

+Unregister
+~~~~~~~~~~
+
+unregister [<enlistment>]::
-+ Remove the specified repository from the list of repositories
-+ registered with Scalar and stop the scheduled background maintenance.
++ Remove the specified repository from the list of repositories
++ registered with Scalar and stop the scheduled background maintenance.
+
SEE ALSO
--------

  6:  087fc9be194 =  6:  82dd253154f scalar: let 'unregister' handle a deleted enlistment directory gracefully
  7:  c272ff4069d !  7:  fb7c931ddb3 scalar: implement 'scalar list'
     @@ contrib/scalar/scalar.txt: an existing Git worktree with Scalar whose name is no
      +~~~~
      +
      +list::
     -+    To see which repositories are currently registered by the service, run
     -+    `scalar list`. This subcommand does not need to be run inside a Scalar
     -+    enlistment.
     ++	To see which repositories are currently registered by the service, run
     ++	`scalar list`. This subcommand does not need to be run inside a Scalar
     ++	enlistment.
      +
       Register
       ~~~~~~~~
  8:  2cbf0b61113 !  8:  f3223c10788 scalar: implement the `clone` subcommand
     @@ Commit message
          experience and experiments of the Microsoft Windows and the Microsoft
          Office development teams.
      
     -    Note: We intentionally use a slightly wasteful `set_config()` function
     -    (which does not reuse a single `strbuf`, for example, though performance
     -    _really_ does not matter here) for convenience and readability.
     -
     -    Also note: since the `scalar clone` command is by far the most commonly
     +    Note: since the `scalar clone` command is by far the most commonly
          called `scalar` subcommand, we document it at the top of the manual
          page.
      
     @@ contrib/scalar/scalar.c: static int unregister_dir(void)
      +
      +	cp.git_cmd = 1;
      +	strvec_pushl(&cp.args, "ls-remote", "--symref", url, "HEAD", NULL);
     -+	strbuf_addstr(&out, "-\n");
      +	if (!pipe_command(&cp, NULL, 0, &out, 0, NULL, 0)) {
     -+		char *ref = out.buf;
     -+
     -+		while ((ref = strstr(ref + 1, "\nref: "))) {
     -+			const char *p;
     -+			char *head, *branch;
     ++		const char *line = out.buf;
      +
     -+			ref += strlen("\nref: ");
     -+			head = strstr(ref, "\tHEAD");
     ++		while (*line) {
     ++			const char *eol = strchrnul(line, '\n'), *p;
     ++			size_t len = eol - line;
     ++			char *branch;
      +
     -+			if (!head || memchr(ref, '\n', head - ref))
     ++			if (!skip_prefix(line, "ref: ", &p) ||
     ++			    !strip_suffix_mem(line, &len, "\tHEAD")) {
     ++				line = eol + (*eol == '\n');
      +				continue;
     ++			}
      +
     -+			if (skip_prefix(ref, "refs/heads/", &p)) {
     -+				branch = xstrndup(p, head - p);
     ++			eol = line + len;
     ++			if (skip_prefix(p, "refs/heads/", &p)) {
     ++				branch = xstrndup(p, eol - p);
      +				strbuf_release(&out);
      +				return branch;
      +			}
      +
      +			error(_("remote HEAD is not a branch: '%.*s'"),
     -+			      (int)(head - ref), ref);
     ++			      (int)(eol - p), p);
      +			strbuf_release(&out);
      +			return NULL;
      +		}
     @@ contrib/scalar/scalar.txt: an existing Git worktree with Scalar whose name is no
      +~~~~~
      +
      +clone [<options>] <url> [<enlistment>]::
     -+    Clones the specified repository, similar to linkgit:git-clone[1]. By
     -+    default, only commit and tree objects are cloned. Once finished, the
     -+    worktree is located at `<enlistment>/src`.
     ++	Clones the specified repository, similar to linkgit:git-clone[1]. By
     ++	default, only commit and tree objects are cloned. Once finished, the
     ++	worktree is located at `<enlistment>/src`.
      ++
      +The sparse-checkout feature is enabled (except when run with `--full-clone`)
      +and the only files present are those in the top-level directory. Use
     @@ contrib/scalar/scalar.txt: an existing Git worktree with Scalar whose name is no
      +
      +-b <name>::
      +--branch <name>::
     -+    Instead of checking out the branch pointed to by the cloned repository's
     -+    HEAD, check out the `<name>` branch instead.
     ++	Instead of checking out the branch pointed to by the cloned
     ++	repository's HEAD, check out the `<name>` branch instead.
      +
      +--[no-]full-clone::
     -+    A sparse-checkout is initialized by default. This behavior can be turned
     -+    off via `--full-clone`.
     ++	A sparse-checkout is initialized by default. This behavior can be
     ++	turned off via `--full-clone`.
      +
       List
       ~~~~
       
       list::
     -     To see which repositories are currently registered by the service, run
     --    `scalar list`. This subcommand does not need to be run inside a Scalar
     --    enlistment.
     -+    `scalar list`. This subcommand, like `clone`, does not need to be run
     -+    inside a Scalar enlistment.
     + 	To see which repositories are currently registered by the service, run
     +-	`scalar list`. This subcommand does not need to be run inside a Scalar
     +-	enlistment.
     ++	`scalar list`. This subcommand, like `clone`, does not need to be run
     ++	inside a Scalar enlistment.
       
       Register
       ~~~~~~~~

9: 9af1c37c2ea ! 9: b3c4b3dccc6 scalar: teach 'clone' to support the --single-branch option
@@ contrib/scalar/scalar.txt: scalar - an opinionated repository management tool
scalar register [<enlistment>]
scalar unregister [<enlistment>]
@@ contrib/scalar/scalar.txt: subdirectories outside your sparse-checkout by using git ls-tree HEAD.
- Instead of checking out the branch pointed to by the cloned repository's
- HEAD, check out the <name> branch instead.
+ Instead of checking out the branch pointed to by the cloned
+ repository's HEAD, check out the <name> branch instead.

+--[no-]single-branch::
-+ Clone only the history leading to the tip of a single branch,
-+ either specified by the --branch option or the primary
-+ branch remote's HEAD points at.
++ Clone only the history leading to the tip of a single branch, either
++ specified by the --branch option or the primary branch remote's
++ HEAD points at.
++
+Further fetches into the resulting repository will only update the +remote-tracking branch for the branch this option was used for the initial
@@ contrib/scalar/scalar.txt: subdirectories outside your sparse-checkout by using
+--single-branch clone was made, no remote-tracking branch is created. +
--[no-]full-clone::
- A sparse-checkout is initialized by default. This behavior can be turned
- off via --full-clone.
+ A sparse-checkout is initialized by default. This behavior can be
+ turned off via --full-clone.

## contrib/scalar/t/t9099-scalar.sh ##
@@ contrib/scalar/t/t9099-scalar.sh: test_expect_success 'set up repository to clone' '
10: c3f16bccd02 ! 10: b7fc2dc29c8 scalar: implement the run command
@@ contrib/scalar/scalar.txt: scalar clone [--single-branch] [--branch <main-branch
DESCRIPTION
-----------
@@ contrib/scalar/scalar.txt: unregister [<enlistment>]::
- Remove the specified repository from the list of repositories
- registered with Scalar and stop the scheduled background maintenance.
+ Remove the specified repository from the list of repositories
+ registered with Scalar and stop the scheduled background maintenance.

+Run
+~~~
+
+scalar run ( all | config | commit-graph | fetch | loose-objects | pack-files ) [<enlistment>]::
-+ Run the given maintenance task (or all tasks, if all was specified).
-+ Except for all and config, this subcommand simply hands off to
-+ linkgit:git-maintenance[1] (mapping fetch to prefetch and
-+ pack-files to incremental-repack).
++ Run the given maintenance task (or all tasks, if all was specified).
++ Except for all and config, this subcommand simply hands off to
++ linkgit:git-maintenance[1] (mapping fetch to prefetch and
++ pack-files to incremental-repack).
++
+These tasks are run automatically as part of the scheduled maintenance,
+as soon as the repository is registered with Scalar. It should therefore
11: 13056f02018 ! 11: 9a834c23d08 scalar: allow reconfiguring an existing enlistment

     @@ contrib/scalar/scalar.c: static int set_recommended_config(void)
      -		{ "core.untrackedCache", "false" },
      +		{ "core.untrackedCache", "false", 1 },
       #endif
     --		{ "core.bare", "false" },
      -		{ "core.logAllRefUpdates", "true" },
      -		{ "credential.https://dev.azure.com.useHttpPath", "true" },
      -		{ "credential.validate", "false" }, /* GCM4W-only */
     @@ contrib/scalar/scalar.c: static int set_recommended_config(void)
      -		{ "feature.experimental", "false" },
      -		{ "fetch.unpackLimit", "1" },
      -		{ "fetch.writeCommitGraph", "false" },
     -+		{ "core.bare", "false", 1 },
      +		{ "core.logAllRefUpdates", "true", 1 },
      +		{ "credential.https://dev.azure.com.useHttpPath", "true", 1 },
      +		{ "credential.validate", "false", 1 }, /* GCM4W-only */

12: 732a28c22fc ! 12: 79e9f5d203a scalar: teach 'reconfigure' to optionally handle all registered enlistments
@@ contrib/scalar/scalar.txt: After a Scalar upgrade, or when the configuration of
reconfigure the enlistment.

+With the --all option, all enlistments currently registered with Scalar
-+will be reconfigured. This option is meant to to be run every time Scalar
-+was upgraded.
++will be reconfigured. This option is meant to to be run every time after
++Scalar is upgraded.
+
SEE ALSO
--------
13: 13afbd68812 ! 13: 94a21982652 scalar: implement the delete command
@@ contrib/scalar/scalar.txt: scalar register [<enlistment>]
DESCRIPTION
-----------
@@ contrib/scalar/scalar.txt: With the --all option, all enlistments currently registered with Scalar
- will be reconfigured. This option is meant to to be run every time Scalar
- was upgraded.
+ will be reconfigured. This option is meant to to be run every time after
+ Scalar is upgraded.

+Delete
+~~~~~~
+
+delete <enlistment>::
-+ This subcommand lets you delete an existing Scalar enlistment from your
-+ local file system, unregistering the repository.
++ This subcommand lets you delete an existing Scalar enlistment from your
++ local file system, unregistering the repository.
+
SEE ALSO
--------
14: 73d08c0c894 ! 14: 707d8e19683 scalar: implement the version command
@@ Commit message

Since Scalar is now tightly coupled with Git, it does not make sense for them to show different versions. Therefore, it shows the same output as
- git versions. For backwards-compatibility with the .NET version,
+ git version. For backwards-compatibility with the .NET version,
scalar version prints to stderr, though (git version prints to stdout instead).

15: 6455b18f1b6 ! 15: 26e23b5c5e5 scalar: accept -C and -c options before the subcommand
@@ contrib/scalar/scalar.txt: The scalar command implements various subcommands,
+The following options can be specified _before_ the subcommand:
+
+-C <directory>::
-+ Before running the subcommand, change the working directory. This
-+ option imitates the same option of linkgit:git[1].
++ Before running the subcommand, change the working directory. This
++ option imitates the same option of linkgit:git[1].
+
+-c <key>=<value>::
-+ For the duration of running the specified subcommand, configure this
-+ setting. This option imitates the same option of linkgit:git[1].
++ For the duration of running the specified subcommand, configure this
++ setting. This option imitates the same option of linkgit:git[1].
+
COMMANDS
--------

gitgitgadget