worktree: copy-on-write creation and ephemeral branches

When many worktrees share one repository -- for example a fleet of agents each needing an isolated checkout -- "git worktree add" is costly at scale. Objects are shared via the common directory, but the working tree is not:
each add rewrites every tracked file, so N worktrees cost N full checkouts of disk and I/O. A branch created only to back one of those worktrees is also left behind when the worktree is removed.

Patch 1 adds "git worktree add --reflink". On a copy-on-write filesystem it populates the new worktree by reflinking the current worktree's files and index, then "git reset --hard" rewrites only the paths that differ from . reflink_file() in copy.c uses FICLONE (Linux), clonefile() (macOS), and ReFS block cloning (FSCTL_DUPLICATE_EXTENTS_TO_FILE, as Git LFS does). Elsewhere it is probed up front and falls back to a normal checkout. worktree.reflink (true/false/auto) sets the default; --no-reflink overrides.

Uncommitted edits to tracked files in the source do not leak into the new worktree. The copied index is refreshed before reset, and because every reflinked file is a new inode the refresh re-validates by content. Only untracked files are carried over. The option stays opt-in for that reason:
it copies the source's untracked and ignored files, which a plain checkout does not.

For many worktrees at one commit, the series does not share a branch. git-worktree(1) shows a single update-ref transaction that creates a ref per worktree (or a detached HEAD). A shared branch still lets a stale index in one worktree silently revert a commit made in another.

Patch 2 treats a branch that "git worktree add " created itself as owned by that worktree. "git worktree remove" deletes it. Branches named with -b/-B, or that already existed, are left alone. --keep-branch keeps an owned branch.

Changes since v1:

* Dropped the patch that allowed one branch to be checked out in several worktrees. Documented the per-ref / detached-HEAD pattern instead (brian m. carlson, Junio).
* Explained, in the commit message and the manual, why a dirty source worktree does not leak tracked content (brian m. carlson).
* Added the ReFS block-cloning arm (brian m. carlson).
* Noted overlayfs as a supported setup when the lower filesystem supports reflinks (brian m. carlson).
* Kept --reflink opt-in. A whole-tree reflink is not invisible: it carries untracked files, and cloning every file can be slower than checkout when the files are small (Junio; qub1n on the ReFS work).
* Rebased onto 47fdf2208c ("The 7th batch"). reflink_file() now takes a repository argument, matching copy_file().
* Added ephemeral-branch reaping for the auto-named branch that "worktree add " creates.

ReFS measurements (32 GiB Dev Drive, Windows build 26300). The USB disk itself would not format as ReFS, so the volume is a Dev Drive VHD on an NTFS partition of that disk. NTFS rejects the clone ioctl (win32 error 1 there, error 50 on C:) and falls back to a full write.

The clone shares the source clusters and does not flush. Four thousand clones of one 8 MiB file, 31.3 GiB of logical names, were read back unbuffered and SHA-256'd in full: 4000 of 4000 matched the source. The source was then overwritten with a different pattern. A read-only FlushFileBuffers failed with win32 error 5; a writable reopen succeeded. After that flush, 4000 of 4000 clones still hashed to the original contents and none hashed to the new source. While the clones were created the free-space counter fell to about 1 GiB, then returned to 28.38 GiB with the clones still present. The backing VHD did not grow. Deleting the clones freed nothing further.

Jason Newton (4):
worktree: add --reflink for copy-on-write worktree creation
worktree: reap ephemeral branches created by "worktree add"
worktree: disable reflink on Windows
worktree: keep the reflink probe referenced on Windows

 Documentation/config/worktree.adoc |  12 ++
 Documentation/git-worktree.adoc    |  75 ++++++-
 builtin/branch.c                   |  28 +++
 builtin/worktree.c                 | 315 ++++++++++++++++++++++++++++-
 copy.c                             |  83 ++++++++
 copy.h                             |  14 ++
 t/t2400-worktree-add.sh            | 166 +++++++++++++++
 t/t3200-branch.sh                  |   2 +-
 t/t3400-rebase.sh                  |   2 +-
 t/t5516-fetch-push.sh              |   2 +-
 10 files changed, 690 insertions(+), 9 deletions(-)

base-commit: 47fdf2208c5b94c8cc01c106b8f7c85efb91c926
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2317%2Fnevion%2Fworktree-reflink-cow-v3
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2317/nevion/worktree-reflink-cow-v3
Pull-Request: https://github.com/git/git/pull/2317

Range-diff vs v2:

1: 83841020f4 ! 1: a163f737af worktree: add --reflink for copy-on-write worktree creation
@@ Commit message
and bcachefs implement, and the one overlayfs forwards when the lower filesystem can clone. This path has been tested on btrfs.

- On macOS it uses clonefile(), the APFS primitive. That path is
- implemented and has not been run here.
+ On macOS it uses clonefile(), the APFS primitive. The t2400
+ REFLINK prerequisite probes that with "cp -c", because BSD cp has no
+ --reflink=always, so the macOS CI jobs exercise this path on APFS.

On Windows it uses ReFS block cloning, FSCTL_DUPLICATE_EXTENTS_TO_FILE, as Git LFS does. The destination is
@@ t/t2400-worktree-add.sh: test_expect_success 'relative worktree sets extension c
+# Reflinks are only available on copy-on-write filesystems (btrfs, XFS, +# bcachefs, APFS, ...). Where they are unavailable, --reflink transparently +# falls back to a regular checkout, which the test above already covers.
++#
++# GNU cp probes with --reflink=always. macOS cp has no such option; -c ++# is clonefile(2), which is what the APFS CI jobs exercise.
+test_lazy_prereq REFLINK '
+ echo probe >reflink-src &&
-+ cp --reflink=always reflink-src reflink-dst 2>/dev/null
++ if test_have_prereq MACOS
++ then
++ cp -c reflink-src reflink-dst 2>/dev/null
++ else
++ cp --reflink=always reflink-src reflink-dst 2>/dev/null
++ fi
+'
+
+test_expect_success REFLINK '--reflink carries untracked files and reconciles changes' '

 2:  6e2acdc5e2 = 2:  042485f0e9 worktree: reap ephemeral branches created by "worktree add"
 -:  ---------- > 3:  91806975a6 worktree: disable reflink on Windows
 -:  ---------- > 4:  876fb541a8 worktree: keep the reflink probe referenced on Windows

gitgitgadget