Skip to contentSkip to conversation

worktree: add --recurse-submodules support to git worktree add

PATCH In review Jimmy Aguilar Mena 3 participants 4 messages Apr 16 – Apr 17

lore
Share & tools
lore
#0 lore raw

cover worktree: add --recurse-submodules support to git worktree add

This series implements the native --recurse-submodules flag for "git worktree add" discussed in the earlier RFC thread.

The approach follows Phillip Wood's and Junio's feedback: each linked worktree gets its own per-worktree submodule gitdir under $GIT_COMMON_DIR/worktrees/<id>/modules/<name>/, so HEAD, refs, and the index are independent per worktree while pack files and loose objects are shared via hardlinks. The gitdir isolation is the same model git worktree already uses for the superproject.

Patch 1 adds the --recurse-submodules flag to builtin/worktree.c and calls "git submodule update --init --recursive" from within the new worktree after checkout.

Patch 2 teaches clone_submodule() in builtin/submodule--helper.c to detect when the main worktree already has the submodule cloned and reuse it via "git clone --local --no-checkout --separate-git-dir" instead of fetching from the remote URL. This avoids redundant network access and disk use: the objects are already present locally.

Patch 3 adds tests to t2405-worktree-submodule.sh covering both the happy path and the gitdir-isolation invariant.

Cleanup is automatic: submodule gitdirs under worktrees/<id>/modules/ are removed when "git worktree remove" calls remove_dir_recursively() on the worktree entry, exactly as with the superproject's per-worktree state.

Changes since the RFC:
- Replaced the shell-script cp -al prototype with a native C implementation in builtin/worktree.c and builtin/submodule--helper.c.
- Per-worktree gitdir isolation is now handled by the existing submodule_name_to_gitdir() path; no extra plumbing is needed.
- Added t2405 tests verifying both behaviour and gitdir placement.

Jimmy Aguilar Mena (3):
worktree: add --recurse-submodules flag to worktree add
submodule--helper: reuse main-worktree gitdir in linked worktrees
t2405: add tests for worktree add --recurse-submodules

 builtin/submodule--helper.c   | 54 +++++++++++++++++++++++++++++++++++
 builtin/worktree.c            | 23 +++++++++++++++
 t/t2405-worktree-submodule.sh | 24 +++++++++++++++-
 3 files changed, 100 insertions(+), 1 deletion(-)
#1 lore raw
Quoted 6 lines · Jimmy Aguilar Mena

The approach follows Phillip Wood's and Junio's feedback: each linked worktree gets its own per-worktree submodule gitdir under $GIT_COMMON_DIR/worktrees/<id>/modules/<name>/, so HEAD, refs, and the index are independent per worktree while pack files and loose objects are shared via hardlinks. The gitdir isolation is the same model git worktree already uses for the superproject.

I do not quite follow. The point of git-native worktree support (which improved a lot compared to its precursor, "git-new-workdir", is that it can work well in a hardlink-challenged platforms. You shouldn't worry about "hardlinking" yourself at all.

After the superproject successfully did "submodule init", you can move the submodule's repository with "absorbgitdirs" to $GIT_DIR/modules/<submodule>/ of the superproject. The primary motivation behind this feature was that you can switch to a commit in the superproject that does *not* have the submodule bound to it at all (and obviously you do not want to lose the submodule repository only because you tentatively switch to such a commit and have to re-download when you switch back), but I think it gives the single instance of submodule repository that you can share across worktrees of the submodule. Because the single directory created with "absorbgitdirs" looks like a bare repository, you should be able to create two worktrees off of that, with their own HEAD etc.

#2 lore raw
Quoted 14 lines · Junio C Hamano
Quoted 6 lines · Jimmy Aguilar Mena

The approach follows Phillip Wood's and Junio's feedback: each linked worktree gets its own per-worktree submodule gitdir under $GIT_COMMON_DIR/worktrees/<id>/modules/<name>/, so HEAD, refs, and the index are independent per worktree while pack files and loose objects are shared via hardlinks. The gitdir isolation is the same model git worktree already uses for the superproject.

I do not quite follow. The point of git-native worktree support (which improved a lot compared to its precursor, "git-new-workdir", is that it can work well in a hardlink-challenged platforms. You shouldn't worry about "hardlinking" yourself at all.

Quoted 12 lines

After the superproject successfully did "submodule init", you can move the submodule's repository with "absorbgitdirs" to $GIT_DIR/modules/<submodule>/ of the superproject. The primary motivation behind this feature was that you can switch to a commit in the superproject that does *not* have the submodule bound to it at all (and obviously you do not want to lose the submodule repository only because you tentatively switch to such a commit and have to re-download when you switch back), but I think it gives the single instance of submodule repository that you can share across worktrees of the submodule. Because the single directory created with "absorbgitdirs" looks like a bare repository, you should be able to create two worktrees off of that, with their own HEAD etc.

I haven't thought much about it but that would mean that "git worktree remove" ought to remove the submodule's worktree when the worktree containing the submodule is removed. Worktrees avoid hardlinks by creating a "commondir" file in the worktree's gitdir which contains the relative path to "$GIT_COMMON_DIR". I think we could probably do the same here and create "$GIT_COMMON_DIR/worktrees/<id>/modules/<name>/commondir" containing "../../../../modules/<name>" if we want to store the submodule's gitdir under the worktree's gitdir. That way removing a worktree's gitdir removes all the gitdirs of its submodules without any extra effort. There are probably other tradeoffs between the two approaches that I've not thought of.

Thanks

Phillip

#3 lore raw
Quoted 28 lines · Phillip Wood
Quoted 14 lines · Junio C Hamano
Quoted 6 lines · Jimmy Aguilar Mena

The approach follows Phillip Wood's and Junio's feedback: each linked worktree gets its own per-worktree submodule gitdir under $GIT_COMMON_DIR/worktrees/<id>/modules/<name>/, so HEAD, refs, and the index are independent per worktree while pack files and loose objects are shared via hardlinks.  The gitdir isolation is the same model git worktree already uses for the superproject.

I do not quite follow.  The point of git-native worktree support (which improved a lot compared to its precursor, "git-new-workdir", is that it can work well in a hardlink-challenged platforms.  You shouldn't worry about "hardlinking" yourself at all.

Quoted 12 lines

After the superproject successfully did "submodule init", you can move the submodule's repository with "absorbgitdirs" to $GIT_DIR/modules/<submodule>/ of the superproject.  The primary motivation behind this feature was that you can switch to a commit in the superproject that does *not* have the submodule bound to it at all (and obviously you do not want to lose the submodule repository only because you tentatively switch to such a commit and have to re-download when you switch back), but I think it gives the single instance of submodule repository that you can share across worktrees of the submodule.  Because the single directory created with "absorbgitdirs" looks like a bare repository, you should be able to create two worktrees off of that, with their own HEAD etc.

Quoted 1 line

I haven't thought much about it but that would mean that "git worktree remove" ought to remove the submodule's worktree when the worktree containing the submodule is removed. Worktrees avoid hardlinks by creating a "commondir" file in the worktree's gitdir which contains the relative path to "$GIT_COMMON_DIR". I think we could probably do the same here and create "$GIT_COMMON_DIR/worktrees/<id>/modules/<name>/ commondir" containing "../../../../modules/<name>" if we want to store the submodule's gitdir under the worktree's gitdir. That way removing a worktree's gitdir removes all the gitdirs of its submodules without any extra effort. There are probably other tradeoffs between the two approaches that I've not thought of.

I've realized that creating the submodule's gitdir under the worktree's gitdir means that "git gc" running in the submodule repository wont see the per-worktree refs and index file and will happily prune those objects. Junio's suggestion avoids that problem.

Thanks

Phillip