Skip to contentSkip to conversation

Unable to de-init stubborn submodule

DISCUSSION Ævar Arnfjörð Bjarmason 3 participants 3 messages Jan 11, 2018

lore
Share & tools
lore
#0 lore raw

What if we wanted to drop sha1collisiondetection/ as a submodule and replace it with a copy of what's now in sha1dc/? I ran into this with another project, but here's a way to reproduce it on git.git:

    (
        rm -rf /tmp/git &&
        git clone [email protected]:git/git.git /tmp/git
        cd /tmp/git &&
        git tag nuke-before &&
        git submodule update --init &&
        git rm -r .gitmodules sha1collisiondetection &&
        git commit -m"Nuke sha1dc submodule" &&
        cp -Rvp sha1dc sha1collisiondetection &&
        git add sha1collisiondetection &&
        git commit -m"Now it's not a submodule" &&
        git tag nuke-after &&
        git reset --hard nuke-before &&
        git submodule update --init && # skip this and the below won't fail
        git reset --hard nuke-after && # Emulate someone doing a pull
        git ls-tree HEAD | grep sha1collisiondetection && # OK, shows "tree" not "commit"
        test $(git rev-parse HEAD) == $(git -C sha1collisiondetection/ log -1 --pretty=format:%H) && echo OK || echo WTF
    )

This results in a really bizarre state where according to ls-tree sha1collisiondetection is a tree at the current commit:

    040000 tree 81583289d96bdde4b366c243ab524ea28d895ea5    sha1collisiondetection

But git still believes there's a submodule there for some reason, and shows the log for the upstream sha1collisiondetection project:

git -C sha1collisiondetection/ log -1
commit 19d97bf (HEAD, origin/master, origin/HEAD, master)
Merge: 3f14d1b c93f0b4
Author: Dan Shumow <[email protected]>
Date: Sat Jul 1 12:36:15 2017 -0700

Merge pull request #37 from avar/fixup-pull-request-34

Fix endian detection logic for Sparc, little endian BSD etc.

Doing:

git submodule deinit sha1collisiondetection

Does nothing to help, then I thought it might be:

git config -f .git/config -l|grep ^submodule submodule.sha1collisiondetection.active=true submodule.sha1collisiondetection.url=https://github.com/cr-marcstevens/sha1collisiondetection.git

But running:

    git config --remove-section submodule.sha1collisiondetection

Doesn't help either, neither does removing the index:

    rm .git/index &&
    git reset --hard

If you then do:

rm -rf .git/modules

You'll get this error:

git -C sha1collisiondetection/ log -1
fatal: Not a git repository: /tmp/git/sha1collisiondetection/../.git/modules/sha1collisiondetection

But I can't see what's still referencing it.

This problem is avoided if, as noted with a comment I skip:

    git submodule update --init

But I shouldn't need to remember to de-init a submodule before moving to a new commit that doesn't have it, least I end up in some seemingly unrecoverable state.

Am I missing something obvious here?

#1 lore raw
Quoted 73 lines · Ævar Arnfjörð Bjarmason

What if we wanted to drop sha1collisiondetection/ as a submodule and replace it with a copy of what's now in sha1dc/? I ran into this with another project, but here's a way to reproduce it on git.git:

    (
        rm -rf /tmp/git &&
        git clone [email protected]:git/git.git /tmp/git
        cd /tmp/git &&
        git tag nuke-before &&
        git submodule update --init &&
        git rm -r .gitmodules sha1collisiondetection &&
        git commit -m"Nuke sha1dc submodule" &&
        cp -Rvp sha1dc sha1collisiondetection &&
        git add sha1collisiondetection &&
        git commit -m"Now it's not a submodule" &&
        git tag nuke-after &&
        git reset --hard nuke-before &&
        git submodule update --init && # skip this and the below won't fail
        git reset --hard nuke-after && # Emulate someone doing a pull
        git ls-tree HEAD | grep sha1collisiondetection && # OK, shows "tree" not "commit"
        test $(git rev-parse HEAD) == $(git -C sha1collisiondetection/ log -1 --pretty=format:%H) && echo OK || echo WTF
    )

This results in a really bizarre state where according to ls-tree sha1collisiondetection is a tree at the current commit:

    040000 tree 81583289d96bdde4b366c243ab524ea28d895ea5    sha1collisiondetection

But git still believes there's a submodule there for some reason, and shows the log for the upstream sha1collisiondetection project:

git -C sha1collisiondetection/ log -1
commit 19d97bf (HEAD, origin/master, origin/HEAD, master)
Merge: 3f14d1b c93f0b4
Author: Dan Shumow <[email protected]>
Date: Sat Jul 1 12:36:15 2017 -0700

Merge pull request #37 from avar/fixup-pull-request-34

Fix endian detection logic for Sparc, little endian BSD etc.

Doing:

git submodule deinit sha1collisiondetection

Does nothing to help, then I thought it might be:

git config -f .git/config -l|grep ^submodule submodule.sha1collisiondetection.active=true submodule.sha1collisiondetection.url=https://github.com/cr-marcstevens/sha1collisiondetection.git

But running:

    git config --remove-section submodule.sha1collisiondetection

Doesn't help either, neither does removing the index:

    rm .git/index &&
    git reset --hard

If you then do:

rm -rf .git/modules

You'll get this error:

git -C sha1collisiondetection/ log -1
fatal: Not a git repository: /tmp/git/sha1collisiondetection/../.git/modules/sha1collisiondetection

But I can't see what's still referencing it.

This problem is avoided if, as noted with a comment I skip:

Quoted 7 lines
    git submodule update --init

But I shouldn't need to remember to de-init a submodule before moving to a new commit that doesn't have it, least I end up in some seemingly unrecoverable state.

Am I missing something obvious here?

One thing you could try is adding --recurse-submodule flags to the reset commands. IIRC reset ignores submodules unless you specify that flag.

Signature · 1 line
Brandon Williams
#2 lore raw
In reply to Ævar Arnfjörð Bjarmason
Quoted 10 lines · Ævar Arnfjörð Bjarmason

What if we wanted to drop sha1collisiondetection/ as a submodule and replace it with a copy of what's now in sha1dc/? I ran into this with another project, but here's a way to reproduce it on git.git:

    (
        rm -rf /tmp/git &&
        git clone [email protected]:git/git.git /tmp/git
        cd /tmp/git &&
        git tag nuke-before &&
        git submodule update --init &&

At this point

    $ tail -n 3 .git/config
    [submodule "sha1collisiondetection"]
        active = true
        url = https://github.com/cr-marcstevens/sha1collisiondetection.git
Quoted 1 line
        git rm -r .gitmodules sha1collisiondetection &&

no need to delete the .gitmodules here. Git detects you're deleting a submodule and adjusts the .gitmodules file (it is empty after just "git rm sha1coll...")

Quoted 6 lines
        git commit -m"Nuke sha1dc submodule" &&
        cp -Rvp sha1dc sha1collisiondetection &&
        git add sha1collisiondetection &&
        git commit -m"Now it's not a submodule" &&
        git tag nuke-after &&
        git reset --hard nuke-before &&

As bmwill said, you may want to reset with --recurse-submodules here, "git ls-tree HEAD |grep sha1c" will show commit and "git status" thinks everything is fine on disk. I have no suspicion to believe otherwise. But we digress, you chose to not use that flag.

Quoted 2 lines
        git submodule update --init && # skip this and the below won't fail
        git reset --hard nuke-after && # Emulate someone doing a pull
$ git reset --hard nuke-after
warning: unable to rmdir 'sha1collisiondetection': Directory not empty
HEAD is now at f683a1b81 Now it's not a submodule
$ git reset --hard nuke-after --recurse-submodules
HEAD is now at f683a1b81 Now it's not a submodule
Quoted 7 lines
        git ls-tree HEAD | grep sha1collisiondetection && # OK, shows "tree" not "commit"
        test $(git rev-parse HEAD) == $(git -C sha1collisiondetection/ log -1 --pretty=format:%H) && echo OK || echo WTF
    )

This results in a really bizarre state where according to ls-tree sha1collisiondetection is a tree at the current commit:

Quoted 6 lines
    040000 tree 81583289d96bdde4b366c243ab524ea28d895ea5    sha1collisiondetection

But git still believes there's a submodule there for some reason, and shows the log for the upstream sha1collisiondetection project:

git -C sha1collisiondetection/ log -1

(A)
That is because the reset without flag do touch submodules kept the submodule in place and the -C in this command tells git to cd into that directory, (which is the submodule, an 'independent' repo) and shows the log of said repo.

Quoted 5 lines

commit 19d97bf (HEAD, origin/master, origin/HEAD, master)
Merge: 3f14d1b c93f0b4
Author: Dan Shumow <[email protected]>
Date: Sat Jul 1 12:36:15 2017 -0700

Quoted 7 lines

Merge pull request #37 from avar/fixup-pull-request-34

Fix endian detection logic for Sparc, little endian BSD etc.

Doing:

git submodule deinit sha1collisiondetection

Doing this after you reset to nuke-after, there are no submodules from the superprojects point of view, hence no submodule is touched/modified. :/ (There just happens to be a stray repository at a place where we'd want to have a tree).

Quoted 9 lines

Does nothing to help, then I thought it might be:

git config -f .git/config -l|grep ^submodule submodule.sha1collisiondetection.active=true submodule.sha1collisiondetection.url=https://github.com/cr-marcstevens/sha1collisiondetection.git

But running:

    git config --remove-section submodule.sha1collisiondetection

This made the submodule not active any more, (note that at the current tree there
is no submodule to begin with... so what effect to we want here?)

Quoted 8 lines

Doesn't help either, neither does removing the index:

    rm .git/index &&
    git reset --hard

If you then do:

rm -rf .git/modules

Getting out the big hammer, eh?

instead of deleting the submodules git repo, remove its worktree:

rm -rf sha1collisiondetection

(This is best done at time (A) above)

Quoted 7 lines

You'll get this error:

git -C sha1collisiondetection/ log -1
fatal: Not a git repository: /tmp/git/sha1collisiondetection/../.git/modules/sha1collisiondetection

But I can't see what's still referencing it.

$ls -la /tmp/git/sha1collisiondetection
...
-rw-r--r-- 1 sbeller eng 47 Jan 11 10:02 .git
...

$ cat sha1collisiondetection/.git
gitdir: ../.git/modules/sha1collisiondetection

This is a crucial file, telling you there is a repo with the worktree at the place where
.git lives and the git dir is at the location indicated (which you removed)

Quoted 9 lines

This problem is avoided if, as noted with a comment I skip:

    git submodule update --init

But I shouldn't need to remember to de-init a submodule before moving to a new commit that doesn't have it, least I end up in some seemingly unrecoverable state.

Am I missing something obvious here?

The "git submodule update/init/deinit" is a "raw" command, (not to be confused with plumbing, I just made up "raw commands" ;) that is very good to deal with submodules, but that tool has sharp edges. Try this:

  $ git clone --recurse-submodules=. [email protected]:git/git.git testgit
  $ cd testgit/
  $ git config --list |grep submod
submodule.active=.
submodule.sha1collisiondetection.url=https://github.com/cr-marcstevens/sha1collisiondetection.git
  $ git config submodule.recurse true
  $ git tag before-nuke
  $ git rm sha1collisiondetection/
  rm 'sha1collisiondetection'
  $ git commit -a -m delete_sub
  $ git tag after-nuke
  $ git checkout before-nuke
  $ git status # and more checks
  $ git checkout after-nuke
  $ git status # and more checks