Problem with fsck and invalid submodule path in history
Hello,
I have a git repository with an error in a submodule path in the history.
The submodule path is “-f”, which is not allowed. But for some reason, it’s in the history of the git, and I’m trying to find a way to manage it without having to rewrite the full history on the main gitlab (if possible)...
To reproduce this unwanted history:
mkdir test-bad-history
cd test-bad-history
echo "Test git submodule problems" > README.md
git init
git add README.md
git commit --message="Start test"
git submodule add https://github.com/leachim6/hello-world.git
git commit --message="Commit new submodule with correct path" # the bad part...
git mv hello-world -- -f
git commit --message="Move submodule to an invalid path" # correct it...
git mv -- -f valid-path
sed -i.bak 's/-f/valid-path/' .gitmodules
git add .gitmodules
git commit --message="Back to a valid path"
After that, even if the git is working correctly, we have a “bad” history if we check with fsck:
$ git fsck
Checking object directories: 100% (256/256), done.
error in blob 19a97d3b70760c74b780c8134e33f5392292c2e6: gitmodulesPath: disallowed submodule path: -fIs it possible to correct it? Must git handle this kind of errors?
Thanks in advance for any help.
--
Olivier Bornet
[email protected]
Quoted 5 lines · Olivier Bornet
I have a git repository with an error in a submodule path in the history.
The submodule path is “-f”, which is not allowed. But for some reason, it’s in the history of the git, and I’m trying to find a way to manage it without having to rewrite the full history on the main gitlab (if possible)...To reproduce this unwanted history:
Quoted 7 lines
After that, even if the git is working correctly, we have a “bad” history if we check with fsck:
$ git fsck Checking object directories: 100% (256/256), done. error in blob 19a97d3b70760c74b780c8134e33f5392292c2e6: gitmodulesPath: disallowed submodule path: -fIs it possible to correct it? Must git handle this kind of errors?
To correct without rewriting history, no.
However, you can tell 'git fsck' to ignore it using the 'fsck.skipList' configuration variable (see 'git help config'; for some reason it's not included in 'git fsck's documentation):
$ cat <<EOF >.git-fsck-skiplist
Quoted 3 lines
# invalid submodule path 19a97d3b70760c74b780c8134e33f5392292c2e6 EOF
$ git config fsck.skipList .git-fsck-skiplist $ git fsck Checking object directories: 100% (256/256), done.
It may or may not be worth committing this file, I'm not quite sure what the best practice is. By committing it others don't have to maintain such a skiplist file themselves, though they still have to set the config variable. OTOH, if anyone sets this config variable and attempts to run 'git fsck' while on a branch that doesn't contain this file, then they will get a 'fatal: could not open object name
list: .git-fsck-skiplist' error.
And it won't help anyone cloning the repository with 'fetch.fsckObjects' enabled.
PATCH 1/1 Documentation/git-fsck.txt: include fsck.* config variables
The 'fsck.skipList' and 'fsck.<msg-id>' config variables might be easier to discover when they are documented in 'git fsck's man page.
--- Documentation/git-fsck.txt | 5 +++++ 1 file changed, 5 insertions(+)
Documentation/git-fsck.txt +5 -0
@@ -104,6 +104,11 @@ care about this output and want to speed it up further. progress status even if the standard error stream is not directed to a terminal. +CONFIGURATION +------------- + +include::config/fsck.txt[] + DISCUSSION ----------
Signature · 1 line
2.22.0.924.g7c03687a56
Re: [PATCH] Documentation/git-fsck.txt: include fsck.* config variables
Quoted 13 lines · SZEDER Gábor
The 'fsck.skipList' and 'fsck.<msg-id>' config variables might be easier to discover when they are documented in 'git fsck's man page.
Signed-off-by SZEDER Gábor <[email protected]>--- Documentation/git-fsck.txt | 5 +++++ 1 file changed, 5 insertions(+)Documentation/git-fsck.txt +0
-0
Documentation/git-fsck.txt care about this output and want to speed it up further. · line 110
Quoted 7 lines
Documentation/git-fsck.txt +4
-0@@ -104,6 +104,11 @@ care about this output and want to speed it up further. progress status even if the standard error stream is not directed to a terminal. +CONFIGURATION +------------- + +include::config/fsck.txt[]
Before this include let's add:
The below documentation is the same as what’s found in git-config(1):
As I did for a similar change in git-gc in b6a8d09f6d ("gc docs: include the "gc.*" section from "config" in "gc"", 2019-04-07). Sometimes we repeat ourselves, it helps the reader to know this isn't some slightly different prose than what's in git-config.
Quoted 3 lines
Documentation/git-fsck.txt +1
-0+ DISCUSSION ----------
Re: [PATCH] Documentation/git-fsck.txt: include fsck.* config variables
Quoted 25 lines · Ævar Arnfjörð Bjarmason
Quoted 13 lines · SZEDER Gábor
The 'fsck.skipList' and 'fsck.<msg-id>' config variables might be easier to discover when they are documented in 'git fsck's man page.
Signed-off-by SZEDER Gábor <[email protected]>--- Documentation/git-fsck.txt | 5 +++++ 1 file changed, 5 insertions(+)Documentation/git-fsck.txt +0
-0Documentation/git-fsck.txt care about this output and want to speed it up further. · line 110
Quoted 7 lines
Documentation/git-fsck.txt +4
-0@@ -104,6 +104,11 @@ care about this output and want to speed it up further. progress status even if the standard error stream is not directed to a terminal. +CONFIGURATION +------------- + +include::config/fsck.txt[]Before this include let's add:
Quoted 2 lines
The below documentation is the same as what’s found in git-config(1):
I actually do not think we would want to do that. I am all for the kind of 'include' proposed by this patch, and we should strive to make it easier for us to make sure the duplicated text are in sync.
But that would mean that the readers will have to see the "is the same as the other one" over and over. If our documentation set is consistent, they should not have to.
I think we *must* make such a note in a total opposite case, i.e. "here are the summary of the most often used options; for full list, see git-config(1)".
Quoted 4 lines
As I did for a similar change in git-gc in
b6a8d09f6d("gc docs: include the "gc.*" section from "config" in "gc"", 2019-04-07). Sometimes we repeat ourselves, it helps the reader to know this isn't some slightly different prose than what's in git-config.
So, I think we should revert that part out of b6a8d09f6d, too.
Re: [PATCH] Documentation/git-fsck.txt: include fsck.* config variables
Quoted 9 lines · Junio C Hamano
Quoted 4 lines
Before this include let's add:
The below documentation is the same as what’s found in git-config(1):
I actually do not think we would want to do that. I am all for the kind of 'include' proposed by this patch, and we should strive to make it easier for us to make sure the duplicated text are in sync.
Quoted 7 lines
But that would mean that the readers will have to see the "is the same as the other one" over and over. If our documentation set is consistent, they should not have to.
I think we *must* make such a note in a total opposite case, i.e. "here are the summary of the most often used options; for full list, see git-config(1)".
I disagree. _We_ know that the content is the same, because we are looking at the source that says "include". But as a user, how do I know when I get to one section or the other that it is something I have already read and can skip over?
Perhaps if Git were entirely consistent here, they could make the assumption that "CONFIG" sections were always duplicated and know this already. But I think even that is asking a bit much. Unless they are intimately familiar with our documentation, they don't know that we are, in fact, consistent. And we are in an uphill battle with every other thing the user has read, which may not agree with our assumptions of consistency. ;)
So IMHO it's worth leaving a note that guides the reader, as long as it's short (and I think this one is).
That said, I think an even _better_ solution would be to avoid includes, and instead make it clear when we are pointing the user to shared content. Then we get them to the right place _and_ explicitly instruct them that concepts/content are shared. For config, for example, I've worked with a previous system that did something like:
- include fsck.* documentation in the git-fsck manpage
- make a master table of config options in git-config.1 with _just_ the names and the associated manpage where the definition can be found. This serves as an index if you don't know where to look.
This would probably involve creating new concept-based pages for some of the groupings (e.g., where does "remote.*" config go?), but I think that would probably help round out our documentation (if there is a concept with related config options but we don't explain it anywhere, that is probably a gap we should fix).
The biggest downside is that chasing down references in manpages sucks. For the HTML documentation we'd ideally hyperlink from the git-config.1 index into each definition, but there's no way to do that with a regular manpage.
-Peff
PS This is an approach I've advocated for a while:
https://public-inbox.org/git/[email protected]/
but haven't actually done much about, so perhaps I should be putting my money where my mouth is. ;)
Re: [PATCH] Documentation/git-fsck.txt: include fsck.* config variables
Quoted 15 lines · Jeff King
Quoted 13 lines · Junio C Hamano
Quoted 4 lines
Before this include let's add:
The below documentation is the same as what’s found in git-config(1):
I actually do not think we would want to do that. I am all for the kind of 'include' proposed by this patch, and we should strive to make it easier for us to make sure the duplicated text are in sync.
But that would mean that the readers will have to see the "is the same as the other one" over and over. If our documentation set is consistent, they should not have to.
Quoted 8 lines
Quoted 3 lines
I think we *must* make such a note in a total opposite case, i.e. "here are the summary of the most often used options; for full list, see git-config(1)".
I disagree. _We_ know that the content is the same, because we are looking at the source that says "include". But as a user, how do I know when I get to one section or the other that it is something I have already read and can skip over?
I want to raise the user expectation so that they would expect from our documentation, unless we say "these are different", we would never say conflicting things in two places.
So,... I disagree.
Hello,
Quoted 5 lines · SZEDER Gábor
However, you can tell 'git fsck' to ignore it using the 'fsck.skipList' configuration variable (see 'git help config'; for some reason it's not included in 'git fsck's documentation):
Thanks for pointing it out. Thanks to it, we have managed to handle the error with our gitlab automatic fsck procedure.
Have a nice day.
--
Olivier Bornet
[email protected]