submodule add + autocrlf + safecrlf

Hi Folks,

When 'git submodule add' uses 'git config' to create a '.gitmodules' file it gets LF newlines that the subsequent 'git add --force .gitmodules' rejects if autocrlf and safecrlf are both enabled. This series adds a test and proposes a fix that simply uses '-c core.safecrlf=false' to disable safecrlf when adding '.gitmodules'.

I'm not excited by allowing a LF file in work tree that has clearly been configured to prefer CRLF, but avoiding that for .gitmodules is probably a separate issue.

-Brad

Brad King (2):
submodule: Demonstrate failure to add with auto/safecrlf
submodule: Tolerate auto/safecrlf when adding .gitmodules

 git-submodule.sh           |    2 +-
 t/t7400-submodule-basic.sh |   13 +++++++++++++
 2 files changed, 14 insertions(+), 1 deletion(-)

1.7.10

If 'core.autocrlf' and 'core.safecrlf' are both enabled then 'git submodule add' fails with an error such as

fatal: LF would be replaced by CRLF in .gitmodules Failed to register submodule 'submod'

because it generates a '.gitmodules' file with LF newlines that are rejected by 'git add' under this configuration. Demonstrate this known breakage with a new test in t7400-submodule-basic covering the case.

---
 t/t7400-submodule-basic.sh |   13 +++++++++++++
 1 file changed, 13 insertions(+)

index 81827e6..5eaeb04 100755
@@ -43,6 +43,7 @@ test_expect_success 'setup - hide init subdirectory' '
 
 test_expect_success 'setup - repository to add submodules to' '
 	git init addtest &&
+	git init addtest-crlf &&
 	git init addtest-ignore
 '
 

@@ -98,6 +99,18 @@ test_expect_success 'submodule add' '
 	test_cmp empty untracked
 '
 
+test_expect_failure 'submodule add with core.autocrlf and core.safecrlf' '
+	(
+		cd addtest-crlf &&
+		git config core.autocrlf true &&
+		git config core.safecrlf true &&
+		git submodule add "$submodurl" submod &&
+		echo ".gitmodules" >expect &&
+		git ls-files -- .gitmodules >actual &&
+		test_cmp expect actual
+	)
+'
+
 test_expect_success 'submodule add to .gitignored path fails' '
 	(
 		cd addtest-ignore &&

1.7.10

I have a suspicion that "git config" should be taught about this kind of thing instead.

Shoudn't your .git/config file that is outside the revision control also end with CRLF if your platform and project prefer CRLF over LF?

That would be reasonable, but is beyond the scope I'm willing to tackle myself.

I don't actually have a project like this so I have no strong opinion on this issue. I discovered the problem by accident and have already worked around it in the obscure case it matters for me.

Perhaps only the first patch in the series is worth inclusion. It can become the beginning of a series if someone wants to address handling crlf in config files. Note that in the case I discovered this the crlf configuration was in ~/.gitconfig so the project knew nothing about it and had no .gitattributes.

-Brad

If that is the case, my preference would be to wait until that "git config" change happens (provided that it is a reasonably way forward---it may turn out to be a bad idea) and do nothing to clutter "git submodule" at all. In the meantime, if there are real projects that are hurt by this, we can tell them to explicitly specify that .gitmodules is a LF text file in their .gitattributes.

Temporarily disable 'core.safecrlf' to add '.gitmodules' so that 'git add' does not reject the LF newlines we write to the file even if both 'core.autocrlf' and 'core.safecrlf' are enabled. This fixes known breakage tested in t7400-submodule-basic.

---
 git-submodule.sh           |    2 +-
 t/t7400-submodule-basic.sh |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

index 5c61ae2..ed9a54a 100755
@@ -303,7 +303,7 @@ Use -f if you really want to add it." >&2
 
 	git config -f .gitmodules submodule."$sm_path".path "$sm_path" &&
 	git config -f .gitmodules submodule."$sm_path".url "$repo" &&
-	git add --force .gitmodules ||
+	git -c core.safecrlf=false add --force .gitmodules ||
 	die "$(eval_gettext "Failed to register submodule '\$sm_path'")"
 }
 

index 5eaeb04..9a4da9b 100755
@@ -99,7 +99,7 @@ test_expect_success 'submodule add' '
 	test_cmp empty untracked
 '
 
-test_expect_failure 'submodule add with core.autocrlf and core.safecrlf' '
+test_expect_success 'submodule add with core.autocrlf and core.safecrlf' '
 	(
 		cd addtest-crlf &&
 		git config core.autocrlf true &&

1.7.10

Am 20.06.2012 16:43, schrieb Brad King:

Hmm, I have no objections against the intention of the patch. But as I understand it this message will reoccur when the user e.g. edits the .gitmodules file later with any editor who just writes lfs and adds it.

I don't know terribly much about crlf support but maybe flagging the .gitmodules file in .gitattributes would be a better solution here? Opinions?

Once a user edits the file with an outside tool it is his/her responsibility to add .gitattributes for the file. In the reported case Git is creating the file and already knows the crlf mode when creating it.

I think Junio's proposal to teach "git config" to respect crlf configuration is a more general solution.

-Brad

I don't think so. It should not be an issue at all that .gitmodules has CRLF or even mixed line endings; the config parser knows that its input is a text file and ignores the CRs already.

The only problem is when adding the file, because git is not aware that .gitmodules is, by definition, a text file. The point of safecrlf was to prevent accidents with files which are really binary, or for which mixed line endings must be preserved; .gitmodules is not such a file. We know that, but we never tell git.

We can paper over the problem by normalizing line endings when config writes it out, but that does not cover the case of somebody editing it manually and introducing mixed line endings. Nor does it help if somebody checks in a .gitmodules file with CRLF, which we then checkout on a LF-based system. Or if we do a merge between two versions with different line endings.

The only sane thing is to have a canonical in-repo representation. Fortunately we already have the infrastructure for that, and in theory it should be as easy as adding ".gitmodules text" to our built-in gitattributes (you could even do "eol=lf", but I don't see a reason not to respect the native line endings in the working tree, given that git can handle the CRLFs just fine).

I say "in theory" there because I am not sure whether specifying a file as definitely text via attributes will actually suppress the safecrlf check or not. IMHO, it should, since safecrlf is really about preventing false positives via autocrlf or text=auto.

I don't see any reason for each individual repo to have to add these attributes manually. This is a git-specific file, and the format is dictated by git. We know that it's a text file, so why not help out the user? We should possibly do the same thing for .gitattributes and .gitignore.

-Peff

OK.

Am 20.06.2012 21:11, schrieb Jeff King:

A quick test shows that unfortunately theory differs from practice here. Adding ".gitmodules text" to the built-in gitattributes lets the test Brad wrote still fail. You have to use ".gitmodules eol=lf" to make it pass. I stopped digging deeper at this point.

I really like this approach. (And in the long run would like to see a ini-file aware merge driver being used for the .gitmodules file too, which would just merge submodules added in different branches instead of producing a conflict)

Am 20.06.2012 20:06, schrieb Brad King:

Yep.