Skip to contentSkip to conversation

dir: Treat a repository without commits as a repository

PATCH Superseded Kyle Meyer 2 participants 16 messages Mar 14, 2019 – Apr 4, 2019

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Kyle Meyer

author
#0 lore raw

cover dir: Treat a repository without commits as a repository

When the working tree contains a repository with no commits, it's treated as an empty directory, not a repository:

    $ git init
    $ git init no-commit && touch no-commit/untracked
    $ git init with-commit && touch with-commit/untracked
    $ git -C with-commit commit --allow-empty -mmsg
    $ git ls-files -o
    no-commit/untracked
    with-commit/

That's admittedly a weird case that is unlikely to happen in the wild, and indeed I didn't observe it in the wild. In DataLad, we rely on ls-files -o to stop at the boundary of a submodule (like it does with "with-commit" above), and we noticed the "repository with no commit" exception on a minimal snippet that was meant for debugging another issue [*].

This series makes the "is repository?" check in treat_directory() use is_nonbare_repository_dir() instead of resolve_gitlink_ref() so that a repository without any commits is treated like a repository rather than an empty directory. This makes the reporting from commands like ls-files consistent for the case shown above, and it also avoids the case where 'git add no-commit' (no trailing slash) adds the untracked files of the no-commit/ repository to the index of the current repository.

[*]: https://github.com/datalad/datalad/issues/3139#issuecomment-460542647

Kyle Meyer (4):
submodule: refuse to add repository with no commits
t3000: move non-submodule repo test to separate file
t3009: test that ls-files -o traverses bogus repo
dir: do not traverse repositories with no commits

 dir.c                                   |  6 ++-
 git-submodule.sh                        | 12 +++++-
 t/t3000-ls-files-others.sh              |  7 ----
 t/t3009-ls-files-others-nonsubmodule.sh | 56 +++++++++++++++++++++++++
 t/t3700-add.sh                          |  1 +
 t/t7400-submodule-basic.sh              | 11 ++++-
 6 files changed, 81 insertions(+), 12 deletions(-)
 create mode 100755 t/t3009-ls-files-others-nonsubmodule.sh
Signature · 1 line
2.21.0
author
#1 lore raw

PATCH 1/4 submodule: refuse to add repository with no commits

When the path given to 'git submodule add' is an existing repository that is not in the index, the repository is passed to 'git add'. If this repository doesn't have any commits, we don't get a useful
result: there is no subproject OID to track, and any untracked files in the sub-repository are added to the current repository.

Detect if the path is a repository with no commits and abort to avoid getting into this state unless --force is used. Reacting to --force isn't very useful, especially because an upcoming commit will make 'git add' fail in this situation, but it allows us to use the same 'git add --dry-run' condition as the ignored path case.

---
 git-submodule.sh           | 12 ++++++++++--
 t/t7400-submodule-basic.sh | 11 ++++++++++-
 2 files changed, 20 insertions(+), 3 deletions(-)
git-submodule.sh +10 -2
index 514ede2596..6c74656027 100755
@@ -234,10 +234,18 @@ cmd_add()
 	if test -z "$force" &&
 		! git add --dry-run --ignore-missing --no-warn-embedded-repo "$sm_path" > /dev/null 2>&1
 	then
-		eval_gettextln "The following path is ignored by one of your .gitignore files:
+		if test -d "$sm_path" &&
+			test -z $(git -C "$sm_path" rev-parse --show-cdup 2>/dev/null) &&
+			! git -C "$sm_path" rev-parse --verify -q HEAD >/dev/null
+		then
+			die "$(eval_gettext "'\$sm_path' does not have any commits")"
+		else
+			eval_gettextln "\
+The following path is ignored by one of your .gitignore files:
 \$sm_path
 Use -f if you really want to add it." >&2
-		exit 1
+			exit 1
+		fi
 	fi
 
 	if test -n "$custom_name"
t/t7400-submodule-basic.sh +10 -1
index aba2d4d6ee..6adf640143 100755
@@ -46,6 +46,15 @@ test_expect_success 'submodule update aborts on missing gitmodules url' '
 	test_must_fail git submodule init
 '
 
+test_expect_success 'add aborts on repository with no commits' '
+	cat >expected <<-\EOF &&
+	'"'repo-no-commits'"' does not have any commits
+	EOF
+	git init repo-no-commits &&
+	test_must_fail git submodule add ../a ./repo-no-commits 2>actual &&
+	test_i18ncmp expected actual
+'
+
 test_expect_success 'setup - repository in init subdirectory' '
 	mkdir init &&
 	(
@@ -809,7 +818,7 @@ test_expect_success '../bar/a/b/c works with relative local path - ../foo/bar.gi
 		cp pristine-.git-config .git/config &&
 		cp pristine-.gitmodules .gitmodules &&
 		mkdir -p a/b/c &&
-		(cd a/b/c && git init) &&
+		(cd a/b/c && git init && test_commit msg) &&
 		git config remote.origin.url ../foo/bar.git &&
 		git submodule add ../bar/a/b/c ./a/b/c &&
 		git submodule init &&
Signature · 1 line
2.21.0
author
#2 lore raw

Re: [PATCH 1/4] submodule: refuse to add repository with no commits

Kyle Meyer <[email protected]> writes:

[...]

Quoted 12 lines
git-submodule.sh +3 -1
index 514ede2596..6c74656027 100755
@@ -234,10 +234,18 @@ cmd_add()
 	if test -z "$force" &&
 		! git add --dry-run --ignore-missing --no-warn-embedded-repo "$sm_path" > /dev/null 2>&1
 	then
-		eval_gettextln "The following path is ignored by one of your .gitignore files:
+		if test -d "$sm_path" &&
+			test -z $(git -C "$sm_path" rev-parse --show-cdup 2>/dev/null) &&
+			! git -C "$sm_path" rev-parse --verify -q HEAD >/dev/null

git-submodule.sh cmd_add() · line 248

Quoted 10 lines
git-submodule.sh +7 -1
+		then
+			die "$(eval_gettext "'\$sm_path' does not have any commits")"
+		else
+			eval_gettextln "\
+The following path is ignored by one of your .gitignore files:
 \$sm_path
 Use -f if you really want to add it." >&2
-		exit 1
+			exit 1
+		fi

I didn't think through this check, which would have been obvious had I ran the added test without the rest of the patches in this series. It assumes that the 'git add --dry-run' call fails, but that failure depends on the last patch of this series. So I'd need to move this patch to the end or find a new place for this "no commits" check.

author
#3 lore raw

v2 cover dir: Treat a repository without commits as a repository

Quoted 24 lines

Kyle Meyer <[email protected]> writes:

[...]

Quoted 20 lines
git-submodule.sh +8 -2
index 514ede2596..6c74656027 100755
@@ -234,10 +234,18 @@ cmd_add()
 	if test -z "$force" &&
 		! git add --dry-run --ignore-missing --no-warn-embedded-repo "$sm_path" > /dev/null 2>&1
 	then
-		eval_gettextln "The following path is ignored by one of your .gitignore files:
+		if test -d "$sm_path" &&
+			test -z $(git -C "$sm_path" rev-parse --show-cdup 2>/dev/null) &&
+			! git -C "$sm_path" rev-parse --verify -q HEAD >/dev/null
+		then
+			die "$(eval_gettext "'\$sm_path' does not have any commits")"
+		else
+			eval_gettextln "\
+The following path is ignored by one of your .gitignore files:
 \$sm_path
 Use -f if you really want to add it." >&2
-		exit 1
Quoted 8 lines

git-submodule.sh cmd_add() · line 248

Quoted 2 lines
git-submodule.sh +2 -0
+			exit 1
+		fi

I didn't think through this check, which would have been obvious had I ran the added test without the rest of the patches in this series. It assumes that the 'git add --dry-run' call fails, but that failure depends on the last patch of this series. So I'd need to move this patch to the end or find a new place for this "no commits" check.

v2 moves the "no commits" check outside of the 'git add --dry-run' failure condition so that the first patch doesn't depend on the final patch in this series.

Kyle Meyer (4):
submodule: refuse to add repository with no commits
t3000: move non-submodule repo test to separate file
t3009: test that ls-files -o traverses bogus repo
dir: do not traverse repositories with no commits

 dir.c                                   |  6 ++-
 git-submodule.sh                        |  7 ++++
 t/t3000-ls-files-others.sh              |  7 ----
 t/t3009-ls-files-others-nonsubmodule.sh | 56 +++++++++++++++++++++++++
 t/t3700-add.sh                          |  1 +
 t/t7400-submodule-basic.sh              | 11 ++++-
 6 files changed, 78 insertions(+), 10 deletions(-)
 create mode 100755 t/t3009-ls-files-others-nonsubmodule.sh

Range-diff against v1:
1: e0db7e3c3c ! 1: b080e2c557 submodule: refuse to add repository with no commits

    @@ -9,10 +9,9 @@
         in the sub-repository are added to the current repository.
     
         Detect if the path is a repository with no commits and abort to avoid
    -    getting into this state unless --force is used.  Reacting to --force
    -    isn't very useful, especially because an upcoming commit will make
    -    'git add' fail in this situation, but it allows us to use the same
    -    'git add --dry-run' condition as the ignored path case.
    +    getting into this state.  Note that this check must come before the
    +    'git add --dry-run' check because an upcoming commit will make 'git
    +    add' fail in this situation.
     
         Signed-off-by: Kyle Meyer <[email protected]>
     
    @@ -20,26 +19,19 @@
      --- a/git-submodule.sh
      +++ b/git-submodule.sh
     @@
    + 		die "$(eval_gettext "'\$sm_path' already exists in the index and is not a submodule")"
    + 	fi
    + 
    ++	if test -d "$sm_path" &&
    ++		test -z $(git -C "$sm_path" rev-parse --show-cdup 2>/dev/null)
    ++	then
    ++	    git -C "$sm_path" rev-parse --verify -q HEAD >/dev/null ||
    ++	    die "$(eval_gettext "'\$sm_path' does not have any commits")"
    ++	fi
    ++
      	if test -z "$force" &&
      		! git add --dry-run --ignore-missing --no-warn-embedded-repo "$sm_path" > /dev/null 2>&1
      	then
    --		eval_gettextln "The following path is ignored by one of your .gitignore files:
    -+		if test -d "$sm_path" &&
    -+			test -z $(git -C "$sm_path" rev-parse --show-cdup 2>/dev/null) &&
    -+			! git -C "$sm_path" rev-parse --verify -q HEAD >/dev/null
    -+		then
    -+			die "$(eval_gettext "'\$sm_path' does not have any commits")"
    -+		else
    -+			eval_gettextln "\
    -+The following path is ignored by one of your .gitignore files:
    - \$sm_path
    - Use -f if you really want to add it." >&2
    --		exit 1
    -+			exit 1
    -+		fi
    - 	fi
    - 
    - 	if test -n "$custom_name"
     
      diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
      --- a/t/t7400-submodule-basic.sh
2:  6eed1f5daf = 2:  c027701842 t3000: move non-submodule repo test to separate file
3:  7ba3209762 = 3:  97f53e30c0 t3009: test that ls-files -o traverses bogus repo
4:  2901375dc1 = 4:  a926b87102 dir: do not traverse repositories with no commits
Signature · 1 line
2.21.0
author
#4 lore raw

PATCH v2 1/4 submodule: refuse to add repository with no commits

When the path given to 'git submodule add' is an existing repository that is not in the index, the repository is passed to 'git add'. If this repository doesn't have any commits, we don't get a useful
result: there is no subproject OID to track, and any untracked files in the sub-repository are added to the current repository.

Detect if the path is a repository with no commits and abort to avoid getting into this state. Note that this check must come before the 'git add --dry-run' check because an upcoming commit will make 'git add' fail in this situation.

---
 git-submodule.sh           |  7 +++++++
 t/t7400-submodule-basic.sh | 11 ++++++++++-
 2 files changed, 17 insertions(+), 1 deletion(-)
git-submodule.sh +7 -0
index 514ede2596..5a9dc9bf7e 100755
@@ -231,6 +231,13 @@ cmd_add()
 		die "$(eval_gettext "'\$sm_path' already exists in the index and is not a submodule")"
 	fi
 
+	if test -d "$sm_path" &&
+		test -z $(git -C "$sm_path" rev-parse --show-cdup 2>/dev/null)
+	then
+	    git -C "$sm_path" rev-parse --verify -q HEAD >/dev/null ||
+	    die "$(eval_gettext "'\$sm_path' does not have any commits")"
+	fi
+
 	if test -z "$force" &&
 		! git add --dry-run --ignore-missing --no-warn-embedded-repo "$sm_path" > /dev/null 2>&1
 	then
t/t7400-submodule-basic.sh +10 -1
index aba2d4d6ee..6adf640143 100755
@@ -46,6 +46,15 @@ test_expect_success 'submodule update aborts on missing gitmodules url' '
 	test_must_fail git submodule init
 '
 
+test_expect_success 'add aborts on repository with no commits' '
+	cat >expected <<-\EOF &&
+	'"'repo-no-commits'"' does not have any commits
+	EOF
+	git init repo-no-commits &&
+	test_must_fail git submodule add ../a ./repo-no-commits 2>actual &&
+	test_i18ncmp expected actual
+'
+
 test_expect_success 'setup - repository in init subdirectory' '
 	mkdir init &&
 	(
@@ -809,7 +818,7 @@ test_expect_success '../bar/a/b/c works with relative local path - ../foo/bar.gi
 		cp pristine-.git-config .git/config &&
 		cp pristine-.gitmodules .gitmodules &&
 		mkdir -p a/b/c &&
-		(cd a/b/c && git init) &&
+		(cd a/b/c && git init && test_commit msg) &&
 		git config remote.origin.url ../foo/bar.git &&
 		git submodule add ../bar/a/b/c ./a/b/c &&
 		git submodule init &&
Signature · 1 line
2.21.0
#5 lore raw

Re: [PATCH v2 1/4] submodule: refuse to add repository with no commits

Quoted 10 lines · Kyle Meyer

When the path given to 'git submodule add' is an existing repository that is not in the index, the repository is passed to 'git add'. If this repository doesn't have any commits, we don't get a useful
result: there is no subproject OID to track, and any untracked files in the sub-repository are added to the current repository.

Detect if the path is a repository with no commits and abort to avoid getting into this state. Note that this check must come before the 'git add --dry-run' check because an upcoming commit will make 'git add' fail in this situation.

OK, in short, in an already-populated repository,

	$ git init junk ;# there was no 'junk' subdirectory
	$ git add junk

would have been a successful no-op, and after the above two commands,

	$ >junk/one
	$ git add junk

would have added junk/one as a normal blob to the top-level project, ignoring the fact that the user wanted to eventually populate junk as a repository and add it as a submodule. You'll be teaching "git add" not to do so, and pay attention to the fact that junk/.git exists, even though junk/.git/HEAD is on an unborn branch.

Which makes sense to me.

Quoted 53 lines
---
 git-submodule.sh           |  7 +++++++
 t/t7400-submodule-basic.sh | 11 ++++++++++-
 2 files changed, 17 insertions(+), 1 deletion(-)
git-submodule.sh +7 -0
index 514ede2596..5a9dc9bf7e 100755
@@ -231,6 +231,13 @@ cmd_add()
 		die "$(eval_gettext "'\$sm_path' already exists in the index and is not a submodule")"
 	fi
 
+	if test -d "$sm_path" &&
+		test -z $(git -C "$sm_path" rev-parse --show-cdup 2>/dev/null)
+	then
+	    git -C "$sm_path" rev-parse --verify -q HEAD >/dev/null ||
+	    die "$(eval_gettext "'\$sm_path' does not have any commits")"
+	fi
+
 	if test -z "$force" &&
 		! git add --dry-run --ignore-missing --no-warn-embedded-repo "$sm_path" > /dev/null 2>&1
 	then
t/t7400-submodule-basic.sh +10 -1
index aba2d4d6ee..6adf640143 100755
@@ -46,6 +46,15 @@ test_expect_success 'submodule update aborts on missing gitmodules url' '
 	test_must_fail git submodule init
 '
 
+test_expect_success 'add aborts on repository with no commits' '
+	cat >expected <<-\EOF &&
+	'"'repo-no-commits'"' does not have any commits
+	EOF
+	git init repo-no-commits &&
+	test_must_fail git submodule add ../a ./repo-no-commits 2>actual &&
+	test_i18ncmp expected actual
+'
+
 test_expect_success 'setup - repository in init subdirectory' '
 	mkdir init &&
 	(
@@ -809,7 +818,7 @@ test_expect_success '../bar/a/b/c works with relative local path - ../foo/bar.gi
 		cp pristine-.git-config .git/config &&
 		cp pristine-.gitmodules .gitmodules &&
 		mkdir -p a/b/c &&
-		(cd a/b/c && git init) &&
+		(cd a/b/c && git init && test_commit msg) &&
 		git config remote.origin.url ../foo/bar.git &&
 		git submodule add ../bar/a/b/c ./a/b/c &&
 		git submodule init &&
author
#6 lore raw

PATCH v2 2/4 t3000: move non-submodule repo test to separate file

In reply to Kyle Meyer

a2d5156c2b (resolve_gitlink_ref: ignore non-repository paths, 2016-01-22) added a test to t3000-ls-files-others.sh to check that 'ls-files -o' does not die() when given a subdirectory that looks like a repository but is actually a subdirectory containing a bogus .git file.

Move this test to a separate file in preparation for testing scenarios with non-submodule repositories that are not bogus.

---
 t/t3000-ls-files-others.sh              |  7 -------
 t/t3009-ls-files-others-nonsubmodule.sh | 21 +++++++++++++++++++++
 2 files changed, 21 insertions(+), 7 deletions(-)
 create mode 100755 t/t3009-ls-files-others-nonsubmodule.sh
t/t3000-ls-files-others.sh +0 -7
index afd4756134..b4f9fc4580 100755
@@ -65,13 +65,6 @@ test_expect_success '--no-empty-directory hides empty directory' '
 	test_cmp expected3 output
 '
 
-test_expect_success 'ls-files --others handles non-submodule .git' '
-	mkdir not-a-submodule &&
-	echo foo >not-a-submodule/.git &&
-	git ls-files -o >output &&
-	test_cmp expected1 output
-'
-
 test_expect_success SYMLINKS 'ls-files --others with symlinked submodule' '
 	git init super &&
 	git init sub &&
t/t3009-ls-files-others-nonsubmodule.sh new file +21 -0
new file mode 100755
index 0000000000..cc66a4a14d
@@ -0,0 +1,21 @@
+#!/bin/sh
+
+test_description='test git ls-files --others with non-submodule repositories'
+
+. ./test-lib.sh
+
+test_expect_success 'setup: expected output' '
+	cat >expected <<-EOF
+	expected
+	output
+	EOF
+'
+
+test_expect_success 'ls-files --others handles non-submodule .git' '
+	mkdir not-a-submodule &&
+	echo foo >not-a-submodule/.git &&
+	git ls-files -o >output &&
+	test_cmp expected output
+'
+
+test_done
Signature · 1 line
2.21.0
#7 lore raw

Re: [PATCH v2 2/4] t3000: move non-submodule repo test to separate file

Quoted 8 lines · Kyle Meyer

a2d5156c2b (resolve_gitlink_ref: ignore non-repository paths, 2016-01-22) added a test to t3000-ls-files-others.sh to check that 'ls-files -o' does not die() when given a subdirectory that looks like a repository but is actually a subdirectory containing a bogus .git file.

Move this test to a separate file in preparation for testing scenarios with non-submodule repositories that are not bogus.

It is unclear to me why this is needed.

Quoted 4 lines

+++ b/t/t3009-ls-files-others-nonsubmodule.sh

t/t3009-ls-files-others-nonsubmodule.sh new file +2 -0
@@ -0,0 +1,21 @@
+#!/bin/sh
+

t/t3009-ls-files-others-nonsubmodule.sh line 12

Quoted 10 lines
t/t3009-ls-files-others-nonsubmodule.sh new file +10 -0
+test_description='test git ls-files --others with non-submodule repositories'
+
+. ./test-lib.sh
+
+test_expect_success 'setup: expected output' '
+	cat >expected <<-EOF
+	expected
+	output
+	EOF
+'

I think this is overkill. Usually we have one expectation for a single test, so having the above inside the actual test below makes more sense.

Or are you planning to add more tests before the test_done we see below, all of which expect the above output? It would make perfect sense if it were the case, but I do not think that is what is happenning here...

Quoted 8 lines
t/t3009-ls-files-others-nonsubmodule.sh new file +8 -0
+test_expect_success 'ls-files --others handles non-submodule .git' '
+	mkdir not-a-submodule &&
+	echo foo >not-a-submodule/.git &&
+	git ls-files -o >output &&
+	test_cmp expected output
+'
+
+test_done
author
#8 lore raw

Re: [PATCH v2 2/4] t3000: move non-submodule repo test to separate file

Quoted 8 lines · Junio C Hamano
Quoted 6 lines · Kyle Meyer

a2d5156c2b (resolve_gitlink_ref: ignore non-repository paths, 2016-01-22) added a test to t3000-ls-files-others.sh to check that 'ls-files -o' does not die() when given a subdirectory that looks like a repository but is actually a subdirectory containing a bogus .git file.

Quoted 4 lines
Quoted 2 lines

Move this test to a separate file in preparation for testing scenarios with non-submodule repositories that are not bogus.

It is unclear to me why this is needed.

It's not needed. My thinking, which I didn't do a good job of spelling out above, is

We're going to be adding a test that checks how 'ls-files -o' handles a few different scenarios involving untracked repositories. This new test should go into a separate file rather than t3000-ls-files-others.sh because it substantially changes the shared directory layout that the t3000 tests work on. Like the upcoming test, the "non-submodule .git" test from t3000 deals with a (bogus) untracked repository, so let's split it off into a separate test file that will be extended with the other scenarios.

Perhaps that's not a good reason to touch t3000, though. I could drop this patch, as well as the next one, and just add the new test file in the final patch.

Quoted 10 lines
Quoted 10 lines

+++ b/t/t3009-ls-files-others-nonsubmodule.sh

t/t3009-ls-files-others-nonsubmodule.sh new file +8 -0
@@ -0,0 +1,21 @@
+#!/bin/sh
+
+test_description='test git ls-files --others with non-submodule repositories'
+
+. ./test-lib.sh
+
+test_expect_success 'setup: expected output' '
+	cat >expected <<-EOF
Quoted 8 lines

t/t3009-ls-files-others-nonsubmodule.sh line 12

Quoted 4 lines
t/t3009-ls-files-others-nonsubmodule.sh new file +4 -0
+	expected
+	output
+	EOF
+'

I think this is overkill. Usually we have one expectation for a single test, so having the above inside the actual test below makes more sense.

OK, I'll move this into the test.

author
#9 lore raw

PATCH v2 3/4 t3009: test that ls-files -o traverses bogus repo

In reply to Kyle Meyer

When a2d5156c2b (resolve_gitlink_ref: ignore non-repository paths, 2016-01-22) added this test, the purpose was to check the 'ls-files -o' didn't die() when processing the bogus repository. The expected output didn't even need to be adjusted for the addition because the bogus repository is treated as an empty directory.

Let's add another scenario to test that 'ls-files -o' lists an untracked file within this directory. Doing so isn't useful for the original purpose of the test, but it will be helpful for highlighting that this traversal does _not_ happen in the non-bogus scenarios that will be added next. Rename the original subdirectory to highlight the difference with the new one.

---
 t/t3009-ls-files-others-nonsubmodule.sh | 21 ++++++++++++++++++---
 1 file changed, 18 insertions(+), 3 deletions(-)
t/t3009-ls-files-others-nonsubmodule.sh +18 -3
index cc66a4a14d..9ed75928aa 100755
@@ -1,6 +1,14 @@
 #!/bin/sh
 
-test_description='test git ls-files --others with non-submodule repositories'
+test_description='test git ls-files --others with non-submodule repositories
+
+This test runs git ls-files --others with the following working tree:
+
+    repo-bogus-no-files/
+      directory with no files aside from a bogus .git file
+    repo-bogus-untracked-file/
+      directory with a bogus .git file and another untracked file
+'
 
 . ./test-lib.sh
 
@@ -8,12 +16,19 @@ test_expect_success 'setup: expected output' '
 	cat >expected <<-EOF
 	expected
 	output
+	repo-bogus-untracked-file/untracked
 	EOF
 '
 
+test_expect_success 'setup: directories' '
+	mkdir repo-bogus-no-files &&
+	echo foo >repo-bogus-no-files/.git &&
+	mkdir repo-bogus-untracked-file &&
+	echo foo >repo-bogus-untracked-file/.git &&
+	: >repo-bogus-untracked-file/untracked
+'
+
 test_expect_success 'ls-files --others handles non-submodule .git' '
-	mkdir not-a-submodule &&
-	echo foo >not-a-submodule/.git &&
 	git ls-files -o >output &&
 	test_cmp expected output
 '
Signature · 1 line
2.21.0
author
#10 lore raw

PATCH v2 4/4 dir: do not traverse repositories with no commits

In reply to Kyle Meyer

When treat_directory() encounters a directory that is not in the index and DIR_NO_GITLINKS is unset, it calls resolve_gitlink_ref() to decide if a directory looks like a repository, in which case the directory won't be traversed. As a result, 'status -uall' and 'ls-files -o' will show only the directory, even when there are untracked files within the directory.

For the unusual case where a repository doesn't have any commits, resolve_gitlink_ref() returns -1 because HEAD cannot be resolved, and the directory is treated as a normal directory (i.e. traversal does not stop at the repository boundary). The status and ls-files commands above list untracked files within the repository rather than showing only the top-level directory.

The above case is a corner case in an already unusual situation of the working tree containing a repository that is not a tracked submodule, but we might as well treat anything that looks like a repository consistently. Loosen the "looks like a repository" criteria in treat_directory() by replacing resolve_gitlink_ref() with is_nonbare_repository_dir(), one of the checks that is performed downstream when resolve_gitlink_ref() is called with an empty repository.

As the required update to t3700-add shows, being looser with the check means that we're stricter when adding empty repositories to the index:

% git add repo
warning: adding embedded git repository: repo
hint: You've added another git repository inside your current repository.
hint: [...]
error: unable to index file 'repo/'
fatal: adding files failed

That error message isn't particularly helpful in this situation, but it seems preferable to the old behavior of adding the repository's untracked files. And if the caller really wants the previous behavior, they can get it by adding a trailing slash.

---
 dir.c                                   |  6 ++++--
 t/t3009-ls-files-others-nonsubmodule.sh | 22 +++++++++++++++++++++-
 t/t3700-add.sh                          |  1 +
 3 files changed, 26 insertions(+), 3 deletions(-)
dir.c +4 -2
index b2cabadf25..a4e59eb351 100644
@@ -1467,9 +1467,11 @@ static enum path_treatment treat_directory(struct dir_struct *dir,
 			return path_none;
 		}
 		if (!(dir->flags & DIR_NO_GITLINKS)) {
-			struct object_id oid;
-			if (resolve_gitlink_ref(dirname, "HEAD", &oid) == 0)
+			struct strbuf sb = STRBUF_INIT;
+			strbuf_addstr(&sb, dirname);
+			if (is_nonbare_repository_dir(&sb))
 				return exclude ? path_excluded : path_untracked;
+			strbuf_release(&sb);
 		}
 		return path_recurse;
 	}
t/t3009-ls-files-others-nonsubmodule.sh +21 -1
index 9ed75928aa..be4e7e26bc 100755
@@ -8,6 +8,14 @@ This test runs git ls-files --others with the following working tree:
       directory with no files aside from a bogus .git file
     repo-bogus-untracked-file/
       directory with a bogus .git file and another untracked file
+    repo-no-commit-no-files/
+      git repository without a commit or a file
+    repo-no-commit-untracked-file/
+      git repository without a commit but with an untracked file
+    repo-with-commit-no-files/
+      git repository with a commit and no untracked files
+    repo-with-commit-untracked-file/
+      git repository with a commit and an untracked file
 '
 
 . ./test-lib.sh
@@ -17,6 +25,10 @@ test_expect_success 'setup: expected output' '
 	expected
 	output
 	repo-bogus-untracked-file/untracked
+	repo-no-commit-no-files/
+	repo-no-commit-untracked-file/
+	repo-with-commit-no-files/
+	repo-with-commit-untracked-file/
 	EOF
 '
 
@@ -25,7 +37,15 @@ test_expect_success 'setup: directories' '
 	echo foo >repo-bogus-no-files/.git &&
 	mkdir repo-bogus-untracked-file &&
 	echo foo >repo-bogus-untracked-file/.git &&
-	: >repo-bogus-untracked-file/untracked
+	: >repo-bogus-untracked-file/untracked &&
+	git init repo-no-commit-no-files &&
+	git init repo-no-commit-untracked-file &&
+	: >repo-no-commit-untracked-file/untracked &&
+	git init repo-with-commit-no-files &&
+	git -C repo-with-commit-no-files commit --allow-empty -mmsg &&
+	git init repo-with-commit-untracked-file &&
+	test_commit -C repo-with-commit-untracked-file msg &&
+	: >repo-with-commit-untracked-file/untracked
 '
 
 test_expect_success 'ls-files --others handles non-submodule .git' '
t/t3700-add.sh +1 -0
index be582a513b..5a8425962b 100755
@@ -396,6 +396,7 @@ test_expect_success 'no file status change if no pathspec is given in subdir' '
 '
 
 test_expect_success 'all statuses changed in folder if . is given' '
+	rm -fr empty &&
 	git add --chmod=+x . &&
 	test $(git ls-files --stage | grep ^100644 | wc -l) -eq 0 &&
 	git add --chmod=-x . &&
Signature · 1 line
2.21.0
#11 lore raw

Re: [PATCH v2 4/4] dir: do not traverse repositories with no commits

Quoted 4 lines · Kyle Meyer

The above case is a corner case in an already unusual situation of the working tree containing a repository that is not a tracked submodule, but we might as well treat anything that looks like a repository consistently.

Makes sense.

Quoted 9 lines

As the required update to t3700-add shows, being looser with the check means that we're stricter when adding empty repositories to the index:

% git add repo
warning: adding embedded git repository: repo
hint: You've added another git repository inside your current repository.
hint: [...]
error: unable to index file 'repo/'
fatal: adding files failed

Hmph, why should we have the initial "warning" with hint? Shouldn't the above result in an outright error? Something like:

    $ git add repo ;# or git add repo/
    error: repo does not have a commit checked out
    fatal: adding files failed

because the entire purpose of "git add repo" (or "git add repo/") when "repo" is a repository is to register the commit that is checked out in that sub-repository to the index of the top-level repository, as part of updating (or starting) a submodule.

I very much like the direction these four patches want to take us. I just have small troubles here and there in their execution, though ;-)

Thanks.

author
#12 lore raw

Re: [PATCH v2 4/4] dir: do not traverse repositories with no commits

Quoted 15 lines · Junio C Hamano
Quoted 9 lines · Kyle Meyer

As the required update to t3700-add shows, being looser with the check means that we're stricter when adding empty repositories to the index:

% git add repo
warning: adding embedded git repository: repo
hint: You've added another git repository inside your current repository.
hint: [...]
error: unable to index file 'repo/'
fatal: adding files failed

Hmph, why should we have the initial "warning" with hint? Shouldn't the above result in an outright error? Something like:

Quoted 8 lines
    $ git add repo ;# or git add repo/
    error: repo does not have a commit checked out
    fatal: adding files failed

because the entire purpose of "git add repo" (or "git add repo/") when "repo" is a repository is to register the commit that is checked out in that sub-repository to the index of the top-level repository, as part of updating (or starting) a submodule.

Right, the hint makes no sense here. I'll look into adjusting the output to be appropriate.

As a minor note: I believe your "git add repo/" comments above describe what should be and not what currently is. The trailing slash results in adding the content from the specified repository to the current repository rather than registering the repository as a submodule:

  $ git --version
  git version 2.21.0
  $ git init
  $ git init a
  $ git -C a commit --allow-empty -mmsg
  $ touch a/f
  $ git add a/
  $ git status -s
  A  a/f

This behavior (and fixing it) was discussed at
https://public-inbox.org/git/[email protected]

author
#13 lore raw

PATCH 2/4 t3000: move non-submodule repo test to separate file

In reply to Kyle Meyer

a2d5156c2b (resolve_gitlink_ref: ignore non-repository paths, 2016-01-22) added a test to t3000-ls-files-others.sh to check that 'ls-files -o' does not die() when given a subdirectory that looks like a repository but is actually a subdirectory containing a bogus .git file.

Move this test to a separate file in preparation for testing scenarios with non-submodule repositories that are not bogus.

---
 t/t3000-ls-files-others.sh              |  7 -------
 t/t3009-ls-files-others-nonsubmodule.sh | 21 +++++++++++++++++++++
 2 files changed, 21 insertions(+), 7 deletions(-)
 create mode 100755 t/t3009-ls-files-others-nonsubmodule.sh
t/t3000-ls-files-others.sh +0 -7
index afd4756134..b4f9fc4580 100755
@@ -65,13 +65,6 @@ test_expect_success '--no-empty-directory hides empty directory' '
 	test_cmp expected3 output
 '
 
-test_expect_success 'ls-files --others handles non-submodule .git' '
-	mkdir not-a-submodule &&
-	echo foo >not-a-submodule/.git &&
-	git ls-files -o >output &&
-	test_cmp expected1 output
-'
-
 test_expect_success SYMLINKS 'ls-files --others with symlinked submodule' '
 	git init super &&
 	git init sub &&
t/t3009-ls-files-others-nonsubmodule.sh new file +21 -0
new file mode 100755
index 0000000000..cc66a4a14d
@@ -0,0 +1,21 @@
+#!/bin/sh
+
+test_description='test git ls-files --others with non-submodule repositories'
+
+. ./test-lib.sh
+
+test_expect_success 'setup: expected output' '
+	cat >expected <<-EOF
+	expected
+	output
+	EOF
+'
+
+test_expect_success 'ls-files --others handles non-submodule .git' '
+	mkdir not-a-submodule &&
+	echo foo >not-a-submodule/.git &&
+	git ls-files -o >output &&
+	test_cmp expected output
+'
+
+test_done
Signature · 1 line
2.21.0
author
#14 lore raw

PATCH 3/4 t3009: test that ls-files -o traverses bogus repo

In reply to Kyle Meyer

When a2d5156c2b (resolve_gitlink_ref: ignore non-repository paths, 2016-01-22) added this test, the purpose was to check the 'ls-files -o' didn't die() when processing the bogus repository. The expected output didn't even need to be adjusted for the addition because the bogus repository is treated as an empty directory.

Let's add another scenario to test that 'ls-files -o' lists an untracked file within this directory. Doing so isn't useful for the original purpose of the test, but it will be helpful for highlighting that this traversal does _not_ happen in the non-bogus scenarios that will be added next. Rename the original subdirectory to highlight the difference with the new one.

---
 t/t3009-ls-files-others-nonsubmodule.sh | 21 ++++++++++++++++++---
 1 file changed, 18 insertions(+), 3 deletions(-)
t/t3009-ls-files-others-nonsubmodule.sh +18 -3
index cc66a4a14d..9ed75928aa 100755
@@ -1,6 +1,14 @@
 #!/bin/sh
 
-test_description='test git ls-files --others with non-submodule repositories'
+test_description='test git ls-files --others with non-submodule repositories
+
+This test runs git ls-files --others with the following working tree:
+
+    repo-bogus-no-files/
+      directory with no files aside from a bogus .git file
+    repo-bogus-untracked-file/
+      directory with a bogus .git file and another untracked file
+'
 
 . ./test-lib.sh
 
@@ -8,12 +16,19 @@ test_expect_success 'setup: expected output' '
 	cat >expected <<-EOF
 	expected
 	output
+	repo-bogus-untracked-file/untracked
 	EOF
 '
 
+test_expect_success 'setup: directories' '
+	mkdir repo-bogus-no-files &&
+	echo foo >repo-bogus-no-files/.git &&
+	mkdir repo-bogus-untracked-file &&
+	echo foo >repo-bogus-untracked-file/.git &&
+	: >repo-bogus-untracked-file/untracked
+'
+
 test_expect_success 'ls-files --others handles non-submodule .git' '
-	mkdir not-a-submodule &&
-	echo foo >not-a-submodule/.git &&
 	git ls-files -o >output &&
 	test_cmp expected output
 '
Signature · 1 line
2.21.0
author
#15 lore raw

PATCH 4/4 dir: do not traverse repositories with no commits

In reply to Kyle Meyer

When treat_directory() encounters a directory that is not in the index and DIR_NO_GITLINKS is unset, it calls resolve_gitlink_ref() to decide if a directory looks like a repository, in which case the directory won't be traversed. As a result, 'status -uall' and 'ls-files -o' will show only the directory, even when there are untracked files within the directory.

For the unusual case where a repository doesn't have any commits, resolve_gitlink_ref() returns -1 because HEAD cannot be resolved, and the directory is treated as a normal directory (i.e. traversal does not stop at the repository boundary). The status and ls-files commands above list untracked files within the repository rather than showing only the top-level directory.

The above case is a corner case in an already unusual situation of the working tree containing a repository that is not a tracked submodule, but we might as well treat anything that looks like a repository consistently. Loosen the "looks like a repository" criteria in treat_directory() by replacing resolve_gitlink_ref() with is_nonbare_repository_dir(), one of the checks that is performed downstream when resolve_gitlink_ref() is called with an empty repository.

As the required update to t3700-add shows, being looser with the check means that we're stricter when adding empty repositories to the index:

% git add repo
warning: adding embedded git repository: repo
hint: You've added another git repository inside your current repository.
hint: [...]
error: unable to index file 'repo/'
fatal: adding files failed

That error message isn't particularly helpful in this situation, but it seems preferable to the old behavior of adding the repository's untracked files. And if the caller really wants the previous behavior, they can get it by adding a trailing slash.

---
 dir.c                                   |  6 ++++--
 t/t3009-ls-files-others-nonsubmodule.sh | 22 +++++++++++++++++++++-
 t/t3700-add.sh                          |  1 +
 3 files changed, 26 insertions(+), 3 deletions(-)
dir.c +4 -2
index b2cabadf25..a4e59eb351 100644
@@ -1467,9 +1467,11 @@ static enum path_treatment treat_directory(struct dir_struct *dir,
 			return path_none;
 		}
 		if (!(dir->flags & DIR_NO_GITLINKS)) {
-			struct object_id oid;
-			if (resolve_gitlink_ref(dirname, "HEAD", &oid) == 0)
+			struct strbuf sb = STRBUF_INIT;
+			strbuf_addstr(&sb, dirname);
+			if (is_nonbare_repository_dir(&sb))
 				return exclude ? path_excluded : path_untracked;
+			strbuf_release(&sb);
 		}
 		return path_recurse;
 	}
t/t3009-ls-files-others-nonsubmodule.sh +21 -1
index 9ed75928aa..be4e7e26bc 100755
@@ -8,6 +8,14 @@ This test runs git ls-files --others with the following working tree:
       directory with no files aside from a bogus .git file
     repo-bogus-untracked-file/
       directory with a bogus .git file and another untracked file
+    repo-no-commit-no-files/
+      git repository without a commit or a file
+    repo-no-commit-untracked-file/
+      git repository without a commit but with an untracked file
+    repo-with-commit-no-files/
+      git repository with a commit and no untracked files
+    repo-with-commit-untracked-file/
+      git repository with a commit and an untracked file
 '
 
 . ./test-lib.sh
@@ -17,6 +25,10 @@ test_expect_success 'setup: expected output' '
 	expected
 	output
 	repo-bogus-untracked-file/untracked
+	repo-no-commit-no-files/
+	repo-no-commit-untracked-file/
+	repo-with-commit-no-files/
+	repo-with-commit-untracked-file/
 	EOF
 '
 
@@ -25,7 +37,15 @@ test_expect_success 'setup: directories' '
 	echo foo >repo-bogus-no-files/.git &&
 	mkdir repo-bogus-untracked-file &&
 	echo foo >repo-bogus-untracked-file/.git &&
-	: >repo-bogus-untracked-file/untracked
+	: >repo-bogus-untracked-file/untracked &&
+	git init repo-no-commit-no-files &&
+	git init repo-no-commit-untracked-file &&
+	: >repo-no-commit-untracked-file/untracked &&
+	git init repo-with-commit-no-files &&
+	git -C repo-with-commit-no-files commit --allow-empty -mmsg &&
+	git init repo-with-commit-untracked-file &&
+	test_commit -C repo-with-commit-untracked-file msg &&
+	: >repo-with-commit-untracked-file/untracked
 '
 
 test_expect_success 'ls-files --others handles non-submodule .git' '
t/t3700-add.sh +1 -0
index be582a513b..5a8425962b 100755
@@ -396,6 +396,7 @@ test_expect_success 'no file status change if no pathspec is given in subdir' '
 '
 
 test_expect_success 'all statuses changed in folder if . is given' '
+	rm -fr empty &&
 	git add --chmod=+x . &&
 	test $(git ls-files --stage | grep ^100644 | wc -l) -eq 0 &&
 	git add --chmod=-x . &&
Signature · 1 line
2.21.0