mingw: align symlinks-related rmdir() behavior with Linux

From: Johannes Schindelin <[email protected]>

When performing a rebase, rmdir() is called on the folder .git/logs. On Unix rmdir() exits without deleting anything in case .git/logs is a symbolic link but the equivalent functions on Windows (_rmdir, _wrmdir and RemoveDirectoryW) do not behave the same and remove the folder if it is symlinked even if it is not empty.

This creates issues when folders in .git/ are symlinks which is especially the case when git-repo[1] is used.

This commit updates mingw_rmdir() so that its behavior is the same as Linux rmdir() in case of symbolic links.

This fixes https://github.com/git-for-windows/git/issues/2967

[1]: git-repo is a python tool built on top of Git which helps manage many Git repositories. It stores all the .git/ folders in a central place by taking advantage of symbolic links.
More information: https://gerrit.googlesource.com/git-repo/

---
mingw: support the git-repo tool better

This addresses an issue, originally reported at
https://github.com/git-for-windows/git/issues/2967, where the git-repo tool [https://gerrit.googlesource.com/git-repo/] replaces folders in .git/ with symlinks and mingw_rmdir() erroneously removes the symlink target directory's contents.

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1008%2Fdscho%2Ffix-rmdir-with-symlinks-on-windows-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1008/dscho/fix-rmdir-with-symlinks-on-windows-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1008

 compat/mingw.c    | 15 +++++++++++++++
 t/t3400-rebase.sh | 10 ++++++++++
 t/test-lib.sh     |  6 ++++++
 3 files changed, 31 insertions(+)

index aa647b367b0..685d3efa3c0 100644
@@ -341,6 +341,21 @@ int mingw_rmdir(const char *pathname)
 {
 	int ret, tries = 0;
 	wchar_t wpathname[MAX_PATH];
+	struct stat st;
+
+	/*
+	* Contrary to Linux rmdir(), Windows' _wrmdir() and _rmdir()
+	* will remove the directory at the path if it is a symbolic link
+	* which leads to issues when symlinks are used in the .git folder
+	* (in the context of git-repo for instance). So before calling _wrmdir()
+	* we first check if the path is a symbolic link. If it is, we exit
+	* and return the same error as Linux rmdir() in this case (ENOTDIR).
+	*/
+	if (!mingw_lstat(pathname, &st) && S_ISLNK(st.st_mode)) {
+		errno = ENOTDIR;
+		return -1;
+	}
+
 	if (xutftowcs_path(wpathname, pathname) < 0)
 		return -1;
 

index 0bb88aa982b..23dbd3c82ed 100755
@@ -406,4 +406,14 @@ test_expect_success 'refuse to switch to branch checked out elsewhere' '
 	test_i18ngrep "already checked out" err
 '
 
+test_expect_success MINGW,SYMLINKS_WINDOWS 'rebase when .git/logs is a symlink' '
+	git checkout main &&
+	mv .git/logs actual_logs &&
+	cmd //c "mklink /D .git\logs ..\actual_logs" &&
+	git rebase -f HEAD^ &&
+	test -L .git/logs &&
+	rm .git/logs &&
+	mv actual_logs .git/logs
+'
+
 test_done

index adaf03543e8..73f6d645b66 100644
@@ -1513,6 +1513,12 @@ test_lazy_prereq SYMLINKS '
 	ln -s x y && test -h y
 '
 
+test_lazy_prereq SYMLINKS_WINDOWS '
+	# test whether symbolic links are enabled on Windows
+	test_have_prereq MINGW &&
+	cmd //c "mklink y x" &> /dev/null && test -h y
+'
+
 test_lazy_prereq FILEMODE '
 	test "$(git config --bool core.filemode)" = true
 '

base-commit: ebf3c04b262aa27fbb97f8a0156c2347fecafafb

gitgitgadget

Hi,

Sorry for missing this, this line should have read

From: Thomas Bétous <[email protected]>

Ciao,
Dscho

"Johannes Schindelin via GitGitGadget" <[email protected]>

The distinction is understandable, and I can see this justifies the patch really nicely.

It is curious why "rebase" causes rmdir on the reflog hierarchy, though. It is also unclear if "rebase" is special in having this behaviour (and if so why), or just an example the problem was observed with and other subcommands may benefit from the same fix.

Will queue with authorship corrected (as you reported in the follow-up).

Thanks.

Hi Junio,

I augmented the commit message in preparation for v2, specifically to address this concern.

Ciao,
Dscho

"issues" is a rather nebulous word. It doesn't help the reader understand what actually goes wrong. Presumably some step later in the rebase fails? Or is the problem that subsequent interaction with git-repo fails because the link which git-repo (presumably) created disappears out from underneath it?

Okay, makes sense so far as the above explanation goes. But it also feels like a band-aid fix to support a use-case which only works by accident on Unix, if I'm reading between the lines correctly. That is, presumably rmdir(".git/logs") is an intended cleanup action but the fact that the cleanup doesn't happen as intended when it is a symlink is not intentional behavior, thus git-repo's reliance upon that behavior is questionable.

I guess this would also help tools which replace .git/worktrees with a symlink since git-worktree -- as a cleanup -- removes its .git/worktrees directory when the last worktree is removed...

Hi Eric,

I added a few lines describing the problem that was observed, what the code specifically does to cause the problem, and then documented also that the regression test specifically verifies that the observed problem won't reoccur.

It is the remove_path() function that tries to delete parent directories recursively until things fail.

Whether git-repo's reliance upon that behavior questionable is outside the purview of this patch. At this point in time, with so many Git versions accommodating that behavior, I would not want to spend much time pondering that question, though: we de facto supported that expectation for too long to change the behavior on Git's side, and therefore adjusting the Windows side to that expectation feels like the most pragmatic way forward.

Yes, indeed.

Okay. I reworded the comment extensively, to address your concern.

Ciao,
Dscho

From: =?UTF-8?q?Thomas=20B=C3=A9tous?= <[email protected]>

When performing a rebase, rmdir() is called on the folder .git/logs. On Unix rmdir() exits without deleting anything in case .git/logs is a symbolic link but the equivalent functions on Windows (_rmdir, _wrmdir and RemoveDirectoryW) do not behave the same and remove the folder if it is symlinked even if it is not empty.

This creates issues when folders in .git/ are symlinks which is especially the case when git-repo[1] is used: It replaces .git/logs/ with a symlink.

One such issue is that the _target_ of that symlink is removed e.g. during a git rebase, where delete_reflog("REBASE_HEAD") will not only try to remove .git/logs/REBASE_HEAD but then recursively try to remove the parent directories until an error occurs, a technique that obviously relies on rmdir() refusing to remove a symlink.

This was reported in https://github.com/git-for-windows/git/issues/2967.

This commit updates mingw_rmdir() so that its behavior is the same as Linux rmdir() in case of symbolic links.

To verify that Git does not regress on the reported issue, this patch adds a regression test for the git rebase symptom, even if the same rmdir() behavior is quite likely to cause potential problems in other Git commands as well.

[1]: git-repo is a python tool built on top of Git which helps manage many Git repositories. It stores all the .git/ folders in a central place by taking advantage of symbolic links.
More information: https://gerrit.googlesource.com/git-repo/

---
mingw: support the git-repo tool better

This addresses an issue, originally reported at
https://github.com/git-for-windows/git/issues/2967, where the git-repo tool [https://gerrit.googlesource.com/git-repo/] replaces folders in .git/ with symlinks and mingw_rmdir() erroneously removes the symlink target directory's contents.

Changes since v1:

* Fixed the authorship
* Augmented the commit message to elaborate on what the issues are, concretely, with the current behavior of mingw_rmdir()
* Added an explanation to the commit message as to what happens during a git rebase that would trigger the rmdir() code path to misbehave
* Adjusted the code comment to talk specifically about the remove_path() caller and why we want to align with Linux' rmdir() behavior

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1008%2Fdscho%2Ffix-rmdir-with-symlinks-on-windows-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1008/dscho/fix-rmdir-with-symlinks-on-windows-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/1008

Range-diff vs v1:

1: c5a7fe0007f ! 1: 0a9cb1bcc14 mingw: align symlinks-related rmdir() behavior with Linux
@@
## Metadata ##
-Author: Johannes Schindelin <[email protected]>
+Author: Thomas Bétous <[email protected]>

## Commit message ##
mingw: align symlinks-related rmdir() behavior with Linux
@@ Commit message
is symlinked even if it is not empty.

This creates issues when folders in .git/ are symlinks which is
- especially the case when git-repo[1] is used.
+ especially the case when git-repo[1] is used: It replaces .git/logs/
+ with a symlink.
+
+ One such issue is that the _target_ of that symlink is removed e.g.
+ during a git rebase, where delete_reflog("REBASE_HEAD") will not
+ only try to remove .git/logs/REBASE_HEAD but then recursively try to
+ remove the parent directories until an error occurs, a technique that
+ obviously relies on rmdir() refusing to remove a symlink.
+
+ This was reported in https://github.com/git-for-windows/git/issues/2967.

This commit updates mingw_rmdir() so that its behavior is the same as Linux rmdir() in case of symbolic links.

- This fixes https://github.com/git-for-windows/git/issues/2967
+ To verify that Git does not regress on the reported issue, this patch
+ adds a regression test for the git rebase symptom, even if the same
+ rmdir() behavior is quite likely to cause potential problems in other
+ Git commands as well.

[1]: git-repo is a python tool built on top of Git which helps manage many Git repositories. It stores all the .git/ folders in a central
@@ compat/mingw.c: int mingw_rmdir(const char *pathname)
+ struct stat st;
+
+ /*
-+ * Contrary to Linux rmdir(), Windows' _wrmdir() and _rmdir()
-+ * will remove the directory at the path if it is a symbolic link
-+ * which leads to issues when symlinks are used in the .git folder
-+ * (in the context of git-repo for instance). So before calling _wrmdir()
-+ * we first check if the path is a symbolic link. If it is, we exit
-+ * and return the same error as Linux rmdir() in this case (ENOTDIR).
-+ */
++ * Contrary to Linux' rmdir(), Windows' _wrmdir() and _rmdir()
++ * (and RemoveDirectoryW()) will attempt to remove the target of a
++ * symbolic link (if it points to a directory).
++ *
++ * This behavior breaks the assumption of e.g. remove_path() which
++ * upon successful deletion of a file will attempt to remove its parent
++ * directories recursively until failure (which usually happens when
++ * the directory is not empty).
++ *
++ * Therefore, before calling _wrmdir(), we first check if the path is
++ * a symbolic link. If it is, we exit and return the same error as
++ * Linux' rmdir() would, i.e. ENOTDIR.
++ */
+ if (!mingw_lstat(pathname, &st) && S_ISLNK(st.st_mode)) {
+ errno = ENOTDIR;
+ return -1;

 compat/mingw.c    | 21 +++++++++++++++++++++
 t/t3400-rebase.sh | 10 ++++++++++
 t/test-lib.sh     |  6 ++++++
 3 files changed, 37 insertions(+)

index aa647b367b0..9e0cd1e097f 100644
@@ -341,6 +341,27 @@ int mingw_rmdir(const char *pathname)
 {
 	int ret, tries = 0;
 	wchar_t wpathname[MAX_PATH];
+	struct stat st;
+
+	/*
+	 * Contrary to Linux' `rmdir()`, Windows' _wrmdir() and _rmdir()
+	 * (and `RemoveDirectoryW()`) will attempt to remove the target of a
+	 * symbolic link (if it points to a directory).
+	 *
+	 * This behavior breaks the assumption of e.g. `remove_path()` which
+	 * upon successful deletion of a file will attempt to remove its parent
+	 * directories recursively until failure (which usually happens when
+	 * the directory is not empty).
+	 *
+	 * Therefore, before calling `_wrmdir()`, we first check if the path is
+	 * a symbolic link. If it is, we exit and return the same error as
+	 * Linux' `rmdir()` would, i.e. `ENOTDIR`.
+	 */
+	if (!mingw_lstat(pathname, &st) && S_ISLNK(st.st_mode)) {
+		errno = ENOTDIR;
+		return -1;
+	}
+
 	if (xutftowcs_path(wpathname, pathname) < 0)
 		return -1;
 

index 0bb88aa982b..23dbd3c82ed 100755
@@ -406,4 +406,14 @@ test_expect_success 'refuse to switch to branch checked out elsewhere' '
 	test_i18ngrep "already checked out" err
 '
 
+test_expect_success MINGW,SYMLINKS_WINDOWS 'rebase when .git/logs is a symlink' '
+	git checkout main &&
+	mv .git/logs actual_logs &&
+	cmd //c "mklink /D .git\logs ..\actual_logs" &&
+	git rebase -f HEAD^ &&
+	test -L .git/logs &&
+	rm .git/logs &&
+	mv actual_logs .git/logs
+'
+
 test_done

index adaf03543e8..73f6d645b66 100644
@@ -1513,6 +1513,12 @@ test_lazy_prereq SYMLINKS '
 	ln -s x y && test -h y
 '
 
+test_lazy_prereq SYMLINKS_WINDOWS '
+	# test whether symbolic links are enabled on Windows
+	test_have_prereq MINGW &&
+	cmd //c "mklink y x" &> /dev/null && test -h y
+'
+
 test_lazy_prereq FILEMODE '
 	test "$(git config --bool core.filemode)" = true
 '

base-commit: ebf3c04b262aa27fbb97f8a0156c2347fecafafb

gitgitgadget