mingw: fix isatty() after dup2()

This patch has been looong in the waiting (well over a year) for being sent to the Git mailing list; it is a fixup for a change of isatty() on Windows that had long made it into git.git's master branch.

Johannes Schindelin (1):
mingw: fix isatty() after dup2()

 compat/mingw.h   |  3 +++
 compat/winansi.c | 12 ++++++++++++
 2 files changed, 15 insertions(+)

base-commit: 4ede3d42dfb57f9a41ac96a1f216c62eb7566cc2
Published-As: https://github.com/gitgitgadget/git/releases/tags/pr-61%2Fdscho%2Fmingw-isatty-and-dup2-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-61/dscho/mingw-isatty-and-dup2-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/61

gitgitgadget

From: Johannes Schindelin <[email protected]>

Since a9b8a09c3c30 (mingw: replace isatty() hack, 2016-12-22), we handle isatty() by special-casing the stdin/stdout/stderr file descriptors, caching the return value. However, we missed the case where dup2() overrides the respective file descriptor.

That poses a problem e.g. where the show builtin asks for a pager very early, the setup_pager() function sets the pager depending on the return value of isatty() and then redirects stdout. Subsequently, cmd_log_init_finish() calls setup_pager() *again*. What should happen now is that isatty() reports that stdout is *not* a TTY and consequently stdout should be left alone.

Let's override dup2() to handle this appropriately.

This fixes https://github.com/git-for-windows/git/issues/1077

---
 compat/mingw.h   |  3 +++
 compat/winansi.c | 12 ++++++++++++
 2 files changed, 15 insertions(+)

index f31dcff2b..b04556ce0 100644
@@ -390,6 +390,9 @@ int mingw_raise(int sig);
 int winansi_isatty(int fd);
 #define isatty winansi_isatty
 
+int winansi_dup2(int oldfd, int newfd);
+#define dup2 winansi_dup2
+
 void winansi_init(void);
 HANDLE winansi_get_osfhandle(int fd);
 

index a11a0f16d..f4f08237f 100644
@@ -474,6 +474,18 @@ static void die_lasterr(const char *fmt, ...)
 	va_end(params);
 }
 
+#undef dup2
+int winansi_dup2(int oldfd, int newfd)
+{
+	int ret = dup2(oldfd, newfd);
+
+	if (!ret && newfd >= 0 && newfd <= 2)
+		fd_is_interactive[newfd] = oldfd < 0 || oldfd > 2 ?
+			0 : fd_is_interactive[oldfd];
+
+	return ret;
+}
+
 static HANDLE duplicate_handle(HANDLE hnd)
 {
 	HANDLE hresult, hproc = GetCurrentProcess();

gitgitgadget