Upstream some more Git for Windows' patches

When rebasing a Git for Windows commit to enable Rust in Windows CI, I noticed just how many of the preceding commits touching adjacent code haven't been upstreamed yet. This patch series rectifies that.

Johannes Schindelin (12):
mingw: include the Python parts in the build
mingw: stop hard-coding CC = gcc
mingw: drop the -D_USE_32BIT_TIME_T option
mingw: only use -Wl,--large-address-aware for 32-bit builds
mingw: avoid over-specifying --pic-executable
mingw: set the prefix and HOST_CPU as per MSYS2's settings
mingw: only enable the MSYS2-specific stuff when compiling in MSYS2
mingw: rely on MSYS2's metadata instead of hard-coding it
windows: skip linking git-<command> for built-ins
mingw: always define ETC_* for MSYS2 environments
mingw: ensure valid CTYPE
mingw: allow git.exe to be used instead of the "Git wrapper"

 compat/mingw.c                      | 68 +++++++++++++++++++++++++++++
 config.mak.uname                    | 61 ++++++++++++--------------
 contrib/buildsystems/CMakeLists.txt |  9 +++-
 meson.build                         | 13 +++++-
 meson_options.txt                   |  4 ++
 t/t0060-path-utils.sh               | 33 +++++++++++++-
 6 files changed, 151 insertions(+), 37 deletions(-)

base-commit: 5b2471720c93ee30e5764a19f3d3b3ae9ec9712a
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2195%2Fdscho%2Fupstream-some-more-git-for-windows-patches-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2195/dscho/upstream-some-more-git-for-windows-patches-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/2195

gitgitgadget

From: Johannes Schindelin <[email protected]>

While Git for Windows does not _ship_ Python (in order to save on bandwidth), MSYS2 provides very fine Python interpreters that users can easily take advantage of, by using Git for Windows within its SDK.

---
 config.mak.uname | 1 +
 1 file changed, 1 insertion(+)

index 9ebd240378..8dd8acfaa5 100644
@@ -781,6 +781,7 @@ ifeq ($(uname_S),MINGW)
 	HAVE_LIBCHARSET_H = YesPlease
 	USE_GETTEXT_SCHEME = fallthrough
 	USE_LIBPCRE = YesPlease
+	NO_PYTHON =
         ifeq (/mingw64,$(subst 32,64,$(subst clangarm,mingw,$(prefix))))
 		# Move system config into top-level /etc/
 		ETC_GITCONFIG = ../etc/gitconfig

gitgitgadget

From: Johannes Schindelin <[email protected]>

This is no longer true in general, not with supporting Clang out of the box.

---
 config.mak.uname | 1 -
 1 file changed, 1 deletion(-)

index 8dd8acfaa5..a464b2009c 100644
@@ -771,7 +771,6 @@ ifeq ($(uname_S),MINGW)
 		COMPAT_CFLAGS += -D_USE_32BIT_TIME_T
 		BASIC_LDFLAGS += -Wl,--large-address-aware
         endif
-	CC = gcc
 	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \
 		-fstack-protector-strong
 	EXTLIBS += -lntdll

gitgitgadget

From: Johannes Schindelin <[email protected]>

This option was added in fa93bb20d72 (MinGW: Fix stat definitions to work with MinGW runtime version 4.0, 2013-09-11), i.e. a _long_ time ago. So long, in fact, that it still targeted MinGW. But we switched to mingw-w64 in 2015, which seems not to share the problem, and therefore does not require a fix.

Even worse: This flag is incompatible with UCRT64, which we are about to support by way of upstreaming mingw-w64-git to the MSYS2 project, see
https://github.com/msys2/MINGW-packages/pull/26470 for details.

So let's send that option into its well-deserved retirement.

---
 config.mak.uname | 1 -
 1 file changed, 1 deletion(-)

index a464b2009c..cc0ed63af3 100644
@@ -768,7 +768,6 @@ ifeq ($(uname_S),MINGW)
 		HOST_CPU = aarch64
 		BASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup
         else
-		COMPAT_CFLAGS += -D_USE_32BIT_TIME_T
 		BASIC_LDFLAGS += -Wl,--large-address-aware
         endif
 	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \

gitgitgadget

From: Johannes Schindelin <[email protected]>

That option only matters there, and is in fact only really understood in those builds; UCRT64 versions of GCC, for example, do not know what to do with that option.

---
 config.mak.uname | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

index cc0ed63af3..b667c693ec 100644
@@ -757,9 +757,8 @@ ifeq ($(uname_S),MINGW)
         ifeq (MINGW32,$(MSYSTEM))
 		prefix = /mingw32
 		HOST_CPU = i686
-		BASIC_LDFLAGS += -Wl,--pic-executable,-e,_mainCRTStartup
-        endif
-        ifeq (MINGW64,$(MSYSTEM))
+		BASIC_LDFLAGS += -Wl,--pic-executable,-e,_mainCRTStartup -Wl,--large-address-aware
+        else ifeq (MINGW64,$(MSYSTEM))
 		prefix = /mingw64
 		HOST_CPU = x86_64
 		BASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup

@@ -768,7 +767,6 @@ ifeq ($(uname_S),MINGW)
 		HOST_CPU = aarch64
 		BASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup
         else
-		BASIC_LDFLAGS += -Wl,--large-address-aware
         endif
 	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \
 		-fstack-protector-strong

gitgitgadget

From: Johannes Schindelin <[email protected]>

In bf2d5d8239e (Don't let ld strip relocations, 2016-01-16) (picked from
https://github.com/git-for-windows/git/pull/612/commits/6a237925bf10),
Git for Windows introduced the -Wl,-pic-executable flag, specifying the exact entry point via -e. This required discerning between i686 and x86_64 code because the former required the symbol to be prefixed with an underscore, the latter did not.

As per https://sourceware.org/bugzilla/show_bug.cgi?id=10865, the specified symbols are already the default, though.

So let's drop the overly-specific definition.

---
 config.mak.uname | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

index b667c693ec..f6387f4c7b 100644
@@ -757,15 +757,15 @@ ifeq ($(uname_S),MINGW)
         ifeq (MINGW32,$(MSYSTEM))
 		prefix = /mingw32
 		HOST_CPU = i686
-		BASIC_LDFLAGS += -Wl,--pic-executable,-e,_mainCRTStartup -Wl,--large-address-aware
+		BASIC_LDFLAGS += -Wl,--pic-executable -Wl,--large-address-aware
         else ifeq (MINGW64,$(MSYSTEM))
 		prefix = /mingw64
 		HOST_CPU = x86_64
-		BASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup
+		BASIC_LDFLAGS += -Wl,--pic-executable
         else ifeq (CLANGARM64,$(MSYSTEM))
 		prefix = /clangarm64
 		HOST_CPU = aarch64
-		BASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup
+		BASIC_LDFLAGS += -Wl,--pic-executable
         else
         endif
 	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \

gitgitgadget

From: Johannes Schindelin <[email protected]>

MSYS2 already defines a couple of helpful environment variables, and we can use those to infer the installation location as well as the CPU. No need for hard-coding ;-)

---
 config.mak.uname | 18 ++++++------------
 1 file changed, 6 insertions(+), 12 deletions(-)

index f6387f4c7b..8363239513 100644
@@ -754,19 +754,13 @@ ifeq ($(uname_S),MINGW)
         ifneq (,$(findstring -O,$(filter-out -O0 -Og,$(CFLAGS))))
 		BASIC_LDFLAGS += -Wl,--dynamicbase
         endif
-        ifeq (MINGW32,$(MSYSTEM))
-		prefix = /mingw32
-		HOST_CPU = i686
-		BASIC_LDFLAGS += -Wl,--pic-executable -Wl,--large-address-aware
-        else ifeq (MINGW64,$(MSYSTEM))
-		prefix = /mingw64
-		HOST_CPU = x86_64
-		BASIC_LDFLAGS += -Wl,--pic-executable
-        else ifeq (CLANGARM64,$(MSYSTEM))
-		prefix = /clangarm64
-		HOST_CPU = aarch64
+        ifneq (,$(MSYSTEM))
+		prefix = $(MINGW_PREFIX)
+		HOST_CPU = $(patsubst %-w64-mingw32,%,$(MINGW_CHOST))
 		BASIC_LDFLAGS += -Wl,--pic-executable
-        else
+                ifeq (MINGW32,$(MSYSTEM))
+			BASIC_LDFLAGS += -Wl,--large-address-aware
+                endif
         endif
 	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \
 		-fstack-protector-strong

gitgitgadget

From: Johannes Schindelin <[email protected]>

The tell-tale is the presence of the MSYSTEM value while compiling, of course. In that case, we want to ensure that MSYSTEM is set when running git.exe, and also enable the magic MSYS2 tty detection.

---
 config.mak.uname | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

index 8363239513..21f53e3f7e 100644
@@ -758,12 +758,12 @@ ifeq ($(uname_S),MINGW)
 		prefix = $(MINGW_PREFIX)
 		HOST_CPU = $(patsubst %-w64-mingw32,%,$(MINGW_CHOST))
 		BASIC_LDFLAGS += -Wl,--pic-executable
+		COMPAT_CFLAGS += -DDETECT_MSYS_TTY
                 ifeq (MINGW32,$(MSYSTEM))
 			BASIC_LDFLAGS += -Wl,--large-address-aware
                 endif
         endif
-	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \
-		-fstack-protector-strong
+	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -fstack-protector-strong
 	EXTLIBS += -lntdll
 	EXTRA_PROGRAMS += headless-git$X
 	INSTALL = /bin/install

gitgitgadget

From: Johannes Schindelin <[email protected]>

MSYS2 defines some helpful environment variables, e.g. MSYSTEM. There is code in Git for Windows to ensure that that MSYSTEM variable is set, hard-coding a default.

However, the existing solution jumps through hoops to reconstruct the proper default, and is even incomplete doing so, as we found out when we extended it to support CLANGARM64.

This is absolutely unnecessary because there is already a perfectly valid MSYSTEM value we can use at build time. This is even true when building the MINGW32 variant on a MINGW64 system because makepkg-mingw will override the MSYSTEM value as per the MINGW_ARCH array.

The same is equally true for the /mingw64, /mingw32 and /clangarm64 prefix: those values are already available via the MINGW_PREFIX environment variable, and we just need to pass that setting through.

Only when MINGW_PREFIX is not set (as is the case in Git for Windows' minimal SDK, where only MSYSTEM is guaranteed to be set correctly), we use as fall-back the top-level directory whose name is the down-cased value of the MSYSTEM variable.

Incidentally, this also broadens the support to all the configurations supported by the MSYS2 project, i.e. clang64 & ucrt64, too.

Note: This keeps the same, hard-coded MSYSTEM platform support for CMake as before, but drops it for Meson (because it is unclear how Meson could do this in a more flexible manner).

---
 config.mak.uname                    | 14 ++++++--------
 contrib/buildsystems/CMakeLists.txt |  9 ++++++++-
 meson.build                         | 13 ++++++++++++-
 meson_options.txt                   |  4 ++++
 4 files changed, 30 insertions(+), 10 deletions(-)

index 21f53e3f7e..3a90995587 100644
@@ -465,14 +465,8 @@ ifeq ($(uname_S),Windows)
 	GIT_VERSION := $(GIT_VERSION).MSVC
 	pathsep = ;
 	# Assume that this is built in Git for Windows' SDK
-        ifeq (MINGW32,$(MSYSTEM))
-		prefix = /mingw32
-        else
-                ifeq (CLANGARM64,$(MSYSTEM))
-			prefix = /clangarm64
-                else
-			prefix = /mingw64
-                endif
+        ifneq (,$(MSYSTEM))
+		prefix = $(MINGW_PREFIX)
         endif
 	# Prepend MSVC 64-bit tool-chain to PATH.
 	#

@@ -755,6 +749,10 @@ ifeq ($(uname_S),MINGW)
 		BASIC_LDFLAGS += -Wl,--dynamicbase
         endif
         ifneq (,$(MSYSTEM))
+                ifeq ($(MINGW_PREFIX),$(filter-out /%,$(MINGW_PREFIX)))
+			# Override if empty or does not start with a slash
+			MINGW_PREFIX := /$(shell echo '$(MSYSTEM)' | tr A-Z a-z)
+                endif
 		prefix = $(MINGW_PREFIX)
 		HOST_CPU = $(patsubst %-w64-mingw32,%,$(MINGW_CHOST))
 		BASIC_LDFLAGS += -Wl,--pic-executable

index a57c4b464f..7285bd9ac2 100644
@@ -256,7 +256,14 @@ if(CMAKE_SYSTEM_NAME STREQUAL "Windows")
 				_CONSOLE DETECT_MSYS_TTY STRIP_EXTENSION=".exe"  NO_SYMLINK_HEAD UNRELIABLE_FSTAT
 				NOGDI OBJECT_CREATION_MODE=1 __USE_MINGW_ANSI_STDIO=0
 				OVERRIDE_STRDUP MMAP_PREVENTS_DELETE USE_WIN32_MMAP
-				HAVE_WPGMPTR ENSURE_MSYSTEM_IS_SET HAVE_RTLGENRANDOM)
+				HAVE_WPGMPTR HAVE_RTLGENRANDOM)
+	if(CMAKE_GENERATOR_PLATFORM STREQUAL "x64")
+		add_compile_definitions(ENSURE_MSYSTEM_IS_SET="MINGW64" MINGW_PREFIX="mingw64")
+	elseif(CMAKE_GENERATOR_PLATFORM STREQUAL "arm64")
+		add_compile_definitions(ENSURE_MSYSTEM_IS_SET="CLANGARM64" MINGW_PREFIX="clangarm64")
+	elseif(CMAKE_GENERATOR_PLATFORM STREQUAL "x86")
+		add_compile_definitions(ENSURE_MSYSTEM_IS_SET="MINGW32" MINGW_PREFIX="mingw32")
+	endif()
 	list(APPEND compat_SOURCES
 		compat/mingw.c
 		compat/winansi.c

index 7073d5844d..6ddc461873 100644
@@ -1318,7 +1318,6 @@ elif host_machine.system() == 'windows'
 
   libgit_c_args += [
     '-DDETECT_MSYS_TTY',
-    '-DENSURE_MSYSTEM_IS_SET',
     '-DNATIVE_CRLF',
     '-DNOGDI',
     '-DNO_POSIX_GOODIES',

@@ -1328,6 +1327,18 @@ elif host_machine.system() == 'windows'
     '-D__USE_MINGW_ANSI_STDIO=0',
   ]
 
+  msystem = get_option('msystem')
+  if msystem != ''
+    mingw_prefix = get_option('mingw_prefix')
+    if mingw_prefix == ''
+      mingw_prefix = '/' + msystem.to_lower()
+    endif
+    libgit_c_args += [
+      '-DENSURE_MSYSTEM_IS_SET="' + msystem + '"',
+      '-DMINGW_PREFIX="' + mingw_prefix + '"'
+    ]
+  endif
+
   libgit_dependencies += compiler.find_library('ntdll')
   libgit_include_directories += 'compat/win32'
   if compiler.get_id() == 'msvc'

index dc88f130d7..becf4689bf 100644
@@ -21,6 +21,10 @@ option('runtime_prefix', type: 'boolean', value: false,
   description: 'Resolve ancillary tooling and support files relative to the location of the runtime binary instead of hard-coding them into the binary.')
 option('sane_tool_path', type: 'array', value: [],
   description: 'An array of paths to pick up tools from in case the normal tools are broken or lacking.')
+option('msystem', type: 'string', value: '',
+  description: 'Fall-back on Windows when MSYSTEM is not set.')
+option('mingw_prefix', type: 'string', value: '',
+  description: 'Fall-back on Windows when MINGW_PREFIX is not set.')
 
 # Build information compiled into Git and other parts like documentation.
 option('build_date', type: 'string', value: '',

gitgitgadget

"Johannes Schindelin via GitGitGadget" <[email protected]>

index 2f7d445eb3..0b63be10b7 100644
@@ -535,7 +535,9 @@ endif
 		compat/win32/pthread.o compat/win32/syslog.o \
 		compat/win32/trace2_win32_process_info.o \
 		compat/win32/dirent.o
-	COMPAT_CFLAGS = -D__USE_MINGW_ACCESS -DDETECT_MSYS_TTY -DNOGDI -DHAVE_STRING_H -Icompat -Icompat/regex -Icompat/win32 -DSTRIP_EXTENSION=\".exe\"
+	COMPAT_CFLAGS = -D__USE_MINGW_ACCESS -DDETECT_MSYS_TTY \
+		-DENSURE_MSYSTEM_IS_SET="\"$(MSYSTEM)\"" -DMINGW_PREFIX="\"$(patsubst /%,%,$(MINGW_PREFIX))\"" \
+		-DNOGDI -DHAVE_STRING_H -Icompat -Icompat/regex -Icompat/win32 -DSTRIP_EXTENSION=\".exe\"
 	BASIC_LDFLAGS = -IGNORE:4217 -IGNORE:4049 -NOLOGO -ENTRY:wmainCRTStartup -SUBSYSTEM:CONSOLE
 	# invalidcontinue.obj allows Git's source code to close the same file
 	# handle twice, or to access the osfhandle of an already-closed stdout

Hi Junio,

Correct. Since Git for Windows does not use Meson to build the project, and since the Visual C-based builds (including CI's windows-meson-build) do not use MSYSTEM, this was not caught earlier. The next iteration will have the fix.

Ciao,
Johannes

From: Johannes Schindelin <[email protected]>

It is merely a historical wart that, say, git-commit exists in the libexec/git-core/ directory, a tribute to the original idea to let Git be essentially a bunch of Unix shell scripts revolving around very few "plumbing" (AKA low-level) commands.

Git has evolved a lot from there. These days, most of Git's functionality is contained within the git executable, in the form of "built-in" commands.

To accommodate for scripts that use the "dashed" form of Git commands, even today, Git provides hard-links that make the git executable available as, say, git-commit, just in case that an old script has not been updated to invoke git commit.

Those hard-links do not come cheap: they take about half a minute for every build of Git on Windows, they are mistaken for taking up huge amounts of space by some Windows Explorer versions that do not understand hard-links, and therefore many a "bug" report had to be addressed.

The "dashed form" has been officially deprecated in Git version 1.5.4, which was released on February 2nd, 2008, i.e. a very long time ago. This deprecation was never finalized by skipping these hard-links, but we can start the process now, in Git for Windows.

---
 config.mak.uname | 2 ++
 1 file changed, 2 insertions(+)

index 3a90995587..88bbe1d78f 100644
@@ -518,6 +518,7 @@ ifeq ($(uname_S),Windows)
 	NO_POSIX_GOODIES = UnfortunatelyYes
 	NATIVE_CRLF = YesPlease
 	DEFAULT_HELP_FORMAT = html
+	SKIP_DASHED_BUILT_INS = YabbaDabbaDoo
 ifeq (/mingw64,$(subst 32,64,$(subst clangarm,mingw,$(prefix))))
 	# Move system config into top-level /etc/
 	ETC_GITCONFIG = ../etc/gitconfig

@@ -710,6 +711,7 @@ ifeq ($(uname_S),MINGW)
 	FSMONITOR_DAEMON_BACKEND = win32
 	FSMONITOR_OS_SETTINGS = win32
 
+	SKIP_DASHED_BUILT_INS = YabbaDabbaDoo
 	RUNTIME_PREFIX = YesPlease
 	HAVE_WPGMPTR = YesWeDo
 	NO_ST_BLOCKS_IN_STRUCT_STAT = YesPlease

gitgitgadget

"Johannes Schindelin via GitGitGadget" <[email protected]>

Good. Perhaps somebody (you do not have to volunteer) can champion their removal from everywhere at Git 3.0 version boundary?

Thanks.

From: Johannes Schindelin <[email protected]>

Special-casing even more configurations simply does not make sense.

---
 config.mak.uname | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

index 88bbe1d78f..2f7d445eb3 100644
@@ -519,7 +519,7 @@ ifeq ($(uname_S),Windows)
 	NATIVE_CRLF = YesPlease
 	DEFAULT_HELP_FORMAT = html
 	SKIP_DASHED_BUILT_INS = YabbaDabbaDoo
-ifeq (/mingw64,$(subst 32,64,$(subst clangarm,mingw,$(prefix))))
+ifneq (,$(MINGW_PREFIX))
 	# Move system config into top-level /etc/
 	ETC_GITCONFIG = ../etc/gitconfig
 	ETC_GITATTRIBUTES = ../etc/gitattributes

@@ -762,6 +762,9 @@ ifeq ($(uname_S),MINGW)
                 ifeq (MINGW32,$(MSYSTEM))
 			BASIC_LDFLAGS += -Wl,--large-address-aware
                 endif
+		# Move system config into top-level /etc/
+		ETC_GITCONFIG = ../etc/gitconfig
+		ETC_GITATTRIBUTES = ../etc/gitattributes
         endif
 	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -fstack-protector-strong
 	EXTLIBS += -lntdll

@@ -772,11 +775,6 @@ ifeq ($(uname_S),MINGW)
 	USE_GETTEXT_SCHEME = fallthrough
 	USE_LIBPCRE = YesPlease
 	NO_PYTHON =
-        ifeq (/mingw64,$(subst 32,64,$(subst clangarm,mingw,$(prefix))))
-		# Move system config into top-level /etc/
-		ETC_GITCONFIG = ../etc/gitconfig
-		ETC_GITATTRIBUTES = ../etc/gitattributes
-        endif
 endif
 ifeq ($(uname_S),QNX)
 	COMPAT_CFLAGS += -DSA_RESTART=0

gitgitgadget

From: Johannes Schindelin <[email protected]>

A change between versions 2.4.1 and 2.6.0 of the MSYS2 runtime modified how Cygwin's runtime (and hence Git for Windows' MSYS2 runtime derivative) handles locales: d16a56306d (Consolidate wctomb/mbtowc calls for POSIX-1.2008, 2016-07-20).

An unintended side-effect is that "cold-calling" into the POSIX emulation will start with a locale based on the current code page, something that Git for Windows is very ill-prepared for, as it expects to be able to pass a command-line containing non-ASCII characters to the shell without having those characters munged.

One symptom of this behavior: when git clone or git fetch shell out to call git-upload-pack with a path that contains non-ASCII characters, the shell tried to interpret the entire command-line (including command-line parameters) as executable path, which obviously must fail.

This fixes https://github.com/git-for-windows/git/issues/1036

---
 compat/mingw.c | 3 +++
 1 file changed, 3 insertions(+)

index e0fbd2c66d..afdc1ef2db 100644
@@ -3185,6 +3185,9 @@ static void setup_windows_environment(void)
 		if (!tmp && (tmp = getenv("USERPROFILE")))
 			setenv("HOME", tmp, 1);
 	}
+
+	if (!getenv("LC_ALL") && !getenv("LC_CTYPE") && !getenv("LANG"))
+		setenv("LC_CTYPE", "C.UTF-8", 1);
 }
 
 int mingw_platform_has_symlinks(void)

gitgitgadget

From: Johannes Schindelin <[email protected]>

Git for Windows wants to add git.exe to the users' PATH, without cluttering the latter with unnecessary executables such as wish.exe. To that end, it invented the concept of its "Git wrapper", i.e. a tiny executable located in C:\Program Files\Git\cmd\git.exe (originally a CMD script) whose sole purpose is to set up a couple of environment variables and then spawn the _actual_ git.exe (which nowadays lives in C:\Program Files\Git\mingw64\bin\git.exe for 64-bit, and the obvious equivalent for 32-bit installations).

Currently, the following environment variables are set unless already initialized:

- MSYSTEM, to make sure that the MSYS2 Bash and the MSYS2 Perl interpreter behave as expected, and

- PLINK_PROTOCOL, to force PuTTY's plink.exe to use the SSH protocol instead of Telnet,

- PATH, to make sure that the bin folder in the user's home directory, as well as the /mingw64/bin and the /usr/bin directories are included. The trick here is that the /mingw64/bin/ and /usr/bin/ directories are relative to the top-level installation directory of Git for Windows (which the included Bash interprets as /, i.e. as the MSYS pseudo root directory).

Using the absence of MSYSTEM as a tell-tale, we can detect in git.exe whether these environment variables have been initialized properly. Therefore we can call C:\Program Files\Git\mingw64\bin\git in-place after this change, without having to call Git through the Git wrapper.

Obviously, above-mentioned directories must be _prepended_ to the PATH variable, otherwise we risk picking up executables from unrelated Git installations. We do that by constructing the new PATH value from scratch, appending $HOME/bin (if HOME is set), then the MSYS2 system directories, and then appending the original PATH.

Side note: this modification of the PATH variable is independent of the modification necessary to reach the executables and scripts in /mingw64/libexec/git-core/, i.e. the GIT_EXEC_PATH. That modification is still performed by Git, elsewhere, long after making the changes described above.

While we _still_ cannot simply hard-link mingw64\bin\git.exe to cmd (because the former depends on a couple of .dll files that are only in mingw64\bin, i.e. calling ...\cmd\git.exe would fail to load due to missing dependencies), at least we can now avoid that extra process of running the Git wrapper (which then has to wait for the spawned git.exe to finish) by calling ...\mingw64\bin\git.exe directly, via its absolute path.

Testing this is in Git's test suite tricky: we set up a "new" MSYS pseudo-root and copy the git.exe file into the appropriate location, then verify that MSYSTEM is set properly, and also that the PATH is modified so that scripts can be found in $HOME/bin, /mingw64/bin/ and /usr/bin/.

This addresses https://github.com/git-for-windows/git/issues/2283

---
 compat/mingw.c        | 65 +++++++++++++++++++++++++++++++++++++++++++
 config.mak.uname      |  8 ++++--
 t/t0060-path-utils.sh | 33 +++++++++++++++++++++-
 3 files changed, 103 insertions(+), 3 deletions(-)

index afdc1ef2db..563e6299bc 100644
@@ -3134,6 +3134,45 @@ int xwcstoutf(char *utf, const wchar_t *wcs, size_t utflen)
 	return -1;
 }
 
+#ifdef ENSURE_MSYSTEM_IS_SET
+#if !defined(RUNTIME_PREFIX) || !defined(HAVE_WPGMPTR) || !defined(MINGW_PREFIX)
+static size_t append_system_bin_dirs(char *path UNUSED, size_t size UNUSED)
+{
+	return 0;
+}
+#else
+static size_t append_system_bin_dirs(char *path, size_t size)
+{
+	char prefix[32768];
+	const char *slash;
+	size_t len = xwcstoutf(prefix, _wpgmptr, sizeof(prefix)), off = 0;
+
+	if (len == 0 || len >= sizeof(prefix) ||
+	    !(slash = find_last_dir_sep(prefix)))
+		return 0;
+	/* strip trailing `git.exe` */
+	len = slash - prefix;
+
+	/* strip trailing `cmd` or `<mingw-prefix>\bin` or `bin` or `libexec\git-core` */
+	if (strip_suffix_mem(prefix, &len, "\\" MINGW_PREFIX "\\libexec\\git-core") ||
+	    strip_suffix_mem(prefix, &len, "\\" MINGW_PREFIX "\\bin"))
+		off += xsnprintf(path + off, size - off,
+				 "%.*s\\" MINGW_PREFIX "\\bin;", (int)len, prefix);
+	else if (strip_suffix_mem(prefix, &len, "\\cmd") ||
+		 strip_suffix_mem(prefix, &len, "\\bin") ||
+		 strip_suffix_mem(prefix, &len, "\\libexec\\git-core"))
+		off += xsnprintf(path + off, size - off,
+				 "%.*s\\" MINGW_PREFIX "\\bin;", (int)len, prefix);
+	else
+		return 0;
+
+	off += xsnprintf(path + off, size - off,
+			 "%.*s\\usr\\bin;", (int)len, prefix);
+	return off;
+}
+#endif
+#endif
+
 static void setup_windows_environment(void)
 {
 	char *tmp = getenv("TMPDIR");

@@ -3186,6 +3225,32 @@ static void setup_windows_environment(void)
 			setenv("HOME", tmp, 1);
 	}
 
+	if (!getenv("PLINK_PROTOCOL"))
+		setenv("PLINK_PROTOCOL", "ssh", 0);
+
+#ifdef ENSURE_MSYSTEM_IS_SET
+	if (!(tmp = getenv("MSYSTEM")) || !tmp[0]) {
+		const char *home = getenv("HOME"), *path = getenv("PATH");
+		char buf[32768];
+		size_t off = 0;
+
+		setenv("MSYSTEM", ENSURE_MSYSTEM_IS_SET, 1);
+
+		if (home)
+			off += xsnprintf(buf + off, sizeof(buf) - off,
+					 "%s\\bin;", home);
+		off += append_system_bin_dirs(buf + off, sizeof(buf) - off);
+		if (path)
+			off += xsnprintf(buf + off, sizeof(buf) - off,
+					 "%s", path);
+		else if (off > 0)
+			buf[off - 1] = '\0';
+		else
+			buf[0] = '\0';
+		setenv("PATH", buf, 1);
+	}
+#endif
+
 	if (!getenv("LC_ALL") && !getenv("LC_CTYPE") && !getenv("LANG"))
 		setenv("LC_CTYPE", "C.UTF-8", 1);
 }

index 2f7d445eb3..0b63be10b7 100644
@@ -535,7 +535,9 @@ endif
 		compat/win32/pthread.o compat/win32/syslog.o \
 		compat/win32/trace2_win32_process_info.o \
 		compat/win32/dirent.o
-	COMPAT_CFLAGS = -D__USE_MINGW_ACCESS -DDETECT_MSYS_TTY -DNOGDI -DHAVE_STRING_H -Icompat -Icompat/regex -Icompat/win32 -DSTRIP_EXTENSION=\".exe\"
+	COMPAT_CFLAGS = -D__USE_MINGW_ACCESS -DDETECT_MSYS_TTY \
+		-DENSURE_MSYSTEM_IS_SET="\"$(MSYSTEM)\"" -DMINGW_PREFIX="\"$(patsubst /%,%,$(MINGW_PREFIX))\"" \
+		-DNOGDI -DHAVE_STRING_H -Icompat -Icompat/regex -Icompat/win32 -DSTRIP_EXTENSION=\".exe\"
 	BASIC_LDFLAGS = -IGNORE:4217 -IGNORE:4049 -NOLOGO -ENTRY:wmainCRTStartup -SUBSYSTEM:CONSOLE
 	# invalidcontinue.obj allows Git's source code to close the same file
 	# handle twice, or to access the osfhandle of an already-closed stdout

@@ -758,7 +760,9 @@ ifeq ($(uname_S),MINGW)
 		prefix = $(MINGW_PREFIX)
 		HOST_CPU = $(patsubst %-w64-mingw32,%,$(MINGW_CHOST))
 		BASIC_LDFLAGS += -Wl,--pic-executable
-		COMPAT_CFLAGS += -DDETECT_MSYS_TTY
+		COMPAT_CFLAGS += -DDETECT_MSYS_TTY \
+			-DENSURE_MSYSTEM_IS_SET="\"$(MSYSTEM)\"" \
+			-DMINGW_PREFIX="\"$(patsubst /%,%,$(MINGW_PREFIX))\""
                 ifeq (MINGW32,$(MSYSTEM))
 			BASIC_LDFLAGS += -Wl,--large-address-aware
                 endif

index 8545cdfab5..56faf5fe73 100755
@@ -602,7 +602,8 @@ test_expect_success !VALGRIND,RUNTIME_PREFIX,CAN_EXEC_IN_PWD 'RUNTIME_PREFIX wor
 	echo "echo HERE" | write_script pretend/libexec/git-core/git-here &&
 	GIT_EXEC_PATH= ./pretend/bin/git here >actual &&
 	echo HERE >expect &&
-	test_cmp expect actual'
+	test_cmp expect actual
+'
 
 test_expect_success !VALGRIND,RUNTIME_PREFIX,CAN_EXEC_IN_PWD '%(prefix)/ works' '
 	git config yes.path "%(prefix)/yes" &&

@@ -611,4 +612,34 @@ test_expect_success !VALGRIND,RUNTIME_PREFIX,CAN_EXEC_IN_PWD '%(prefix)/ works'
 	test_cmp expect actual
 '
 
+test_expect_success MINGW,RUNTIME_PREFIX 'MSYSTEM/PATH is adjusted if necessary' '
+	if test -z "$MINGW_PREFIX"
+	then
+		MINGW_PREFIX="/$(echo "${MSYSTEM:-MINGW64}" | tr A-Z a-z)"
+	fi &&
+	mkdir -p "$HOME"/bin pretend"$MINGW_PREFIX"/bin \
+		pretend"$MINGW_PREFIX"/libexec/git-core pretend/usr/bin &&
+	cp "$GIT_EXEC_PATH"/git.exe pretend"$MINGW_PREFIX"/bin/ &&
+	cp "$GIT_EXEC_PATH"/git.exe pretend"$MINGW_PREFIX"/libexec/git-core/ &&
+	# copy the .dll files, if any (happens when building via CMake)
+	if test -n "$(ls "$GIT_EXEC_PATH"/*.dll 2>/dev/null)"
+	then
+		cp "$GIT_EXEC_PATH"/*.dll pretend"$MINGW_PREFIX"/bin/ &&
+		cp "$GIT_EXEC_PATH"/*.dll pretend"$MINGW_PREFIX"/libexec/git-core/
+	fi &&
+	echo "env | grep MSYSTEM=" | write_script "$HOME"/bin/git-test-home &&
+	echo "echo ${MINGW_PREFIX#/}" | write_script pretend"$MINGW_PREFIX"/bin/git-test-bin &&
+	echo "echo usr" | write_script pretend/usr/bin/git-test-bin2 &&
+
+	(
+		MSYSTEM= &&
+		GIT_EXEC_PATH= &&
+		pretend"$MINGW_PREFIX"/libexec/git-core/git.exe test-home >actual &&
+		pretend"$MINGW_PREFIX"/libexec/git-core/git.exe test-bin >>actual &&
+		pretend"$MINGW_PREFIX"/bin/git.exe test-bin2 >>actual
+	) &&
+	test_write_lines MSYSTEM=$MSYSTEM "${MINGW_PREFIX#/}" usr >expect &&
+	test_cmp expect actual
+'
+
 test_done

gitgitgadget

"Johannes Schindelin via GitGitGadget" <[email protected]>

Hi Junio,

"sane state" is quite the strong wording for such a minor issue.

The MSYSTEM variable is useful as an indicator, and is therefore used in many scripts. Git itself only uses it as a tell-tale that a GUI should be used in git bisect visualize, otherwise Git is totally fine with empty or even non-sensical values.

But it _is_ a convenient thing to have, together with MINGW_PREFIX to ensure that the respective /*/bin/ directories are prepended to the PATH variable. That's what this is all about.

Sure it can. Just like PATH or HOME can be empty, or invalid. But the person building the project has to go out of their way to make it so, therefore I don't want to spend any more brain cells on that highly unlikely scenario.

Ciao,
Johannes

Well I didn't know how important that variable is, and didn't expect or wish to offend you by use of that word. Perhaps "sane"->"usual" would have been better? Sorry about uninformed choice of phrasing.

Again, I didn't know how unlikely it was, so I judged it based on the existing practice (quoted) below.

When rebasing a Git for Windows commit to enable Rust in Windows CI, I noticed just how many of the preceding commits touching adjacent code haven't been upstreamed yet. This patch series rectifies that.

Changes since v1:

* Fixed the meson-side handling of the MINGW_PREFIX constant (this was not noticed in the Git for Windows project because it does not use Meson to build the project).

Johannes Schindelin (12):
mingw: include the Python parts in the build
mingw: stop hard-coding CC = gcc
mingw: drop the -D_USE_32BIT_TIME_T option
mingw: only use -Wl,--large-address-aware for 32-bit builds
mingw: avoid over-specifying --pic-executable
mingw: set the prefix and HOST_CPU as per MSYS2's settings
mingw: only enable the MSYS2-specific stuff when compiling in MSYS2
mingw: rely on MSYS2's metadata instead of hard-coding it
windows: skip linking git-<command> for built-ins
mingw: always define ETC_* for MSYS2 environments
mingw: ensure valid CTYPE
mingw: allow git.exe to be used instead of the "Git wrapper"

 compat/mingw.c                      | 68 +++++++++++++++++++++++++++++
 config.mak.uname                    | 61 ++++++++++++--------------
 contrib/buildsystems/CMakeLists.txt |  9 +++-
 meson.build                         | 15 ++++++-
 meson_options.txt                   |  4 ++
 t/t0060-path-utils.sh               | 33 +++++++++++++-
 6 files changed, 153 insertions(+), 37 deletions(-)

base-commit: 5b2471720c93ee30e5764a19f3d3b3ae9ec9712a
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2195%2Fdscho%2Fupstream-some-more-git-for-windows-patches-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2195/dscho/upstream-some-more-git-for-windows-patches-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/2195

Range-diff vs v1:

  1:  fd594ffa6c =  1:  fd594ffa6c mingw: include the Python parts in the build
  2:  47fc974582 =  2:  47fc974582 mingw: stop hard-coding `CC = gcc`
  3:  53f1fd5763 =  3:  53f1fd5763 mingw: drop the -D_USE_32BIT_TIME_T option
  4:  ad96aedda6 =  4:  ad96aedda6 mingw: only use -Wl,--large-address-aware for 32-bit builds
  5:  d67201f9b3 =  5:  d67201f9b3 mingw: avoid over-specifying `--pic-executable`
  6:  b49c42c50d =  6:  b49c42c50d mingw: set the prefix and HOST_CPU as per MSYS2's settings
  7:  b00d242621 =  7:  b00d242621 mingw: only enable the MSYS2-specific stuff when compiling in MSYS2
  8:  1593d1d1a0 !  8:  9de4ea7fc1 mingw: rely on MSYS2's metadata instead of hard-coding it

@@ Commit message
as before, but drops it for Meson (because it is unclear how Meson could do this in a more flexible manner).

+ Helped-by: Junio C Hamano <[email protected]>

## config.mak.uname ##
@@ meson.build: elif host_machine.system() == 'windows'
+ if msystem != ''
+ mingw_prefix = get_option('mingw_prefix')
+ if mingw_prefix == ''
-+ mingw_prefix = '/' + msystem.to_lower()
++ mingw_prefix = msystem.to_lower()
++ elif mingw_prefix.startswith('/')
++ mingw_prefix = mingw_prefix.substring(1)
+ endif
+ libgit_c_args += [
+ '-DENSURE_MSYSTEM_IS_SET="' + msystem + '"',

  9:  c4e97449c1 =  9:  60a2999b75 windows: skip linking `git-<command>` for built-ins
 10:  625f01ad29 = 10:  1dd6ac9295 mingw: always define `ETC_*` for MSYS2 environments
 11:  39a138a773 = 11:  d44d05c897 mingw: ensure valid CTYPE
 12:  f822133191 = 12:  679d1fdd41 mingw: allow `git.exe` to be used instead of the "Git wrapper"

gitgitgadget

From: Johannes Schindelin <[email protected]>

While Git for Windows does not _ship_ Python (in order to save on bandwidth), MSYS2 provides very fine Python interpreters that users can easily take advantage of, by using Git for Windows within its SDK.

---
 config.mak.uname | 1 +
 1 file changed, 1 insertion(+)

index 9ebd240378..8dd8acfaa5 100644
@@ -781,6 +781,7 @@ ifeq ($(uname_S),MINGW)
 	HAVE_LIBCHARSET_H = YesPlease
 	USE_GETTEXT_SCHEME = fallthrough
 	USE_LIBPCRE = YesPlease
+	NO_PYTHON =
         ifeq (/mingw64,$(subst 32,64,$(subst clangarm,mingw,$(prefix))))
 		# Move system config into top-level /etc/
 		ETC_GITCONFIG = ../etc/gitconfig

gitgitgadget

From: Johannes Schindelin <[email protected]>

This is no longer true in general, not with supporting Clang out of the box.

---
 config.mak.uname | 1 -
 1 file changed, 1 deletion(-)

index 8dd8acfaa5..a464b2009c 100644
@@ -771,7 +771,6 @@ ifeq ($(uname_S),MINGW)
 		COMPAT_CFLAGS += -D_USE_32BIT_TIME_T
 		BASIC_LDFLAGS += -Wl,--large-address-aware
         endif
-	CC = gcc
 	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \
 		-fstack-protector-strong
 	EXTLIBS += -lntdll

gitgitgadget

From: Johannes Schindelin <[email protected]>

This option was added in fa93bb20d72 (MinGW: Fix stat definitions to work with MinGW runtime version 4.0, 2013-09-11), i.e. a _long_ time ago. So long, in fact, that it still targeted MinGW. But we switched to mingw-w64 in 2015, which seems not to share the problem, and therefore does not require a fix.

Even worse: This flag is incompatible with UCRT64, which we are about to support by way of upstreaming mingw-w64-git to the MSYS2 project, see
https://github.com/msys2/MINGW-packages/pull/26470 for details.

So let's send that option into its well-deserved retirement.

---
 config.mak.uname | 1 -
 1 file changed, 1 deletion(-)

index a464b2009c..cc0ed63af3 100644
@@ -768,7 +768,6 @@ ifeq ($(uname_S),MINGW)
 		HOST_CPU = aarch64
 		BASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup
         else
-		COMPAT_CFLAGS += -D_USE_32BIT_TIME_T
 		BASIC_LDFLAGS += -Wl,--large-address-aware
         endif
 	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \

gitgitgadget

From: Johannes Schindelin <[email protected]>

That option only matters there, and is in fact only really understood in those builds; UCRT64 versions of GCC, for example, do not know what to do with that option.

---
 config.mak.uname | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

index cc0ed63af3..b667c693ec 100644
@@ -757,9 +757,8 @@ ifeq ($(uname_S),MINGW)
         ifeq (MINGW32,$(MSYSTEM))
 		prefix = /mingw32
 		HOST_CPU = i686
-		BASIC_LDFLAGS += -Wl,--pic-executable,-e,_mainCRTStartup
-        endif
-        ifeq (MINGW64,$(MSYSTEM))
+		BASIC_LDFLAGS += -Wl,--pic-executable,-e,_mainCRTStartup -Wl,--large-address-aware
+        else ifeq (MINGW64,$(MSYSTEM))
 		prefix = /mingw64
 		HOST_CPU = x86_64
 		BASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup

@@ -768,7 +767,6 @@ ifeq ($(uname_S),MINGW)
 		HOST_CPU = aarch64
 		BASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup
         else
-		BASIC_LDFLAGS += -Wl,--large-address-aware
         endif
 	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \
 		-fstack-protector-strong

gitgitgadget

From: Johannes Schindelin <[email protected]>

In bf2d5d8239e (Don't let ld strip relocations, 2016-01-16) (picked from
https://github.com/git-for-windows/git/pull/612/commits/6a237925bf10),
Git for Windows introduced the -Wl,-pic-executable flag, specifying the exact entry point via -e. This required discerning between i686 and x86_64 code because the former required the symbol to be prefixed with an underscore, the latter did not.

As per https://sourceware.org/bugzilla/show_bug.cgi?id=10865, the specified symbols are already the default, though.

So let's drop the overly-specific definition.

---
 config.mak.uname | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

index b667c693ec..f6387f4c7b 100644
@@ -757,15 +757,15 @@ ifeq ($(uname_S),MINGW)
         ifeq (MINGW32,$(MSYSTEM))
 		prefix = /mingw32
 		HOST_CPU = i686
-		BASIC_LDFLAGS += -Wl,--pic-executable,-e,_mainCRTStartup -Wl,--large-address-aware
+		BASIC_LDFLAGS += -Wl,--pic-executable -Wl,--large-address-aware
         else ifeq (MINGW64,$(MSYSTEM))
 		prefix = /mingw64
 		HOST_CPU = x86_64
-		BASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup
+		BASIC_LDFLAGS += -Wl,--pic-executable
         else ifeq (CLANGARM64,$(MSYSTEM))
 		prefix = /clangarm64
 		HOST_CPU = aarch64
-		BASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup
+		BASIC_LDFLAGS += -Wl,--pic-executable
         else
         endif
 	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \

gitgitgadget

From: Johannes Schindelin <[email protected]>

MSYS2 already defines a couple of helpful environment variables, and we can use those to infer the installation location as well as the CPU. No need for hard-coding ;-)

---
 config.mak.uname | 18 ++++++------------
 1 file changed, 6 insertions(+), 12 deletions(-)

index f6387f4c7b..8363239513 100644
@@ -754,19 +754,13 @@ ifeq ($(uname_S),MINGW)
         ifneq (,$(findstring -O,$(filter-out -O0 -Og,$(CFLAGS))))
 		BASIC_LDFLAGS += -Wl,--dynamicbase
         endif
-        ifeq (MINGW32,$(MSYSTEM))
-		prefix = /mingw32
-		HOST_CPU = i686
-		BASIC_LDFLAGS += -Wl,--pic-executable -Wl,--large-address-aware
-        else ifeq (MINGW64,$(MSYSTEM))
-		prefix = /mingw64
-		HOST_CPU = x86_64
-		BASIC_LDFLAGS += -Wl,--pic-executable
-        else ifeq (CLANGARM64,$(MSYSTEM))
-		prefix = /clangarm64
-		HOST_CPU = aarch64
+        ifneq (,$(MSYSTEM))
+		prefix = $(MINGW_PREFIX)
+		HOST_CPU = $(patsubst %-w64-mingw32,%,$(MINGW_CHOST))
 		BASIC_LDFLAGS += -Wl,--pic-executable
-        else
+                ifeq (MINGW32,$(MSYSTEM))
+			BASIC_LDFLAGS += -Wl,--large-address-aware
+                endif
         endif
 	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \
 		-fstack-protector-strong

gitgitgadget

From: Johannes Schindelin <[email protected]>

The tell-tale is the presence of the MSYSTEM value while compiling, of course. In that case, we want to ensure that MSYSTEM is set when running git.exe, and also enable the magic MSYS2 tty detection.

---
 config.mak.uname | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

index 8363239513..21f53e3f7e 100644
@@ -758,12 +758,12 @@ ifeq ($(uname_S),MINGW)
 		prefix = $(MINGW_PREFIX)
 		HOST_CPU = $(patsubst %-w64-mingw32,%,$(MINGW_CHOST))
 		BASIC_LDFLAGS += -Wl,--pic-executable
+		COMPAT_CFLAGS += -DDETECT_MSYS_TTY
                 ifeq (MINGW32,$(MSYSTEM))
 			BASIC_LDFLAGS += -Wl,--large-address-aware
                 endif
         endif
-	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \
-		-fstack-protector-strong
+	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -fstack-protector-strong
 	EXTLIBS += -lntdll
 	EXTRA_PROGRAMS += headless-git$X
 	INSTALL = /bin/install

gitgitgadget

From: Johannes Schindelin <[email protected]>

MSYS2 defines some helpful environment variables, e.g. MSYSTEM. There is code in Git for Windows to ensure that that MSYSTEM variable is set, hard-coding a default.

However, the existing solution jumps through hoops to reconstruct the proper default, and is even incomplete doing so, as we found out when we extended it to support CLANGARM64.

This is absolutely unnecessary because there is already a perfectly valid MSYSTEM value we can use at build time. This is even true when building the MINGW32 variant on a MINGW64 system because makepkg-mingw will override the MSYSTEM value as per the MINGW_ARCH array.

The same is equally true for the /mingw64, /mingw32 and /clangarm64 prefix: those values are already available via the MINGW_PREFIX environment variable, and we just need to pass that setting through.

Only when MINGW_PREFIX is not set (as is the case in Git for Windows' minimal SDK, where only MSYSTEM is guaranteed to be set correctly), we use as fall-back the top-level directory whose name is the down-cased value of the MSYSTEM variable.

Incidentally, this also broadens the support to all the configurations supported by the MSYS2 project, i.e. clang64 & ucrt64, too.

Note: This keeps the same, hard-coded MSYSTEM platform support for CMake as before, but drops it for Meson (because it is unclear how Meson could do this in a more flexible manner).

---
 config.mak.uname                    | 14 ++++++--------
 contrib/buildsystems/CMakeLists.txt |  9 ++++++++-
 meson.build                         | 15 ++++++++++++++-
 meson_options.txt                   |  4 ++++
 4 files changed, 32 insertions(+), 10 deletions(-)

index 21f53e3f7e..3a90995587 100644
@@ -465,14 +465,8 @@ ifeq ($(uname_S),Windows)
 	GIT_VERSION := $(GIT_VERSION).MSVC
 	pathsep = ;
 	# Assume that this is built in Git for Windows' SDK
-        ifeq (MINGW32,$(MSYSTEM))
-		prefix = /mingw32
-        else
-                ifeq (CLANGARM64,$(MSYSTEM))
-			prefix = /clangarm64
-                else
-			prefix = /mingw64
-                endif
+        ifneq (,$(MSYSTEM))
+		prefix = $(MINGW_PREFIX)
         endif
 	# Prepend MSVC 64-bit tool-chain to PATH.
 	#

@@ -755,6 +749,10 @@ ifeq ($(uname_S),MINGW)
 		BASIC_LDFLAGS += -Wl,--dynamicbase
         endif
         ifneq (,$(MSYSTEM))
+                ifeq ($(MINGW_PREFIX),$(filter-out /%,$(MINGW_PREFIX)))
+			# Override if empty or does not start with a slash
+			MINGW_PREFIX := /$(shell echo '$(MSYSTEM)' | tr A-Z a-z)
+                endif
 		prefix = $(MINGW_PREFIX)
 		HOST_CPU = $(patsubst %-w64-mingw32,%,$(MINGW_CHOST))
 		BASIC_LDFLAGS += -Wl,--pic-executable

index a57c4b464f..7285bd9ac2 100644
@@ -256,7 +256,14 @@ if(CMAKE_SYSTEM_NAME STREQUAL "Windows")
 				_CONSOLE DETECT_MSYS_TTY STRIP_EXTENSION=".exe"  NO_SYMLINK_HEAD UNRELIABLE_FSTAT
 				NOGDI OBJECT_CREATION_MODE=1 __USE_MINGW_ANSI_STDIO=0
 				OVERRIDE_STRDUP MMAP_PREVENTS_DELETE USE_WIN32_MMAP
-				HAVE_WPGMPTR ENSURE_MSYSTEM_IS_SET HAVE_RTLGENRANDOM)
+				HAVE_WPGMPTR HAVE_RTLGENRANDOM)
+	if(CMAKE_GENERATOR_PLATFORM STREQUAL "x64")
+		add_compile_definitions(ENSURE_MSYSTEM_IS_SET="MINGW64" MINGW_PREFIX="mingw64")
+	elseif(CMAKE_GENERATOR_PLATFORM STREQUAL "arm64")
+		add_compile_definitions(ENSURE_MSYSTEM_IS_SET="CLANGARM64" MINGW_PREFIX="clangarm64")
+	elseif(CMAKE_GENERATOR_PLATFORM STREQUAL "x86")
+		add_compile_definitions(ENSURE_MSYSTEM_IS_SET="MINGW32" MINGW_PREFIX="mingw32")
+	endif()
 	list(APPEND compat_SOURCES
 		compat/mingw.c
 		compat/winansi.c

index 7073d5844d..a8aba81e29 100644
@@ -1318,7 +1318,6 @@ elif host_machine.system() == 'windows'
 
   libgit_c_args += [
     '-DDETECT_MSYS_TTY',
-    '-DENSURE_MSYSTEM_IS_SET',
     '-DNATIVE_CRLF',
     '-DNOGDI',
     '-DNO_POSIX_GOODIES',

@@ -1328,6 +1327,20 @@ elif host_machine.system() == 'windows'
     '-D__USE_MINGW_ANSI_STDIO=0',
   ]
 
+  msystem = get_option('msystem')
+  if msystem != ''
+    mingw_prefix = get_option('mingw_prefix')
+    if mingw_prefix == ''
+      mingw_prefix = msystem.to_lower()
+    elif mingw_prefix.startswith('/')
+      mingw_prefix = mingw_prefix.substring(1)
+    endif
+    libgit_c_args += [
+      '-DENSURE_MSYSTEM_IS_SET="' + msystem + '"',
+      '-DMINGW_PREFIX="' + mingw_prefix + '"'
+    ]
+  endif
+
   libgit_dependencies += compiler.find_library('ntdll')
   libgit_include_directories += 'compat/win32'
   if compiler.get_id() == 'msvc'

index dc88f130d7..becf4689bf 100644
@@ -21,6 +21,10 @@ option('runtime_prefix', type: 'boolean', value: false,
   description: 'Resolve ancillary tooling and support files relative to the location of the runtime binary instead of hard-coding them into the binary.')
 option('sane_tool_path', type: 'array', value: [],
   description: 'An array of paths to pick up tools from in case the normal tools are broken or lacking.')
+option('msystem', type: 'string', value: '',
+  description: 'Fall-back on Windows when MSYSTEM is not set.')
+option('mingw_prefix', type: 'string', value: '',
+  description: 'Fall-back on Windows when MINGW_PREFIX is not set.')
 
 # Build information compiled into Git and other parts like documentation.
 option('build_date', type: 'string', value: '',

gitgitgadget

Am 12.08.26 um 09:52 schrieb Johannes Schindelin via GitGitGadget:

Haven't heard anything on this topic for a few weeks. Can we conclude it anytime soon? I do not like too many topics that are almost there hanging around only because they miss the finishing touches.

Thanks.

Hi Hannes,

Ah, right, the following hunks do touch the ENSURE_MSYSTEM_IS_SET stuff. But they _also_ add the MINGW_PREFIX stuff.

Will disentangle.

Ciao,
Johannes

Hi Hannes,

Gah. My assessment is incorrect. At this point in the patch series, the MINGW_PREFIX constant isn't used anywhere in the C code. So you were absolutely right, those hunks do need to move to 12/12 wholesale.

Sorry about the slalom,
Johannes

From: Johannes Schindelin <[email protected]>

It is merely a historical wart that, say, git-commit exists in the libexec/git-core/ directory, a tribute to the original idea to let Git be essentially a bunch of Unix shell scripts revolving around very few "plumbing" (AKA low-level) commands.

Git has evolved a lot from there. These days, most of Git's functionality is contained within the git executable, in the form of "built-in" commands.

To accommodate for scripts that use the "dashed" form of Git commands, even today, Git provides hard-links that make the git executable available as, say, git-commit, just in case that an old script has not been updated to invoke git commit.

Those hard-links do not come cheap: they take about half a minute for every build of Git on Windows, they are mistaken for taking up huge amounts of space by some Windows Explorer versions that do not understand hard-links, and therefore many a "bug" report had to be addressed.

The "dashed form" has been officially deprecated in Git version 1.5.4, which was released on February 2nd, 2008, i.e. a very long time ago. This deprecation was never finalized by skipping these hard-links, but we can start the process now, in Git for Windows.

---
 config.mak.uname | 2 ++
 1 file changed, 2 insertions(+)

index 3a90995587..88bbe1d78f 100644
@@ -518,6 +518,7 @@ ifeq ($(uname_S),Windows)
 	NO_POSIX_GOODIES = UnfortunatelyYes
 	NATIVE_CRLF = YesPlease
 	DEFAULT_HELP_FORMAT = html
+	SKIP_DASHED_BUILT_INS = YabbaDabbaDoo
 ifeq (/mingw64,$(subst 32,64,$(subst clangarm,mingw,$(prefix))))
 	# Move system config into top-level /etc/
 	ETC_GITCONFIG = ../etc/gitconfig

@@ -710,6 +711,7 @@ ifeq ($(uname_S),MINGW)
 	FSMONITOR_DAEMON_BACKEND = win32
 	FSMONITOR_OS_SETTINGS = win32
 
+	SKIP_DASHED_BUILT_INS = YabbaDabbaDoo
 	RUNTIME_PREFIX = YesPlease
 	HAVE_WPGMPTR = YesWeDo
 	NO_ST_BLOCKS_IN_STRUCT_STAT = YesPlease

gitgitgadget

From: Johannes Schindelin <[email protected]>

Special-casing even more configurations simply does not make sense.

---
 config.mak.uname | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

index 88bbe1d78f..2f7d445eb3 100644
@@ -519,7 +519,7 @@ ifeq ($(uname_S),Windows)
 	NATIVE_CRLF = YesPlease
 	DEFAULT_HELP_FORMAT = html
 	SKIP_DASHED_BUILT_INS = YabbaDabbaDoo
-ifeq (/mingw64,$(subst 32,64,$(subst clangarm,mingw,$(prefix))))
+ifneq (,$(MINGW_PREFIX))
 	# Move system config into top-level /etc/
 	ETC_GITCONFIG = ../etc/gitconfig
 	ETC_GITATTRIBUTES = ../etc/gitattributes

@@ -762,6 +762,9 @@ ifeq ($(uname_S),MINGW)
                 ifeq (MINGW32,$(MSYSTEM))
 			BASIC_LDFLAGS += -Wl,--large-address-aware
                 endif
+		# Move system config into top-level /etc/
+		ETC_GITCONFIG = ../etc/gitconfig
+		ETC_GITATTRIBUTES = ../etc/gitattributes
         endif
 	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -fstack-protector-strong
 	EXTLIBS += -lntdll

@@ -772,11 +775,6 @@ ifeq ($(uname_S),MINGW)
 	USE_GETTEXT_SCHEME = fallthrough
 	USE_LIBPCRE = YesPlease
 	NO_PYTHON =
-        ifeq (/mingw64,$(subst 32,64,$(subst clangarm,mingw,$(prefix))))
-		# Move system config into top-level /etc/
-		ETC_GITCONFIG = ../etc/gitconfig
-		ETC_GITATTRIBUTES = ../etc/gitattributes
-        endif
 endif
 ifeq ($(uname_S),QNX)
 	COMPAT_CFLAGS += -DSA_RESTART=0

gitgitgadget

From: Johannes Schindelin <[email protected]>

A change between versions 2.4.1 and 2.6.0 of the MSYS2 runtime modified how Cygwin's runtime (and hence Git for Windows' MSYS2 runtime derivative) handles locales: d16a56306d (Consolidate wctomb/mbtowc calls for POSIX-1.2008, 2016-07-20).

An unintended side-effect is that "cold-calling" into the POSIX emulation will start with a locale based on the current code page, something that Git for Windows is very ill-prepared for, as it expects to be able to pass a command-line containing non-ASCII characters to the shell without having those characters munged.

One symptom of this behavior: when git clone or git fetch shell out to call git-upload-pack with a path that contains non-ASCII characters, the shell tried to interpret the entire command-line (including command-line parameters) as executable path, which obviously must fail.

This fixes https://github.com/git-for-windows/git/issues/1036

---
 compat/mingw.c | 3 +++
 1 file changed, 3 insertions(+)

index e0fbd2c66d..afdc1ef2db 100644
@@ -3185,6 +3185,9 @@ static void setup_windows_environment(void)
 		if (!tmp && (tmp = getenv("USERPROFILE")))
 			setenv("HOME", tmp, 1);
 	}
+
+	if (!getenv("LC_ALL") && !getenv("LC_CTYPE") && !getenv("LANG"))
+		setenv("LC_CTYPE", "C.UTF-8", 1);
 }
 
 int mingw_platform_has_symlinks(void)

gitgitgadget

From: Johannes Schindelin <[email protected]>

Git for Windows wants to add git.exe to the users' PATH, without cluttering the latter with unnecessary executables such as wish.exe. To that end, it invented the concept of its "Git wrapper", i.e. a tiny executable located in C:\Program Files\Git\cmd\git.exe (originally a CMD script) whose sole purpose is to set up a couple of environment variables and then spawn the _actual_ git.exe (which nowadays lives in C:\Program Files\Git\mingw64\bin\git.exe for 64-bit, and the obvious equivalent for 32-bit installations).

Currently, the following environment variables are set unless already initialized:

- MSYSTEM, to make sure that the MSYS2 Bash and the MSYS2 Perl interpreter behave as expected, and

- PLINK_PROTOCOL, to force PuTTY's plink.exe to use the SSH protocol instead of Telnet,

- PATH, to make sure that the bin folder in the user's home directory, as well as the /mingw64/bin and the /usr/bin directories are included. The trick here is that the /mingw64/bin/ and /usr/bin/ directories are relative to the top-level installation directory of Git for Windows (which the included Bash interprets as /, i.e. as the MSYS pseudo root directory).

Using the absence of MSYSTEM as a tell-tale, we can detect in git.exe whether these environment variables have been initialized properly. Therefore we can call C:\Program Files\Git\mingw64\bin\git in-place after this change, without having to call Git through the Git wrapper.

Obviously, above-mentioned directories must be _prepended_ to the PATH variable, otherwise we risk picking up executables from unrelated Git installations. We do that by constructing the new PATH value from scratch, appending $HOME/bin (if HOME is set), then the MSYS2 system directories, and then appending the original PATH.

Side note: this modification of the PATH variable is independent of the modification necessary to reach the executables and scripts in /mingw64/libexec/git-core/, i.e. the GIT_EXEC_PATH. That modification is still performed by Git, elsewhere, long after making the changes described above.

While we _still_ cannot simply hard-link mingw64\bin\git.exe to cmd (because the former depends on a couple of .dll files that are only in mingw64\bin, i.e. calling ...\cmd\git.exe would fail to load due to missing dependencies), at least we can now avoid that extra process of running the Git wrapper (which then has to wait for the spawned git.exe to finish) by calling ...\mingw64\bin\git.exe directly, via its absolute path.

Testing this is in Git's test suite tricky: we set up a "new" MSYS pseudo-root and copy the git.exe file into the appropriate location, then verify that MSYSTEM is set properly, and also that the PATH is modified so that scripts can be found in $HOME/bin, /mingw64/bin/ and /usr/bin/.

This addresses https://github.com/git-for-windows/git/issues/2283

---
 compat/mingw.c        | 65 +++++++++++++++++++++++++++++++++++++++++++
 config.mak.uname      |  8 ++++--
 t/t0060-path-utils.sh | 33 +++++++++++++++++++++-
 3 files changed, 103 insertions(+), 3 deletions(-)

index afdc1ef2db..563e6299bc 100644
@@ -3134,6 +3134,45 @@ int xwcstoutf(char *utf, const wchar_t *wcs, size_t utflen)
 	return -1;
 }
 
+#ifdef ENSURE_MSYSTEM_IS_SET
+#if !defined(RUNTIME_PREFIX) || !defined(HAVE_WPGMPTR) || !defined(MINGW_PREFIX)
+static size_t append_system_bin_dirs(char *path UNUSED, size_t size UNUSED)
+{
+	return 0;
+}
+#else
+static size_t append_system_bin_dirs(char *path, size_t size)
+{
+	char prefix[32768];
+	const char *slash;
+	size_t len = xwcstoutf(prefix, _wpgmptr, sizeof(prefix)), off = 0;
+
+	if (len == 0 || len >= sizeof(prefix) ||
+	    !(slash = find_last_dir_sep(prefix)))
+		return 0;
+	/* strip trailing `git.exe` */
+	len = slash - prefix;
+
+	/* strip trailing `cmd` or `<mingw-prefix>\bin` or `bin` or `libexec\git-core` */
+	if (strip_suffix_mem(prefix, &len, "\\" MINGW_PREFIX "\\libexec\\git-core") ||
+	    strip_suffix_mem(prefix, &len, "\\" MINGW_PREFIX "\\bin"))
+		off += xsnprintf(path + off, size - off,
+				 "%.*s\\" MINGW_PREFIX "\\bin;", (int)len, prefix);
+	else if (strip_suffix_mem(prefix, &len, "\\cmd") ||
+		 strip_suffix_mem(prefix, &len, "\\bin") ||
+		 strip_suffix_mem(prefix, &len, "\\libexec\\git-core"))
+		off += xsnprintf(path + off, size - off,
+				 "%.*s\\" MINGW_PREFIX "\\bin;", (int)len, prefix);
+	else
+		return 0;
+
+	off += xsnprintf(path + off, size - off,
+			 "%.*s\\usr\\bin;", (int)len, prefix);
+	return off;
+}
+#endif
+#endif
+
 static void setup_windows_environment(void)
 {
 	char *tmp = getenv("TMPDIR");

@@ -3186,6 +3225,32 @@ static void setup_windows_environment(void)
 			setenv("HOME", tmp, 1);
 	}
 
+	if (!getenv("PLINK_PROTOCOL"))
+		setenv("PLINK_PROTOCOL", "ssh", 0);
+
+#ifdef ENSURE_MSYSTEM_IS_SET
+	if (!(tmp = getenv("MSYSTEM")) || !tmp[0]) {
+		const char *home = getenv("HOME"), *path = getenv("PATH");
+		char buf[32768];
+		size_t off = 0;
+
+		setenv("MSYSTEM", ENSURE_MSYSTEM_IS_SET, 1);
+
+		if (home)
+			off += xsnprintf(buf + off, sizeof(buf) - off,
+					 "%s\\bin;", home);
+		off += append_system_bin_dirs(buf + off, sizeof(buf) - off);
+		if (path)
+			off += xsnprintf(buf + off, sizeof(buf) - off,
+					 "%s", path);
+		else if (off > 0)
+			buf[off - 1] = '\0';
+		else
+			buf[0] = '\0';
+		setenv("PATH", buf, 1);
+	}
+#endif
+
 	if (!getenv("LC_ALL") && !getenv("LC_CTYPE") && !getenv("LANG"))
 		setenv("LC_CTYPE", "C.UTF-8", 1);
 }

index 2f7d445eb3..0b63be10b7 100644
@@ -535,7 +535,9 @@ endif
 		compat/win32/pthread.o compat/win32/syslog.o \
 		compat/win32/trace2_win32_process_info.o \
 		compat/win32/dirent.o
-	COMPAT_CFLAGS = -D__USE_MINGW_ACCESS -DDETECT_MSYS_TTY -DNOGDI -DHAVE_STRING_H -Icompat -Icompat/regex -Icompat/win32 -DSTRIP_EXTENSION=\".exe\"
+	COMPAT_CFLAGS = -D__USE_MINGW_ACCESS -DDETECT_MSYS_TTY \
+		-DENSURE_MSYSTEM_IS_SET="\"$(MSYSTEM)\"" -DMINGW_PREFIX="\"$(patsubst /%,%,$(MINGW_PREFIX))\"" \
+		-DNOGDI -DHAVE_STRING_H -Icompat -Icompat/regex -Icompat/win32 -DSTRIP_EXTENSION=\".exe\"
 	BASIC_LDFLAGS = -IGNORE:4217 -IGNORE:4049 -NOLOGO -ENTRY:wmainCRTStartup -SUBSYSTEM:CONSOLE
 	# invalidcontinue.obj allows Git's source code to close the same file
 	# handle twice, or to access the osfhandle of an already-closed stdout

@@ -758,7 +760,9 @@ ifeq ($(uname_S),MINGW)
 		prefix = $(MINGW_PREFIX)
 		HOST_CPU = $(patsubst %-w64-mingw32,%,$(MINGW_CHOST))
 		BASIC_LDFLAGS += -Wl,--pic-executable
-		COMPAT_CFLAGS += -DDETECT_MSYS_TTY
+		COMPAT_CFLAGS += -DDETECT_MSYS_TTY \
+			-DENSURE_MSYSTEM_IS_SET="\"$(MSYSTEM)\"" \
+			-DMINGW_PREFIX="\"$(patsubst /%,%,$(MINGW_PREFIX))\""
                 ifeq (MINGW32,$(MSYSTEM))
 			BASIC_LDFLAGS += -Wl,--large-address-aware
                 endif

index 8545cdfab5..56faf5fe73 100755
@@ -602,7 +602,8 @@ test_expect_success !VALGRIND,RUNTIME_PREFIX,CAN_EXEC_IN_PWD 'RUNTIME_PREFIX wor
 	echo "echo HERE" | write_script pretend/libexec/git-core/git-here &&
 	GIT_EXEC_PATH= ./pretend/bin/git here >actual &&
 	echo HERE >expect &&
-	test_cmp expect actual'
+	test_cmp expect actual
+'
 
 test_expect_success !VALGRIND,RUNTIME_PREFIX,CAN_EXEC_IN_PWD '%(prefix)/ works' '
 	git config yes.path "%(prefix)/yes" &&

@@ -611,4 +612,34 @@ test_expect_success !VALGRIND,RUNTIME_PREFIX,CAN_EXEC_IN_PWD '%(prefix)/ works'
 	test_cmp expect actual
 '
 
+test_expect_success MINGW,RUNTIME_PREFIX 'MSYSTEM/PATH is adjusted if necessary' '
+	if test -z "$MINGW_PREFIX"
+	then
+		MINGW_PREFIX="/$(echo "${MSYSTEM:-MINGW64}" | tr A-Z a-z)"
+	fi &&
+	mkdir -p "$HOME"/bin pretend"$MINGW_PREFIX"/bin \
+		pretend"$MINGW_PREFIX"/libexec/git-core pretend/usr/bin &&
+	cp "$GIT_EXEC_PATH"/git.exe pretend"$MINGW_PREFIX"/bin/ &&
+	cp "$GIT_EXEC_PATH"/git.exe pretend"$MINGW_PREFIX"/libexec/git-core/ &&
+	# copy the .dll files, if any (happens when building via CMake)
+	if test -n "$(ls "$GIT_EXEC_PATH"/*.dll 2>/dev/null)"
+	then
+		cp "$GIT_EXEC_PATH"/*.dll pretend"$MINGW_PREFIX"/bin/ &&
+		cp "$GIT_EXEC_PATH"/*.dll pretend"$MINGW_PREFIX"/libexec/git-core/
+	fi &&
+	echo "env | grep MSYSTEM=" | write_script "$HOME"/bin/git-test-home &&
+	echo "echo ${MINGW_PREFIX#/}" | write_script pretend"$MINGW_PREFIX"/bin/git-test-bin &&
+	echo "echo usr" | write_script pretend/usr/bin/git-test-bin2 &&
+
+	(
+		MSYSTEM= &&
+		GIT_EXEC_PATH= &&
+		pretend"$MINGW_PREFIX"/libexec/git-core/git.exe test-home >actual &&
+		pretend"$MINGW_PREFIX"/libexec/git-core/git.exe test-bin >>actual &&
+		pretend"$MINGW_PREFIX"/bin/git.exe test-bin2 >>actual
+	) &&
+	test_write_lines MSYSTEM=$MSYSTEM "${MINGW_PREFIX#/}" usr >expect &&
+	test_cmp expect actual
+'
+
 test_done

gitgitgadget

Am 12.08.26 um 09:52 schrieb Johannes Schindelin via GitGitGadget:

I have been building my own Windows version with a subset of these patches since November 25 already, specifically with 02-06, 08, and 09/12.

After building with all of these patches instead of just the subset, I don't observe any negative effects after a quick check. CI is happy as well (https://github.com/j6t/git/actions/runs/31831084178).

-- Hannes

When rebasing a Git for Windows commit to enable Rust in Windows CI, I noticed just how many of the preceding commits touching adjacent code haven't been upstreamed yet. This patch series rectifies that.

Changes since v2:

* Moved some hunks that belonged into 12/12 out of 08/12.
* Dropped a fly-by style cleanup from t0060.

Changes since v1:

* Fixed the meson-side handling of the MINGW_PREFIX constant (this was not noticed in the Git for Windows project because it does not use Meson to build the project).

Johannes Schindelin (12):
mingw: include the Python parts in the build
mingw: stop hard-coding CC = gcc
mingw: drop the -D_USE_32BIT_TIME_T option
mingw: only use -Wl,--large-address-aware for 32-bit builds
mingw: avoid over-specifying --pic-executable
mingw: set the prefix and HOST_CPU as per MSYS2's settings
mingw: only enable the MSYS2-specific stuff when compiling in MSYS2
mingw: rely on MSYS2's metadata instead of hard-coding it
windows: skip linking git-<command> for built-ins
mingw: always define ETC_* for MSYS2 environments
mingw: ensure valid CTYPE
mingw: allow git.exe to be used instead of the "Git wrapper"

 compat/mingw.c                      | 68 +++++++++++++++++++++++++++++
 config.mak.uname                    | 61 ++++++++++++--------------
 contrib/buildsystems/CMakeLists.txt |  9 +++-
 meson.build                         | 15 ++++++-
 meson_options.txt                   |  4 ++
 t/t0060-path-utils.sh               | 30 +++++++++++++
 6 files changed, 151 insertions(+), 36 deletions(-)

base-commit: 5b2471720c93ee30e5764a19f3d3b3ae9ec9712a
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2195%2Fdscho%2Fupstream-some-more-git-for-windows-patches-v3
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2195/dscho/upstream-some-more-git-for-windows-patches-v3
Pull-Request: https://github.com/gitgitgadget/git/pull/2195

Range-diff vs v2:

  1:  fd594ffa6c =  1:  fd594ffa6c mingw: include the Python parts in the build
  2:  47fc974582 =  2:  47fc974582 mingw: stop hard-coding `CC = gcc`
  3:  53f1fd5763 =  3:  53f1fd5763 mingw: drop the -D_USE_32BIT_TIME_T option
  4:  ad96aedda6 =  4:  ad96aedda6 mingw: only use -Wl,--large-address-aware for 32-bit builds
  5:  d67201f9b3 =  5:  d67201f9b3 mingw: avoid over-specifying `--pic-executable`
  6:  b49c42c50d =  6:  b49c42c50d mingw: set the prefix and HOST_CPU as per MSYS2's settings
  7:  b00d242621 =  7:  b00d242621 mingw: only enable the MSYS2-specific stuff when compiling in MSYS2
  8:  9de4ea7fc1 !  8:  4cc14553dc mingw: rely on MSYS2's metadata instead of hard-coding it
     @@ config.mak.uname: ifeq ($(uname_S),MINGW)
       		prefix = $(MINGW_PREFIX)
       		HOST_CPU = $(patsubst %-w64-mingw32,%,$(MINGW_CHOST))
       		BASIC_LDFLAGS += -Wl,--pic-executable
     -
     - ## contrib/buildsystems/CMakeLists.txt ##
     -@@ contrib/buildsystems/CMakeLists.txt: if(CMAKE_SYSTEM_NAME STREQUAL "Windows")
     - 				_CONSOLE DETECT_MSYS_TTY STRIP_EXTENSION=".exe"  NO_SYMLINK_HEAD UNRELIABLE_FSTAT
     - 				NOGDI OBJECT_CREATION_MODE=1 __USE_MINGW_ANSI_STDIO=0
     - 				OVERRIDE_STRDUP MMAP_PREVENTS_DELETE USE_WIN32_MMAP
     --				HAVE_WPGMPTR ENSURE_MSYSTEM_IS_SET HAVE_RTLGENRANDOM)
     -+				HAVE_WPGMPTR HAVE_RTLGENRANDOM)
     -+	if(CMAKE_GENERATOR_PLATFORM STREQUAL "x64")
     -+		add_compile_definitions(ENSURE_MSYSTEM_IS_SET="MINGW64" MINGW_PREFIX="mingw64")
     -+	elseif(CMAKE_GENERATOR_PLATFORM STREQUAL "arm64")
     -+		add_compile_definitions(ENSURE_MSYSTEM_IS_SET="CLANGARM64" MINGW_PREFIX="clangarm64")
     -+	elseif(CMAKE_GENERATOR_PLATFORM STREQUAL "x86")
     -+		add_compile_definitions(ENSURE_MSYSTEM_IS_SET="MINGW32" MINGW_PREFIX="mingw32")
     -+	endif()
     - 	list(APPEND compat_SOURCES
     - 		compat/mingw.c
     - 		compat/winansi.c
     -
     - ## meson.build ##
     -@@ meson.build: elif host_machine.system() == 'windows'
     - 
     -   libgit_c_args += [
     -     '-DDETECT_MSYS_TTY',
     --    '-DENSURE_MSYSTEM_IS_SET',
     -     '-DNATIVE_CRLF',
     -     '-DNOGDI',
     -     '-DNO_POSIX_GOODIES',
     -@@ meson.build: elif host_machine.system() == 'windows'
     -     '-D__USE_MINGW_ANSI_STDIO=0',
     -   ]
     - 
     -+  msystem = get_option('msystem')
     -+  if msystem != ''
     -+    mingw_prefix = get_option('mingw_prefix')
     -+    if mingw_prefix == ''
     -+      mingw_prefix = msystem.to_lower()
     -+    elif mingw_prefix.startswith('/')
     -+      mingw_prefix = mingw_prefix.substring(1)
     -+    endif
     -+    libgit_c_args += [
     -+      '-DENSURE_MSYSTEM_IS_SET="' + msystem + '"',
     -+      '-DMINGW_PREFIX="' + mingw_prefix + '"'
     -+    ]
     -+  endif
     -+
     -   libgit_dependencies += compiler.find_library('ntdll')
     -   libgit_include_directories += 'compat/win32'
     -   if compiler.get_id() == 'msvc'
     -
     - ## meson_options.txt ##
     -@@ meson_options.txt: option('runtime_prefix', type: 'boolean', value: false,
     -   description: 'Resolve ancillary tooling and support files relative to the location of the runtime binary instead of hard-coding them into the binary.')
     - option('sane_tool_path', type: 'array', value: [],
     -   description: 'An array of paths to pick up tools from in case the normal tools are broken or lacking.')
     -+option('msystem', type: 'string', value: '',
     -+  description: 'Fall-back on Windows when MSYSTEM is not set.')
     -+option('mingw_prefix', type: 'string', value: '',
     -+  description: 'Fall-back on Windows when MINGW_PREFIX is not set.')
     - 
     - # Build information compiled into Git and other parts like documentation.
     - option('build_date', type: 'string', value: '',
  9:  60a2999b75 =  9:  19c600cbee windows: skip linking `git-<command>` for built-ins
 10:  1dd6ac9295 = 10:  b8cc067e92 mingw: always define `ETC_*` for MSYS2 environments
 11:  d44d05c897 = 11:  09c90015fb mingw: ensure valid CTYPE
 12:  679d1fdd41 ! 12:  4a4f24a1ba mingw: allow `git.exe` to be used instead of the "Git wrapper"

@@ config.mak.uname: ifeq ($(uname_S),MINGW)
BASIC_LDFLAGS += -Wl,--large-address-aware
endif

- ## t/t0060-path-utils.sh ##
-@@ t/t0060-path-utils.sh: test_expect_success !VALGRIND,RUNTIME_PREFIX,CAN_EXEC_IN_PWD 'RUNTIME_PREFIX wor
- echo "echo HERE" | write_script pretend/libexec/git-core/git-here &&
- GIT_EXEC_PATH= ./pretend/bin/git here >actual &&
- echo HERE >expect &&
-- test_cmp expect actual'
-+ test_cmp expect actual
-+'
+ ## contrib/buildsystems/CMakeLists.txt ##
+@@ contrib/buildsystems/CMakeLists.txt: if(CMAKE_SYSTEM_NAME STREQUAL "Windows")
+ _CONSOLE DETECT_MSYS_TTY STRIP_EXTENSION=".exe" NO_SYMLINK_HEAD UNRELIABLE_FSTAT
+ NOGDI OBJECT_CREATION_MODE=1 __USE_MINGW_ANSI_STDIO=0
+ OVERRIDE_STRDUP MMAP_PREVENTS_DELETE USE_WIN32_MMAP
+- HAVE_WPGMPTR ENSURE_MSYSTEM_IS_SET HAVE_RTLGENRANDOM)
++ HAVE_WPGMPTR HAVE_RTLGENRANDOM)
++ if(CMAKE_GENERATOR_PLATFORM STREQUAL "x64")
++ add_compile_definitions(ENSURE_MSYSTEM_IS_SET="MINGW64" MINGW_PREFIX="mingw64")
++ elseif(CMAKE_GENERATOR_PLATFORM STREQUAL "arm64")
++ add_compile_definitions(ENSURE_MSYSTEM_IS_SET="CLANGARM64" MINGW_PREFIX="clangarm64")
++ elseif(CMAKE_GENERATOR_PLATFORM STREQUAL "x86")
++ add_compile_definitions(ENSURE_MSYSTEM_IS_SET="MINGW32" MINGW_PREFIX="mingw32")
++ endif()
+ list(APPEND compat_SOURCES
+ compat/mingw.c
+ compat/winansi.c
+
+ ## meson.build ##
+@@ meson.build: elif host_machine.system() == 'windows'
+
+ libgit_c_args += [
+ '-DDETECT_MSYS_TTY',
+- '-DENSURE_MSYSTEM_IS_SET',
+ '-DNATIVE_CRLF',
+ '-DNOGDI',
+ '-DNO_POSIX_GOODIES',
+@@ meson.build: elif host_machine.system() == 'windows'
+ '-D__USE_MINGW_ANSI_STDIO=0',
+ ]

- test_expect_success !VALGRIND,RUNTIME_PREFIX,CAN_EXEC_IN_PWD '%(prefix)/ works' '
- git config yes.path "%(prefix)/yes" &&
++ msystem = get_option('msystem')
++ if msystem != ''
++ mingw_prefix = get_option('mingw_prefix')
++ if mingw_prefix == ''
++ mingw_prefix = msystem.to_lower()
++ elif mingw_prefix.startswith('/')
++ mingw_prefix = mingw_prefix.substring(1)
++ endif
++ libgit_c_args += [
++ '-DENSURE_MSYSTEM_IS_SET="' + msystem + '"',
++ '-DMINGW_PREFIX="' + mingw_prefix + '"'
++ ]
++ endif
++
+ libgit_dependencies += compiler.find_library('ntdll')
+ libgit_include_directories += 'compat/win32'
+ if compiler.get_id() == 'msvc'
+
+ ## meson_options.txt ##
+@@ meson_options.txt: option('runtime_prefix', type: 'boolean', value: false,
+ description: 'Resolve ancillary tooling and support files relative to the location of the runtime binary instead of hard-coding them into the binary.')
+ option('sane_tool_path', type: 'array', value: [],
+ description: 'An array of paths to pick up tools from in case the normal tools are broken or lacking.')
++option('msystem', type: 'string', value: '',
++ description: 'Fall-back on Windows when MSYSTEM is not set.') ++option('mingw_prefix', type: 'string', value: '',
++ description: 'Fall-back on Windows when MINGW_PREFIX is not set.')
+
+ # Build information compiled into Git and other parts like documentation.
+ option('build_date', type: 'string', value: '',
+
+ ## t/t0060-path-utils.sh ##
@@ t/t0060-path-utils.sh: test_expect_success !VALGRIND,RUNTIME_PREFIX,CAN_EXEC_IN_PWD '%(prefix)/ works'
test_cmp expect actual
'

gitgitgadget

From: Johannes Schindelin <[email protected]>

While Git for Windows does not _ship_ Python (in order to save on bandwidth), MSYS2 provides very fine Python interpreters that users can easily take advantage of, by using Git for Windows within its SDK.

---
 config.mak.uname | 1 +
 1 file changed, 1 insertion(+)

index 9ebd240378..8dd8acfaa5 100644
@@ -781,6 +781,7 @@ ifeq ($(uname_S),MINGW)
 	HAVE_LIBCHARSET_H = YesPlease
 	USE_GETTEXT_SCHEME = fallthrough
 	USE_LIBPCRE = YesPlease
+	NO_PYTHON =
         ifeq (/mingw64,$(subst 32,64,$(subst clangarm,mingw,$(prefix))))
 		# Move system config into top-level /etc/
 		ETC_GITCONFIG = ../etc/gitconfig

gitgitgadget

From: Johannes Schindelin <[email protected]>

This is no longer true in general, not with supporting Clang out of the box.

---
 config.mak.uname | 1 -
 1 file changed, 1 deletion(-)

index 8dd8acfaa5..a464b2009c 100644
@@ -771,7 +771,6 @@ ifeq ($(uname_S),MINGW)
 		COMPAT_CFLAGS += -D_USE_32BIT_TIME_T
 		BASIC_LDFLAGS += -Wl,--large-address-aware
         endif
-	CC = gcc
 	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \
 		-fstack-protector-strong
 	EXTLIBS += -lntdll

gitgitgadget

From: Johannes Schindelin <[email protected]>

This option was added in fa93bb20d72 (MinGW: Fix stat definitions to work with MinGW runtime version 4.0, 2013-09-11), i.e. a _long_ time ago. So long, in fact, that it still targeted MinGW. But we switched to mingw-w64 in 2015, which seems not to share the problem, and therefore does not require a fix.

Even worse: This flag is incompatible with UCRT64, which we are about to support by way of upstreaming mingw-w64-git to the MSYS2 project, see
https://github.com/msys2/MINGW-packages/pull/26470 for details.

So let's send that option into its well-deserved retirement.

---
 config.mak.uname | 1 -
 1 file changed, 1 deletion(-)

index a464b2009c..cc0ed63af3 100644
@@ -768,7 +768,6 @@ ifeq ($(uname_S),MINGW)
 		HOST_CPU = aarch64
 		BASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup
         else
-		COMPAT_CFLAGS += -D_USE_32BIT_TIME_T
 		BASIC_LDFLAGS += -Wl,--large-address-aware
         endif
 	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \

gitgitgadget

From: Johannes Schindelin <[email protected]>

That option only matters there, and is in fact only really understood in those builds; UCRT64 versions of GCC, for example, do not know what to do with that option.

---
 config.mak.uname | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

index cc0ed63af3..b667c693ec 100644
@@ -757,9 +757,8 @@ ifeq ($(uname_S),MINGW)
         ifeq (MINGW32,$(MSYSTEM))
 		prefix = /mingw32
 		HOST_CPU = i686
-		BASIC_LDFLAGS += -Wl,--pic-executable,-e,_mainCRTStartup
-        endif
-        ifeq (MINGW64,$(MSYSTEM))
+		BASIC_LDFLAGS += -Wl,--pic-executable,-e,_mainCRTStartup -Wl,--large-address-aware
+        else ifeq (MINGW64,$(MSYSTEM))
 		prefix = /mingw64
 		HOST_CPU = x86_64
 		BASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup

@@ -768,7 +767,6 @@ ifeq ($(uname_S),MINGW)
 		HOST_CPU = aarch64
 		BASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup
         else
-		BASIC_LDFLAGS += -Wl,--large-address-aware
         endif
 	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \
 		-fstack-protector-strong

gitgitgadget

From: Johannes Schindelin <[email protected]>

In bf2d5d8239e (Don't let ld strip relocations, 2016-01-16) (picked from
https://github.com/git-for-windows/git/pull/612/commits/6a237925bf10),
Git for Windows introduced the -Wl,-pic-executable flag, specifying the exact entry point via -e. This required discerning between i686 and x86_64 code because the former required the symbol to be prefixed with an underscore, the latter did not.

As per https://sourceware.org/bugzilla/show_bug.cgi?id=10865, the specified symbols are already the default, though.

So let's drop the overly-specific definition.

---
 config.mak.uname | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

index b667c693ec..f6387f4c7b 100644
@@ -757,15 +757,15 @@ ifeq ($(uname_S),MINGW)
         ifeq (MINGW32,$(MSYSTEM))
 		prefix = /mingw32
 		HOST_CPU = i686
-		BASIC_LDFLAGS += -Wl,--pic-executable,-e,_mainCRTStartup -Wl,--large-address-aware
+		BASIC_LDFLAGS += -Wl,--pic-executable -Wl,--large-address-aware
         else ifeq (MINGW64,$(MSYSTEM))
 		prefix = /mingw64
 		HOST_CPU = x86_64
-		BASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup
+		BASIC_LDFLAGS += -Wl,--pic-executable
         else ifeq (CLANGARM64,$(MSYSTEM))
 		prefix = /clangarm64
 		HOST_CPU = aarch64
-		BASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup
+		BASIC_LDFLAGS += -Wl,--pic-executable
         else
         endif
 	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \

gitgitgadget

From: Johannes Schindelin <[email protected]>

MSYS2 already defines a couple of helpful environment variables, and we can use those to infer the installation location as well as the CPU. No need for hard-coding ;-)

---
 config.mak.uname | 18 ++++++------------
 1 file changed, 6 insertions(+), 12 deletions(-)

index f6387f4c7b..8363239513 100644
@@ -754,19 +754,13 @@ ifeq ($(uname_S),MINGW)
         ifneq (,$(findstring -O,$(filter-out -O0 -Og,$(CFLAGS))))
 		BASIC_LDFLAGS += -Wl,--dynamicbase
         endif
-        ifeq (MINGW32,$(MSYSTEM))
-		prefix = /mingw32
-		HOST_CPU = i686
-		BASIC_LDFLAGS += -Wl,--pic-executable -Wl,--large-address-aware
-        else ifeq (MINGW64,$(MSYSTEM))
-		prefix = /mingw64
-		HOST_CPU = x86_64
-		BASIC_LDFLAGS += -Wl,--pic-executable
-        else ifeq (CLANGARM64,$(MSYSTEM))
-		prefix = /clangarm64
-		HOST_CPU = aarch64
+        ifneq (,$(MSYSTEM))
+		prefix = $(MINGW_PREFIX)
+		HOST_CPU = $(patsubst %-w64-mingw32,%,$(MINGW_CHOST))
 		BASIC_LDFLAGS += -Wl,--pic-executable
-        else
+                ifeq (MINGW32,$(MSYSTEM))
+			BASIC_LDFLAGS += -Wl,--large-address-aware
+                endif
         endif
 	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \
 		-fstack-protector-strong

gitgitgadget

From: Johannes Schindelin <[email protected]>

The tell-tale is the presence of the MSYSTEM value while compiling, of course. In that case, we want to ensure that MSYSTEM is set when running git.exe, and also enable the magic MSYS2 tty detection.

---
 config.mak.uname | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

index 8363239513..21f53e3f7e 100644
@@ -758,12 +758,12 @@ ifeq ($(uname_S),MINGW)
 		prefix = $(MINGW_PREFIX)
 		HOST_CPU = $(patsubst %-w64-mingw32,%,$(MINGW_CHOST))
 		BASIC_LDFLAGS += -Wl,--pic-executable
+		COMPAT_CFLAGS += -DDETECT_MSYS_TTY
                 ifeq (MINGW32,$(MSYSTEM))
 			BASIC_LDFLAGS += -Wl,--large-address-aware
                 endif
         endif
-	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \
-		-fstack-protector-strong
+	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -fstack-protector-strong
 	EXTLIBS += -lntdll
 	EXTRA_PROGRAMS += headless-git$X
 	INSTALL = /bin/install

gitgitgadget

From: Johannes Schindelin <[email protected]>

MSYS2 defines some helpful environment variables, e.g. MSYSTEM. There is code in Git for Windows to ensure that that MSYSTEM variable is set, hard-coding a default.

However, the existing solution jumps through hoops to reconstruct the proper default, and is even incomplete doing so, as we found out when we extended it to support CLANGARM64.

This is absolutely unnecessary because there is already a perfectly valid MSYSTEM value we can use at build time. This is even true when building the MINGW32 variant on a MINGW64 system because makepkg-mingw will override the MSYSTEM value as per the MINGW_ARCH array.

The same is equally true for the /mingw64, /mingw32 and /clangarm64 prefix: those values are already available via the MINGW_PREFIX environment variable, and we just need to pass that setting through.

Only when MINGW_PREFIX is not set (as is the case in Git for Windows' minimal SDK, where only MSYSTEM is guaranteed to be set correctly), we use as fall-back the top-level directory whose name is the down-cased value of the MSYSTEM variable.

Incidentally, this also broadens the support to all the configurations supported by the MSYS2 project, i.e. clang64 & ucrt64, too.

Note: This keeps the same, hard-coded MSYSTEM platform support for CMake as before, but drops it for Meson (because it is unclear how Meson could do this in a more flexible manner).

---
 config.mak.uname | 14 ++++++--------
 1 file changed, 6 insertions(+), 8 deletions(-)

index 21f53e3f7e..3a90995587 100644
@@ -465,14 +465,8 @@ ifeq ($(uname_S),Windows)
 	GIT_VERSION := $(GIT_VERSION).MSVC
 	pathsep = ;
 	# Assume that this is built in Git for Windows' SDK
-        ifeq (MINGW32,$(MSYSTEM))
-		prefix = /mingw32
-        else
-                ifeq (CLANGARM64,$(MSYSTEM))
-			prefix = /clangarm64
-                else
-			prefix = /mingw64
-                endif
+        ifneq (,$(MSYSTEM))
+		prefix = $(MINGW_PREFIX)
         endif
 	# Prepend MSVC 64-bit tool-chain to PATH.
 	#

@@ -755,6 +749,10 @@ ifeq ($(uname_S),MINGW)
 		BASIC_LDFLAGS += -Wl,--dynamicbase
         endif
         ifneq (,$(MSYSTEM))
+                ifeq ($(MINGW_PREFIX),$(filter-out /%,$(MINGW_PREFIX)))
+			# Override if empty or does not start with a slash
+			MINGW_PREFIX := /$(shell echo '$(MSYSTEM)' | tr A-Z a-z)
+                endif
 		prefix = $(MINGW_PREFIX)
 		HOST_CPU = $(patsubst %-w64-mingw32,%,$(MINGW_CHOST))
 		BASIC_LDFLAGS += -Wl,--pic-executable

gitgitgadget

Am 09.09.26 um 21:17 schrieb Johannes Schindelin via GitGitGadget:

I think this paragraph is now stale and should be removed from this commit message. I don't know if it makes sense to insert in 12/12, though.

Except for this, this is looking good now.

-- Hannes

Hi Hannes,

D'oh, of course! It _was_ unclear to me how Meson could do this, but I obviously found a way to support this _somewhat_.

Yes, this should also have a Helped-by: Johannes Sixt <[email protected]>.

Junio, can you squash those changes before merging to next, or do you want me to send another iteration before I go to bed?

Ciao,
Johannes

Those changes meaning removal of the "Note:" paragraph from 08/12 or anything more elaborate?

From: Johannes Schindelin <[email protected]>

It is merely a historical wart that, say, git-commit exists in the libexec/git-core/ directory, a tribute to the original idea to let Git be essentially a bunch of Unix shell scripts revolving around very few "plumbing" (AKA low-level) commands.

Git has evolved a lot from there. These days, most of Git's functionality is contained within the git executable, in the form of "built-in" commands.

To accommodate for scripts that use the "dashed" form of Git commands, even today, Git provides hard-links that make the git executable available as, say, git-commit, just in case that an old script has not been updated to invoke git commit.

Those hard-links do not come cheap: they take about half a minute for every build of Git on Windows, they are mistaken for taking up huge amounts of space by some Windows Explorer versions that do not understand hard-links, and therefore many a "bug" report had to be addressed.

The "dashed form" has been officially deprecated in Git version 1.5.4, which was released on February 2nd, 2008, i.e. a very long time ago. This deprecation was never finalized by skipping these hard-links, but we can start the process now, in Git for Windows.

---
 config.mak.uname | 2 ++
 1 file changed, 2 insertions(+)

index 3a90995587..88bbe1d78f 100644
@@ -518,6 +518,7 @@ ifeq ($(uname_S),Windows)
 	NO_POSIX_GOODIES = UnfortunatelyYes
 	NATIVE_CRLF = YesPlease
 	DEFAULT_HELP_FORMAT = html
+	SKIP_DASHED_BUILT_INS = YabbaDabbaDoo
 ifeq (/mingw64,$(subst 32,64,$(subst clangarm,mingw,$(prefix))))
 	# Move system config into top-level /etc/
 	ETC_GITCONFIG = ../etc/gitconfig

@@ -710,6 +711,7 @@ ifeq ($(uname_S),MINGW)
 	FSMONITOR_DAEMON_BACKEND = win32
 	FSMONITOR_OS_SETTINGS = win32
 
+	SKIP_DASHED_BUILT_INS = YabbaDabbaDoo
 	RUNTIME_PREFIX = YesPlease
 	HAVE_WPGMPTR = YesWeDo
 	NO_ST_BLOCKS_IN_STRUCT_STAT = YesPlease

gitgitgadget

From: Johannes Schindelin <[email protected]>

Special-casing even more configurations simply does not make sense.

---
 config.mak.uname | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

index 88bbe1d78f..2f7d445eb3 100644
@@ -519,7 +519,7 @@ ifeq ($(uname_S),Windows)
 	NATIVE_CRLF = YesPlease
 	DEFAULT_HELP_FORMAT = html
 	SKIP_DASHED_BUILT_INS = YabbaDabbaDoo
-ifeq (/mingw64,$(subst 32,64,$(subst clangarm,mingw,$(prefix))))
+ifneq (,$(MINGW_PREFIX))
 	# Move system config into top-level /etc/
 	ETC_GITCONFIG = ../etc/gitconfig
 	ETC_GITATTRIBUTES = ../etc/gitattributes

@@ -762,6 +762,9 @@ ifeq ($(uname_S),MINGW)
                 ifeq (MINGW32,$(MSYSTEM))
 			BASIC_LDFLAGS += -Wl,--large-address-aware
                 endif
+		# Move system config into top-level /etc/
+		ETC_GITCONFIG = ../etc/gitconfig
+		ETC_GITATTRIBUTES = ../etc/gitattributes
         endif
 	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -fstack-protector-strong
 	EXTLIBS += -lntdll

@@ -772,11 +775,6 @@ ifeq ($(uname_S),MINGW)
 	USE_GETTEXT_SCHEME = fallthrough
 	USE_LIBPCRE = YesPlease
 	NO_PYTHON =
-        ifeq (/mingw64,$(subst 32,64,$(subst clangarm,mingw,$(prefix))))
-		# Move system config into top-level /etc/
-		ETC_GITCONFIG = ../etc/gitconfig
-		ETC_GITATTRIBUTES = ../etc/gitattributes
-        endif
 endif
 ifeq ($(uname_S),QNX)
 	COMPAT_CFLAGS += -DSA_RESTART=0

gitgitgadget

From: Johannes Schindelin <[email protected]>

A change between versions 2.4.1 and 2.6.0 of the MSYS2 runtime modified how Cygwin's runtime (and hence Git for Windows' MSYS2 runtime derivative) handles locales: d16a56306d (Consolidate wctomb/mbtowc calls for POSIX-1.2008, 2016-07-20).

An unintended side-effect is that "cold-calling" into the POSIX emulation will start with a locale based on the current code page, something that Git for Windows is very ill-prepared for, as it expects to be able to pass a command-line containing non-ASCII characters to the shell without having those characters munged.

One symptom of this behavior: when git clone or git fetch shell out to call git-upload-pack with a path that contains non-ASCII characters, the shell tried to interpret the entire command-line (including command-line parameters) as executable path, which obviously must fail.

This fixes https://github.com/git-for-windows/git/issues/1036

---
 compat/mingw.c | 3 +++
 1 file changed, 3 insertions(+)

index e0fbd2c66d..afdc1ef2db 100644
@@ -3185,6 +3185,9 @@ static void setup_windows_environment(void)
 		if (!tmp && (tmp = getenv("USERPROFILE")))
 			setenv("HOME", tmp, 1);
 	}
+
+	if (!getenv("LC_ALL") && !getenv("LC_CTYPE") && !getenv("LANG"))
+		setenv("LC_CTYPE", "C.UTF-8", 1);
 }
 
 int mingw_platform_has_symlinks(void)

gitgitgadget

From: Johannes Schindelin <[email protected]>

Git for Windows wants to add git.exe to the users' PATH, without cluttering the latter with unnecessary executables such as wish.exe. To that end, it invented the concept of its "Git wrapper", i.e. a tiny executable located in C:\Program Files\Git\cmd\git.exe (originally a CMD script) whose sole purpose is to set up a couple of environment variables and then spawn the _actual_ git.exe (which nowadays lives in C:\Program Files\Git\mingw64\bin\git.exe for 64-bit, and the obvious equivalent for 32-bit installations).

Currently, the following environment variables are set unless already initialized:

- MSYSTEM, to make sure that the MSYS2 Bash and the MSYS2 Perl interpreter behave as expected, and

- PLINK_PROTOCOL, to force PuTTY's plink.exe to use the SSH protocol instead of Telnet,

- PATH, to make sure that the bin folder in the user's home directory, as well as the /mingw64/bin and the /usr/bin directories are included. The trick here is that the /mingw64/bin/ and /usr/bin/ directories are relative to the top-level installation directory of Git for Windows (which the included Bash interprets as /, i.e. as the MSYS pseudo root directory).

Using the absence of MSYSTEM as a tell-tale, we can detect in git.exe whether these environment variables have been initialized properly. Therefore we can call C:\Program Files\Git\mingw64\bin\git in-place after this change, without having to call Git through the Git wrapper.

Obviously, above-mentioned directories must be _prepended_ to the PATH variable, otherwise we risk picking up executables from unrelated Git installations. We do that by constructing the new PATH value from scratch, appending $HOME/bin (if HOME is set), then the MSYS2 system directories, and then appending the original PATH.

Side note: this modification of the PATH variable is independent of the modification necessary to reach the executables and scripts in /mingw64/libexec/git-core/, i.e. the GIT_EXEC_PATH. That modification is still performed by Git, elsewhere, long after making the changes described above.

While we _still_ cannot simply hard-link mingw64\bin\git.exe to cmd (because the former depends on a couple of .dll files that are only in mingw64\bin, i.e. calling ...\cmd\git.exe would fail to load due to missing dependencies), at least we can now avoid that extra process of running the Git wrapper (which then has to wait for the spawned git.exe to finish) by calling ...\mingw64\bin\git.exe directly, via its absolute path.

Testing this is in Git's test suite tricky: we set up a "new" MSYS pseudo-root and copy the git.exe file into the appropriate location, then verify that MSYSTEM is set properly, and also that the PATH is modified so that scripts can be found in $HOME/bin, /mingw64/bin/ and /usr/bin/.

This addresses https://github.com/git-for-windows/git/issues/2283

---
 compat/mingw.c                      | 65 +++++++++++++++++++++++++++++
 config.mak.uname                    |  8 +++-
 contrib/buildsystems/CMakeLists.txt |  9 +++-
 meson.build                         | 15 ++++++-
 meson_options.txt                   |  4 ++
 t/t0060-path-utils.sh               | 30 +++++++++++++
 6 files changed, 127 insertions(+), 4 deletions(-)

index afdc1ef2db..563e6299bc 100644
@@ -3134,6 +3134,45 @@ int xwcstoutf(char *utf, const wchar_t *wcs, size_t utflen)
 	return -1;
 }
 
+#ifdef ENSURE_MSYSTEM_IS_SET
+#if !defined(RUNTIME_PREFIX) || !defined(HAVE_WPGMPTR) || !defined(MINGW_PREFIX)
+static size_t append_system_bin_dirs(char *path UNUSED, size_t size UNUSED)
+{
+	return 0;
+}
+#else
+static size_t append_system_bin_dirs(char *path, size_t size)
+{
+	char prefix[32768];
+	const char *slash;
+	size_t len = xwcstoutf(prefix, _wpgmptr, sizeof(prefix)), off = 0;
+
+	if (len == 0 || len >= sizeof(prefix) ||
+	    !(slash = find_last_dir_sep(prefix)))
+		return 0;
+	/* strip trailing `git.exe` */
+	len = slash - prefix;
+
+	/* strip trailing `cmd` or `<mingw-prefix>\bin` or `bin` or `libexec\git-core` */
+	if (strip_suffix_mem(prefix, &len, "\\" MINGW_PREFIX "\\libexec\\git-core") ||
+	    strip_suffix_mem(prefix, &len, "\\" MINGW_PREFIX "\\bin"))
+		off += xsnprintf(path + off, size - off,
+				 "%.*s\\" MINGW_PREFIX "\\bin;", (int)len, prefix);
+	else if (strip_suffix_mem(prefix, &len, "\\cmd") ||
+		 strip_suffix_mem(prefix, &len, "\\bin") ||
+		 strip_suffix_mem(prefix, &len, "\\libexec\\git-core"))
+		off += xsnprintf(path + off, size - off,
+				 "%.*s\\" MINGW_PREFIX "\\bin;", (int)len, prefix);
+	else
+		return 0;
+
+	off += xsnprintf(path + off, size - off,
+			 "%.*s\\usr\\bin;", (int)len, prefix);
+	return off;
+}
+#endif
+#endif
+
 static void setup_windows_environment(void)
 {
 	char *tmp = getenv("TMPDIR");

@@ -3186,6 +3225,32 @@ static void setup_windows_environment(void)
 			setenv("HOME", tmp, 1);
 	}
 
+	if (!getenv("PLINK_PROTOCOL"))
+		setenv("PLINK_PROTOCOL", "ssh", 0);
+
+#ifdef ENSURE_MSYSTEM_IS_SET
+	if (!(tmp = getenv("MSYSTEM")) || !tmp[0]) {
+		const char *home = getenv("HOME"), *path = getenv("PATH");
+		char buf[32768];
+		size_t off = 0;
+
+		setenv("MSYSTEM", ENSURE_MSYSTEM_IS_SET, 1);
+
+		if (home)
+			off += xsnprintf(buf + off, sizeof(buf) - off,
+					 "%s\\bin;", home);
+		off += append_system_bin_dirs(buf + off, sizeof(buf) - off);
+		if (path)
+			off += xsnprintf(buf + off, sizeof(buf) - off,
+					 "%s", path);
+		else if (off > 0)
+			buf[off - 1] = '\0';
+		else
+			buf[0] = '\0';
+		setenv("PATH", buf, 1);
+	}
+#endif
+
 	if (!getenv("LC_ALL") && !getenv("LC_CTYPE") && !getenv("LANG"))
 		setenv("LC_CTYPE", "C.UTF-8", 1);
 }

index 2f7d445eb3..0b63be10b7 100644
@@ -535,7 +535,9 @@ endif
 		compat/win32/pthread.o compat/win32/syslog.o \
 		compat/win32/trace2_win32_process_info.o \
 		compat/win32/dirent.o
-	COMPAT_CFLAGS = -D__USE_MINGW_ACCESS -DDETECT_MSYS_TTY -DNOGDI -DHAVE_STRING_H -Icompat -Icompat/regex -Icompat/win32 -DSTRIP_EXTENSION=\".exe\"
+	COMPAT_CFLAGS = -D__USE_MINGW_ACCESS -DDETECT_MSYS_TTY \
+		-DENSURE_MSYSTEM_IS_SET="\"$(MSYSTEM)\"" -DMINGW_PREFIX="\"$(patsubst /%,%,$(MINGW_PREFIX))\"" \
+		-DNOGDI -DHAVE_STRING_H -Icompat -Icompat/regex -Icompat/win32 -DSTRIP_EXTENSION=\".exe\"
 	BASIC_LDFLAGS = -IGNORE:4217 -IGNORE:4049 -NOLOGO -ENTRY:wmainCRTStartup -SUBSYSTEM:CONSOLE
 	# invalidcontinue.obj allows Git's source code to close the same file
 	# handle twice, or to access the osfhandle of an already-closed stdout

@@ -758,7 +760,9 @@ ifeq ($(uname_S),MINGW)
 		prefix = $(MINGW_PREFIX)
 		HOST_CPU = $(patsubst %-w64-mingw32,%,$(MINGW_CHOST))
 		BASIC_LDFLAGS += -Wl,--pic-executable
-		COMPAT_CFLAGS += -DDETECT_MSYS_TTY
+		COMPAT_CFLAGS += -DDETECT_MSYS_TTY \
+			-DENSURE_MSYSTEM_IS_SET="\"$(MSYSTEM)\"" \
+			-DMINGW_PREFIX="\"$(patsubst /%,%,$(MINGW_PREFIX))\""
                 ifeq (MINGW32,$(MSYSTEM))
 			BASIC_LDFLAGS += -Wl,--large-address-aware
                 endif

index a57c4b464f..7285bd9ac2 100644
@@ -256,7 +256,14 @@ if(CMAKE_SYSTEM_NAME STREQUAL "Windows")
 				_CONSOLE DETECT_MSYS_TTY STRIP_EXTENSION=".exe"  NO_SYMLINK_HEAD UNRELIABLE_FSTAT
 				NOGDI OBJECT_CREATION_MODE=1 __USE_MINGW_ANSI_STDIO=0
 				OVERRIDE_STRDUP MMAP_PREVENTS_DELETE USE_WIN32_MMAP
-				HAVE_WPGMPTR ENSURE_MSYSTEM_IS_SET HAVE_RTLGENRANDOM)
+				HAVE_WPGMPTR HAVE_RTLGENRANDOM)
+	if(CMAKE_GENERATOR_PLATFORM STREQUAL "x64")
+		add_compile_definitions(ENSURE_MSYSTEM_IS_SET="MINGW64" MINGW_PREFIX="mingw64")
+	elseif(CMAKE_GENERATOR_PLATFORM STREQUAL "arm64")
+		add_compile_definitions(ENSURE_MSYSTEM_IS_SET="CLANGARM64" MINGW_PREFIX="clangarm64")
+	elseif(CMAKE_GENERATOR_PLATFORM STREQUAL "x86")
+		add_compile_definitions(ENSURE_MSYSTEM_IS_SET="MINGW32" MINGW_PREFIX="mingw32")
+	endif()
 	list(APPEND compat_SOURCES
 		compat/mingw.c
 		compat/winansi.c

index 7073d5844d..a8aba81e29 100644
@@ -1318,7 +1318,6 @@ elif host_machine.system() == 'windows'
 
   libgit_c_args += [
     '-DDETECT_MSYS_TTY',
-    '-DENSURE_MSYSTEM_IS_SET',
     '-DNATIVE_CRLF',
     '-DNOGDI',
     '-DNO_POSIX_GOODIES',

@@ -1328,6 +1327,20 @@ elif host_machine.system() == 'windows'
     '-D__USE_MINGW_ANSI_STDIO=0',
   ]
 
+  msystem = get_option('msystem')
+  if msystem != ''
+    mingw_prefix = get_option('mingw_prefix')
+    if mingw_prefix == ''
+      mingw_prefix = msystem.to_lower()
+    elif mingw_prefix.startswith('/')
+      mingw_prefix = mingw_prefix.substring(1)
+    endif
+    libgit_c_args += [
+      '-DENSURE_MSYSTEM_IS_SET="' + msystem + '"',
+      '-DMINGW_PREFIX="' + mingw_prefix + '"'
+    ]
+  endif
+
   libgit_dependencies += compiler.find_library('ntdll')
   libgit_include_directories += 'compat/win32'
   if compiler.get_id() == 'msvc'

index dc88f130d7..becf4689bf 100644
@@ -21,6 +21,10 @@ option('runtime_prefix', type: 'boolean', value: false,
   description: 'Resolve ancillary tooling and support files relative to the location of the runtime binary instead of hard-coding them into the binary.')
 option('sane_tool_path', type: 'array', value: [],
   description: 'An array of paths to pick up tools from in case the normal tools are broken or lacking.')
+option('msystem', type: 'string', value: '',
+  description: 'Fall-back on Windows when MSYSTEM is not set.')
+option('mingw_prefix', type: 'string', value: '',
+  description: 'Fall-back on Windows when MINGW_PREFIX is not set.')
 
 # Build information compiled into Git and other parts like documentation.
 option('build_date', type: 'string', value: '',

index 8545cdfab5..02906055d3 100755
@@ -611,4 +611,34 @@ test_expect_success !VALGRIND,RUNTIME_PREFIX,CAN_EXEC_IN_PWD '%(prefix)/ works'
 	test_cmp expect actual
 '
 
+test_expect_success MINGW,RUNTIME_PREFIX 'MSYSTEM/PATH is adjusted if necessary' '
+	if test -z "$MINGW_PREFIX"
+	then
+		MINGW_PREFIX="/$(echo "${MSYSTEM:-MINGW64}" | tr A-Z a-z)"
+	fi &&
+	mkdir -p "$HOME"/bin pretend"$MINGW_PREFIX"/bin \
+		pretend"$MINGW_PREFIX"/libexec/git-core pretend/usr/bin &&
+	cp "$GIT_EXEC_PATH"/git.exe pretend"$MINGW_PREFIX"/bin/ &&
+	cp "$GIT_EXEC_PATH"/git.exe pretend"$MINGW_PREFIX"/libexec/git-core/ &&
+	# copy the .dll files, if any (happens when building via CMake)
+	if test -n "$(ls "$GIT_EXEC_PATH"/*.dll 2>/dev/null)"
+	then
+		cp "$GIT_EXEC_PATH"/*.dll pretend"$MINGW_PREFIX"/bin/ &&
+		cp "$GIT_EXEC_PATH"/*.dll pretend"$MINGW_PREFIX"/libexec/git-core/
+	fi &&
+	echo "env | grep MSYSTEM=" | write_script "$HOME"/bin/git-test-home &&
+	echo "echo ${MINGW_PREFIX#/}" | write_script pretend"$MINGW_PREFIX"/bin/git-test-bin &&
+	echo "echo usr" | write_script pretend/usr/bin/git-test-bin2 &&
+
+	(
+		MSYSTEM= &&
+		GIT_EXEC_PATH= &&
+		pretend"$MINGW_PREFIX"/libexec/git-core/git.exe test-home >actual &&
+		pretend"$MINGW_PREFIX"/libexec/git-core/git.exe test-bin >>actual &&
+		pretend"$MINGW_PREFIX"/bin/git.exe test-bin2 >>actual
+	) &&
+	test_write_lines MSYSTEM=$MSYSTEM "${MINGW_PREFIX#/}" usr >expect &&
+	test_cmp expect actual
+'
+
 test_done

gitgitgadget

"Johannes Schindelin via GitGitGadget" <[email protected]>

index 56faf5fe73..02906055d3 100755
@@ -602,8 +602,7 @@ test_expect_success !VALGRIND,RUNTIME_PREFIX,CAN_EXEC_IN_PWD 'RUNTIME_PREFIX wor
 	echo "echo HERE" | write_script pretend/libexec/git-core/git-here &&
 	GIT_EXEC_PATH= ./pretend/bin/git here >actual &&
 	echo HERE >expect &&
-	test_cmp expect actual
-'
+	test_cmp expect actual'
 
 test_expect_success !VALGRIND,RUNTIME_PREFIX,CAN_EXEC_IN_PWD '%(prefix)/ works' '
 	git config yes.path "%(prefix)/yes" &&

With the imminent switch of Git for Windows from the old MINGW64 environment (that was deprecated by MSYS2) to the UCRT64 environment, this patch series became quite urgent; I will have to work around this patch series not yet being in master via setup-git-for-windows-sdk to be able to switch without breaking Git's CI builds.

Originally, I noticed that these patches had not been upstreamed yet when rebasing a Git for Windows commit to enable Rust in Windows CI.

Changes since v3:

* Reinstated the fly-by style fix as a separate commit.
* Added Helped-by trailers
* Moved and reworded a paragraph from the commit message of 08/12 to 12/12; This should have been done as part of v2.

Changes since v2:

* Moved some hunks that belonged into 12/12 out of 08/12.
* Dropped a fly-by style cleanup from t0060.

Changes since v1:

* Fixed the meson-side handling of the MINGW_PREFIX constant (this was not noticed in the Git for Windows project because it does not use Meson to build the project).

Johannes Schindelin (13):
mingw: include the Python parts in the build
mingw: stop hard-coding CC = gcc
mingw: drop the -D_USE_32BIT_TIME_T option
mingw: only use -Wl,--large-address-aware for 32-bit builds
mingw: avoid over-specifying --pic-executable
mingw: set the prefix and HOST_CPU as per MSYS2's settings
mingw: only enable the MSYS2-specific stuff when compiling in MSYS2
mingw: rely on MSYS2's metadata instead of hard-coding it
windows: skip linking git-<command> for built-ins
mingw: always define ETC_* for MSYS2 environments
mingw: ensure valid CTYPE
mingw: allow git.exe to be used instead of the "Git wrapper"
t0060: adjust the code style

 compat/mingw.c                      | 68 +++++++++++++++++++++++++++++
 config.mak.uname                    | 61 ++++++++++++--------------
 contrib/buildsystems/CMakeLists.txt |  9 +++-
 meson.build                         | 15 ++++++-
 meson_options.txt                   |  4 ++
 t/t0060-path-utils.sh               | 33 +++++++++++++-
 6 files changed, 153 insertions(+), 37 deletions(-)

base-commit: 5b2471720c93ee30e5764a19f3d3b3ae9ec9712a
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2195%2Fdscho%2Fupstream-some-more-git-for-windows-patches-v4
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2195/dscho/upstream-some-more-git-for-windows-patches-v4
Pull-Request: https://github.com/gitgitgadget/git/pull/2195

Range-diff vs v3:

1: fd594ffa6c ! 1: 0b8ca0e3b9 mingw: include the Python parts in the build
@@ Commit message
bandwidth), MSYS2 provides very fine Python interpreters that users can easily take advantage of, by using Git for Windows within its SDK.

+ Helped-by: Johannes Sixt <[email protected]>

## config.mak.uname ##
2: 47fc974582 ! 2: 9894892ff5 mingw: stop hard-coding CC = gcc
@@ Commit message
This is no longer true in general, not with supporting Clang out of the box.

+ Helped-by: Johannes Sixt <[email protected]>

## config.mak.uname ##
3: 53f1fd5763 ! 3: 340dd18f27 mingw: drop the -D_USE_32BIT_TIME_T option
@@ Commit message

So let's send that option into its well-deserved retirement.

+ Helped-by: Johannes Sixt <[email protected]>

## config.mak.uname ##
4: ad96aedda6 ! 4: 37c6563d74 mingw: only use -Wl,--large-address-aware for 32-bit builds
@@ Commit message
those builds; UCRT64 versions of GCC, for example, do not know what to do with that option.

+ Helped-by: Johannes Sixt <[email protected]>

## config.mak.uname ##
5: d67201f9b3 ! 5: 0063f2d96a mingw: avoid over-specifying --pic-executable
@@ Commit message

So let's drop the overly-specific definition.

+ Helped-by: Johannes Sixt <[email protected]>

## config.mak.uname ##
6: b49c42c50d ! 6: e03279a0df mingw: set the prefix and HOST_CPU as per MSYS2's settings
@@ Commit message
can use those to infer the installation location as well as the CPU. No need for hard-coding ;-)

+ Helped-by: Johannes Sixt <[email protected]>

## config.mak.uname ##
7: b00d242621 ! 7: 29caf78bdd mingw: only enable the MSYS2-specific stuff when compiling in MSYS2
@@ Commit message
course. In that case, we want to ensure that MSYSTEM is set when
running git.exe, and also enable the magic MSYS2 tty detection.

+ Helped-by: Johannes Sixt <[email protected]>

## config.mak.uname ##
8: 4cc14553dc ! 8: a9665a3304 mingw: rely on MSYS2's metadata instead of hard-coding it
@@ Commit message
Incidentally, this also broadens the support to all the configurations supported by the MSYS2 project, i.e. clang64 & ucrt64, too.

- Note: This keeps the same, hard-coded MSYSTEM platform support for CMake
- as before, but drops it for Meson (because it is unclear how Meson could
- do this in a more flexible manner).
-
+ Helped-by: Johannes Sixt <[email protected]>

9: 19c600cbee ! 9: 0d2990aa14 windows: skip linking git-<command> for built-ins
@@ Commit message
This deprecation was never finalized by skipping these hard-links, but we can start the process now, in Git for Windows.

+ Helped-by: Johannes Sixt <[email protected]>

## config.mak.uname ##
10: b8cc067e92 ! 10: b5a78f65b6 mingw: always define ETC_* for MSYS2 environments
@@ Commit message

Special-casing even more configurations simply does not make sense.

+ Helped-by: Johannes Sixt <[email protected]>

## config.mak.uname ##
11: 09c90015fb ! 11: 1bbd0dc6aa mingw: ensure valid CTYPE
@@ Commit message

This fixes https://github.com/git-for-windows/git/issues/1036

+ Helped-by: Johannes Sixt <[email protected]>

## compat/mingw.c ##
12: 4a4f24a1ba ! 12: e8115c158c mingw: allow git.exe to be used instead of the "Git wrapper"
@@ Commit message

This addresses https://github.com/git-for-windows/git/issues/2283

+ Note: This keeps the same, hard-coded MSYSTEM platform support for CMake
+ as before, and introduces an msystem' and mingw-prefix` knob for Meson
+ (read: neither CMake nor Meson will automatically inherit the setting
+ from the current build environment).
+
+ Helped-by: Johannes Sixt <[email protected]>

## compat/mingw.c ##
-: ---------- > 13: 88403ffc9e t0060: adjust the code style

gitgitgadget

From: Johannes Schindelin <[email protected]>

While Git for Windows does not _ship_ Python (in order to save on bandwidth), MSYS2 provides very fine Python interpreters that users can easily take advantage of, by using Git for Windows within its SDK.

---
 config.mak.uname | 1 +
 1 file changed, 1 insertion(+)

index 9ebd240378..8dd8acfaa5 100644
@@ -781,6 +781,7 @@ ifeq ($(uname_S),MINGW)
 	HAVE_LIBCHARSET_H = YesPlease
 	USE_GETTEXT_SCHEME = fallthrough
 	USE_LIBPCRE = YesPlease
+	NO_PYTHON =
         ifeq (/mingw64,$(subst 32,64,$(subst clangarm,mingw,$(prefix))))
 		# Move system config into top-level /etc/
 		ETC_GITCONFIG = ../etc/gitconfig

gitgitgadget

From: Johannes Schindelin <[email protected]>

This is no longer true in general, not with supporting Clang out of the box.

---
 config.mak.uname | 1 -
 1 file changed, 1 deletion(-)

index 8dd8acfaa5..a464b2009c 100644
@@ -771,7 +771,6 @@ ifeq ($(uname_S),MINGW)
 		COMPAT_CFLAGS += -D_USE_32BIT_TIME_T
 		BASIC_LDFLAGS += -Wl,--large-address-aware
         endif
-	CC = gcc
 	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \
 		-fstack-protector-strong
 	EXTLIBS += -lntdll

gitgitgadget

From: Johannes Schindelin <[email protected]>

This option was added in fa93bb20d72 (MinGW: Fix stat definitions to work with MinGW runtime version 4.0, 2013-09-11), i.e. a _long_ time ago. So long, in fact, that it still targeted MinGW. But we switched to mingw-w64 in 2015, which seems not to share the problem, and therefore does not require a fix.

Even worse: This flag is incompatible with UCRT64, which we are about to support by way of upstreaming mingw-w64-git to the MSYS2 project, see
https://github.com/msys2/MINGW-packages/pull/26470 for details.

So let's send that option into its well-deserved retirement.

---
 config.mak.uname | 1 -
 1 file changed, 1 deletion(-)

index a464b2009c..cc0ed63af3 100644
@@ -768,7 +768,6 @@ ifeq ($(uname_S),MINGW)
 		HOST_CPU = aarch64
 		BASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup
         else
-		COMPAT_CFLAGS += -D_USE_32BIT_TIME_T
 		BASIC_LDFLAGS += -Wl,--large-address-aware
         endif
 	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \

gitgitgadget

From: Johannes Schindelin <[email protected]>

That option only matters there, and is in fact only really understood in those builds; UCRT64 versions of GCC, for example, do not know what to do with that option.

---
 config.mak.uname | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

index cc0ed63af3..b667c693ec 100644
@@ -757,9 +757,8 @@ ifeq ($(uname_S),MINGW)
         ifeq (MINGW32,$(MSYSTEM))
 		prefix = /mingw32
 		HOST_CPU = i686
-		BASIC_LDFLAGS += -Wl,--pic-executable,-e,_mainCRTStartup
-        endif
-        ifeq (MINGW64,$(MSYSTEM))
+		BASIC_LDFLAGS += -Wl,--pic-executable,-e,_mainCRTStartup -Wl,--large-address-aware
+        else ifeq (MINGW64,$(MSYSTEM))
 		prefix = /mingw64
 		HOST_CPU = x86_64
 		BASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup

@@ -768,7 +767,6 @@ ifeq ($(uname_S),MINGW)
 		HOST_CPU = aarch64
 		BASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup
         else
-		BASIC_LDFLAGS += -Wl,--large-address-aware
         endif
 	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \
 		-fstack-protector-strong

gitgitgadget

From: Johannes Schindelin <[email protected]>

In bf2d5d8239e (Don't let ld strip relocations, 2016-01-16) (picked from
https://github.com/git-for-windows/git/pull/612/commits/6a237925bf10),
Git for Windows introduced the -Wl,-pic-executable flag, specifying the exact entry point via -e. This required discerning between i686 and x86_64 code because the former required the symbol to be prefixed with an underscore, the latter did not.

As per https://sourceware.org/bugzilla/show_bug.cgi?id=10865, the specified symbols are already the default, though.

So let's drop the overly-specific definition.

---
 config.mak.uname | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

index b667c693ec..f6387f4c7b 100644
@@ -757,15 +757,15 @@ ifeq ($(uname_S),MINGW)
         ifeq (MINGW32,$(MSYSTEM))
 		prefix = /mingw32
 		HOST_CPU = i686
-		BASIC_LDFLAGS += -Wl,--pic-executable,-e,_mainCRTStartup -Wl,--large-address-aware
+		BASIC_LDFLAGS += -Wl,--pic-executable -Wl,--large-address-aware
         else ifeq (MINGW64,$(MSYSTEM))
 		prefix = /mingw64
 		HOST_CPU = x86_64
-		BASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup
+		BASIC_LDFLAGS += -Wl,--pic-executable
         else ifeq (CLANGARM64,$(MSYSTEM))
 		prefix = /clangarm64
 		HOST_CPU = aarch64
-		BASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup
+		BASIC_LDFLAGS += -Wl,--pic-executable
         else
         endif
 	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \

gitgitgadget

From: Johannes Schindelin <[email protected]>

MSYS2 already defines a couple of helpful environment variables, and we can use those to infer the installation location as well as the CPU. No need for hard-coding ;-)

---
 config.mak.uname | 18 ++++++------------
 1 file changed, 6 insertions(+), 12 deletions(-)

index f6387f4c7b..8363239513 100644
@@ -754,19 +754,13 @@ ifeq ($(uname_S),MINGW)
         ifneq (,$(findstring -O,$(filter-out -O0 -Og,$(CFLAGS))))
 		BASIC_LDFLAGS += -Wl,--dynamicbase
         endif
-        ifeq (MINGW32,$(MSYSTEM))
-		prefix = /mingw32
-		HOST_CPU = i686
-		BASIC_LDFLAGS += -Wl,--pic-executable -Wl,--large-address-aware
-        else ifeq (MINGW64,$(MSYSTEM))
-		prefix = /mingw64
-		HOST_CPU = x86_64
-		BASIC_LDFLAGS += -Wl,--pic-executable
-        else ifeq (CLANGARM64,$(MSYSTEM))
-		prefix = /clangarm64
-		HOST_CPU = aarch64
+        ifneq (,$(MSYSTEM))
+		prefix = $(MINGW_PREFIX)
+		HOST_CPU = $(patsubst %-w64-mingw32,%,$(MINGW_CHOST))
 		BASIC_LDFLAGS += -Wl,--pic-executable
-        else
+                ifeq (MINGW32,$(MSYSTEM))
+			BASIC_LDFLAGS += -Wl,--large-address-aware
+                endif
         endif
 	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \
 		-fstack-protector-strong

gitgitgadget

From: Johannes Schindelin <[email protected]>

The tell-tale is the presence of the MSYSTEM value while compiling, of course. In that case, we want to ensure that MSYSTEM is set when running git.exe, and also enable the magic MSYS2 tty detection.

---
 config.mak.uname | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

index 8363239513..21f53e3f7e 100644
@@ -758,12 +758,12 @@ ifeq ($(uname_S),MINGW)
 		prefix = $(MINGW_PREFIX)
 		HOST_CPU = $(patsubst %-w64-mingw32,%,$(MINGW_CHOST))
 		BASIC_LDFLAGS += -Wl,--pic-executable
+		COMPAT_CFLAGS += -DDETECT_MSYS_TTY
                 ifeq (MINGW32,$(MSYSTEM))
 			BASIC_LDFLAGS += -Wl,--large-address-aware
                 endif
         endif
-	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \
-		-fstack-protector-strong
+	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -fstack-protector-strong
 	EXTLIBS += -lntdll
 	EXTRA_PROGRAMS += headless-git$X
 	INSTALL = /bin/install

gitgitgadget

From: Johannes Schindelin <[email protected]>

MSYS2 defines some helpful environment variables, e.g. MSYSTEM. There is code in Git for Windows to ensure that that MSYSTEM variable is set, hard-coding a default.

However, the existing solution jumps through hoops to reconstruct the proper default, and is even incomplete doing so, as we found out when we extended it to support CLANGARM64.

This is absolutely unnecessary because there is already a perfectly valid MSYSTEM value we can use at build time. This is even true when building the MINGW32 variant on a MINGW64 system because makepkg-mingw will override the MSYSTEM value as per the MINGW_ARCH array.

The same is equally true for the /mingw64, /mingw32 and /clangarm64 prefix: those values are already available via the MINGW_PREFIX environment variable, and we just need to pass that setting through.

Only when MINGW_PREFIX is not set (as is the case in Git for Windows' minimal SDK, where only MSYSTEM is guaranteed to be set correctly), we use as fall-back the top-level directory whose name is the down-cased value of the MSYSTEM variable.

Incidentally, this also broadens the support to all the configurations supported by the MSYS2 project, i.e. clang64 & ucrt64, too.

---
 config.mak.uname | 14 ++++++--------
 1 file changed, 6 insertions(+), 8 deletions(-)

index 21f53e3f7e..3a90995587 100644
@@ -465,14 +465,8 @@ ifeq ($(uname_S),Windows)
 	GIT_VERSION := $(GIT_VERSION).MSVC
 	pathsep = ;
 	# Assume that this is built in Git for Windows' SDK
-        ifeq (MINGW32,$(MSYSTEM))
-		prefix = /mingw32
-        else
-                ifeq (CLANGARM64,$(MSYSTEM))
-			prefix = /clangarm64
-                else
-			prefix = /mingw64
-                endif
+        ifneq (,$(MSYSTEM))
+		prefix = $(MINGW_PREFIX)
         endif
 	# Prepend MSVC 64-bit tool-chain to PATH.
 	#

@@ -755,6 +749,10 @@ ifeq ($(uname_S),MINGW)
 		BASIC_LDFLAGS += -Wl,--dynamicbase
         endif
         ifneq (,$(MSYSTEM))
+                ifeq ($(MINGW_PREFIX),$(filter-out /%,$(MINGW_PREFIX)))
+			# Override if empty or does not start with a slash
+			MINGW_PREFIX := /$(shell echo '$(MSYSTEM)' | tr A-Z a-z)
+                endif
 		prefix = $(MINGW_PREFIX)
 		HOST_CPU = $(patsubst %-w64-mingw32,%,$(MINGW_CHOST))
 		BASIC_LDFLAGS += -Wl,--pic-executable

gitgitgadget

From: Johannes Schindelin <[email protected]>

It is merely a historical wart that, say, git-commit exists in the libexec/git-core/ directory, a tribute to the original idea to let Git be essentially a bunch of Unix shell scripts revolving around very few "plumbing" (AKA low-level) commands.

Git has evolved a lot from there. These days, most of Git's functionality is contained within the git executable, in the form of "built-in" commands.

To accommodate for scripts that use the "dashed" form of Git commands, even today, Git provides hard-links that make the git executable available as, say, git-commit, just in case that an old script has not been updated to invoke git commit.

Those hard-links do not come cheap: they take about half a minute for every build of Git on Windows, they are mistaken for taking up huge amounts of space by some Windows Explorer versions that do not understand hard-links, and therefore many a "bug" report had to be addressed.

The "dashed form" has been officially deprecated in Git version 1.5.4, which was released on February 2nd, 2008, i.e. a very long time ago. This deprecation was never finalized by skipping these hard-links, but we can start the process now, in Git for Windows.

---
 config.mak.uname | 2 ++
 1 file changed, 2 insertions(+)

index 3a90995587..88bbe1d78f 100644
@@ -518,6 +518,7 @@ ifeq ($(uname_S),Windows)
 	NO_POSIX_GOODIES = UnfortunatelyYes
 	NATIVE_CRLF = YesPlease
 	DEFAULT_HELP_FORMAT = html
+	SKIP_DASHED_BUILT_INS = YabbaDabbaDoo
 ifeq (/mingw64,$(subst 32,64,$(subst clangarm,mingw,$(prefix))))
 	# Move system config into top-level /etc/
 	ETC_GITCONFIG = ../etc/gitconfig

@@ -710,6 +711,7 @@ ifeq ($(uname_S),MINGW)
 	FSMONITOR_DAEMON_BACKEND = win32
 	FSMONITOR_OS_SETTINGS = win32
 
+	SKIP_DASHED_BUILT_INS = YabbaDabbaDoo
 	RUNTIME_PREFIX = YesPlease
 	HAVE_WPGMPTR = YesWeDo
 	NO_ST_BLOCKS_IN_STRUCT_STAT = YesPlease

gitgitgadget

From: Johannes Schindelin <[email protected]>

Special-casing even more configurations simply does not make sense.

---
 config.mak.uname | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

index 88bbe1d78f..2f7d445eb3 100644
@@ -519,7 +519,7 @@ ifeq ($(uname_S),Windows)
 	NATIVE_CRLF = YesPlease
 	DEFAULT_HELP_FORMAT = html
 	SKIP_DASHED_BUILT_INS = YabbaDabbaDoo
-ifeq (/mingw64,$(subst 32,64,$(subst clangarm,mingw,$(prefix))))
+ifneq (,$(MINGW_PREFIX))
 	# Move system config into top-level /etc/
 	ETC_GITCONFIG = ../etc/gitconfig
 	ETC_GITATTRIBUTES = ../etc/gitattributes

@@ -762,6 +762,9 @@ ifeq ($(uname_S),MINGW)
                 ifeq (MINGW32,$(MSYSTEM))
 			BASIC_LDFLAGS += -Wl,--large-address-aware
                 endif
+		# Move system config into top-level /etc/
+		ETC_GITCONFIG = ../etc/gitconfig
+		ETC_GITATTRIBUTES = ../etc/gitattributes
         endif
 	COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -fstack-protector-strong
 	EXTLIBS += -lntdll

@@ -772,11 +775,6 @@ ifeq ($(uname_S),MINGW)
 	USE_GETTEXT_SCHEME = fallthrough
 	USE_LIBPCRE = YesPlease
 	NO_PYTHON =
-        ifeq (/mingw64,$(subst 32,64,$(subst clangarm,mingw,$(prefix))))
-		# Move system config into top-level /etc/
-		ETC_GITCONFIG = ../etc/gitconfig
-		ETC_GITATTRIBUTES = ../etc/gitattributes
-        endif
 endif
 ifeq ($(uname_S),QNX)
 	COMPAT_CFLAGS += -DSA_RESTART=0

gitgitgadget

From: Johannes Schindelin <[email protected]>

A change between versions 2.4.1 and 2.6.0 of the MSYS2 runtime modified how Cygwin's runtime (and hence Git for Windows' MSYS2 runtime derivative) handles locales: d16a56306d (Consolidate wctomb/mbtowc calls for POSIX-1.2008, 2016-07-20).

An unintended side-effect is that "cold-calling" into the POSIX emulation will start with a locale based on the current code page, something that Git for Windows is very ill-prepared for, as it expects to be able to pass a command-line containing non-ASCII characters to the shell without having those characters munged.

One symptom of this behavior: when git clone or git fetch shell out to call git-upload-pack with a path that contains non-ASCII characters, the shell tried to interpret the entire command-line (including command-line parameters) as executable path, which obviously must fail.

This fixes https://github.com/git-for-windows/git/issues/1036

---
 compat/mingw.c | 3 +++
 1 file changed, 3 insertions(+)

index e0fbd2c66d..afdc1ef2db 100644
@@ -3185,6 +3185,9 @@ static void setup_windows_environment(void)
 		if (!tmp && (tmp = getenv("USERPROFILE")))
 			setenv("HOME", tmp, 1);
 	}
+
+	if (!getenv("LC_ALL") && !getenv("LC_CTYPE") && !getenv("LANG"))
+		setenv("LC_CTYPE", "C.UTF-8", 1);
 }
 
 int mingw_platform_has_symlinks(void)

gitgitgadget

From: Johannes Schindelin <[email protected]>

Git for Windows wants to add git.exe to the users' PATH, without cluttering the latter with unnecessary executables such as wish.exe. To that end, it invented the concept of its "Git wrapper", i.e. a tiny executable located in C:\Program Files\Git\cmd\git.exe (originally a CMD script) whose sole purpose is to set up a couple of environment variables and then spawn the _actual_ git.exe (which nowadays lives in C:\Program Files\Git\mingw64\bin\git.exe for 64-bit, and the obvious equivalent for 32-bit installations).

Currently, the following environment variables are set unless already initialized:

- MSYSTEM, to make sure that the MSYS2 Bash and the MSYS2 Perl interpreter behave as expected, and

- PLINK_PROTOCOL, to force PuTTY's plink.exe to use the SSH protocol instead of Telnet,

- PATH, to make sure that the bin folder in the user's home directory, as well as the /mingw64/bin and the /usr/bin directories are included. The trick here is that the /mingw64/bin/ and /usr/bin/ directories are relative to the top-level installation directory of Git for Windows (which the included Bash interprets as /, i.e. as the MSYS pseudo root directory).

Using the absence of MSYSTEM as a tell-tale, we can detect in git.exe whether these environment variables have been initialized properly. Therefore we can call C:\Program Files\Git\mingw64\bin\git in-place after this change, without having to call Git through the Git wrapper.

Obviously, above-mentioned directories must be _prepended_ to the PATH variable, otherwise we risk picking up executables from unrelated Git installations. We do that by constructing the new PATH value from scratch, appending $HOME/bin (if HOME is set), then the MSYS2 system directories, and then appending the original PATH.

Side note: this modification of the PATH variable is independent of the modification necessary to reach the executables and scripts in /mingw64/libexec/git-core/, i.e. the GIT_EXEC_PATH. That modification is still performed by Git, elsewhere, long after making the changes described above.

While we _still_ cannot simply hard-link mingw64\bin\git.exe to cmd (because the former depends on a couple of .dll files that are only in mingw64\bin, i.e. calling ...\cmd\git.exe would fail to load due to missing dependencies), at least we can now avoid that extra process of running the Git wrapper (which then has to wait for the spawned git.exe to finish) by calling ...\mingw64\bin\git.exe directly, via its absolute path.

Testing this is in Git's test suite tricky: we set up a "new" MSYS pseudo-root and copy the git.exe file into the appropriate location, then verify that MSYSTEM is set properly, and also that the PATH is modified so that scripts can be found in $HOME/bin, /mingw64/bin/ and /usr/bin/.

This addresses https://github.com/git-for-windows/git/issues/2283

Note: This keeps the same, hard-coded MSYSTEM platform support for CMake as before, and introduces an msystem' and mingw-prefix` knob for Meson (read: neither CMake nor Meson will automatically inherit the setting from the current build environment).

---
 compat/mingw.c                      | 65 +++++++++++++++++++++++++++++
 config.mak.uname                    |  8 +++-
 contrib/buildsystems/CMakeLists.txt |  9 +++-
 meson.build                         | 15 ++++++-
 meson_options.txt                   |  4 ++
 t/t0060-path-utils.sh               | 30 +++++++++++++
 6 files changed, 127 insertions(+), 4 deletions(-)

index afdc1ef2db..563e6299bc 100644
@@ -3134,6 +3134,45 @@ int xwcstoutf(char *utf, const wchar_t *wcs, size_t utflen)
 	return -1;
 }
 
+#ifdef ENSURE_MSYSTEM_IS_SET
+#if !defined(RUNTIME_PREFIX) || !defined(HAVE_WPGMPTR) || !defined(MINGW_PREFIX)
+static size_t append_system_bin_dirs(char *path UNUSED, size_t size UNUSED)
+{
+	return 0;
+}
+#else
+static size_t append_system_bin_dirs(char *path, size_t size)
+{
+	char prefix[32768];
+	const char *slash;
+	size_t len = xwcstoutf(prefix, _wpgmptr, sizeof(prefix)), off = 0;
+
+	if (len == 0 || len >= sizeof(prefix) ||
+	    !(slash = find_last_dir_sep(prefix)))
+		return 0;
+	/* strip trailing `git.exe` */
+	len = slash - prefix;
+
+	/* strip trailing `cmd` or `<mingw-prefix>\bin` or `bin` or `libexec\git-core` */
+	if (strip_suffix_mem(prefix, &len, "\\" MINGW_PREFIX "\\libexec\\git-core") ||
+	    strip_suffix_mem(prefix, &len, "\\" MINGW_PREFIX "\\bin"))
+		off += xsnprintf(path + off, size - off,
+				 "%.*s\\" MINGW_PREFIX "\\bin;", (int)len, prefix);
+	else if (strip_suffix_mem(prefix, &len, "\\cmd") ||
+		 strip_suffix_mem(prefix, &len, "\\bin") ||
+		 strip_suffix_mem(prefix, &len, "\\libexec\\git-core"))
+		off += xsnprintf(path + off, size - off,
+				 "%.*s\\" MINGW_PREFIX "\\bin;", (int)len, prefix);
+	else
+		return 0;
+
+	off += xsnprintf(path + off, size - off,
+			 "%.*s\\usr\\bin;", (int)len, prefix);
+	return off;
+}
+#endif
+#endif
+
 static void setup_windows_environment(void)
 {
 	char *tmp = getenv("TMPDIR");

@@ -3186,6 +3225,32 @@ static void setup_windows_environment(void)
 			setenv("HOME", tmp, 1);
 	}
 
+	if (!getenv("PLINK_PROTOCOL"))
+		setenv("PLINK_PROTOCOL", "ssh", 0);
+
+#ifdef ENSURE_MSYSTEM_IS_SET
+	if (!(tmp = getenv("MSYSTEM")) || !tmp[0]) {
+		const char *home = getenv("HOME"), *path = getenv("PATH");
+		char buf[32768];
+		size_t off = 0;
+
+		setenv("MSYSTEM", ENSURE_MSYSTEM_IS_SET, 1);
+
+		if (home)
+			off += xsnprintf(buf + off, sizeof(buf) - off,
+					 "%s\\bin;", home);
+		off += append_system_bin_dirs(buf + off, sizeof(buf) - off);
+		if (path)
+			off += xsnprintf(buf + off, sizeof(buf) - off,
+					 "%s", path);
+		else if (off > 0)
+			buf[off - 1] = '\0';
+		else
+			buf[0] = '\0';
+		setenv("PATH", buf, 1);
+	}
+#endif
+
 	if (!getenv("LC_ALL") && !getenv("LC_CTYPE") && !getenv("LANG"))
 		setenv("LC_CTYPE", "C.UTF-8", 1);
 }

index 2f7d445eb3..0b63be10b7 100644
@@ -535,7 +535,9 @@ endif
 		compat/win32/pthread.o compat/win32/syslog.o \
 		compat/win32/trace2_win32_process_info.o \
 		compat/win32/dirent.o
-	COMPAT_CFLAGS = -D__USE_MINGW_ACCESS -DDETECT_MSYS_TTY -DNOGDI -DHAVE_STRING_H -Icompat -Icompat/regex -Icompat/win32 -DSTRIP_EXTENSION=\".exe\"
+	COMPAT_CFLAGS = -D__USE_MINGW_ACCESS -DDETECT_MSYS_TTY \
+		-DENSURE_MSYSTEM_IS_SET="\"$(MSYSTEM)\"" -DMINGW_PREFIX="\"$(patsubst /%,%,$(MINGW_PREFIX))\"" \
+		-DNOGDI -DHAVE_STRING_H -Icompat -Icompat/regex -Icompat/win32 -DSTRIP_EXTENSION=\".exe\"
 	BASIC_LDFLAGS = -IGNORE:4217 -IGNORE:4049 -NOLOGO -ENTRY:wmainCRTStartup -SUBSYSTEM:CONSOLE
 	# invalidcontinue.obj allows Git's source code to close the same file
 	# handle twice, or to access the osfhandle of an already-closed stdout

@@ -758,7 +760,9 @@ ifeq ($(uname_S),MINGW)
 		prefix = $(MINGW_PREFIX)
 		HOST_CPU = $(patsubst %-w64-mingw32,%,$(MINGW_CHOST))
 		BASIC_LDFLAGS += -Wl,--pic-executable
-		COMPAT_CFLAGS += -DDETECT_MSYS_TTY
+		COMPAT_CFLAGS += -DDETECT_MSYS_TTY \
+			-DENSURE_MSYSTEM_IS_SET="\"$(MSYSTEM)\"" \
+			-DMINGW_PREFIX="\"$(patsubst /%,%,$(MINGW_PREFIX))\""
                 ifeq (MINGW32,$(MSYSTEM))
 			BASIC_LDFLAGS += -Wl,--large-address-aware
                 endif

index a57c4b464f..7285bd9ac2 100644
@@ -256,7 +256,14 @@ if(CMAKE_SYSTEM_NAME STREQUAL "Windows")
 				_CONSOLE DETECT_MSYS_TTY STRIP_EXTENSION=".exe"  NO_SYMLINK_HEAD UNRELIABLE_FSTAT
 				NOGDI OBJECT_CREATION_MODE=1 __USE_MINGW_ANSI_STDIO=0
 				OVERRIDE_STRDUP MMAP_PREVENTS_DELETE USE_WIN32_MMAP
-				HAVE_WPGMPTR ENSURE_MSYSTEM_IS_SET HAVE_RTLGENRANDOM)
+				HAVE_WPGMPTR HAVE_RTLGENRANDOM)
+	if(CMAKE_GENERATOR_PLATFORM STREQUAL "x64")
+		add_compile_definitions(ENSURE_MSYSTEM_IS_SET="MINGW64" MINGW_PREFIX="mingw64")
+	elseif(CMAKE_GENERATOR_PLATFORM STREQUAL "arm64")
+		add_compile_definitions(ENSURE_MSYSTEM_IS_SET="CLANGARM64" MINGW_PREFIX="clangarm64")
+	elseif(CMAKE_GENERATOR_PLATFORM STREQUAL "x86")
+		add_compile_definitions(ENSURE_MSYSTEM_IS_SET="MINGW32" MINGW_PREFIX="mingw32")
+	endif()
 	list(APPEND compat_SOURCES
 		compat/mingw.c
 		compat/winansi.c

index 7073d5844d..a8aba81e29 100644
@@ -1318,7 +1318,6 @@ elif host_machine.system() == 'windows'
 
   libgit_c_args += [
     '-DDETECT_MSYS_TTY',
-    '-DENSURE_MSYSTEM_IS_SET',
     '-DNATIVE_CRLF',
     '-DNOGDI',
     '-DNO_POSIX_GOODIES',

@@ -1328,6 +1327,20 @@ elif host_machine.system() == 'windows'
     '-D__USE_MINGW_ANSI_STDIO=0',
   ]
 
+  msystem = get_option('msystem')
+  if msystem != ''
+    mingw_prefix = get_option('mingw_prefix')
+    if mingw_prefix == ''
+      mingw_prefix = msystem.to_lower()
+    elif mingw_prefix.startswith('/')
+      mingw_prefix = mingw_prefix.substring(1)
+    endif
+    libgit_c_args += [
+      '-DENSURE_MSYSTEM_IS_SET="' + msystem + '"',
+      '-DMINGW_PREFIX="' + mingw_prefix + '"'
+    ]
+  endif
+
   libgit_dependencies += compiler.find_library('ntdll')
   libgit_include_directories += 'compat/win32'
   if compiler.get_id() == 'msvc'

index dc88f130d7..becf4689bf 100644
@@ -21,6 +21,10 @@ option('runtime_prefix', type: 'boolean', value: false,
   description: 'Resolve ancillary tooling and support files relative to the location of the runtime binary instead of hard-coding them into the binary.')
 option('sane_tool_path', type: 'array', value: [],
   description: 'An array of paths to pick up tools from in case the normal tools are broken or lacking.')
+option('msystem', type: 'string', value: '',
+  description: 'Fall-back on Windows when MSYSTEM is not set.')
+option('mingw_prefix', type: 'string', value: '',
+  description: 'Fall-back on Windows when MINGW_PREFIX is not set.')
 
 # Build information compiled into Git and other parts like documentation.
 option('build_date', type: 'string', value: '',

index 8545cdfab5..02906055d3 100755
@@ -611,4 +611,34 @@ test_expect_success !VALGRIND,RUNTIME_PREFIX,CAN_EXEC_IN_PWD '%(prefix)/ works'
 	test_cmp expect actual
 '
 
+test_expect_success MINGW,RUNTIME_PREFIX 'MSYSTEM/PATH is adjusted if necessary' '
+	if test -z "$MINGW_PREFIX"
+	then
+		MINGW_PREFIX="/$(echo "${MSYSTEM:-MINGW64}" | tr A-Z a-z)"
+	fi &&
+	mkdir -p "$HOME"/bin pretend"$MINGW_PREFIX"/bin \
+		pretend"$MINGW_PREFIX"/libexec/git-core pretend/usr/bin &&
+	cp "$GIT_EXEC_PATH"/git.exe pretend"$MINGW_PREFIX"/bin/ &&
+	cp "$GIT_EXEC_PATH"/git.exe pretend"$MINGW_PREFIX"/libexec/git-core/ &&
+	# copy the .dll files, if any (happens when building via CMake)
+	if test -n "$(ls "$GIT_EXEC_PATH"/*.dll 2>/dev/null)"
+	then
+		cp "$GIT_EXEC_PATH"/*.dll pretend"$MINGW_PREFIX"/bin/ &&
+		cp "$GIT_EXEC_PATH"/*.dll pretend"$MINGW_PREFIX"/libexec/git-core/
+	fi &&
+	echo "env | grep MSYSTEM=" | write_script "$HOME"/bin/git-test-home &&
+	echo "echo ${MINGW_PREFIX#/}" | write_script pretend"$MINGW_PREFIX"/bin/git-test-bin &&
+	echo "echo usr" | write_script pretend/usr/bin/git-test-bin2 &&
+
+	(
+		MSYSTEM= &&
+		GIT_EXEC_PATH= &&
+		pretend"$MINGW_PREFIX"/libexec/git-core/git.exe test-home >actual &&
+		pretend"$MINGW_PREFIX"/libexec/git-core/git.exe test-bin >>actual &&
+		pretend"$MINGW_PREFIX"/bin/git.exe test-bin2 >>actual
+	) &&
+	test_write_lines MSYSTEM=$MSYSTEM "${MINGW_PREFIX#/}" usr >expect &&
+	test_cmp expect actual
+'
+
 test_done

gitgitgadget

From: Johannes Schindelin <[email protected]>

These days, the test cases are less free-form than in the wild old days of the Git project.

---
 t/t0060-path-utils.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

index 02906055d3..56faf5fe73 100755
@@ -602,7 +602,8 @@ test_expect_success !VALGRIND,RUNTIME_PREFIX,CAN_EXEC_IN_PWD 'RUNTIME_PREFIX wor
 	echo "echo HERE" | write_script pretend/libexec/git-core/git-here &&
 	GIT_EXEC_PATH= ./pretend/bin/git here >actual &&
 	echo HERE >expect &&
-	test_cmp expect actual'
+	test_cmp expect actual
+'
 
 test_expect_success !VALGRIND,RUNTIME_PREFIX,CAN_EXEC_IN_PWD '%(prefix)/ works' '
 	git config yes.path "%(prefix)/yes" &&

gitgitgadget

"Johannes Schindelin via GitGitGadget" <[email protected]>

Having a reroll by the author makes my life easier ;-) Will replace, and with J6t's earlier blessing, we can mark it for 'next'.

Thanks, both.