Skip to contentSkip to conversation

Platform compatibility fixes

PATCH Superseded Patrick Steinhardt 6 participants 58 messages Oct 14, 2024 – Oct 28, 2024 Kbuild

lore
Share & tools
lore
#0 lore raw

cover Platform compatibility fixes

Hi,

this patch series includes platform compatibility fixes that I have found while working on the Meson build system. These patches have been split out of the original patch series at [1] such that it can land independent of the changes in there.

I'll reroll the Meson patch series eventually and build on top of this patch series here and a second, independent patch series that updates clar [2].

Thanks!

Patrick

[1]: https://lore.kernel.org/git/[email protected]/
[2]: https://lore.kernel.org/git/[email protected]/

Patrick Steinhardt (10):
t/test-lib: fix quoting of TEST_RESULTS_SAN_FILE t/test-lib: wire up NO_ICONV prerequisite
t/lib-gitweb: test against the build version of gitweb t/lib-gpg: fix setup of GNUPGHOME in MinGW
t1401: make invocation of tar(1) work with Win32-provided one
t3404: work around platform-specific behaviour on macOS 10.15 t5500, t5601: skip tests which exercise paths with '[::1]' on Cygwin
t7300: work around platform-specific behaviour with long paths on
MinGW
builtin/credential-cache: fix missing parameter for stub function
http: fix build error on FreeBSD

 Makefile                                     |   1 +
 builtin/credential-cache.c                   |   3 +-
 contrib/buildsystems/CMakeLists.txt          |   6 ++
 http.c                                       |  10 +-
 t/lib-gitweb.sh                              |   2 +-
 t/lib-gpg.sh                                 |   2 +-
 t/t0028-working-tree-encoding.sh             |   6 ++
 t/t1401-symbolic-ref.sh                      |   2 +-
 t/t2082-parallel-checkout-attributes.sh      |   2 +-
 t/t3404-rebase-interactive.sh                |  26 +++--
 t/t3434-rebase-i18n.sh                       |   6 ++
 t/t3900-i18n-commit.sh                       |   6 ++
 t/t3901-i18n-patch.sh                        |   6 ++
 t/t4041-diff-submodule-option.sh             |  16 ++-
 t/t4059-diff-submodule-not-initialized.sh    |  16 ++-
 t/t4060-diff-submodule-option-diff-format.sh |  17 ++--
 t/t4201-shortlog.sh                          |   8 +-
 t/t4205-log-pretty-formats.sh                | 102 +++++++++++--------
 t/t4210-log-i18n.sh                          |   6 ++
 t/t4254-am-corrupt.sh                        |   6 ++
 t/t5100-mailinfo.sh                          |  14 ++-
 t/t5500-fetch-pack.sh                        |  14 ++-
 t/t5550-http-fetch-dumb.sh                   |   4 +-
 t/t5601-clone.sh                             |  11 +-
 t/t6006-rev-list-format.sh                   |  54 ++++++----
 t/t7102-reset.sh                             |  40 +++++---
 t/t7300-clean.sh                             |   2 +-
 t/t8005-blame-i18n.sh                        |   6 ++
 t/t9300-fast-import.sh                       |   2 +-
 t/t9350-fast-export.sh                       |  10 +-
 t/test-lib.sh                                |   3 +-
 31 files changed, 273 insertions(+), 136 deletions(-)
Signature · 1 line
2.47.0.dirty
#1 lore raw

PATCH 1/10 t/test-lib: fix quoting of TEST_RESULTS_SAN_FILE

When assembling our LSAN_OPTIONS that configure the leak sanitizer we end up prepending the string with various different colon-separated options via calls to prepend_var. One of the settings we add is the path where the sanitizer should store logs, which can be an arbitrary filesystem path.

Naturally, filesystem paths may contain whitespace characters. And while it does seem as if we were quoting the value, we use escaped quotes and consequently split up the value if it does contain spaces. This leads to the following error in t0000 when having a value with whitespaces:

.../t/test-lib.sh: eval: line 64: unexpected EOF while looking for matching `"' ++ return 1
error: last command exited with $?=1
not ok 5 - subtest: 3 passing tests

The error itself is a bit puzzling at first. The basic problem is that the code sees the leading escaped quote during eval, but because we truncate everything after the space character it doesn't see the trailing escaped quote and thus fails to parse the string.

Properly quote the value to fix the issue while using single-quotes to quote the inner value passed to eval. The issue can be reproduced by t0000 with such a path that contains spaces.

---
 t/test-lib.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
t/test-lib.sh +1 -1
index b1a8ee5c002..241198ba95f 100644
@@ -1572,7 +1572,7 @@ then
 
 	prepend_var LSAN_OPTIONS : dedup_token_length=9999
 	prepend_var LSAN_OPTIONS : log_exe_name=1
-	prepend_var LSAN_OPTIONS : log_path=\"$TEST_RESULTS_SAN_FILE\"
+	prepend_var LSAN_OPTIONS : log_path="'$TEST_RESULTS_SAN_FILE'"
 	export LSAN_OPTIONS
 
 elif test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check" ||
Signature · 1 line
2.47.0.dirty
#3 lore raw

PATCH 2/10 t/test-lib: wire up NO_ICONV prerequisite

In reply to Patrick Steinhardt

The iconv library is used by Git to reencode files, commit messages and other things. As such it is a rather integral part, but given that many platforms nowadays use UTF-8 everywhere you can live without support for reencoding in many situations. It is thus optional to build Git with iconv, and some of our platforms wired up in "config.mak.uname" disable it. But while we support building without it, running our test suite with "NO_ICONV=Yes" causes many test failures.

Wire up a new test prerequisite ICONV that gets populated via our GIT-BUILD-OPTIONS. Annotate failing tests accordingly.

Note that this commit does not do a deep dive into every single test to assess whether the failure is expected or not. Most of the tests do smell like the expected kind of failure though.

Further note that there are several "!MINGW" conditions in t4201, and all of these fail due to iconv-related errors. This is quite likely a leftover from times before dce7d29551 (msvc: support building Git using MS Visual C++, 2019-06-25), which switched Windows-based builds over from "NO_ICONV=YesPlease" to "NEEDS_LIBICONV=YesPlease". Consequently, adapt those tests to also use the new ICONV prerequisite.

---
 Makefile                                     |   1 +
 contrib/buildsystems/CMakeLists.txt          |   6 ++
 t/t0028-working-tree-encoding.sh             |   6 ++
 t/t2082-parallel-checkout-attributes.sh      |   2 +-
 t/t3434-rebase-i18n.sh                       |   6 ++
 t/t3900-i18n-commit.sh                       |   6 ++
 t/t3901-i18n-patch.sh                        |   6 ++
 t/t4041-diff-submodule-option.sh             |  16 ++-
 t/t4059-diff-submodule-not-initialized.sh    |  16 ++-
 t/t4060-diff-submodule-option-diff-format.sh |  17 ++--
 t/t4201-shortlog.sh                          |   8 +-
 t/t4205-log-pretty-formats.sh                | 102 +++++++++++--------
 t/t4210-log-i18n.sh                          |   6 ++
 t/t4254-am-corrupt.sh                        |   6 ++
 t/t5100-mailinfo.sh                          |  14 ++-
 t/t5550-http-fetch-dumb.sh                   |   4 +-
 t/t6006-rev-list-format.sh                   |  54 ++++++----
 t/t7102-reset.sh                             |  40 +++++---
 t/t8005-blame-i18n.sh                        |   6 ++
 t/t9300-fast-import.sh                       |   2 +-
 t/t9350-fast-export.sh                       |  10 +-
 t/test-lib.sh                                |   1 +
 22 files changed, 229 insertions(+), 106 deletions(-)
Makefile +1 -0
index feeed6f9321..5db10347341 100644
@@ -3171,6 +3171,7 @@ GIT-BUILD-OPTIONS: FORCE
 	@echo PYTHON_PATH=\''$(subst ','\'',$(PYTHON_PATH_SQ))'\' >>$@+
 	@echo TAR=\''$(subst ','\'',$(subst ','\'',$(TAR)))'\' >>$@+
 	@echo NO_CURL=\''$(subst ','\'',$(subst ','\'',$(NO_CURL)))'\' >>$@+
+	@echo NO_ICONV=\''$(subst ','\'',$(subst ','\'',$(NO_ICONV)))'\' >>$@+
 	@echo NO_EXPAT=\''$(subst ','\'',$(subst ','\'',$(NO_EXPAT)))'\' >>$@+
 	@echo USE_LIBPCRE2=\''$(subst ','\'',$(subst ','\'',$(USE_LIBPCRE2)))'\' >>$@+
 	@echo NO_PERL=\''$(subst ','\'',$(subst ','\'',$(NO_PERL)))'\' >>$@+
contrib/buildsystems/CMakeLists.txt +6 -0
index 62af7b33d2f..1384c0eb6d3 100644
@@ -1109,6 +1109,7 @@ set(DIFF diff)
 set(PYTHON_PATH /usr/bin/python)
 set(TAR tar)
 set(NO_CURL )
+set(NO_ICONV )
 set(NO_EXPAT )
 set(USE_LIBPCRE2 )
 set(NO_PERL )
@@ -1122,6 +1123,10 @@ if(NOT CURL_FOUND)
 	set(NO_CURL 1)
 endif()
 
+if(NOT Iconv_FOUND)
+	SET(NO_ICONV 1)
+endif()
+
 if(NOT EXPAT_FOUND)
 	set(NO_EXPAT 1)
 endif()
@@ -1145,6 +1150,7 @@ file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS "DIFF='${DIFF}'\n")
 file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS "PYTHON_PATH='${PYTHON_PATH}'\n")
 file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS "TAR='${TAR}'\n")
 file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS "NO_CURL='${NO_CURL}'\n")
+file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS "NO_ICONV='${NO_ICONV}'\n")
 file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS "NO_EXPAT='${NO_EXPAT}'\n")
 file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS "NO_PERL='${NO_PERL}'\n")
 file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS "NO_PTHREADS='${NO_PTHREADS}'\n")
t/t0028-working-tree-encoding.sh +6 -0
index ad151a34670..510da4ca12d 100755
@@ -12,6 +12,12 @@ TEST_CREATE_REPO_NO_TEMPLATE=1
 
 GIT_TRACE_WORKING_TREE_ENCODING=1 && export GIT_TRACE_WORKING_TREE_ENCODING
 
+if ! test_have_prereq ICONV
+then
+	skip_all='skipping working tree encoding tests; iconv not available'
+	test_done
+fi
+
 test_expect_success 'setup test files' '
 	git config core.eol lf &&
 
t/t2082-parallel-checkout-attributes.sh +1 -1
index aec55496eb1..a040aa54cee 100755
@@ -34,7 +34,7 @@ test_expect_success 'parallel-checkout with ident' '
 	)
 '
 
-test_expect_success 'parallel-checkout with re-encoding' '
+test_expect_success ICONV 'parallel-checkout with re-encoding' '
 	set_checkout_config 2 0 &&
 	git init encoding &&
 	(
t/t3434-rebase-i18n.sh +6 -0
index 26a48d6b103..97fc9a23f21 100755
@@ -20,6 +20,12 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
 TEST_PASSES_SANITIZE_LEAK=true
 . ./test-lib.sh
 
+if ! test_have_prereq ICONV
+then
+	skip_all='skipping rebase i18n tests; iconv not available'
+	test_done
+fi
+
 compare_msg () {
 	iconv -f "$2" -t "$3" "$TEST_DIRECTORY/t3434/$1" >expect &&
 	git cat-file commit HEAD >raw &&
t/t3900-i18n-commit.sh +6 -0
index db7b403bc15..9d4b5ab1f95 100755
@@ -8,6 +8,12 @@ test_description='commit and log output encodings'
 TEST_PASSES_SANITIZE_LEAK=true
 . ./test-lib.sh
 
+if ! test_have_prereq ICONV
+then
+	skip_all='skipping commit i18n tests; iconv not available'
+	test_done
+fi
+
 compare_with () {
 	git show -s $1 | sed -e '1,/^$/d' -e 's/^    //' >current &&
 	case "$3" in
t/t3901-i18n-patch.sh +6 -0
index 5f0b9afc3fa..e0659c92935 100755
@@ -11,6 +11,12 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
 TEST_PASSES_SANITIZE_LEAK=true
 . ./test-lib.sh
 
+if ! test_have_prereq ICONV
+then
+	skip_all='skipping patch i18n tests; iconv not available'
+	test_done
+fi
+
 check_encoding () {
 	# Make sure characters are not corrupted
 	cnt="$1" header="$2" i=1 j=0
t/t4041-diff-submodule-option.sh +11 -5
index 8fc40e75eb3..aa149e0085e 100755
@@ -15,12 +15,18 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
 TEST_PASSES_SANITIZE_LEAK=true
 . ./test-lib.sh
 
-# Tested non-UTF-8 encoding
-test_encoding="ISO8859-1"
+# Test non-UTF-8 encoding in case iconv is available.
+if test_have_prereq ICONV
+then
+	test_encoding="ISO8859-1"
+	# String "added" in German (translated with Google Translate), encoded in UTF-8,
+	# used in sample commit log messages in add_file() function below.
+	added=$(printf "hinzugef\303\274gt")
+else
+	test_encoding="UTF-8"
+	added="added"
+fi
 
-# String "added" in German (translated with Google Translate), encoded in UTF-8,
-# used in sample commit log messages in add_file() function below.
-added=$(printf "hinzugef\303\274gt")
 add_file () {
 	(
 		cd "$1" &&
t/t4059-diff-submodule-not-initialized.sh +11 -5
index 668f5263038..28fd3cdb154 100755
@@ -12,12 +12,18 @@ initialized previously but the checkout has since been removed.
 TEST_PASSES_SANITIZE_LEAK=true
 . ./test-lib.sh
 
-# Tested non-UTF-8 encoding
-test_encoding="ISO8859-1"
 
-# String "added" in German (translated with Google Translate), encoded in UTF-8,
-# used in sample commit log messages in add_file() function below.
-added=$(printf "hinzugef\303\274gt")
+# Test non-UTF-8 encoding in case iconv is available.
+if test_have_prereq ICONV
+then
+	test_encoding="ISO8859-1"
+	# String "added" in German (translated with Google Translate), encoded in UTF-8,
+	# used in sample commit log messages in add_file() function below.
+	added=$(printf "hinzugef\303\274gt")
+else
+	test_encoding="UTF-8"
+	added="added"
+fi
 
 add_file () {
 	(
t/t4060-diff-submodule-option-diff-format.sh +11 -6
index 8ce67442d96..918334fa4c8 100755
@@ -13,12 +13,17 @@ This test tries to verify the sanity of --submodule=diff option of git diff.
 TEST_PASSES_SANITIZE_LEAK=true
 . ./test-lib.sh
 
-# Tested non-UTF-8 encoding
-test_encoding="ISO8859-1"
-
-# String "added" in German (translated with Google Translate), encoded in UTF-8,
-# used in sample commit log messages in add_file() function below.
-added=$(printf "hinzugef\303\274gt")
+# Test non-UTF-8 encoding in case iconv is available.
+if test_have_prereq ICONV
+then
+	test_encoding="ISO8859-1"
+	# String "added" in German (translated with Google Translate), encoded in UTF-8,
+	# used in sample commit log messages in add_file() function below.
+	added=$(printf "hinzugef\303\274gt")
+else
+	test_encoding="UTF-8"
+	added="added"
+fi
 
 add_file () {
 	(
t/t4201-shortlog.sh +4 -4
index c20c8857244..680e707ba1e 100755
@@ -105,7 +105,7 @@ test_expect_success 'output from user-defined format is re-wrapped' '
 	test_cmp expect log.predictable
 '
 
-test_expect_success !MINGW 'shortlog wrapping' '
+test_expect_success ICONV 'shortlog wrapping' '
 	cat >expect <<\EOF &&
 A U Thor (5):
       Test
@@ -126,13 +126,13 @@ EOF
 	test_cmp expect out
 '
 
-test_expect_success !MINGW 'shortlog from non-git directory' '
+test_expect_success ICONV 'shortlog from non-git directory' '
 	git log --no-expand-tabs HEAD >log &&
 	GIT_DIR=non-existing git shortlog -w <log >out &&
 	test_cmp expect out
 '
 
-test_expect_success !MINGW 'shortlog can read --format=raw output' '
+test_expect_success ICONV 'shortlog can read --format=raw output' '
 	git log --format=raw HEAD >log &&
 	GIT_DIR=non-existing git shortlog -w <log >out &&
 	test_cmp expect out
@@ -182,7 +182,7 @@ $DSCHO (2):
 
 EOF
 
-test_expect_success !MINGW 'shortlog encoding' '
+test_expect_success ICONV 'shortlog encoding' '
 	git reset --hard "$commit" &&
 	git config --unset i18n.commitencoding &&
 	echo 2 > a1 &&
t/t4205-log-pretty-formats.sh +59 -43

310 lines not on this page · raw message

t/t4210-log-i18n.sh +6 -0
index 7120030b5c6..4a12b2b4979 100755
@@ -5,6 +5,12 @@ test_description='test log with i18n features'
 TEST_PASSES_SANITIZE_LEAK=true
 . ./lib-gettext.sh
 
+if ! test_have_prereq ICONV
+then
+	skip_all='skipping log i18n tests; iconv not available'
+	test_done
+fi
+
 # two forms of é
 utf8_e=$(printf '\303\251')
 latin1_e=$(printf '\351')
t/t4254-am-corrupt.sh +6 -0
index 661feb60709..cb03522d021 100755
@@ -5,6 +5,12 @@ test_description='git am with corrupt input'
 TEST_PASSES_SANITIZE_LEAK=true
 . ./test-lib.sh
 
+if ! test_have_prereq ICONV
+then
+	skip_all='skipping am encoding corruption tests; iconv not available'
+	test_done
+fi
+
 make_mbox_with_nul () {
 	space=' '
 	q_nul_in_subject=
t/t5100-mailinfo.sh +12 -2
index 065156c1f39..23b2f218725 100755
@@ -28,7 +28,12 @@ check_mailinfo () {
 
 for mail in 00*
 do
-	test_expect_success "mailinfo $mail" '
+	case "$mail" in
+	0004)
+		prereq=ICONV;;
+	esac
+
+	test_expect_success $prereq "mailinfo $mail" '
 		check_mailinfo "$mail" "" &&
 		if test -f "$DATA/msg$mail--scissors"
 		then
@@ -56,7 +61,12 @@ test_expect_success 'split box with rfc2047 samples' \
 
 for mail in rfc2047/00*
 do
-	test_expect_success "mailinfo $mail" '
+	case "$mail" in
+	rfc2047/0001)
+		prereq=ICONV;;
+	esac
+
+	test_expect_success $prereq "mailinfo $mail" '
 		git mailinfo -u "$mail-msg" "$mail-patch" <"$mail" >"$mail-info" &&
 		echo msg &&
 		test_cmp "$DATA/empty" "$mail-msg" &&
t/t5550-http-fetch-dumb.sh +2 -2
index 58189c9f7dc..3c873de17ec 100755
@@ -344,12 +344,12 @@ test_expect_success 'git client shows text/plain with a charset' '
 	grep "this is the error message" stderr
 '
 
-test_expect_success 'http error messages are reencoded' '
+test_expect_success ICONV 'http error messages are reencoded' '
 	test_must_fail git clone "$HTTPD_URL/error/utf16" 2>stderr &&
 	grep "this is the error message" stderr
 '
 
-test_expect_success 'reencoding is robust to whitespace oddities' '
+test_expect_success ICONV 'reencoding is robust to whitespace oddities' '
 	test_must_fail git clone "$HTTPD_URL/error/odd-spacing" 2>stderr &&
 	grep "this is the error message" stderr
 '
t/t6006-rev-list-format.sh +37 -17

72 lines not on this page · raw message

t/t7102-reset.sh +25 -15
index 2add26d7684..e9a6cc72658 100755
@@ -13,21 +13,31 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
 TEST_PASSES_SANITIZE_LEAK=true
 . ./test-lib.sh
 
-commit_msg () {
-	# String "modify 2nd file (changed)" partly in German
-	# (translated with Google Translate),
-	# encoded in UTF-8, used as a commit log message below.
-	msg="modify 2nd file (ge\303\244ndert)\n"
-	if test -n "$1"
-	then
-		printf "$msg" | iconv -f utf-8 -t "$1"
-	else
-		printf "$msg"
-	fi
-}
-
-# Tested non-UTF-8 encoding
-test_encoding="ISO8859-1"
+if test_have_prereq ICONV
+then
+	commit_msg () {
+		# String "modify 2nd file (changed)" partly in German
+		# (translated with Google Translate),
+		# encoded in UTF-8, used as a commit log message below.
+		msg="modify 2nd file (ge\303\244ndert)\n"
+		if test -n "$1"
+		then
+			printf "$msg" | iconv -f utf-8 -t "$1"
+		else
+			printf "$msg"
+		fi
+	}
+
+	# Tested non-UTF-8 encoding
+	test_encoding="ISO8859-1"
+else
+	commit_msg () {
+		echo "modify 2nd file (geandert)"
+	}
+
+	# Tested non-UTF-8 encoding
+	test_encoding="UTF-8"
+fi
 
 test_expect_success 'creating initial files and commits' '
 	test_tick &&
t/t8005-blame-i18n.sh +6 -0
index 75da219ed1b..7a1f581c240 100755
@@ -3,6 +3,12 @@
 test_description='git blame encoding conversion'
 . ./test-lib.sh
 
+if ! test_have_prereq ICONV
+then
+	skip_all='skipping blame i18n tests; iconv not available'
+	test_done
+fi
+
 . "$TEST_DIRECTORY"/t8005/utf8.txt
 . "$TEST_DIRECTORY"/t8005/euc-japan.txt
 . "$TEST_DIRECTORY"/t8005/sjis.txt
t/t9300-fast-import.sh +1 -1
index 3b3c371740a..6224f54d4d2 100755
@@ -3676,7 +3676,7 @@ test_expect_success !MINGW 'W: get-mark & empty orphan commit with erroneous thi
 ### series X (other new features)
 ###
 
-test_expect_success 'X: handling encoding' '
+test_expect_success ICONV 'X: handling encoding' '
 	test_tick &&
 	cat >input <<-INPUT_END &&
 	commit refs/heads/encoding
t/t9350-fast-export.sh +5 -5
index 2bdc02b4599..9595dfef2ee 100755
@@ -125,7 +125,7 @@ test_expect_success 'fast-export --show-original-ids | git fast-import' '
 	test $MUSS = $(git rev-parse --verify refs/tags/muss)
 '
 
-test_expect_success 'reencoding iso-8859-7' '
+test_expect_success ICONV 'reencoding iso-8859-7' '
 
 	test_when_finished "git reset --hard HEAD~1" &&
 	test_config i18n.commitencoding iso-8859-7 &&
@@ -421,7 +421,7 @@ M 100644 :1 there
 
 EOF
 
-test_expect_success 'dropping tag of filtered out object' '
+test_expect_success ICONV 'dropping tag of filtered out object' '
 (
 	cd limit-by-paths &&
 	git fast-export --tag-of-filtered-object=drop mytag -- there > output &&
@@ -438,7 +438,7 @@ msg
 
 EOF
 
-test_expect_success 'rewriting tag of filtered out object' '
+test_expect_success ICONV 'rewriting tag of filtered out object' '
 (
 	cd limit-by-paths &&
 	git fast-export --tag-of-filtered-object=rewrite mytag -- there > output &&
@@ -667,7 +667,7 @@ M 100644 :13 file
 
 EOF
 
-test_expect_success 'avoid uninteresting refs' '
+test_expect_success ICONV 'avoid uninteresting refs' '
 	> tmp-marks &&
 	git fast-export --import-marks=tmp-marks \
 		--export-marks=tmp-marks main > /dev/null &&
@@ -686,7 +686,7 @@ from :14
 
 EOF
 
-test_expect_success 'refs are updated even if no commits need to be exported' '
+test_expect_success ICONV 'refs are updated even if no commits need to be exported' '
 	> tmp-marks &&
 	git fast-export --import-marks=tmp-marks \
 		--export-marks=tmp-marks main > /dev/null &&
t/test-lib.sh +1 -0
index 241198ba95f..a278181a056 100644
@@ -1743,6 +1743,7 @@ esac
 
 ( COLUMNS=1 && test $COLUMNS = 1 ) && test_set_prereq COLUMNS_CAN_BE_1
 test -z "$NO_CURL" && test_set_prereq LIBCURL
+test -z "$NO_ICONV" && test_set_prereq ICONV
 test -z "$NO_PERL" && test_set_prereq PERL
 test -z "$NO_PTHREADS" && test_set_prereq PTHREADS
 test -z "$NO_PYTHON" && test_set_prereq PYTHON
Signature · 1 line
2.47.0.dirty
#4 lore raw

PATCH 3/10 t/lib-gitweb: test against the build version of gitweb

In reply to Patrick Steinhardt

When testing gitweb we set up the CGI script as "gitweb.perl", which is the source file of the build target "gitweb.cgi". This works alright as long as we run in-tree tests. But we're about to make out-of-tree tests a reality, and there things will break because "gitweb.perl" will not be found in the build directory.

Fix this by using "gitweb.cgi" instead.

---
 t/lib-gitweb.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
t/lib-gitweb.sh +1 -1
index 1f32ca66ea5..c64c87fb87d 100644
@@ -49,7 +49,7 @@ EOF
 		error "Cannot find gitweb at $GITWEB_TEST_INSTALLED."
 		say "# Testing $SCRIPT_NAME"
 	else # normal case, use source version of gitweb
-		SCRIPT_NAME="$GIT_BUILD_DIR/gitweb/gitweb.perl"
+		SCRIPT_NAME="$GIT_BUILD_DIR/gitweb/gitweb.cgi"
 	fi
 	export SCRIPT_NAME
 }
Signature · 1 line
2.47.0.dirty
#7 lore raw

PATCH 4/10 t/lib-gpg: fix setup of GNUPGHOME in MinGW

In reply to Patrick Steinhardt

In "t/lib-gpg.sh" we set up the "GNUPGHOME" environment variable to point to a test-specific directory. This is done by using "$PWD/gpghome" as value, where "$PWD" is the current test's trash directory.

This is broken for MinGW though because "$PWD" will use Windows-style paths that contain drive letters. What we really want in this context is a Unix-style path, which we can get by using $(pwd) instead. It is somewhat puzzling that nobody ever hit this issue, but it may easily be that nobody ever tests on Windows with GnuPG installed, which would make us skip those tests.

Adapt the code accordingly to fix tests using this library.

---
 t/lib-gpg.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
t/lib-gpg.sh +1 -1
index add11e88fc0..3845b6ac449 100644
@@ -6,7 +6,7 @@
 #   executed in an eval'ed subshell that changes the working directory to a
 #   temporary one.
 
-GNUPGHOME="$PWD/gpghome"
+GNUPGHOME="$(pwd)/gpghome"
 export GNUPGHOME
 
 test_lazy_prereq GPG '
Signature · 1 line
2.47.0.dirty
#9 lore raw

PATCH 5/10 t1401: make invocation of tar(1) work with Win32-provided one

In reply to Patrick Steinhardt

Windows nowadays provides a tar(1) binary in "C:\Windows\system32". This version of tar(1) doesn't seem to handle the case where directory paths end with a trailing forward slash. And as we do that in t1401 the result is that the test fails.

Drop the trailing slash. Other tests that use tar(1) work alright, this is the only instance where it has been failing.

---
 t/t1401-symbolic-ref.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
t/t1401-symbolic-ref.sh +1 -1
index 5c60d6f812d..90af3f955c0 100755
@@ -16,7 +16,7 @@ reset_to_sane() {
 test_expect_success 'setup' '
 	git symbolic-ref HEAD refs/heads/foo &&
 	test_commit file &&
-	"$TAR" cf .git.tar .git/
+	"$TAR" cf .git.tar .git
 '
 
 test_expect_success 'symbolic-ref read/write roundtrip' '
Signature · 1 line
2.47.0.dirty
#12 lore raw

PATCH 6/10 t3404: work around platform-specific behaviour on macOS 10.15

In reply to Patrick Steinhardt

Two of our tests in t3404 use indented HERE docs where leading tabs on some of the lines are actually relevant. The tabs do get removed though, and we try to fix this up by using sed(1) to replace leading tabs in the actual output, as well. But macOS 10.15 uses an oldish version of sed(1) that has BSD lineage, which does not understand "\t", and thus we fail to strip those leading tabs and fail the test.

Address this issue by using q_to_tab such that we do not have to strip leading tabs from the actual output.

---
 t/t3404-rebase-interactive.sh | 26 ++++++++++++--------------
 1 file changed, 12 insertions(+), 14 deletions(-)
t/t3404-rebase-interactive.sh +12 -14
index f171af3061d..7ce75237803 100755
@@ -1917,18 +1917,17 @@ test_expect_success '--update-refs updates refs correctly' '
 	test_cmp_rev HEAD~1 refs/heads/third &&
 	test_cmp_rev HEAD refs/heads/no-conflict-branch &&
 
-	cat >expect <<-\EOF &&
+	q_to_tab >expect <<-\EOF &&
 	Successfully rebased and updated refs/heads/update-refs.
 	Updated the following refs with --update-refs:
-		refs/heads/first
-		refs/heads/no-conflict-branch
-		refs/heads/second
-		refs/heads/third
+	Qrefs/heads/first
+	Qrefs/heads/no-conflict-branch
+	Qrefs/heads/second
+	Qrefs/heads/third
 	EOF
 
 	# Clear "Rebasing (X/Y)" progress lines and drop leading tabs.
-	sed -e "s/Rebasing.*Successfully/Successfully/g" -e "s/^\t//g" \
-		<err >err.trimmed &&
+	sed "s/Rebasing.*Successfully/Successfully/g" <err >err.trimmed &&
 	test_cmp expect err.trimmed
 '
 
@@ -2178,19 +2177,18 @@ test_expect_success '--update-refs: check failed ref update' '
 	test_must_fail git rebase --continue 2>err &&
 	grep "update_ref failed for ref '\''refs/heads/second'\''" err &&
 
-	cat >expect <<-\EOF &&
+	q_to_tab >expect <<-\EOF &&
 	Updated the following refs with --update-refs:
-		refs/heads/first
-		refs/heads/no-conflict-branch
-		refs/heads/third
+	Qrefs/heads/first
+	Qrefs/heads/no-conflict-branch
+	Qrefs/heads/third
 	Failed to update the following refs with --update-refs:
-		refs/heads/second
+	Qrefs/heads/second
 	EOF
 
 	# Clear "Rebasing (X/Y)" progress lines and drop leading tabs.
 	tail -n 6 err >err.last &&
-	sed -e "s/Rebasing.*Successfully/Successfully/g" -e "s/^\t//g" \
-		<err.last >err.trimmed &&
+	sed "s/Rebasing.*Successfully/Successfully/g" <err.last >err.trimmed &&
 	test_cmp expect err.trimmed
 '
 
Signature · 1 line
2.47.0.dirty
#13 lore raw

PATCH 7/10 t5500, t5601: skip tests which exercise paths with '[::1]' on Cygwin

In reply to Patrick Steinhardt

Parsing repositories which contain '[::1]' is broken on Cygwin. It seems as if Cygwin is confusing those as drive letter prefixes or something like this, but I couldn't deduce the actual root cause.

Mark those tests as broken for now.

---
 t/t5500-fetch-pack.sh | 14 ++++++++++----
 t/t5601-clone.sh      | 11 +++++++++--
 2 files changed, 19 insertions(+), 6 deletions(-)
t/t5500-fetch-pack.sh +10 -4
index 605f17240c1..416522c86ad 100755
@@ -774,7 +774,7 @@ do
 	# file with scheme
 	for p in file
 	do
-		test_expect_success !MINGW "fetch-pack --diag-url $p://$h/$r" '
+		test_expect_success !WINDOWS "fetch-pack --diag-url $p://$h/$r" '
 			check_prot_path $p://$h/$r $p "/$r"
 		'
 		test_expect_success MINGW "fetch-pack --diag-url $p://$h/$r" '
@@ -784,7 +784,7 @@ do
 			check_prot_path $p:///$r $p "/$r"
 		'
 		# No "/~" -> "~" conversion for file
-		test_expect_success !MINGW "fetch-pack --diag-url $p://$h/~$r" '
+		test_expect_success !WINDOWS "fetch-pack --diag-url $p://$h/~$r" '
 			check_prot_path $p://$h/~$r $p "/~$r"
 		'
 		test_expect_success MINGW "fetch-pack --diag-url $p://$h/~$r" '
@@ -806,11 +806,17 @@ do
 	p=ssh
 	for h in host [::1]
 	do
-		test_expect_success "fetch-pack --diag-url $h:$r" '
+		expectation="success"
+		if test_have_prereq CYGWIN && test "$h" = "[::1]"
+		then
+			expectation="failure"
+		fi
+
+		test_expect_$expectation "fetch-pack --diag-url $h:$r" '
 			check_prot_host_port_path $h:$r $p "$h" NONE "$r"
 		'
 		# Do "/~" -> "~" conversion
-		test_expect_success "fetch-pack --diag-url $h:/~$r" '
+		test_expect_$expectation "fetch-pack --diag-url $h:/~$r" '
 			check_prot_host_port_path $h:/~$r $p "$h" NONE "~$r"
 		'
 	done
t/t5601-clone.sh +9 -2
index 5d7ea147f1a..9fe665eadfb 100755
@@ -530,10 +530,17 @@ do
 	'
 done
 
+# Parsing of paths that look like IPv6 addresses is broken on Cygwin.
+expectation_for_ipv6_tests=success
+if test_have_prereq CYGWIN
+then
+	expectation_for_ipv6_tests=failure
+fi
+
 #ipv6
 for repo in rep rep/home/project 123
 do
-	test_expect_success "clone [::1]:$repo" '
+	test_expect_$expectation_for_ipv6_tests "clone [::1]:$repo" '
 		test_clone_url [::1]:$repo ::1 "$repo"
 	'
 done
@@ -542,7 +549,7 @@ test_expect_success "clone host:/~repo" '
 	test_clone_url host:/~repo host "~repo"
 '
 
-test_expect_success "clone [::1]:/~repo" '
+test_expect_$expectation_for_ipv6_tests "clone [::1]:/~repo" '
 	test_clone_url [::1]:/~repo ::1 "~repo"
 '
 
Signature · 1 line
2.47.0.dirty
#15 lore raw

PATCH 8/10 t7300: work around platform-specific behaviour with long paths on MinGW

In reply to Patrick Steinhardt

Windows by default has a restriction in place to only allow paths up to 260 characters. This restriction can nowadays be lifted by setting a registry key, but is still active by default.

In t7300 we have one test that exercises the behaviour of git-clean(1) with such long paths. Interestingly enough, this test fails on my system that uses Windows 10 with mingw-w64 installed via MSYS2: instead of observing ENAMETOOLONG, we observe ENOENT. This behaviour is consistent across multiple different environments I have tried.

I cannot say why exactly we observe a different error here, but I would not be surprised if this was either dependent on the Windows version, the version of MinGW, the current working directory of Git or any kind of combination of these.

Work around the issue by handling both errors.

---
 t/t7300-clean.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
t/t7300-clean.sh +1 -1
index 0aae0dee670..12ab25296b0 100755
@@ -747,7 +747,7 @@ test_expect_success MINGW 'handle clean & core.longpaths = false nicely' '
 	test_must_fail git clean -xdf 2>.git/err &&
 	# grepping for a strerror string is unportable but it is OK here with
 	# MINGW prereq
-	test_grep "too long" .git/err
+	test_grep -e "too long" -e "No such file or directory" .git/err
 '
 
 test_expect_success 'clean untracked paths by pathspec' '
Signature · 1 line
2.47.0.dirty
#16 lore raw

PATCH 9/10 builtin/credential-cache: fix missing parameter for stub function

In reply to Patrick Steinhardt

When not compiling the credential cache we may use a stub function for cmd_credential_cache(). With commit 9b1cb5070f (builtin: add a repository parameter for builtin functions, 2024-09-13), we have added a new parameter to all of those top-level cmd_*() functions, and did indeed adapt the non-stubbed-out cmd_credential_cache(). But we didn't adapt the stubbed-out variant, so the code does not compile.

Fix this by adding the missing parameter.

---
 builtin/credential-cache.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
builtin/credential-cache.c +2 -1
index 5de8b9123bf..c51f7fc2ade 100644
@@ -189,7 +189,8 @@ int cmd_credential_cache(int argc,
 
 #else
 
-int cmd_credential_cache(int argc, const char **argv, const char *prefix)
+int cmd_credential_cache(int argc, const char **argv, const char *prefix,
+			 struct repository *repo UNUSED)
 {
 	const char * const usage[] = {
 		"git credential-cache [options] <action>",
Signature · 1 line
2.47.0.dirty
#18 lore raw

PATCH 10/10 http: fix build error on FreeBSD

In reply to Patrick Steinhardt

The result parameter passed to http_request_reauth() may either point to a struct strbuf or a FILE *, where the target parameter tells us which of either it actually is. To accommodate for both types the pointer is a void *, which we then pass directly to functions without doing a cast.

This is fine on most platforms, but it breaks on FreeBSD because fileno() is implemented as a macro that tries to directly access the FILE * structure.

Fix this issue by storing the FILE * in a local variable before we pass it on to other functions.

---
 http.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)
http.c +6 -4
index d59e59f66b1..72973175a85 100644
@@ -2290,17 +2290,19 @@ static int http_request_reauth(const char *url,
 		case HTTP_REQUEST_STRBUF:
 			strbuf_reset(result);
 			break;
-		case HTTP_REQUEST_FILE:
-			if (fflush(result)) {
+		case HTTP_REQUEST_FILE: {
+			FILE *f = result;
+			if (fflush(f)) {
 				error_errno("unable to flush a file");
 				return HTTP_START_FAILED;
 			}
-			rewind(result);
-			if (ftruncate(fileno(result), 0) < 0) {
+			rewind(f);
+			if (ftruncate(fileno(f), 0) < 0) {
 				error_errno("unable to truncate a file");
 				return HTTP_START_FAILED;
 			}
 			break;
+		}
 		default:
 			BUG("Unknown http_request target");
 		}
Signature · 1 line
2.47.0.dirty
#2 lore raw

Re: [PATCH 01/10] t/test-lib: fix quoting of TEST_RESULTS_SAN_FILE

In reply to Patrick Steinhardt
Quoted 14 lines · Patrick Steinhardt

Properly quote the value to fix the issue while using single-quotes to quote the inner value passed to eval. The issue can be reproduced by t0000 with such a path that contains spaces.

---
 t/test-lib.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
t/test-lib.sh +0 -0
index b1a8ee5c002..241198ba95f 100644

t/test-lib.sh then · line 1575

Quoted 5 lines
t/test-lib.sh +1 -1
@@ -1572,7 +1572,7 @@ then
 
 	prepend_var LSAN_OPTIONS : dedup_token_length=9999
 	prepend_var LSAN_OPTIONS : log_exe_name=1
-	prepend_var LSAN_OPTIONS : log_path=\"$TEST_RESULTS_SAN_FILE\"
+	prepend_var LSAN_OPTIONS : log_path="'$TEST_RESULTS_SAN_FILE'"

Makes sense.

Thanks,
Taylor

#5 lore raw

Re: [PATCH 03/10] t/lib-gitweb: test against the build version of gitweb

In reply to Patrick Steinhardt
Quoted 18 lines · Patrick Steinhardt

When testing gitweb we set up the CGI script as "gitweb.perl", which is the source file of the build target "gitweb.cgi". This works alright as long as we run in-tree tests. But we're about to make out-of-tree tests a reality, and there things will break because "gitweb.perl" will not be found in the build directory.

Fix this by using "gitweb.cgi" instead.

---
 t/lib-gitweb.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
t/lib-gitweb.sh +0 -0
index 1f32ca66ea5..c64c87fb87d 100644

t/lib-gitweb.sh EOF · line 52

Quoted 5 lines
t/lib-gitweb.sh +1 -1
@@ -49,7 +49,7 @@ EOF
 		error "Cannot find gitweb at $GITWEB_TEST_INSTALLED."
 		say "# Testing $SCRIPT_NAME"
 	else # normal case, use source version of gitweb
-		SCRIPT_NAME="$GIT_BUILD_DIR/gitweb/gitweb.perl"
+		SCRIPT_NAME="$GIT_BUILD_DIR/gitweb/gitweb.cgi"

This one I am a bit confused above. Could the same thing not be accomplished with the '$GITWEB_TEST_INSTALLED' variable above?

If not, why not? And likewise, I believe the "source version" part of this comment would need an update.

Thanks,
Taylor

#8 lore raw

Re: [PATCH 04/10] t/lib-gpg: fix setup of GNUPGHOME in MinGW

In reply to Patrick Steinhardt
Quoted 10 lines · Patrick Steinhardt

In "t/lib-gpg.sh" we set up the "GNUPGHOME" environment variable to point to a test-specific directory. This is done by using "$PWD/gpghome" as value, where "$PWD" is the current test's trash directory.

This is broken for MinGW though because "$PWD" will use Windows-style paths that contain drive letters. What we really want in this context is a Unix-style path, which we can get by using $(pwd) instead. It is somewhat puzzling that nobody ever hit this issue, but it may easily be that nobody ever tests on Windows with GnuPG installed, which would make us skip those tests.

Heh. I am sure that the story of finding such a failure was probably an interesting one as a result ;-).

Quoted 12 lines

Adapt the code accordingly to fix tests using this library.

---
 t/lib-gpg.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
t/lib-gpg.sh +0 -0
index add11e88fc0..3845b6ac449 100644

t/lib-gpg.sh line 9

Quoted 5 lines
t/lib-gpg.sh +1 -1
@@ -6,7 +6,7 @@
 #   executed in an eval'ed subshell that changes the working directory to a
 #   temporary one.
 
-GNUPGHOME="$PWD/gpghome"
+GNUPGHOME="$(pwd)/gpghome"

Makes sense. Thanks for finding and fixing.

Thanks,
Taylor

#14 lore raw

Re: [PATCH 07/10] t5500, t5601: skip tests which exercise paths with '[::1]' on Cygwin

In reply to Patrick Steinhardt

t/t5500-fetch-pack.sh do · line 815

Quoted 12 lines · Patrick Steinhardt
t/t5500-fetch-pack.sh +7 -1
@@ -806,11 +806,17 @@ do
 	p=ssh
 	for h in host [::1]
 	do
-		test_expect_success "fetch-pack --diag-url $h:$r" '
+		expectation="success"
+		if test_have_prereq CYGWIN && test "$h" = "[::1]"
+		then
+			expectation="failure"
+		fi
+
+		test_expect_$expectation "fetch-pack --diag-url $h:$r" '

Very clever :-).

I am not sure what is going on here since I have almost no familiarity with Cygwin. But this feels like the sort of thing that Johannes Schindelin (CC'd) would know like the back of their hand, so perhaps he can illuminate it for us.

Thanks,
Taylor

#17 lore raw

Re: [PATCH 09/10] builtin/credential-cache: fix missing parameter for stub function

In reply to Patrick Steinhardt
Quoted 19 lines · Patrick Steinhardt

When not compiling the credential cache we may use a stub function for cmd_credential_cache(). With commit 9b1cb5070f (builtin: add a repository parameter for builtin functions, 2024-09-13), we have added a new parameter to all of those top-level cmd_*() functions, and did indeed adapt the non-stubbed-out cmd_credential_cache(). But we didn't adapt the stubbed-out variant, so the code does not compile.

Fix this by adding the missing parameter.

---
 builtin/credential-cache.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
builtin/credential-cache.c +0 -0
index 5de8b9123bf..c51f7fc2ade 100644

builtin/credential-cache.c int cmd_credential_cache(int argc, · line 193

Quoted 6 lines
builtin/credential-cache.c +2 -1
@@ -189,7 +189,8 @@ int cmd_credential_cache(int argc,
 
 #else
 
-int cmd_credential_cache(int argc, const char **argv, const char *prefix)
+int cmd_credential_cache(int argc, const char **argv, const char *prefix,
+			 struct repository *repo UNUSED)

Oops. Very good catch, thanks.

Thanks,
Taylor

#19 lore raw

Re: [PATCH 10/10] http: fix build error on FreeBSD

In reply to Patrick Steinhardt
Quoted 9 lines · Patrick Steinhardt

The result parameter passed to http_request_reauth() may either point to a struct strbuf or a FILE *, where the target parameter tells us which of either it actually is. To accommodate for both types the pointer is a void *, which we then pass directly to functions without doing a cast.

This is fine on most platforms, but it breaks on FreeBSD because fileno() is implemented as a macro that tries to directly access the FILE * structure.

Hah. I'm sure this was another fun debugging story ;-). Reading this made me chuckle aloud, but the fix you wrote here makes perfect sense to me.

Thanks,
Taylor

#10 lore raw

Re: [PATCH 05/10] t1401: make invocation of tar(1) work with Win32-provided one

In reply to Patrick Steinhardt
Quoted 8 lines · Patrick Steinhardt

Windows nowadays provides a tar(1) binary in "C:\Windows\system32". This version of tar(1) doesn't seem to handle the case where directory paths end with a trailing forward slash. And as we do that in t1401 the result is that the test fails.

Drop the trailing slash. Other tests that use tar(1) work alright, this is the only instance where it has been failing.
---

Missing sign-off.

#11 lore raw

Re: [PATCH 05/10] t1401: make invocation of tar(1) work with Win32-provided one

Quoted 11 lines · Eric Sunshine
Quoted 8 lines · Patrick Steinhardt

Windows nowadays provides a tar(1) binary in "C:\Windows\system32". This version of tar(1) doesn't seem to handle the case where directory paths end with a trailing forward slash. And as we do that in t1401 the result is that the test fails.

Drop the trailing slash. Other tests that use tar(1) work alright, this is the only instance where it has been failing.
---

Missing sign-off.

Very good catch. It sounds like we will see a reroll of this series, then.

Thanks,
Taylor

#6 lore raw

Re: [PATCH 03/10] t/lib-gitweb: test against the build version of gitweb

In reply to Taylor Blau
Quoted 28 lines · Taylor Blau
Quoted 18 lines · Patrick Steinhardt

When testing gitweb we set up the CGI script as "gitweb.perl", which is the source file of the build target "gitweb.cgi". This works alright as long as we run in-tree tests. But we're about to make out-of-tree tests a reality, and there things will break because "gitweb.perl" will not be found in the build directory.

Fix this by using "gitweb.cgi" instead.

---
 t/lib-gitweb.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
t/lib-gitweb.sh +0 -0
index 1f32ca66ea5..c64c87fb87d 100644

t/lib-gitweb.sh EOF · line 52

Quoted 5 lines
t/lib-gitweb.sh +1 -1
@@ -49,7 +49,7 @@ EOF
 		error "Cannot find gitweb at $GITWEB_TEST_INSTALLED."
 		say "# Testing $SCRIPT_NAME"
 	else # normal case, use source version of gitweb
-		SCRIPT_NAME="$GIT_BUILD_DIR/gitweb/gitweb.perl"
+		SCRIPT_NAME="$GIT_BUILD_DIR/gitweb/gitweb.cgi"

This one I am a bit confused above. Could the same thing not be accomplished with the '$GITWEB_TEST_INSTALLED' variable above?

Quoted 2 lines

If not, why not? And likewise, I believe the "source version" part of this comment would need an update.

In theory we can, but it goes against the spirit of its intent. We do not want to test against an installed version, we want to test against the built version. And the built version is not "gitweb.perl", which still has the wrong shebang and replacement markers inside, but it is "gitweb.cgi".

This used to work because with our Makefile, "gitweb.cgi" and "gitweb.perl" live next to each other in the same directory. But as soon as "$GIT_BUILD_DIR" points somewhere else that is not the source tree, it fails because we cannot find "gitweb.perl" anymore.

Now this change here means that you cannot test gitweb without building Git first anymore. But I doubt that really worked in practice anyway, and I also doubt that this is something we want to cater towards in the first place.

I'll update the commit message accordinly.

Patrick

#20 lore raw

Re: [PATCH 10/10] http: fix build error on FreeBSD

In reply to Taylor Blau
Quoted 7 lines · Taylor Blau
Quoted 6 lines · Patrick Steinhardt

The result parameter passed to http_request_reauth() may either point to a struct strbuf or a FILE *, where the target parameter tells us which of either it actually is. To accommodate for both types the pointer is a void *, which we then pass directly to functions without doing a cast.

Quoted 7 lines
Quoted 3 lines

This is fine on most platforms, but it breaks on FreeBSD because fileno() is implemented as a macro that tries to directly access the FILE * structure.

Hah. I'm sure this was another fun debugging story ;-). Reading this made me chuckle aloud, but the fix you wrote here makes perfect sense to me.

I'm sure I've either got less hair or grown more gray hair over the last couple weeks due to all of these weird compatibility issues. Kind of puzzling that either nobody else has hit those issues, or alternatively that nobody seems to care. *shrug*

Patrick

#21 lore raw

v2 cover Platform compatibility fixes

In reply to Patrick Steinhardt

Hi,

this is the second version of my patch series that includes various different platform compatibility fixes.

Changes compared to v1:

- Provide a more in-depth explanation of the gitweb fix.

- Fix up a stale comment in "lib-gitweb.sh".

- Add a missing signoff.

Thanks!

Patrick

Patrick Steinhardt (10):
t/test-lib: fix quoting of TEST_RESULTS_SAN_FILE t/test-lib: wire up NO_ICONV prerequisite
t/lib-gitweb: test against the build version of gitweb t/lib-gpg: fix setup of GNUPGHOME in MinGW
t1401: make invocation of tar(1) work with Win32-provided one
t3404: work around platform-specific behaviour on macOS 10.15 t5500, t5601: skip tests which exercise paths with '[::1]' on Cygwin
t7300: work around platform-specific behaviour with long paths on
MinGW
builtin/credential-cache: fix missing parameter for stub function
http: fix build error on FreeBSD

 Makefile                                     |   1 +
 builtin/credential-cache.c                   |   3 +-
 contrib/buildsystems/CMakeLists.txt          |   6 ++
 http.c                                       |  10 +-
 t/lib-gitweb.sh                              |   4 +-
 t/lib-gpg.sh                                 |   2 +-
 t/t0028-working-tree-encoding.sh             |   6 ++
 t/t1401-symbolic-ref.sh                      |   2 +-
 t/t2082-parallel-checkout-attributes.sh      |   2 +-
 t/t3404-rebase-interactive.sh                |  26 +++--
 t/t3434-rebase-i18n.sh                       |   6 ++
 t/t3900-i18n-commit.sh                       |   6 ++
 t/t3901-i18n-patch.sh                        |   6 ++
 t/t4041-diff-submodule-option.sh             |  16 ++-
 t/t4059-diff-submodule-not-initialized.sh    |  16 ++-
 t/t4060-diff-submodule-option-diff-format.sh |  17 ++--
 t/t4201-shortlog.sh                          |   8 +-
 t/t4205-log-pretty-formats.sh                | 102 +++++++++++--------
 t/t4210-log-i18n.sh                          |   6 ++
 t/t4254-am-corrupt.sh                        |   6 ++
 t/t5100-mailinfo.sh                          |  14 ++-
 t/t5500-fetch-pack.sh                        |  14 ++-
 t/t5550-http-fetch-dumb.sh                   |   4 +-
 t/t5601-clone.sh                             |  11 +-
 t/t6006-rev-list-format.sh                   |  54 ++++++----
 t/t7102-reset.sh                             |  40 +++++---
 t/t7300-clean.sh                             |   2 +-
 t/t8005-blame-i18n.sh                        |   6 ++
 t/t9300-fast-import.sh                       |   2 +-
 t/t9350-fast-export.sh                       |  10 +-
 t/test-lib.sh                                |   3 +-
 31 files changed, 274 insertions(+), 137 deletions(-)

Range-diff against v1:

 1:  601670912ee =  1:  a514f5d14a7 t/test-lib: fix quoting of TEST_RESULTS_SAN_FILE
 2:  f1d74c46aa2 =  2:  f6a8582c34a t/test-lib: wire up NO_ICONV prerequisite
 3:  eda73d81139 !  3:  2e2d208ef1b t/lib-gitweb: test against the build version of gitweb

@@ Commit message
t/lib-gitweb: test against the build version of gitweb

When testing gitweb we set up the CGI script as "gitweb.perl", which is
- the source file of the build target "gitweb.cgi". This works alright as
- long as we run in-tree tests. But we're about to make out-of-tree tests
- a reality, and there things will break because "gitweb.perl" will not be
- found in the build directory.
+ the source file of the build target "gitweb.cgi". This file doesn't have
+ a patched shebang and still contains ++REPLACEMENT++ markers, but
+ things generally work because we replace the configuration with our own
+ test configuration.

- Fix this by using "gitweb.cgi" instead.
+ But this only works as long as "$GIT_BUILD_DIR" actually points to the
+ source tree, because "gitweb.cgi" and "gitweb.perl" happen to sit next
+ to each other. This is not the case though once you have out-of-tree
+ builds like with CMake, where the source and built versions live in
+ different directories. Consequently, "$GIT_BUILD_DIR/gitweb/gitweb.perl"
+ won't exist there.
+
+ While we could ask build systems with out-of-tree builds to instead set
+ up GITWEB_TEST_INSTALLED, which allows us to override the location of
+ the script, it goes against the spirit of this environment variable. We
+ _don't_ want to test against an installed version, we want to use the
+ version we have just built.
+
+ Fix this by using "gitweb.cgi" instead. This means that you cannot run
+ test scripts without building that file, but in general we do expect
+ developers to build stuff before they test it anyway.

## t/lib-gitweb.sh ##
@@ t/lib-gitweb.sh: EOF
+ test -f "$SCRIPT_NAME" ||
error "Cannot find gitweb at $GITWEB_TEST_INSTALLED." say "# Testing $SCRIPT_NAME"
- else # normal case, use source version of gitweb
+- else # normal case, use source version of gitweb
- SCRIPT_NAME="$GIT_BUILD_DIR/gitweb/gitweb.perl"
++ else # normal case, use built version of gitweb
+ SCRIPT_NAME="$GIT_BUILD_DIR/gitweb/gitweb.cgi"
fi
export SCRIPT_NAME

 4:  f5700647839 =  4:  cc94f6fa648 t/lib-gpg: fix setup of GNUPGHOME in MinGW
 5:  d082cd92af3 !  5:  58691dd652b t1401: make invocation of tar(1) work with Win32-provided one

@@ Commit message
Drop the trailing slash. Other tests that use tar(1) work alright, this is the only instance where it has been failing.

+ Signed-off-by: Patrick Steinhardt <[email protected]>
+
## t/t1401-symbolic-ref.sh ##
@@ t/t1401-symbolic-ref.sh: reset_to_sane() {
test_expect_success 'setup' '

 6:  a82a6b3df70 =  6:  1daadd82766 t3404: work around platform-specific behaviour on macOS 10.15
 7:  e1b9617f943 =  7:  374f47bf3de t5500, t5601: skip tests which exercise paths with '[::1]' on Cygwin
 8:  b94fd876adb =  8:  13c06a8129b t7300: work around platform-specific behaviour with long paths on MinGW
 9:  8cce69e5ba6 =  9:  5fd78b6d535 builtin/credential-cache: fix missing parameter for stub function
10:  deb30e12a58 = 10:  9963dc73988 http: fix build error on FreeBSD

base-commit: ef8ce8f3d4344fd3af049c17eeba5cd20d98b69f

Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
#22 lore raw

PATCH v2 1/10 t/test-lib: fix quoting of TEST_RESULTS_SAN_FILE

When assembling our LSAN_OPTIONS that configure the leak sanitizer we end up prepending the string with various different colon-separated options via calls to prepend_var. One of the settings we add is the path where the sanitizer should store logs, which can be an arbitrary filesystem path.

Naturally, filesystem paths may contain whitespace characters. And while it does seem as if we were quoting the value, we use escaped quotes and consequently split up the value if it does contain spaces. This leads to the following error in t0000 when having a value with whitespaces:

.../t/test-lib.sh: eval: line 64: unexpected EOF while looking for matching `"' ++ return 1
error: last command exited with $?=1
not ok 5 - subtest: 3 passing tests

The error itself is a bit puzzling at first. The basic problem is that the code sees the leading escaped quote during eval, but because we truncate everything after the space character it doesn't see the trailing escaped quote and thus fails to parse the string.

Properly quote the value to fix the issue while using single-quotes to quote the inner value passed to eval. The issue can be reproduced by t0000 with such a path that contains spaces.

---
 t/test-lib.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
t/test-lib.sh +1 -1
index b1a8ee5c002..241198ba95f 100644
@@ -1572,7 +1572,7 @@ then
 
 	prepend_var LSAN_OPTIONS : dedup_token_length=9999
 	prepend_var LSAN_OPTIONS : log_exe_name=1
-	prepend_var LSAN_OPTIONS : log_path=\"$TEST_RESULTS_SAN_FILE\"
+	prepend_var LSAN_OPTIONS : log_path="'$TEST_RESULTS_SAN_FILE'"
 	export LSAN_OPTIONS
 
 elif test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check" ||
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
#23 lore raw

PATCH v2 2/10 t/test-lib: wire up NO_ICONV prerequisite

In reply to Patrick Steinhardt

The iconv library is used by Git to reencode files, commit messages and other things. As such it is a rather integral part, but given that many platforms nowadays use UTF-8 everywhere you can live without support for reencoding in many situations. It is thus optional to build Git with iconv, and some of our platforms wired up in "config.mak.uname" disable it. But while we support building without it, running our test suite with "NO_ICONV=Yes" causes many test failures.

Wire up a new test prerequisite ICONV that gets populated via our GIT-BUILD-OPTIONS. Annotate failing tests accordingly.

Note that this commit does not do a deep dive into every single test to assess whether the failure is expected or not. Most of the tests do smell like the expected kind of failure though.

Further note that there are several "!MINGW" conditions in t4201, and all of these fail due to iconv-related errors. This is quite likely a leftover from times before dce7d29551 (msvc: support building Git using MS Visual C++, 2019-06-25), which switched Windows-based builds over from "NO_ICONV=YesPlease" to "NEEDS_LIBICONV=YesPlease". Consequently, adapt those tests to also use the new ICONV prerequisite.

---
 Makefile                                     |   1 +
 contrib/buildsystems/CMakeLists.txt          |   6 ++
 t/t0028-working-tree-encoding.sh             |   6 ++
 t/t2082-parallel-checkout-attributes.sh      |   2 +-
 t/t3434-rebase-i18n.sh                       |   6 ++
 t/t3900-i18n-commit.sh                       |   6 ++
 t/t3901-i18n-patch.sh                        |   6 ++
 t/t4041-diff-submodule-option.sh             |  16 ++-
 t/t4059-diff-submodule-not-initialized.sh    |  16 ++-
 t/t4060-diff-submodule-option-diff-format.sh |  17 ++--
 t/t4201-shortlog.sh                          |   8 +-
 t/t4205-log-pretty-formats.sh                | 102 +++++++++++--------
 t/t4210-log-i18n.sh                          |   6 ++
 t/t4254-am-corrupt.sh                        |   6 ++
 t/t5100-mailinfo.sh                          |  14 ++-
 t/t5550-http-fetch-dumb.sh                   |   4 +-
 t/t6006-rev-list-format.sh                   |  54 ++++++----
 t/t7102-reset.sh                             |  40 +++++---
 t/t8005-blame-i18n.sh                        |   6 ++
 t/t9300-fast-import.sh                       |   2 +-
 t/t9350-fast-export.sh                       |  10 +-
 t/test-lib.sh                                |   1 +
 22 files changed, 229 insertions(+), 106 deletions(-)
Makefile +1 -0
index feeed6f9321..5db10347341 100644
@@ -3171,6 +3171,7 @@ GIT-BUILD-OPTIONS: FORCE
 	@echo PYTHON_PATH=\''$(subst ','\'',$(PYTHON_PATH_SQ))'\' >>$@+
 	@echo TAR=\''$(subst ','\'',$(subst ','\'',$(TAR)))'\' >>$@+
 	@echo NO_CURL=\''$(subst ','\'',$(subst ','\'',$(NO_CURL)))'\' >>$@+
+	@echo NO_ICONV=\''$(subst ','\'',$(subst ','\'',$(NO_ICONV)))'\' >>$@+
 	@echo NO_EXPAT=\''$(subst ','\'',$(subst ','\'',$(NO_EXPAT)))'\' >>$@+
 	@echo USE_LIBPCRE2=\''$(subst ','\'',$(subst ','\'',$(USE_LIBPCRE2)))'\' >>$@+
 	@echo NO_PERL=\''$(subst ','\'',$(subst ','\'',$(NO_PERL)))'\' >>$@+
contrib/buildsystems/CMakeLists.txt +6 -0
index 62af7b33d2f..1384c0eb6d3 100644
@@ -1109,6 +1109,7 @@ set(DIFF diff)
 set(PYTHON_PATH /usr/bin/python)
 set(TAR tar)
 set(NO_CURL )
+set(NO_ICONV )
 set(NO_EXPAT )
 set(USE_LIBPCRE2 )
 set(NO_PERL )
@@ -1122,6 +1123,10 @@ if(NOT CURL_FOUND)
 	set(NO_CURL 1)
 endif()
 
+if(NOT Iconv_FOUND)
+	SET(NO_ICONV 1)
+endif()
+
 if(NOT EXPAT_FOUND)
 	set(NO_EXPAT 1)
 endif()
@@ -1145,6 +1150,7 @@ file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS "DIFF='${DIFF}'\n")
 file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS "PYTHON_PATH='${PYTHON_PATH}'\n")
 file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS "TAR='${TAR}'\n")
 file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS "NO_CURL='${NO_CURL}'\n")
+file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS "NO_ICONV='${NO_ICONV}'\n")
 file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS "NO_EXPAT='${NO_EXPAT}'\n")
 file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS "NO_PERL='${NO_PERL}'\n")
 file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS "NO_PTHREADS='${NO_PTHREADS}'\n")
t/t0028-working-tree-encoding.sh +6 -0
index ad151a34670..510da4ca12d 100755
@@ -12,6 +12,12 @@ TEST_CREATE_REPO_NO_TEMPLATE=1
 
 GIT_TRACE_WORKING_TREE_ENCODING=1 && export GIT_TRACE_WORKING_TREE_ENCODING
 
+if ! test_have_prereq ICONV
+then
+	skip_all='skipping working tree encoding tests; iconv not available'
+	test_done
+fi
+
 test_expect_success 'setup test files' '
 	git config core.eol lf &&
 
t/t2082-parallel-checkout-attributes.sh +1 -1
index aec55496eb1..a040aa54cee 100755
@@ -34,7 +34,7 @@ test_expect_success 'parallel-checkout with ident' '
 	)
 '
 
-test_expect_success 'parallel-checkout with re-encoding' '
+test_expect_success ICONV 'parallel-checkout with re-encoding' '
 	set_checkout_config 2 0 &&
 	git init encoding &&
 	(
t/t3434-rebase-i18n.sh +6 -0
index 26a48d6b103..97fc9a23f21 100755
@@ -20,6 +20,12 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
 TEST_PASSES_SANITIZE_LEAK=true
 . ./test-lib.sh
 
+if ! test_have_prereq ICONV
+then
+	skip_all='skipping rebase i18n tests; iconv not available'
+	test_done
+fi
+
 compare_msg () {
 	iconv -f "$2" -t "$3" "$TEST_DIRECTORY/t3434/$1" >expect &&
 	git cat-file commit HEAD >raw &&
t/t3900-i18n-commit.sh +6 -0
index db7b403bc15..9d4b5ab1f95 100755
@@ -8,6 +8,12 @@ test_description='commit and log output encodings'
 TEST_PASSES_SANITIZE_LEAK=true
 . ./test-lib.sh
 
+if ! test_have_prereq ICONV
+then
+	skip_all='skipping commit i18n tests; iconv not available'
+	test_done
+fi
+
 compare_with () {
 	git show -s $1 | sed -e '1,/^$/d' -e 's/^    //' >current &&
 	case "$3" in
t/t3901-i18n-patch.sh +6 -0
index 5f0b9afc3fa..e0659c92935 100755
@@ -11,6 +11,12 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
 TEST_PASSES_SANITIZE_LEAK=true
 . ./test-lib.sh
 
+if ! test_have_prereq ICONV
+then
+	skip_all='skipping patch i18n tests; iconv not available'
+	test_done
+fi
+
 check_encoding () {
 	# Make sure characters are not corrupted
 	cnt="$1" header="$2" i=1 j=0
t/t4041-diff-submodule-option.sh +11 -5
index 8fc40e75eb3..aa149e0085e 100755
@@ -15,12 +15,18 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
 TEST_PASSES_SANITIZE_LEAK=true
 . ./test-lib.sh
 
-# Tested non-UTF-8 encoding
-test_encoding="ISO8859-1"
+# Test non-UTF-8 encoding in case iconv is available.
+if test_have_prereq ICONV
+then
+	test_encoding="ISO8859-1"
+	# String "added" in German (translated with Google Translate), encoded in UTF-8,
+	# used in sample commit log messages in add_file() function below.
+	added=$(printf "hinzugef\303\274gt")
+else
+	test_encoding="UTF-8"
+	added="added"
+fi
 
-# String "added" in German (translated with Google Translate), encoded in UTF-8,
-# used in sample commit log messages in add_file() function below.
-added=$(printf "hinzugef\303\274gt")
 add_file () {
 	(
 		cd "$1" &&
t/t4059-diff-submodule-not-initialized.sh +11 -5
index 668f5263038..28fd3cdb154 100755
@@ -12,12 +12,18 @@ initialized previously but the checkout has since been removed.
 TEST_PASSES_SANITIZE_LEAK=true
 . ./test-lib.sh
 
-# Tested non-UTF-8 encoding
-test_encoding="ISO8859-1"
 
-# String "added" in German (translated with Google Translate), encoded in UTF-8,
-# used in sample commit log messages in add_file() function below.
-added=$(printf "hinzugef\303\274gt")
+# Test non-UTF-8 encoding in case iconv is available.
+if test_have_prereq ICONV
+then
+	test_encoding="ISO8859-1"
+	# String "added" in German (translated with Google Translate), encoded in UTF-8,
+	# used in sample commit log messages in add_file() function below.
+	added=$(printf "hinzugef\303\274gt")
+else
+	test_encoding="UTF-8"
+	added="added"
+fi
 
 add_file () {
 	(
t/t4060-diff-submodule-option-diff-format.sh +11 -6
index 8ce67442d96..918334fa4c8 100755
@@ -13,12 +13,17 @@ This test tries to verify the sanity of --submodule=diff option of git diff.
 TEST_PASSES_SANITIZE_LEAK=true
 . ./test-lib.sh
 
-# Tested non-UTF-8 encoding
-test_encoding="ISO8859-1"
-
-# String "added" in German (translated with Google Translate), encoded in UTF-8,
-# used in sample commit log messages in add_file() function below.
-added=$(printf "hinzugef\303\274gt")
+# Test non-UTF-8 encoding in case iconv is available.
+if test_have_prereq ICONV
+then
+	test_encoding="ISO8859-1"
+	# String "added" in German (translated with Google Translate), encoded in UTF-8,
+	# used in sample commit log messages in add_file() function below.
+	added=$(printf "hinzugef\303\274gt")
+else
+	test_encoding="UTF-8"
+	added="added"
+fi
 
 add_file () {
 	(
t/t4201-shortlog.sh +4 -4
index c20c8857244..680e707ba1e 100755
@@ -105,7 +105,7 @@ test_expect_success 'output from user-defined format is re-wrapped' '
 	test_cmp expect log.predictable
 '
 
-test_expect_success !MINGW 'shortlog wrapping' '
+test_expect_success ICONV 'shortlog wrapping' '
 	cat >expect <<\EOF &&
 A U Thor (5):
       Test
@@ -126,13 +126,13 @@ EOF
 	test_cmp expect out
 '
 
-test_expect_success !MINGW 'shortlog from non-git directory' '
+test_expect_success ICONV 'shortlog from non-git directory' '
 	git log --no-expand-tabs HEAD >log &&
 	GIT_DIR=non-existing git shortlog -w <log >out &&
 	test_cmp expect out
 '
 
-test_expect_success !MINGW 'shortlog can read --format=raw output' '
+test_expect_success ICONV 'shortlog can read --format=raw output' '
 	git log --format=raw HEAD >log &&
 	GIT_DIR=non-existing git shortlog -w <log >out &&
 	test_cmp expect out
@@ -182,7 +182,7 @@ $DSCHO (2):
 
 EOF
 
-test_expect_success !MINGW 'shortlog encoding' '
+test_expect_success ICONV 'shortlog encoding' '
 	git reset --hard "$commit" &&
 	git config --unset i18n.commitencoding &&
 	echo 2 > a1 &&
t/t4205-log-pretty-formats.sh +59 -43

310 lines not on this page · raw message

t/t4210-log-i18n.sh +6 -0
index 7120030b5c6..4a12b2b4979 100755
@@ -5,6 +5,12 @@ test_description='test log with i18n features'
 TEST_PASSES_SANITIZE_LEAK=true
 . ./lib-gettext.sh
 
+if ! test_have_prereq ICONV
+then
+	skip_all='skipping log i18n tests; iconv not available'
+	test_done
+fi
+
 # two forms of é
 utf8_e=$(printf '\303\251')
 latin1_e=$(printf '\351')
t/t4254-am-corrupt.sh +6 -0
index 661feb60709..cb03522d021 100755
@@ -5,6 +5,12 @@ test_description='git am with corrupt input'
 TEST_PASSES_SANITIZE_LEAK=true
 . ./test-lib.sh
 
+if ! test_have_prereq ICONV
+then
+	skip_all='skipping am encoding corruption tests; iconv not available'
+	test_done
+fi
+
 make_mbox_with_nul () {
 	space=' '
 	q_nul_in_subject=
t/t5100-mailinfo.sh +12 -2
index 065156c1f39..23b2f218725 100755
@@ -28,7 +28,12 @@ check_mailinfo () {
 
 for mail in 00*
 do
-	test_expect_success "mailinfo $mail" '
+	case "$mail" in
+	0004)
+		prereq=ICONV;;
+	esac
+
+	test_expect_success $prereq "mailinfo $mail" '
 		check_mailinfo "$mail" "" &&
 		if test -f "$DATA/msg$mail--scissors"
 		then
@@ -56,7 +61,12 @@ test_expect_success 'split box with rfc2047 samples' \
 
 for mail in rfc2047/00*
 do
-	test_expect_success "mailinfo $mail" '
+	case "$mail" in
+	rfc2047/0001)
+		prereq=ICONV;;
+	esac
+
+	test_expect_success $prereq "mailinfo $mail" '
 		git mailinfo -u "$mail-msg" "$mail-patch" <"$mail" >"$mail-info" &&
 		echo msg &&
 		test_cmp "$DATA/empty" "$mail-msg" &&
t/t5550-http-fetch-dumb.sh +2 -2
index 58189c9f7dc..3c873de17ec 100755
@@ -344,12 +344,12 @@ test_expect_success 'git client shows text/plain with a charset' '
 	grep "this is the error message" stderr
 '
 
-test_expect_success 'http error messages are reencoded' '
+test_expect_success ICONV 'http error messages are reencoded' '
 	test_must_fail git clone "$HTTPD_URL/error/utf16" 2>stderr &&
 	grep "this is the error message" stderr
 '
 
-test_expect_success 'reencoding is robust to whitespace oddities' '
+test_expect_success ICONV 'reencoding is robust to whitespace oddities' '
 	test_must_fail git clone "$HTTPD_URL/error/odd-spacing" 2>stderr &&
 	grep "this is the error message" stderr
 '
t/t6006-rev-list-format.sh +37 -17

72 lines not on this page · raw message

t/t7102-reset.sh +25 -15
index 2add26d7684..e9a6cc72658 100755
@@ -13,21 +13,31 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
 TEST_PASSES_SANITIZE_LEAK=true
 . ./test-lib.sh
 
-commit_msg () {
-	# String "modify 2nd file (changed)" partly in German
-	# (translated with Google Translate),
-	# encoded in UTF-8, used as a commit log message below.
-	msg="modify 2nd file (ge\303\244ndert)\n"
-	if test -n "$1"
-	then
-		printf "$msg" | iconv -f utf-8 -t "$1"
-	else
-		printf "$msg"
-	fi
-}
-
-# Tested non-UTF-8 encoding
-test_encoding="ISO8859-1"
+if test_have_prereq ICONV
+then
+	commit_msg () {
+		# String "modify 2nd file (changed)" partly in German
+		# (translated with Google Translate),
+		# encoded in UTF-8, used as a commit log message below.
+		msg="modify 2nd file (ge\303\244ndert)\n"
+		if test -n "$1"
+		then
+			printf "$msg" | iconv -f utf-8 -t "$1"
+		else
+			printf "$msg"
+		fi
+	}
+
+	# Tested non-UTF-8 encoding
+	test_encoding="ISO8859-1"
+else
+	commit_msg () {
+		echo "modify 2nd file (geandert)"
+	}
+
+	# Tested non-UTF-8 encoding
+	test_encoding="UTF-8"
+fi
 
 test_expect_success 'creating initial files and commits' '
 	test_tick &&
t/t8005-blame-i18n.sh +6 -0
index 75da219ed1b..7a1f581c240 100755
@@ -3,6 +3,12 @@
 test_description='git blame encoding conversion'
 . ./test-lib.sh
 
+if ! test_have_prereq ICONV
+then
+	skip_all='skipping blame i18n tests; iconv not available'
+	test_done
+fi
+
 . "$TEST_DIRECTORY"/t8005/utf8.txt
 . "$TEST_DIRECTORY"/t8005/euc-japan.txt
 . "$TEST_DIRECTORY"/t8005/sjis.txt
t/t9300-fast-import.sh +1 -1
index 3b3c371740a..6224f54d4d2 100755
@@ -3676,7 +3676,7 @@ test_expect_success !MINGW 'W: get-mark & empty orphan commit with erroneous thi
 ### series X (other new features)
 ###
 
-test_expect_success 'X: handling encoding' '
+test_expect_success ICONV 'X: handling encoding' '
 	test_tick &&
 	cat >input <<-INPUT_END &&
 	commit refs/heads/encoding
t/t9350-fast-export.sh +5 -5
index 2bdc02b4599..9595dfef2ee 100755
@@ -125,7 +125,7 @@ test_expect_success 'fast-export --show-original-ids | git fast-import' '
 	test $MUSS = $(git rev-parse --verify refs/tags/muss)
 '
 
-test_expect_success 'reencoding iso-8859-7' '
+test_expect_success ICONV 'reencoding iso-8859-7' '
 
 	test_when_finished "git reset --hard HEAD~1" &&
 	test_config i18n.commitencoding iso-8859-7 &&
@@ -421,7 +421,7 @@ M 100644 :1 there
 
 EOF
 
-test_expect_success 'dropping tag of filtered out object' '
+test_expect_success ICONV 'dropping tag of filtered out object' '
 (
 	cd limit-by-paths &&
 	git fast-export --tag-of-filtered-object=drop mytag -- there > output &&
@@ -438,7 +438,7 @@ msg
 
 EOF
 
-test_expect_success 'rewriting tag of filtered out object' '
+test_expect_success ICONV 'rewriting tag of filtered out object' '
 (
 	cd limit-by-paths &&
 	git fast-export --tag-of-filtered-object=rewrite mytag -- there > output &&
@@ -667,7 +667,7 @@ M 100644 :13 file
 
 EOF
 
-test_expect_success 'avoid uninteresting refs' '
+test_expect_success ICONV 'avoid uninteresting refs' '
 	> tmp-marks &&
 	git fast-export --import-marks=tmp-marks \
 		--export-marks=tmp-marks main > /dev/null &&
@@ -686,7 +686,7 @@ from :14
 
 EOF
 
-test_expect_success 'refs are updated even if no commits need to be exported' '
+test_expect_success ICONV 'refs are updated even if no commits need to be exported' '
 	> tmp-marks &&
 	git fast-export --import-marks=tmp-marks \
 		--export-marks=tmp-marks main > /dev/null &&
t/test-lib.sh +1 -0
index 241198ba95f..a278181a056 100644
@@ -1743,6 +1743,7 @@ esac
 
 ( COLUMNS=1 && test $COLUMNS = 1 ) && test_set_prereq COLUMNS_CAN_BE_1
 test -z "$NO_CURL" && test_set_prereq LIBCURL
+test -z "$NO_ICONV" && test_set_prereq ICONV
 test -z "$NO_PERL" && test_set_prereq PERL
 test -z "$NO_PTHREADS" && test_set_prereq PTHREADS
 test -z "$NO_PYTHON" && test_set_prereq PYTHON
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
#29 lore raw

PATCH v2 3/10 t/lib-gitweb: test against the build version of gitweb

In reply to Patrick Steinhardt

When testing gitweb we set up the CGI script as "gitweb.perl", which is the source file of the build target "gitweb.cgi". This file doesn't have a patched shebang and still contains ++REPLACEMENT++ markers, but things generally work because we replace the configuration with our own test configuration.

But this only works as long as "$GIT_BUILD_DIR" actually points to the source tree, because "gitweb.cgi" and "gitweb.perl" happen to sit next to each other. This is not the case though once you have out-of-tree builds like with CMake, where the source and built versions live in different directories. Consequently, "$GIT_BUILD_DIR/gitweb/gitweb.perl" won't exist there.

While we could ask build systems with out-of-tree builds to instead set up GITWEB_TEST_INSTALLED, which allows us to override the location of the script, it goes against the spirit of this environment variable. We _don't_ want to test against an installed version, we want to use the version we have just built.

Fix this by using "gitweb.cgi" instead. This means that you cannot run test scripts without building that file, but in general we do expect developers to build stuff before they test it anyway.

---
 t/lib-gitweb.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
t/lib-gitweb.sh +2 -2
index 1f32ca66ea5..7f9808ec202 100644
@@ -48,8 +48,8 @@ EOF
 		test -f "$SCRIPT_NAME" ||
 		error "Cannot find gitweb at $GITWEB_TEST_INSTALLED."
 		say "# Testing $SCRIPT_NAME"
-	else # normal case, use source version of gitweb
-		SCRIPT_NAME="$GIT_BUILD_DIR/gitweb/gitweb.perl"
+	else # normal case, use built version of gitweb
+		SCRIPT_NAME="$GIT_BUILD_DIR/gitweb/gitweb.cgi"
 	fi
 	export SCRIPT_NAME
 }
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
#30 lore raw

PATCH v2 4/10 t/lib-gpg: fix setup of GNUPGHOME in MinGW

In reply to Patrick Steinhardt

In "t/lib-gpg.sh" we set up the "GNUPGHOME" environment variable to point to a test-specific directory. This is done by using "$PWD/gpghome" as value, where "$PWD" is the current test's trash directory.

This is broken for MinGW though because "$PWD" will use Windows-style paths that contain drive letters. What we really want in this context is a Unix-style path, which we can get by using $(pwd) instead. It is somewhat puzzling that nobody ever hit this issue, but it may easily be that nobody ever tests on Windows with GnuPG installed, which would make us skip those tests.

Adapt the code accordingly to fix tests using this library.

---
 t/lib-gpg.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
t/lib-gpg.sh +1 -1
index add11e88fc0..3845b6ac449 100644
@@ -6,7 +6,7 @@
 #   executed in an eval'ed subshell that changes the working directory to a
 #   temporary one.
 
-GNUPGHOME="$PWD/gpghome"
+GNUPGHOME="$(pwd)/gpghome"
 export GNUPGHOME
 
 test_lazy_prereq GPG '
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
#32 lore raw

PATCH v2 5/10 t1401: make invocation of tar(1) work with Win32-provided one

In reply to Patrick Steinhardt

Windows nowadays provides a tar(1) binary in "C:\Windows\system32". This version of tar(1) doesn't seem to handle the case where directory paths end with a trailing forward slash. And as we do that in t1401 the result is that the test fails.

Drop the trailing slash. Other tests that use tar(1) work alright, this is the only instance where it has been failing.

---
 t/t1401-symbolic-ref.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
t/t1401-symbolic-ref.sh +1 -1
index 5c60d6f812d..90af3f955c0 100755
@@ -16,7 +16,7 @@ reset_to_sane() {
 test_expect_success 'setup' '
 	git symbolic-ref HEAD refs/heads/foo &&
 	test_commit file &&
-	"$TAR" cf .git.tar .git/
+	"$TAR" cf .git.tar .git
 '
 
 test_expect_success 'symbolic-ref read/write roundtrip' '
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
#34 lore raw

PATCH v2 6/10 t3404: work around platform-specific behaviour on macOS 10.15

In reply to Patrick Steinhardt

Two of our tests in t3404 use indented HERE docs where leading tabs on some of the lines are actually relevant. The tabs do get removed though, and we try to fix this up by using sed(1) to replace leading tabs in the actual output, as well. But macOS 10.15 uses an oldish version of sed(1) that has BSD lineage, which does not understand "\t", and thus we fail to strip those leading tabs and fail the test.

Address this issue by using q_to_tab such that we do not have to strip leading tabs from the actual output.

---
 t/t3404-rebase-interactive.sh | 26 ++++++++++++--------------
 1 file changed, 12 insertions(+), 14 deletions(-)
t/t3404-rebase-interactive.sh +12 -14
index f171af3061d..7ce75237803 100755
@@ -1917,18 +1917,17 @@ test_expect_success '--update-refs updates refs correctly' '
 	test_cmp_rev HEAD~1 refs/heads/third &&
 	test_cmp_rev HEAD refs/heads/no-conflict-branch &&
 
-	cat >expect <<-\EOF &&
+	q_to_tab >expect <<-\EOF &&
 	Successfully rebased and updated refs/heads/update-refs.
 	Updated the following refs with --update-refs:
-		refs/heads/first
-		refs/heads/no-conflict-branch
-		refs/heads/second
-		refs/heads/third
+	Qrefs/heads/first
+	Qrefs/heads/no-conflict-branch
+	Qrefs/heads/second
+	Qrefs/heads/third
 	EOF
 
 	# Clear "Rebasing (X/Y)" progress lines and drop leading tabs.
-	sed -e "s/Rebasing.*Successfully/Successfully/g" -e "s/^\t//g" \
-		<err >err.trimmed &&
+	sed "s/Rebasing.*Successfully/Successfully/g" <err >err.trimmed &&
 	test_cmp expect err.trimmed
 '
 
@@ -2178,19 +2177,18 @@ test_expect_success '--update-refs: check failed ref update' '
 	test_must_fail git rebase --continue 2>err &&
 	grep "update_ref failed for ref '\''refs/heads/second'\''" err &&
 
-	cat >expect <<-\EOF &&
+	q_to_tab >expect <<-\EOF &&
 	Updated the following refs with --update-refs:
-		refs/heads/first
-		refs/heads/no-conflict-branch
-		refs/heads/third
+	Qrefs/heads/first
+	Qrefs/heads/no-conflict-branch
+	Qrefs/heads/third
 	Failed to update the following refs with --update-refs:
-		refs/heads/second
+	Qrefs/heads/second
 	EOF
 
 	# Clear "Rebasing (X/Y)" progress lines and drop leading tabs.
 	tail -n 6 err >err.last &&
-	sed -e "s/Rebasing.*Successfully/Successfully/g" -e "s/^\t//g" \
-		<err.last >err.trimmed &&
+	sed "s/Rebasing.*Successfully/Successfully/g" <err.last >err.trimmed &&
 	test_cmp expect err.trimmed
 '
 
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
#36 lore raw

PATCH v2 7/10 t5500, t5601: skip tests which exercise paths with '[::1]' on Cygwin

In reply to Patrick Steinhardt

Parsing repositories which contain '[::1]' is broken on Cygwin. It seems as if Cygwin is confusing those as drive letter prefixes or something like this, but I couldn't deduce the actual root cause.

Mark those tests as broken for now.

---
 t/t5500-fetch-pack.sh | 14 ++++++++++----
 t/t5601-clone.sh      | 11 +++++++++--
 2 files changed, 19 insertions(+), 6 deletions(-)
t/t5500-fetch-pack.sh +10 -4
index 605f17240c1..416522c86ad 100755
@@ -774,7 +774,7 @@ do
 	# file with scheme
 	for p in file
 	do
-		test_expect_success !MINGW "fetch-pack --diag-url $p://$h/$r" '
+		test_expect_success !WINDOWS "fetch-pack --diag-url $p://$h/$r" '
 			check_prot_path $p://$h/$r $p "/$r"
 		'
 		test_expect_success MINGW "fetch-pack --diag-url $p://$h/$r" '
@@ -784,7 +784,7 @@ do
 			check_prot_path $p:///$r $p "/$r"
 		'
 		# No "/~" -> "~" conversion for file
-		test_expect_success !MINGW "fetch-pack --diag-url $p://$h/~$r" '
+		test_expect_success !WINDOWS "fetch-pack --diag-url $p://$h/~$r" '
 			check_prot_path $p://$h/~$r $p "/~$r"
 		'
 		test_expect_success MINGW "fetch-pack --diag-url $p://$h/~$r" '
@@ -806,11 +806,17 @@ do
 	p=ssh
 	for h in host [::1]
 	do
-		test_expect_success "fetch-pack --diag-url $h:$r" '
+		expectation="success"
+		if test_have_prereq CYGWIN && test "$h" = "[::1]"
+		then
+			expectation="failure"
+		fi
+
+		test_expect_$expectation "fetch-pack --diag-url $h:$r" '
 			check_prot_host_port_path $h:$r $p "$h" NONE "$r"
 		'
 		# Do "/~" -> "~" conversion
-		test_expect_success "fetch-pack --diag-url $h:/~$r" '
+		test_expect_$expectation "fetch-pack --diag-url $h:/~$r" '
 			check_prot_host_port_path $h:/~$r $p "$h" NONE "~$r"
 		'
 	done
t/t5601-clone.sh +9 -2
index 5d7ea147f1a..9fe665eadfb 100755
@@ -530,10 +530,17 @@ do
 	'
 done
 
+# Parsing of paths that look like IPv6 addresses is broken on Cygwin.
+expectation_for_ipv6_tests=success
+if test_have_prereq CYGWIN
+then
+	expectation_for_ipv6_tests=failure
+fi
+
 #ipv6
 for repo in rep rep/home/project 123
 do
-	test_expect_success "clone [::1]:$repo" '
+	test_expect_$expectation_for_ipv6_tests "clone [::1]:$repo" '
 		test_clone_url [::1]:$repo ::1 "$repo"
 	'
 done
@@ -542,7 +549,7 @@ test_expect_success "clone host:/~repo" '
 	test_clone_url host:/~repo host "~repo"
 '
 
-test_expect_success "clone [::1]:/~repo" '
+test_expect_$expectation_for_ipv6_tests "clone [::1]:/~repo" '
 	test_clone_url [::1]:/~repo ::1 "~repo"
 '
 
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
#37 lore raw

PATCH v2 8/10 t7300: work around platform-specific behaviour with long paths on MinGW

In reply to Patrick Steinhardt

Windows by default has a restriction in place to only allow paths up to 260 characters. This restriction can nowadays be lifted by setting a registry key, but is still active by default.

In t7300 we have one test that exercises the behaviour of git-clean(1) with such long paths. Interestingly enough, this test fails on my system that uses Windows 10 with mingw-w64 installed via MSYS2: instead of observing ENAMETOOLONG, we observe ENOENT. This behaviour is consistent across multiple different environments I have tried.

I cannot say why exactly we observe a different error here, but I would not be surprised if this was either dependent on the Windows version, the version of MinGW, the current working directory of Git or any kind of combination of these.

Work around the issue by handling both errors.

---
 t/t7300-clean.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
t/t7300-clean.sh +1 -1
index 0aae0dee670..12ab25296b0 100755
@@ -747,7 +747,7 @@ test_expect_success MINGW 'handle clean & core.longpaths = false nicely' '
 	test_must_fail git clean -xdf 2>.git/err &&
 	# grepping for a strerror string is unportable but it is OK here with
 	# MINGW prereq
-	test_grep "too long" .git/err
+	test_grep -e "too long" -e "No such file or directory" .git/err
 '
 
 test_expect_success 'clean untracked paths by pathspec' '
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
#40 lore raw

PATCH v2 9/10 builtin/credential-cache: fix missing parameter for stub function

In reply to Patrick Steinhardt

When not compiling the credential cache we may use a stub function for cmd_credential_cache(). With commit 9b1cb5070f (builtin: add a repository parameter for builtin functions, 2024-09-13), we have added a new parameter to all of those top-level cmd_*() functions, and did indeed adapt the non-stubbed-out cmd_credential_cache(). But we didn't adapt the stubbed-out variant, so the code does not compile.

Fix this by adding the missing parameter.

---
 builtin/credential-cache.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
builtin/credential-cache.c +2 -1
index 5de8b9123bf..c51f7fc2ade 100644
@@ -189,7 +189,8 @@ int cmd_credential_cache(int argc,
 
 #else
 
-int cmd_credential_cache(int argc, const char **argv, const char *prefix)
+int cmd_credential_cache(int argc, const char **argv, const char *prefix,
+			 struct repository *repo UNUSED)
 {
 	const char * const usage[] = {
 		"git credential-cache [options] <action>",
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
#41 lore raw

PATCH v2 10/10 http: fix build error on FreeBSD

In reply to Patrick Steinhardt

The result parameter passed to http_request_reauth() may either point to a struct strbuf or a FILE *, where the target parameter tells us which of either it actually is. To accommodate for both types the pointer is a void *, which we then pass directly to functions without doing a cast.

This is fine on most platforms, but it breaks on FreeBSD because fileno() is implemented as a macro that tries to directly access the FILE * structure.

Fix this issue by storing the FILE * in a local variable before we pass it on to other functions.

---
 http.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)
http.c +6 -4
index d59e59f66b1..72973175a85 100644
@@ -2290,17 +2290,19 @@ static int http_request_reauth(const char *url,
 		case HTTP_REQUEST_STRBUF:
 			strbuf_reset(result);
 			break;
-		case HTTP_REQUEST_FILE:
-			if (fflush(result)) {
+		case HTTP_REQUEST_FILE: {
+			FILE *f = result;
+			if (fflush(f)) {
 				error_errno("unable to flush a file");
 				return HTTP_START_FAILED;
 			}
-			rewind(result);
-			if (ftruncate(fileno(result), 0) < 0) {
+			rewind(f);
+			if (ftruncate(fileno(f), 0) < 0) {
 				error_errno("unable to truncate a file");
 				return HTTP_START_FAILED;
 			}
 			break;
+		}
 		default:
 			BUG("Unknown http_request target");
 		}
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
#24 lore raw

Re: [PATCH v2 02/10] t/test-lib: wire up NO_ICONV prerequisite

In reply to Patrick Steinhardt
Quoted 6 lines · Patrick Steinhardt

Further note that there are several "!MINGW" conditions in t4201, and all of these fail due to iconv-related errors. This is quite likely a leftover from times before dce7d29551 (msvc: support building Git using MS Visual C++, 2019-06-25), which switched Windows-based builds over from "NO_ICONV=YesPlease" to "NEEDS_LIBICONV=YesPlease". Consequently, adapt those tests to also use the new ICONV prerequisite.

This appears to break CI on Windows when I merged this into 'jch':

    https://github.com/ttaylorr/git/actions/runs/11355564982/job/31585450667

I'm going to temporarily eject this from 'jch' and 'seen' until we can properly deal with this.

Thanks,
Taylor

#25 lore raw

Re: [PATCH v2 02/10] t/test-lib: wire up NO_ICONV prerequisite

Quoted 8 lines · Taylor Blau
Quoted 6 lines · Patrick Steinhardt

Further note that there are several "!MINGW" conditions in t4201, and all of these fail due to iconv-related errors. This is quite likely a leftover from times before dce7d29551 (msvc: support building Git using MS Visual C++, 2019-06-25), which switched Windows-based builds over from "NO_ICONV=YesPlease" to "NEEDS_LIBICONV=YesPlease". Consequently, adapt those tests to also use the new ICONV prerequisite.

Quoted 6 lines

This appears to break CI on Windows when I merged this into 'jch':

    https://github.com/ttaylorr/git/actions/runs/11355564982/job/31585450667

I'm going to temporarily eject this from 'jch' and 'seen' until we can properly deal with this.

Ugh, I'm looking forward to the Windows jobs for GitLab CI being merged down to next so that I can finally see such regressions before they hit our trees. Anyway, thanks for the heads up, will have a look.

Patrick

#33 lore raw

Re: [PATCH v2 05/10] t1401: make invocation of tar(1) work with Win32-provided one

In reply to Patrick Steinhardt
Quoted 4 lines · Patrick Steinhardt

Windows nowadays provides a tar(1) binary in "C:\Windows\system32". This version of tar(1) doesn't seem to handle the case where directory paths end with a trailing forward slash. And as we do that in t1401 the result is that the test fails.

Native explorer.exe support for .tar (and a bunch of other archive formats) was added as part of KB5031455 [1] not too long ago. It just uses libarchive under the hood, and was a huge QoL improvement for me when I was running Windows full-time at home!

[1] https://support.microsoft.com/en-us/topic/october-31-2023-kb5031455-os-builds-22621-2506-and-22631-2506-preview-6513c5ec-c5a2-4aaf-97f5-44c13d29e0d4

#31 lore raw

Re: [PATCH v2 04/10] t/lib-gpg: fix setup of GNUPGHOME in MinGW

In reply to Patrick Steinhardt
Quoted 10 lines · Patrick Steinhardt

In "t/lib-gpg.sh" we set up the "GNUPGHOME" environment variable to point to a test-specific directory. This is done by using "$PWD/gpghome" as value, where "$PWD" is the current test's trash directory.

This is broken for MinGW though because "$PWD" will use Windows-style paths that contain drive letters. What we really want in this context is a Unix-style path, which we can get by using $(pwd) instead. It is somewhat puzzling that nobody ever hit this issue, but it may easily be that nobody ever tests on Windows with GnuPG installed, which would make us skip those tests.

Thanks for the explanation! I didn't know "$PWD" and "$(pwd)" would behave differently under MinGW.

#35 lore raw

Re: [PATCH v2 06/10] t3404: work around platform-specific behaviour on macOS 10.15

In reply to Patrick Steinhardt
Quoted 6 lines · Patrick Steinhardt

Two of our tests in t3404 use indented HERE docs where leading tabs on some of the lines are actually relevant. The tabs do get removed though, and we try to fix this up by using sed(1) to replace leading tabs in the actual output, as well. But macOS 10.15 uses an oldish version of sed(1) that has BSD lineage, which does not understand "\t", and thus we fail to strip those leading tabs and fail the test.

The subtle differences in BSD and GNU-sed have caused me some headache in the past. I always work around the problem locally by installing the gnu-sed package via Homebrew, but it's better to change the test here.

#38 lore raw

Re: [PATCH v2 08/10] t7300: work around platform-specific behaviour with long paths on MinGW

In reply to Patrick Steinhardt
Quoted 10 lines · Patrick Steinhardt

Windows by default has a restriction in place to only allow paths up to 260 characters. This restriction can nowadays be lifted by setting a registry key, but is still active by default.

In t7300 we have one test that exercises the behaviour of git-clean(1) with such long paths. Interestingly enough, this test fails on my system that uses Windows 10 with mingw-w64 installed via MSYS2: instead of observing ENAMETOOLONG, we observe ENOENT. This behaviour is consistent across multiple different environments I have tried.

Quoted 4 lines

I cannot say why exactly we observe a different error here, but I would not be surprised if this was either dependent on the Windows version, the version of MinGW, the current working directory of Git or any kind of combination of these.

That's very strange. I'll try it on my Windows 11 23H2 system to see if I can observe the same behaviour.

I think it's fine to just catch both cases in the test anyway.

#42 lore raw

v3 cover Platform compatibility fixes

In reply to Patrick Steinhardt

Hi,

this is the third version of this series that fixes various different issues on more or less esoteric platforms.

There's only a single change compared to v2, namely that I restore the !MINGW prerequisite in t4201. I've been a bit overeager to convert it to use ICONV as that platform is seemingly still broken here.

I've kicked off another pipeline at GitHub [1] while the GitLab CI support for Windows hasn't yet landed to verify that things work as expected now. Tests pass now.

Thanks!

[1]: https://github.com/git/git/pull/1815

Patrick

Patrick Steinhardt (10):
t/test-lib: fix quoting of TEST_RESULTS_SAN_FILE t/test-lib: wire up NO_ICONV prerequisite
t/lib-gitweb: test against the build version of gitweb t/lib-gpg: fix setup of GNUPGHOME in MinGW
t1401: make invocation of tar(1) work with Win32-provided one
t3404: work around platform-specific behaviour on macOS 10.15 t5500, t5601: skip tests which exercise paths with '[::1]' on Cygwin
t7300: work around platform-specific behaviour with long paths on
MinGW
builtin/credential-cache: fix missing parameter for stub function
http: fix build error on FreeBSD

 Makefile                                     |   1 +
 builtin/credential-cache.c                   |   3 +-
 contrib/buildsystems/CMakeLists.txt          |   6 ++
 http.c                                       |  10 +-
 t/lib-gitweb.sh                              |   4 +-
 t/lib-gpg.sh                                 |   2 +-
 t/t0028-working-tree-encoding.sh             |   6 ++
 t/t1401-symbolic-ref.sh                      |   2 +-
 t/t2082-parallel-checkout-attributes.sh      |   2 +-
 t/t3404-rebase-interactive.sh                |  26 +++--
 t/t3434-rebase-i18n.sh                       |   6 ++
 t/t3900-i18n-commit.sh                       |   6 ++
 t/t3901-i18n-patch.sh                        |   6 ++
 t/t4041-diff-submodule-option.sh             |  16 ++-
 t/t4059-diff-submodule-not-initialized.sh    |  16 ++-
 t/t4060-diff-submodule-option-diff-format.sh |  17 ++--
 t/t4201-shortlog.sh                          |   8 +-
 t/t4205-log-pretty-formats.sh                | 102 +++++++++++--------
 t/t4210-log-i18n.sh                          |   6 ++
 t/t4254-am-corrupt.sh                        |   6 ++
 t/t5100-mailinfo.sh                          |  14 ++-
 t/t5500-fetch-pack.sh                        |  14 ++-
 t/t5550-http-fetch-dumb.sh                   |   4 +-
 t/t5601-clone.sh                             |  11 +-
 t/t6006-rev-list-format.sh                   |  54 ++++++----
 t/t7102-reset.sh                             |  40 +++++---
 t/t7300-clean.sh                             |   2 +-
 t/t8005-blame-i18n.sh                        |   6 ++
 t/t9300-fast-import.sh                       |   2 +-
 t/t9350-fast-export.sh                       |  10 +-
 t/test-lib.sh                                |   3 +-
 31 files changed, 274 insertions(+), 137 deletions(-)

Range-diff against v2:

 1:  a514f5d14a7 =  1:  a514f5d14a7 t/test-lib: fix quoting of TEST_RESULTS_SAN_FILE
 2:  f6a8582c34a !  2:  c046e5f03bf t/test-lib: wire up NO_ICONV prerequisite

@@ Commit message
assess whether the failure is expected or not. Most of the tests do smell like the expected kind of failure though.

- Further note that there are several "!MINGW" conditions in t4201, and
- all of these fail due to iconv-related errors. This is quite likely a
- leftover from times before dce7d29551 (msvc: support building Git using
- MS Visual C++, 2019-06-25), which switched Windows-based builds over
- from "NO_ICONV=YesPlease" to "NEEDS_LIBICONV=YesPlease". Consequently,
- adapt those tests to also use the new ICONV prerequisite.
-

## Makefile ##
@@ t/t4201-shortlog.sh: test_expect_success 'output from user-defined format is re-
'

-test_expect_success !MINGW 'shortlog wrapping' '
-+test_expect_success ICONV 'shortlog wrapping' ' ++test_expect_success !MINGW,ICONV 'shortlog wrapping' '
cat >expect <<\EOF &&
A U Thor (5):
Test
@@ t/t4201-shortlog.sh: EOF
'

-test_expect_success !MINGW 'shortlog from non-git directory' '
-+test_expect_success ICONV 'shortlog from non-git directory' ' ++test_expect_success !MINGW,ICONV 'shortlog from non-git directory' '
git log --no-expand-tabs HEAD >log && GIT_DIR=non-existing git shortlog -w <log >out && test_cmp expect out
'

-test_expect_success !MINGW 'shortlog can read --format=raw output' '
-+test_expect_success ICONV 'shortlog can read --format=raw output' ' ++test_expect_success !MINGW,ICONV 'shortlog can read --format=raw output' '
git log --format=raw HEAD >log &&
GIT_DIR=non-existing git shortlog -w <log >out &&
test_cmp expect out
@@ t/t4201-shortlog.sh: $DSCHO (2):
EOF

-test_expect_success !MINGW 'shortlog encoding' '
-+test_expect_success ICONV 'shortlog encoding' ' ++test_expect_success !MINGW,ICONV 'shortlog encoding' '
git reset --hard "$commit" &&
git config --unset i18n.commitencoding && echo 2 > a1 &&

 3:  2e2d208ef1b =  3:  52d0b252a1c t/lib-gitweb: test against the build version of gitweb
 4:  cc94f6fa648 =  4:  89957caf6ad t/lib-gpg: fix setup of GNUPGHOME in MinGW
 5:  58691dd652b =  5:  df2fb1b9607 t1401: make invocation of tar(1) work with Win32-provided one
 6:  1daadd82766 =  6:  29fffa5d86b t3404: work around platform-specific behaviour on macOS 10.15
 7:  374f47bf3de =  7:  de4c0c786bd t5500, t5601: skip tests which exercise paths with '[::1]' on Cygwin
 8:  13c06a8129b =  8:  ee67336621c t7300: work around platform-specific behaviour with long paths on MinGW
 9:  5fd78b6d535 =  9:  aaa6b5a175d builtin/credential-cache: fix missing parameter for stub function
10:  9963dc73988 = 10:  c22a94bab41 http: fix build error on FreeBSD
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
#43 lore raw

PATCH v3 1/10 t/test-lib: fix quoting of TEST_RESULTS_SAN_FILE

When assembling our LSAN_OPTIONS that configure the leak sanitizer we end up prepending the string with various different colon-separated options via calls to prepend_var. One of the settings we add is the path where the sanitizer should store logs, which can be an arbitrary filesystem path.

Naturally, filesystem paths may contain whitespace characters. And while it does seem as if we were quoting the value, we use escaped quotes and consequently split up the value if it does contain spaces. This leads to the following error in t0000 when having a value with whitespaces:

.../t/test-lib.sh: eval: line 64: unexpected EOF while looking for matching `"' ++ return 1
error: last command exited with $?=1
not ok 5 - subtest: 3 passing tests

The error itself is a bit puzzling at first. The basic problem is that the code sees the leading escaped quote during eval, but because we truncate everything after the space character it doesn't see the trailing escaped quote and thus fails to parse the string.

Properly quote the value to fix the issue while using single-quotes to quote the inner value passed to eval. The issue can be reproduced by t0000 with such a path that contains spaces.

---
 t/test-lib.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
t/test-lib.sh +1 -1
index b1a8ee5c002..241198ba95f 100644
@@ -1572,7 +1572,7 @@ then
 
 	prepend_var LSAN_OPTIONS : dedup_token_length=9999
 	prepend_var LSAN_OPTIONS : log_exe_name=1
-	prepend_var LSAN_OPTIONS : log_path=\"$TEST_RESULTS_SAN_FILE\"
+	prepend_var LSAN_OPTIONS : log_path="'$TEST_RESULTS_SAN_FILE'"
 	export LSAN_OPTIONS
 
 elif test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check" ||
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
#44 lore raw

PATCH v3 2/10 t/test-lib: wire up NO_ICONV prerequisite

In reply to Patrick Steinhardt

The iconv library is used by Git to reencode files, commit messages and other things. As such it is a rather integral part, but given that many platforms nowadays use UTF-8 everywhere you can live without support for reencoding in many situations. It is thus optional to build Git with iconv, and some of our platforms wired up in "config.mak.uname" disable it. But while we support building without it, running our test suite with "NO_ICONV=Yes" causes many test failures.

Wire up a new test prerequisite ICONV that gets populated via our GIT-BUILD-OPTIONS. Annotate failing tests accordingly.

Note that this commit does not do a deep dive into every single test to assess whether the failure is expected or not. Most of the tests do smell like the expected kind of failure though.

---
 Makefile                                     |   1 +
 contrib/buildsystems/CMakeLists.txt          |   6 ++
 t/t0028-working-tree-encoding.sh             |   6 ++
 t/t2082-parallel-checkout-attributes.sh      |   2 +-
 t/t3434-rebase-i18n.sh                       |   6 ++
 t/t3900-i18n-commit.sh                       |   6 ++
 t/t3901-i18n-patch.sh                        |   6 ++
 t/t4041-diff-submodule-option.sh             |  16 ++-
 t/t4059-diff-submodule-not-initialized.sh    |  16 ++-
 t/t4060-diff-submodule-option-diff-format.sh |  17 ++--
 t/t4201-shortlog.sh                          |   8 +-
 t/t4205-log-pretty-formats.sh                | 102 +++++++++++--------
 t/t4210-log-i18n.sh                          |   6 ++
 t/t4254-am-corrupt.sh                        |   6 ++
 t/t5100-mailinfo.sh                          |  14 ++-
 t/t5550-http-fetch-dumb.sh                   |   4 +-
 t/t6006-rev-list-format.sh                   |  54 ++++++----
 t/t7102-reset.sh                             |  40 +++++---
 t/t8005-blame-i18n.sh                        |   6 ++
 t/t9300-fast-import.sh                       |   2 +-
 t/t9350-fast-export.sh                       |  10 +-
 t/test-lib.sh                                |   1 +
 22 files changed, 229 insertions(+), 106 deletions(-)
Makefile +1 -0
index feeed6f9321..5db10347341 100644
@@ -3171,6 +3171,7 @@ GIT-BUILD-OPTIONS: FORCE
 	@echo PYTHON_PATH=\''$(subst ','\'',$(PYTHON_PATH_SQ))'\' >>$@+
 	@echo TAR=\''$(subst ','\'',$(subst ','\'',$(TAR)))'\' >>$@+
 	@echo NO_CURL=\''$(subst ','\'',$(subst ','\'',$(NO_CURL)))'\' >>$@+
+	@echo NO_ICONV=\''$(subst ','\'',$(subst ','\'',$(NO_ICONV)))'\' >>$@+
 	@echo NO_EXPAT=\''$(subst ','\'',$(subst ','\'',$(NO_EXPAT)))'\' >>$@+
 	@echo USE_LIBPCRE2=\''$(subst ','\'',$(subst ','\'',$(USE_LIBPCRE2)))'\' >>$@+
 	@echo NO_PERL=\''$(subst ','\'',$(subst ','\'',$(NO_PERL)))'\' >>$@+
contrib/buildsystems/CMakeLists.txt +6 -0
index 62af7b33d2f..1384c0eb6d3 100644
@@ -1109,6 +1109,7 @@ set(DIFF diff)
 set(PYTHON_PATH /usr/bin/python)
 set(TAR tar)
 set(NO_CURL )
+set(NO_ICONV )
 set(NO_EXPAT )
 set(USE_LIBPCRE2 )
 set(NO_PERL )
@@ -1122,6 +1123,10 @@ if(NOT CURL_FOUND)
 	set(NO_CURL 1)
 endif()
 
+if(NOT Iconv_FOUND)
+	SET(NO_ICONV 1)
+endif()
+
 if(NOT EXPAT_FOUND)
 	set(NO_EXPAT 1)
 endif()
@@ -1145,6 +1150,7 @@ file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS "DIFF='${DIFF}'\n")
 file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS "PYTHON_PATH='${PYTHON_PATH}'\n")
 file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS "TAR='${TAR}'\n")
 file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS "NO_CURL='${NO_CURL}'\n")
+file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS "NO_ICONV='${NO_ICONV}'\n")
 file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS "NO_EXPAT='${NO_EXPAT}'\n")
 file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS "NO_PERL='${NO_PERL}'\n")
 file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS "NO_PTHREADS='${NO_PTHREADS}'\n")
t/t0028-working-tree-encoding.sh +6 -0
index ad151a34670..510da4ca12d 100755
@@ -12,6 +12,12 @@ TEST_CREATE_REPO_NO_TEMPLATE=1
 
 GIT_TRACE_WORKING_TREE_ENCODING=1 && export GIT_TRACE_WORKING_TREE_ENCODING
 
+if ! test_have_prereq ICONV
+then
+	skip_all='skipping working tree encoding tests; iconv not available'
+	test_done
+fi
+
 test_expect_success 'setup test files' '
 	git config core.eol lf &&
 
t/t2082-parallel-checkout-attributes.sh +1 -1
index aec55496eb1..a040aa54cee 100755
@@ -34,7 +34,7 @@ test_expect_success 'parallel-checkout with ident' '
 	)
 '
 
-test_expect_success 'parallel-checkout with re-encoding' '
+test_expect_success ICONV 'parallel-checkout with re-encoding' '
 	set_checkout_config 2 0 &&
 	git init encoding &&
 	(
t/t3434-rebase-i18n.sh +6 -0
index 26a48d6b103..97fc9a23f21 100755
@@ -20,6 +20,12 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
 TEST_PASSES_SANITIZE_LEAK=true
 . ./test-lib.sh
 
+if ! test_have_prereq ICONV
+then
+	skip_all='skipping rebase i18n tests; iconv not available'
+	test_done
+fi
+
 compare_msg () {
 	iconv -f "$2" -t "$3" "$TEST_DIRECTORY/t3434/$1" >expect &&
 	git cat-file commit HEAD >raw &&
t/t3900-i18n-commit.sh +6 -0
index db7b403bc15..9d4b5ab1f95 100755
@@ -8,6 +8,12 @@ test_description='commit and log output encodings'
 TEST_PASSES_SANITIZE_LEAK=true
 . ./test-lib.sh
 
+if ! test_have_prereq ICONV
+then
+	skip_all='skipping commit i18n tests; iconv not available'
+	test_done
+fi
+
 compare_with () {
 	git show -s $1 | sed -e '1,/^$/d' -e 's/^    //' >current &&
 	case "$3" in
t/t3901-i18n-patch.sh +6 -0
index 5f0b9afc3fa..e0659c92935 100755
@@ -11,6 +11,12 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
 TEST_PASSES_SANITIZE_LEAK=true
 . ./test-lib.sh
 
+if ! test_have_prereq ICONV
+then
+	skip_all='skipping patch i18n tests; iconv not available'
+	test_done
+fi
+
 check_encoding () {
 	# Make sure characters are not corrupted
 	cnt="$1" header="$2" i=1 j=0
t/t4041-diff-submodule-option.sh +11 -5
index 8fc40e75eb3..aa149e0085e 100755
@@ -15,12 +15,18 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
 TEST_PASSES_SANITIZE_LEAK=true
 . ./test-lib.sh
 
-# Tested non-UTF-8 encoding
-test_encoding="ISO8859-1"
+# Test non-UTF-8 encoding in case iconv is available.
+if test_have_prereq ICONV
+then
+	test_encoding="ISO8859-1"
+	# String "added" in German (translated with Google Translate), encoded in UTF-8,
+	# used in sample commit log messages in add_file() function below.
+	added=$(printf "hinzugef\303\274gt")
+else
+	test_encoding="UTF-8"
+	added="added"
+fi
 
-# String "added" in German (translated with Google Translate), encoded in UTF-8,
-# used in sample commit log messages in add_file() function below.
-added=$(printf "hinzugef\303\274gt")
 add_file () {
 	(
 		cd "$1" &&
t/t4059-diff-submodule-not-initialized.sh +11 -5
index 668f5263038..28fd3cdb154 100755
@@ -12,12 +12,18 @@ initialized previously but the checkout has since been removed.
 TEST_PASSES_SANITIZE_LEAK=true
 . ./test-lib.sh
 
-# Tested non-UTF-8 encoding
-test_encoding="ISO8859-1"
 
-# String "added" in German (translated with Google Translate), encoded in UTF-8,
-# used in sample commit log messages in add_file() function below.
-added=$(printf "hinzugef\303\274gt")
+# Test non-UTF-8 encoding in case iconv is available.
+if test_have_prereq ICONV
+then
+	test_encoding="ISO8859-1"
+	# String "added" in German (translated with Google Translate), encoded in UTF-8,
+	# used in sample commit log messages in add_file() function below.
+	added=$(printf "hinzugef\303\274gt")
+else
+	test_encoding="UTF-8"
+	added="added"
+fi
 
 add_file () {
 	(
t/t4060-diff-submodule-option-diff-format.sh +11 -6
index 8ce67442d96..918334fa4c8 100755
@@ -13,12 +13,17 @@ This test tries to verify the sanity of --submodule=diff option of git diff.
 TEST_PASSES_SANITIZE_LEAK=true
 . ./test-lib.sh
 
-# Tested non-UTF-8 encoding
-test_encoding="ISO8859-1"
-
-# String "added" in German (translated with Google Translate), encoded in UTF-8,
-# used in sample commit log messages in add_file() function below.
-added=$(printf "hinzugef\303\274gt")
+# Test non-UTF-8 encoding in case iconv is available.
+if test_have_prereq ICONV
+then
+	test_encoding="ISO8859-1"
+	# String "added" in German (translated with Google Translate), encoded in UTF-8,
+	# used in sample commit log messages in add_file() function below.
+	added=$(printf "hinzugef\303\274gt")
+else
+	test_encoding="UTF-8"
+	added="added"
+fi
 
 add_file () {
 	(
t/t4201-shortlog.sh +4 -4
index c20c8857244..0ecb0597c9a 100755
@@ -105,7 +105,7 @@ test_expect_success 'output from user-defined format is re-wrapped' '
 	test_cmp expect log.predictable
 '
 
-test_expect_success !MINGW 'shortlog wrapping' '
+test_expect_success !MINGW,ICONV 'shortlog wrapping' '
 	cat >expect <<\EOF &&
 A U Thor (5):
       Test
@@ -126,13 +126,13 @@ EOF
 	test_cmp expect out
 '
 
-test_expect_success !MINGW 'shortlog from non-git directory' '
+test_expect_success !MINGW,ICONV 'shortlog from non-git directory' '
 	git log --no-expand-tabs HEAD >log &&
 	GIT_DIR=non-existing git shortlog -w <log >out &&
 	test_cmp expect out
 '
 
-test_expect_success !MINGW 'shortlog can read --format=raw output' '
+test_expect_success !MINGW,ICONV 'shortlog can read --format=raw output' '
 	git log --format=raw HEAD >log &&
 	GIT_DIR=non-existing git shortlog -w <log >out &&
 	test_cmp expect out
@@ -182,7 +182,7 @@ $DSCHO (2):
 
 EOF
 
-test_expect_success !MINGW 'shortlog encoding' '
+test_expect_success !MINGW,ICONV 'shortlog encoding' '
 	git reset --hard "$commit" &&
 	git config --unset i18n.commitencoding &&
 	echo 2 > a1 &&
t/t4205-log-pretty-formats.sh +59 -43

310 lines not on this page · raw message

t/t4210-log-i18n.sh +6 -0
index 7120030b5c6..4a12b2b4979 100755
@@ -5,6 +5,12 @@ test_description='test log with i18n features'
 TEST_PASSES_SANITIZE_LEAK=true
 . ./lib-gettext.sh
 
+if ! test_have_prereq ICONV
+then
+	skip_all='skipping log i18n tests; iconv not available'
+	test_done
+fi
+
 # two forms of é
 utf8_e=$(printf '\303\251')
 latin1_e=$(printf '\351')
t/t4254-am-corrupt.sh +6 -0
index 661feb60709..cb03522d021 100755
@@ -5,6 +5,12 @@ test_description='git am with corrupt input'
 TEST_PASSES_SANITIZE_LEAK=true
 . ./test-lib.sh
 
+if ! test_have_prereq ICONV
+then
+	skip_all='skipping am encoding corruption tests; iconv not available'
+	test_done
+fi
+
 make_mbox_with_nul () {
 	space=' '
 	q_nul_in_subject=
t/t5100-mailinfo.sh +12 -2
index 065156c1f39..23b2f218725 100755
@@ -28,7 +28,12 @@ check_mailinfo () {
 
 for mail in 00*
 do
-	test_expect_success "mailinfo $mail" '
+	case "$mail" in
+	0004)
+		prereq=ICONV;;
+	esac
+
+	test_expect_success $prereq "mailinfo $mail" '
 		check_mailinfo "$mail" "" &&
 		if test -f "$DATA/msg$mail--scissors"
 		then
@@ -56,7 +61,12 @@ test_expect_success 'split box with rfc2047 samples' \
 
 for mail in rfc2047/00*
 do
-	test_expect_success "mailinfo $mail" '
+	case "$mail" in
+	rfc2047/0001)
+		prereq=ICONV;;
+	esac
+
+	test_expect_success $prereq "mailinfo $mail" '
 		git mailinfo -u "$mail-msg" "$mail-patch" <"$mail" >"$mail-info" &&
 		echo msg &&
 		test_cmp "$DATA/empty" "$mail-msg" &&
t/t5550-http-fetch-dumb.sh +2 -2
index 58189c9f7dc..3c873de17ec 100755
@@ -344,12 +344,12 @@ test_expect_success 'git client shows text/plain with a charset' '
 	grep "this is the error message" stderr
 '
 
-test_expect_success 'http error messages are reencoded' '
+test_expect_success ICONV 'http error messages are reencoded' '
 	test_must_fail git clone "$HTTPD_URL/error/utf16" 2>stderr &&
 	grep "this is the error message" stderr
 '
 
-test_expect_success 'reencoding is robust to whitespace oddities' '
+test_expect_success ICONV 'reencoding is robust to whitespace oddities' '
 	test_must_fail git clone "$HTTPD_URL/error/odd-spacing" 2>stderr &&
 	grep "this is the error message" stderr
 '
t/t6006-rev-list-format.sh +37 -17

72 lines not on this page · raw message

t/t7102-reset.sh +25 -15
index 2add26d7684..e9a6cc72658 100755
@@ -13,21 +13,31 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
 TEST_PASSES_SANITIZE_LEAK=true
 . ./test-lib.sh
 
-commit_msg () {
-	# String "modify 2nd file (changed)" partly in German
-	# (translated with Google Translate),
-	# encoded in UTF-8, used as a commit log message below.
-	msg="modify 2nd file (ge\303\244ndert)\n"
-	if test -n "$1"
-	then
-		printf "$msg" | iconv -f utf-8 -t "$1"
-	else
-		printf "$msg"
-	fi
-}
-
-# Tested non-UTF-8 encoding
-test_encoding="ISO8859-1"
+if test_have_prereq ICONV
+then
+	commit_msg () {
+		# String "modify 2nd file (changed)" partly in German
+		# (translated with Google Translate),
+		# encoded in UTF-8, used as a commit log message below.
+		msg="modify 2nd file (ge\303\244ndert)\n"
+		if test -n "$1"
+		then
+			printf "$msg" | iconv -f utf-8 -t "$1"
+		else
+			printf "$msg"
+		fi
+	}
+
+	# Tested non-UTF-8 encoding
+	test_encoding="ISO8859-1"
+else
+	commit_msg () {
+		echo "modify 2nd file (geandert)"
+	}
+
+	# Tested non-UTF-8 encoding
+	test_encoding="UTF-8"
+fi
 
 test_expect_success 'creating initial files and commits' '
 	test_tick &&
t/t8005-blame-i18n.sh +6 -0
index 75da219ed1b..7a1f581c240 100755
@@ -3,6 +3,12 @@
 test_description='git blame encoding conversion'
 . ./test-lib.sh
 
+if ! test_have_prereq ICONV
+then
+	skip_all='skipping blame i18n tests; iconv not available'
+	test_done
+fi
+
 . "$TEST_DIRECTORY"/t8005/utf8.txt
 . "$TEST_DIRECTORY"/t8005/euc-japan.txt
 . "$TEST_DIRECTORY"/t8005/sjis.txt
t/t9300-fast-import.sh +1 -1
index 3b3c371740a..6224f54d4d2 100755
@@ -3676,7 +3676,7 @@ test_expect_success !MINGW 'W: get-mark & empty orphan commit with erroneous thi
 ### series X (other new features)
 ###
 
-test_expect_success 'X: handling encoding' '
+test_expect_success ICONV 'X: handling encoding' '
 	test_tick &&
 	cat >input <<-INPUT_END &&
 	commit refs/heads/encoding
t/t9350-fast-export.sh +5 -5
index 2bdc02b4599..9595dfef2ee 100755
@@ -125,7 +125,7 @@ test_expect_success 'fast-export --show-original-ids | git fast-import' '
 	test $MUSS = $(git rev-parse --verify refs/tags/muss)
 '
 
-test_expect_success 'reencoding iso-8859-7' '
+test_expect_success ICONV 'reencoding iso-8859-7' '
 
 	test_when_finished "git reset --hard HEAD~1" &&
 	test_config i18n.commitencoding iso-8859-7 &&
@@ -421,7 +421,7 @@ M 100644 :1 there
 
 EOF
 
-test_expect_success 'dropping tag of filtered out object' '
+test_expect_success ICONV 'dropping tag of filtered out object' '
 (
 	cd limit-by-paths &&
 	git fast-export --tag-of-filtered-object=drop mytag -- there > output &&
@@ -438,7 +438,7 @@ msg
 
 EOF
 
-test_expect_success 'rewriting tag of filtered out object' '
+test_expect_success ICONV 'rewriting tag of filtered out object' '
 (
 	cd limit-by-paths &&
 	git fast-export --tag-of-filtered-object=rewrite mytag -- there > output &&
@@ -667,7 +667,7 @@ M 100644 :13 file
 
 EOF
 
-test_expect_success 'avoid uninteresting refs' '
+test_expect_success ICONV 'avoid uninteresting refs' '
 	> tmp-marks &&
 	git fast-export --import-marks=tmp-marks \
 		--export-marks=tmp-marks main > /dev/null &&
@@ -686,7 +686,7 @@ from :14
 
 EOF
 
-test_expect_success 'refs are updated even if no commits need to be exported' '
+test_expect_success ICONV 'refs are updated even if no commits need to be exported' '
 	> tmp-marks &&
 	git fast-export --import-marks=tmp-marks \
 		--export-marks=tmp-marks main > /dev/null &&
t/test-lib.sh +1 -0
index 241198ba95f..a278181a056 100644
@@ -1743,6 +1743,7 @@ esac
 
 ( COLUMNS=1 && test $COLUMNS = 1 ) && test_set_prereq COLUMNS_CAN_BE_1
 test -z "$NO_CURL" && test_set_prereq LIBCURL
+test -z "$NO_ICONV" && test_set_prereq ICONV
 test -z "$NO_PERL" && test_set_prereq PERL
 test -z "$NO_PTHREADS" && test_set_prereq PTHREADS
 test -z "$NO_PYTHON" && test_set_prereq PYTHON
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
#50 lore raw

PATCH v3 3/10 t/lib-gitweb: test against the build version of gitweb

In reply to Patrick Steinhardt

When testing gitweb we set up the CGI script as "gitweb.perl", which is the source file of the build target "gitweb.cgi". This file doesn't have a patched shebang and still contains ++REPLACEMENT++ markers, but things generally work because we replace the configuration with our own test configuration.

But this only works as long as "$GIT_BUILD_DIR" actually points to the source tree, because "gitweb.cgi" and "gitweb.perl" happen to sit next to each other. This is not the case though once you have out-of-tree builds like with CMake, where the source and built versions live in different directories. Consequently, "$GIT_BUILD_DIR/gitweb/gitweb.perl" won't exist there.

While we could ask build systems with out-of-tree builds to instead set up GITWEB_TEST_INSTALLED, which allows us to override the location of the script, it goes against the spirit of this environment variable. We _don't_ want to test against an installed version, we want to use the version we have just built.

Fix this by using "gitweb.cgi" instead. This means that you cannot run test scripts without building that file, but in general we do expect developers to build stuff before they test it anyway.

---
 t/lib-gitweb.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
t/lib-gitweb.sh +2 -2
index 1f32ca66ea5..7f9808ec202 100644
@@ -48,8 +48,8 @@ EOF
 		test -f "$SCRIPT_NAME" ||
 		error "Cannot find gitweb at $GITWEB_TEST_INSTALLED."
 		say "# Testing $SCRIPT_NAME"
-	else # normal case, use source version of gitweb
-		SCRIPT_NAME="$GIT_BUILD_DIR/gitweb/gitweb.perl"
+	else # normal case, use built version of gitweb
+		SCRIPT_NAME="$GIT_BUILD_DIR/gitweb/gitweb.cgi"
 	fi
 	export SCRIPT_NAME
 }
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
#51 lore raw

PATCH v3 4/10 t/lib-gpg: fix setup of GNUPGHOME in MinGW

In reply to Patrick Steinhardt

In "t/lib-gpg.sh" we set up the "GNUPGHOME" environment variable to point to a test-specific directory. This is done by using "$PWD/gpghome" as value, where "$PWD" is the current test's trash directory.

This is broken for MinGW though because "$PWD" will use Windows-style paths that contain drive letters. What we really want in this context is a Unix-style path, which we can get by using $(pwd) instead. It is somewhat puzzling that nobody ever hit this issue, but it may easily be that nobody ever tests on Windows with GnuPG installed, which would make us skip those tests.

Adapt the code accordingly to fix tests using this library.

---
 t/lib-gpg.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
t/lib-gpg.sh +1 -1
index add11e88fc0..3845b6ac449 100644
@@ -6,7 +6,7 @@
 #   executed in an eval'ed subshell that changes the working directory to a
 #   temporary one.
 
-GNUPGHOME="$PWD/gpghome"
+GNUPGHOME="$(pwd)/gpghome"
 export GNUPGHOME
 
 test_lazy_prereq GPG '
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
#52 lore raw

PATCH v3 5/10 t1401: make invocation of tar(1) work with Win32-provided one

In reply to Patrick Steinhardt

Windows nowadays provides a tar(1) binary in "C:\Windows\system32". This version of tar(1) doesn't seem to handle the case where directory paths end with a trailing forward slash. And as we do that in t1401 the result is that the test fails.

Drop the trailing slash. Other tests that use tar(1) work alright, this is the only instance where it has been failing.

---
 t/t1401-symbolic-ref.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
t/t1401-symbolic-ref.sh +1 -1
index 5c60d6f812d..90af3f955c0 100755
@@ -16,7 +16,7 @@ reset_to_sane() {
 test_expect_success 'setup' '
 	git symbolic-ref HEAD refs/heads/foo &&
 	test_commit file &&
-	"$TAR" cf .git.tar .git/
+	"$TAR" cf .git.tar .git
 '
 
 test_expect_success 'symbolic-ref read/write roundtrip' '
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
#53 lore raw

PATCH v3 6/10 t3404: work around platform-specific behaviour on macOS 10.15

In reply to Patrick Steinhardt

Two of our tests in t3404 use indented HERE docs where leading tabs on some of the lines are actually relevant. The tabs do get removed though, and we try to fix this up by using sed(1) to replace leading tabs in the actual output, as well. But macOS 10.15 uses an oldish version of sed(1) that has BSD lineage, which does not understand "\t", and thus we fail to strip those leading tabs and fail the test.

Address this issue by using q_to_tab such that we do not have to strip leading tabs from the actual output.

---
 t/t3404-rebase-interactive.sh | 26 ++++++++++++--------------
 1 file changed, 12 insertions(+), 14 deletions(-)
t/t3404-rebase-interactive.sh +12 -14
index f171af3061d..7ce75237803 100755
@@ -1917,18 +1917,17 @@ test_expect_success '--update-refs updates refs correctly' '
 	test_cmp_rev HEAD~1 refs/heads/third &&
 	test_cmp_rev HEAD refs/heads/no-conflict-branch &&
 
-	cat >expect <<-\EOF &&
+	q_to_tab >expect <<-\EOF &&
 	Successfully rebased and updated refs/heads/update-refs.
 	Updated the following refs with --update-refs:
-		refs/heads/first
-		refs/heads/no-conflict-branch
-		refs/heads/second
-		refs/heads/third
+	Qrefs/heads/first
+	Qrefs/heads/no-conflict-branch
+	Qrefs/heads/second
+	Qrefs/heads/third
 	EOF
 
 	# Clear "Rebasing (X/Y)" progress lines and drop leading tabs.
-	sed -e "s/Rebasing.*Successfully/Successfully/g" -e "s/^\t//g" \
-		<err >err.trimmed &&
+	sed "s/Rebasing.*Successfully/Successfully/g" <err >err.trimmed &&
 	test_cmp expect err.trimmed
 '
 
@@ -2178,19 +2177,18 @@ test_expect_success '--update-refs: check failed ref update' '
 	test_must_fail git rebase --continue 2>err &&
 	grep "update_ref failed for ref '\''refs/heads/second'\''" err &&
 
-	cat >expect <<-\EOF &&
+	q_to_tab >expect <<-\EOF &&
 	Updated the following refs with --update-refs:
-		refs/heads/first
-		refs/heads/no-conflict-branch
-		refs/heads/third
+	Qrefs/heads/first
+	Qrefs/heads/no-conflict-branch
+	Qrefs/heads/third
 	Failed to update the following refs with --update-refs:
-		refs/heads/second
+	Qrefs/heads/second
 	EOF
 
 	# Clear "Rebasing (X/Y)" progress lines and drop leading tabs.
 	tail -n 6 err >err.last &&
-	sed -e "s/Rebasing.*Successfully/Successfully/g" -e "s/^\t//g" \
-		<err.last >err.trimmed &&
+	sed "s/Rebasing.*Successfully/Successfully/g" <err.last >err.trimmed &&
 	test_cmp expect err.trimmed
 '
 
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
#54 lore raw

PATCH v3 7/10 t5500, t5601: skip tests which exercise paths with '[::1]' on Cygwin

In reply to Patrick Steinhardt

Parsing repositories which contain '[::1]' is broken on Cygwin. It seems as if Cygwin is confusing those as drive letter prefixes or something like this, but I couldn't deduce the actual root cause.

Mark those tests as broken for now.

---
 t/t5500-fetch-pack.sh | 14 ++++++++++----
 t/t5601-clone.sh      | 11 +++++++++--
 2 files changed, 19 insertions(+), 6 deletions(-)
t/t5500-fetch-pack.sh +10 -4
index 605f17240c1..416522c86ad 100755
@@ -774,7 +774,7 @@ do
 	# file with scheme
 	for p in file
 	do
-		test_expect_success !MINGW "fetch-pack --diag-url $p://$h/$r" '
+		test_expect_success !WINDOWS "fetch-pack --diag-url $p://$h/$r" '
 			check_prot_path $p://$h/$r $p "/$r"
 		'
 		test_expect_success MINGW "fetch-pack --diag-url $p://$h/$r" '
@@ -784,7 +784,7 @@ do
 			check_prot_path $p:///$r $p "/$r"
 		'
 		# No "/~" -> "~" conversion for file
-		test_expect_success !MINGW "fetch-pack --diag-url $p://$h/~$r" '
+		test_expect_success !WINDOWS "fetch-pack --diag-url $p://$h/~$r" '
 			check_prot_path $p://$h/~$r $p "/~$r"
 		'
 		test_expect_success MINGW "fetch-pack --diag-url $p://$h/~$r" '
@@ -806,11 +806,17 @@ do
 	p=ssh
 	for h in host [::1]
 	do
-		test_expect_success "fetch-pack --diag-url $h:$r" '
+		expectation="success"
+		if test_have_prereq CYGWIN && test "$h" = "[::1]"
+		then
+			expectation="failure"
+		fi
+
+		test_expect_$expectation "fetch-pack --diag-url $h:$r" '
 			check_prot_host_port_path $h:$r $p "$h" NONE "$r"
 		'
 		# Do "/~" -> "~" conversion
-		test_expect_success "fetch-pack --diag-url $h:/~$r" '
+		test_expect_$expectation "fetch-pack --diag-url $h:/~$r" '
 			check_prot_host_port_path $h:/~$r $p "$h" NONE "~$r"
 		'
 	done
t/t5601-clone.sh +9 -2
index 5d7ea147f1a..9fe665eadfb 100755
@@ -530,10 +530,17 @@ do
 	'
 done
 
+# Parsing of paths that look like IPv6 addresses is broken on Cygwin.
+expectation_for_ipv6_tests=success
+if test_have_prereq CYGWIN
+then
+	expectation_for_ipv6_tests=failure
+fi
+
 #ipv6
 for repo in rep rep/home/project 123
 do
-	test_expect_success "clone [::1]:$repo" '
+	test_expect_$expectation_for_ipv6_tests "clone [::1]:$repo" '
 		test_clone_url [::1]:$repo ::1 "$repo"
 	'
 done
@@ -542,7 +549,7 @@ test_expect_success "clone host:/~repo" '
 	test_clone_url host:/~repo host "~repo"
 '
 
-test_expect_success "clone [::1]:/~repo" '
+test_expect_$expectation_for_ipv6_tests "clone [::1]:/~repo" '
 	test_clone_url [::1]:/~repo ::1 "~repo"
 '
 
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
#55 lore raw

PATCH v3 8/10 t7300: work around platform-specific behaviour with long paths on MinGW

In reply to Patrick Steinhardt

Windows by default has a restriction in place to only allow paths up to 260 characters. This restriction can nowadays be lifted by setting a registry key, but is still active by default.

In t7300 we have one test that exercises the behaviour of git-clean(1) with such long paths. Interestingly enough, this test fails on my system that uses Windows 10 with mingw-w64 installed via MSYS2: instead of observing ENAMETOOLONG, we observe ENOENT. This behaviour is consistent across multiple different environments I have tried.

I cannot say why exactly we observe a different error here, but I would not be surprised if this was either dependent on the Windows version, the version of MinGW, the current working directory of Git or any kind of combination of these.

Work around the issue by handling both errors.

---
 t/t7300-clean.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
t/t7300-clean.sh +1 -1
index 0aae0dee670..12ab25296b0 100755
@@ -747,7 +747,7 @@ test_expect_success MINGW 'handle clean & core.longpaths = false nicely' '
 	test_must_fail git clean -xdf 2>.git/err &&
 	# grepping for a strerror string is unportable but it is OK here with
 	# MINGW prereq
-	test_grep "too long" .git/err
+	test_grep -e "too long" -e "No such file or directory" .git/err
 '
 
 test_expect_success 'clean untracked paths by pathspec' '
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
#56 lore raw

PATCH v3 9/10 builtin/credential-cache: fix missing parameter for stub function

In reply to Patrick Steinhardt

When not compiling the credential cache we may use a stub function for cmd_credential_cache(). With commit 9b1cb5070f (builtin: add a repository parameter for builtin functions, 2024-09-13), we have added a new parameter to all of those top-level cmd_*() functions, and did indeed adapt the non-stubbed-out cmd_credential_cache(). But we didn't adapt the stubbed-out variant, so the code does not compile.

Fix this by adding the missing parameter.

---
 builtin/credential-cache.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
builtin/credential-cache.c +2 -1
index 5de8b9123bf..c51f7fc2ade 100644
@@ -189,7 +189,8 @@ int cmd_credential_cache(int argc,
 
 #else
 
-int cmd_credential_cache(int argc, const char **argv, const char *prefix)
+int cmd_credential_cache(int argc, const char **argv, const char *prefix,
+			 struct repository *repo UNUSED)
 {
 	const char * const usage[] = {
 		"git credential-cache [options] <action>",
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
#57 lore raw

PATCH v3 10/10 http: fix build error on FreeBSD

In reply to Patrick Steinhardt

The result parameter passed to http_request_reauth() may either point to a struct strbuf or a FILE *, where the target parameter tells us which of either it actually is. To accommodate for both types the pointer is a void *, which we then pass directly to functions without doing a cast.

This is fine on most platforms, but it breaks on FreeBSD because fileno() is implemented as a macro that tries to directly access the FILE * structure.

Fix this issue by storing the FILE * in a local variable before we pass it on to other functions.

---
 http.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)
http.c +6 -4
index d59e59f66b1..72973175a85 100644
@@ -2290,17 +2290,19 @@ static int http_request_reauth(const char *url,
 		case HTTP_REQUEST_STRBUF:
 			strbuf_reset(result);
 			break;
-		case HTTP_REQUEST_FILE:
-			if (fflush(result)) {
+		case HTTP_REQUEST_FILE: {
+			FILE *f = result;
+			if (fflush(f)) {
 				error_errno("unable to flush a file");
 				return HTTP_START_FAILED;
 			}
-			rewind(result);
-			if (ftruncate(fileno(result), 0) < 0) {
+			rewind(f);
+			if (ftruncate(fileno(f), 0) < 0) {
 				error_errno("unable to truncate a file");
 				return HTTP_START_FAILED;
 			}
 			break;
+		}
 		default:
 			BUG("Unknown http_request target");
 		}
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
#39 lore raw

Re: [PATCH v2 08/10] t7300: work around platform-specific behaviour with long paths on MinGW

In reply to James Liu
Quoted 11 lines · James Liu
Quoted 10 lines · Patrick Steinhardt

Windows by default has a restriction in place to only allow paths up to 260 characters. This restriction can nowadays be lifted by setting a registry key, but is still active by default.

In t7300 we have one test that exercises the behaviour of git-clean(1) with such long paths. Interestingly enough, this test fails on my system that uses Windows 10 with mingw-w64 installed via MSYS2: instead of observing ENAMETOOLONG, we observe ENOENT. This behaviour is consistent across multiple different environments I have tried.

Quoted 7 lines
Quoted 4 lines

I cannot say why exactly we observe a different error here, but I would not be surprised if this was either dependent on the Windows version, the version of MinGW, the current working directory of Git or any kind of combination of these.

That's very strange. I'll try it on my Windows 11 23H2 system to see if I can observe the same behaviour.

Hmm maybe not. MSYS2 on ARM64 isn't fully baked, and I don't have an x86 Windows machine handy.

#26 lore raw

Re: [PATCH v2 02/10] t/test-lib: wire up NO_ICONV prerequisite

In reply to Patrick Steinhardt
Quoted 11 lines · Patrick Steinhardt
Quoted 10 lines · Taylor Blau
Quoted 6 lines · Patrick Steinhardt

Further note that there are several "!MINGW" conditions in t4201, and all of these fail due to iconv-related errors. This is quite likely a leftover from times before dce7d29551 (msvc: support building Git using MS Visual C++, 2019-06-25), which switched Windows-based builds over from "NO_ICONV=YesPlease" to "NEEDS_LIBICONV=YesPlease". Consequently, adapt those tests to also use the new ICONV prerequisite.

This appears to break CI on Windows when I merged this into 'jch':

Quoted 8 lines
Quoted 4 lines
    https://github.com/ttaylorr/git/actions/runs/11355564982/job/31585450667

I'm going to temporarily eject this from 'jch' and 'seen' until we can properly deal with this.

Ugh, I'm looking forward to the Windows jobs for GitLab CI being merged down to next so that I can finally see such regressions before they hit our trees. Anyway, thanks for the heads up, will have a look.

It's OK. Ejecting a topic out of 'seen' is relatively easy as it requires the following (after removing the line out of Meta/redo-seen.sh):

git checkout -B seen jch
sh Meta/redo-seen.sh

I was mostly confused why my build of 'seen' passed 'make test' locally, but failed CI when pushed to GitHub. Of course, I'm not testing on a Windows machine, and you didn't have easy access to Windows CI runs on GitLab (for now), so the result makes sense.

Thanks again for working on it.

Thanks,
Taylor

#27 lore raw

Re: [PATCH v2 02/10] t/test-lib: wire up NO_ICONV prerequisite

Quoted 24 lines · Taylor Blau
Quoted 11 lines · Patrick Steinhardt
Quoted 10 lines · Taylor Blau
Quoted 6 lines · Patrick Steinhardt

Further note that there are several "!MINGW" conditions in t4201, and all of these fail due to iconv-related errors. This is quite likely a leftover from times before dce7d29551 (msvc: support building Git using MS Visual C++, 2019-06-25), which switched Windows-based builds over from "NO_ICONV=YesPlease" to "NEEDS_LIBICONV=YesPlease". Consequently, adapt those tests to also use the new ICONV prerequisite.

This appears to break CI on Windows when I merged this into 'jch':

Quoted 8 lines
Quoted 4 lines
    https://github.com/ttaylorr/git/actions/runs/11355564982/job/31585450667

I'm going to temporarily eject this from 'jch' and 'seen' until we can properly deal with this.

Ugh, I'm looking forward to the Windows jobs for GitLab CI being merged down to next so that I can finally see such regressions before they hit our trees. Anyway, thanks for the heads up, will have a look.

It's OK. Ejecting a topic out of 'seen' is relatively easy as it requires the following (after removing the line out of Meta/redo-seen.sh):

Quoted 7 lines

git checkout -B seen jch
sh Meta/redo-seen.sh

I was mostly confused why my build of 'seen' passed 'make test' locally, but failed CI when pushed to GitHub. Of course, I'm not testing on a Windows machine, and you didn't have easy access to Windows CI runs on GitLab (for now), so the result makes sense.

Speaking of... I have 'ps/ci-gitlab-windows' tagged for 'next' in the next integration round. There is some duplication of patches between that topic and this one (as well as ps/build).

How do you want me to handle the dependency?

Thanks,
Taylor

#45 lore raw

Re: [PATCH v3 02/10] t/test-lib: wire up NO_ICONV prerequisite

In reply to Patrick Steinhardt
Quoted 10 lines · Patrick Steinhardt

The iconv library is used by Git to reencode files, commit messages and other things. As such it is a rather integral part, but given that many platforms nowadays use UTF-8 everywhere you can live without support for reencoding in many situations. It is thus optional to build Git with iconv, and some of our platforms wired up in "config.mak.uname" disable it. But while we support building without it, running our test suite with "NO_ICONV=Yes" causes many test failures.

Wire up a new test prerequisite ICONV that gets populated via our GIT-BUILD-OPTIONS. Annotate failing tests accordingly.

I don't really have a strong opinion on this patch, but this has been a part of POSIX since the 2001 revision, so everyone except Windows should have it by default (and Git for Windows ships with their own version). Perhaps if people are not seeing the tests fail, that's because everyone is building with iconv enabled?

I will also add that as much as we might like people to be using UTF-8 everywhere (and to be clear, I very much would), many repositories still have commits encoded in older ways (such as the Linux kernel) and there are unfortunately many people using working-tree-encoding to convert to UTF-16LE-BOM for compatibility with legacy Windows software, so in general, only users in highly restricted Unix-only environments would be able to confidently run without support for more encodings than UTF-8.

I think this patch is fine for now, but given what I mentioned above, maybe we want to add a weather balloon in a future series to see if anyone is compiling with NO_ICONV. After all, if everybody has easy access to iconv(3), then we might be able to drop support for NO_ICONV and the work maintaining it entails without negatively impacting anyone.

Signature · 2 lines
brian m. carlson (they/them or he/him)
Toronto, Ontario, CA
#46 lore raw

Re: [PATCH v3 02/10] t/test-lib: wire up NO_ICONV prerequisite

Quoted 5 lines · brian m. carlson

I think this patch is fine for now, but given what I mentioned above, maybe we want to add a weather balloon in a future series to see if anyone is compiling with NO_ICONV. After all, if everybody has easy access to iconv(3), then we might be able to drop support for NO_ICONV and the work maintaining it entails without negatively impacting anyone.

As you say, for now I'd want this patch to land in our tree to unbreak our test suite with NO_ICONV. It fixes the status quo, and whether or not we want to drop NO_ICONV certainly is a bigger discussion.

We do have one platform that doesn't have iconv, namely QNX. I rather doubt that anybody has access to that platform, or that anybody really uses it. So having a test balloon out there could help us figure out whether there are any users of NO_ICONV left. But when it is used I don't mind that flag staying around.

Patrick

#28 lore raw

Re: [PATCH v2 02/10] t/test-lib: wire up NO_ICONV prerequisite

In reply to Taylor Blau
Quoted 33 lines · Taylor Blau
Quoted 24 lines · Taylor Blau
Quoted 11 lines · Patrick Steinhardt
Quoted 10 lines · Taylor Blau
Quoted 6 lines · Patrick Steinhardt

Further note that there are several "!MINGW" conditions in t4201, and all of these fail due to iconv-related errors. This is quite likely a leftover from times before dce7d29551 (msvc: support building Git using MS Visual C++, 2019-06-25), which switched Windows-based builds over from "NO_ICONV=YesPlease" to "NEEDS_LIBICONV=YesPlease". Consequently, adapt those tests to also use the new ICONV prerequisite.

This appears to break CI on Windows when I merged this into 'jch':

Quoted 8 lines
Quoted 4 lines
    https://github.com/ttaylorr/git/actions/runs/11355564982/job/31585450667

I'm going to temporarily eject this from 'jch' and 'seen' until we can properly deal with this.

Ugh, I'm looking forward to the Windows jobs for GitLab CI being merged down to next so that I can finally see such regressions before they hit our trees. Anyway, thanks for the heads up, will have a look.

It's OK. Ejecting a topic out of 'seen' is relatively easy as it requires the following (after removing the line out of Meta/redo-seen.sh):

Quoted 7 lines

git checkout -B seen jch
sh Meta/redo-seen.sh

I was mostly confused why my build of 'seen' passed 'make test' locally, but failed CI when pushed to GitHub. Of course, I'm not testing on a Windows machine, and you didn't have easy access to Windows CI runs on GitLab (for now), so the result makes sense.

Quoted 5 lines

Speaking of... I have 'ps/ci-gitlab-windows' tagged for 'next' in the next integration round. There is some duplication of patches between that topic and this one (as well as ps/build).

How do you want me to handle the dependency?

I'll soon reroll ps/build to depend on all the other series, and will evict the duplicate patches from there. I'm happy to send another version of this series here that depends on ps/ci-gitlab-windows, but wouldn't mind either if the patch landed via both series.

Patrick

#47 lore raw

Re: [PATCH v3 02/10] t/test-lib: wire up NO_ICONV prerequisite

In reply to Patrick Steinhardt
Quoted 23 lines · Patrick Steinhardt

The iconv library is used by Git to reencode files, commit messages and other things. As such it is a rather integral part, but given that many platforms nowadays use UTF-8 everywhere you can live without support for reencoding in many situations. It is thus optional to build Git with iconv, and some of our platforms wired up in "config.mak.uname" disable it. But while we support building without it, running our test suite with "NO_ICONV=Yes" causes many test failures.

Wire up a new test prerequisite ICONV that gets populated via our GIT-BUILD-OPTIONS. Annotate failing tests accordingly.

Note that this commit does not do a deep dive into every single test to assess whether the failure is expected or not. Most of the tests do smell like the expected kind of failure though.

---

t/t6006-rev-list-format.sh +0 -0
index f1623b1c06d..2a01a62a2f3 100755

[snip]

t/t7102-reset.sh

Quoted 22 lines
t/t6006-rev-list-format.sh +2 -2
@@ -198,7 +218,7 @@ Thu, 7 Apr 2005 15:13:13 -0700
 1112911993
 EOF
 
-test_format encoding %e <<EOF
+test_format ICONV encoding %e <<EOF
 commit $head2
 $test_encoding
 commit $head1
@@ -374,7 +394,7 @@ test_expect_success 'setup complex body' '
 	head3_short=$(git rev-parse --short $head3)
 '
 
-test_format complex-encoding %e <<EOF
+test_format ICONV complex-encoding %e <<EOF
 commit $head3
 $test_encoding
 commit $head2
t/t7102-reset.sh +0 -0
index 2add26d7684..e9a6cc72658 100755

These two changes subtly break t6006 (in such a way that it still passes, but is clearly not testing what we want):

./t6006-rev-list-format.sh: line 92: test_expect_%e: command not found

#48 lore raw

PATCH 1/1 t6006: fix prereq handling with `test_format ()`

In df383b5842 (t/test-lib: wire up NO_ICONV prerequisite, 2024-10-16) we have introduced a new NO_ICONV prerequisite that makes us skip tests in case Git is not compiled with support for iconv. This change subtly broke t6006: while the test suite still passes, some of its tests won't execute because they run into an error.

    ./t6006-rev-list-format.sh: line 92: test_expect_%e: command not found

The broken tests use test_format (), and the mentioned commit simply prepended the new prerequisite to its arguments. But that does not work, as the function is not aware of prereqs at all and will now treat all of its arguments incorrectly.

Fix this by making the function aware of prereqs by accepting an optional fourth argument. Adapt the callsites accordingly.

---

Oops, good catch. @Taylor, let's maybe queue this fix on top of ps/platform-compat-fixes, which currently sits in next.

Thanks!

Patrick

 t/t6006-rev-list-format.sh | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
t/t6006-rev-list-format.sh +4 -4
index 2a01a62a2f..b0ec2fe865 100755
@@ -75,7 +75,7 @@ test_expect_success 'setup' '
 	git config --unset i18n.commitEncoding
 '
 
-# usage: test_format [argument...] name format_string [failure] <expected_output
+# usage: test_format [argument...] name format_string [success|failure] [prereq] <expected_output
 test_format () {
 	local args=
 	while true
@@ -89,7 +89,7 @@ test_format () {
 		esac
 	done
 	cat >expect.$1
-	test_expect_${3:-success} "format $1" "
+	test_expect_${3:-success} $4 "format $1" "
 		git rev-list $args --pretty=format:'$2' main >output.$1 &&
 		test_cmp expect.$1 output.$1
 	"
@@ -218,7 +218,7 @@ Thu, 7 Apr 2005 15:13:13 -0700
 1112911993
 EOF
 
-test_format ICONV encoding %e <<EOF
+test_format encoding %e success ICONV <<EOF
 commit $head2
 $test_encoding
 commit $head1
@@ -394,7 +394,7 @@ test_expect_success 'setup complex body' '
 	head3_short=$(git rev-parse --short $head3)
 '
 
-test_format ICONV complex-encoding %e <<EOF
+test_format complex-encoding %e success ICONV <<EOF
 commit $head3
 $test_encoding
 commit $head2
Signature · 1 line
2.47.0.118.gfd3785337b.dirty
#49 lore raw

Re: [PATCH] t6006: fix prereq handling with `test_format ()`

Quoted 2 lines · Patrick Steinhardt

Oops, good catch. @Taylor, let's maybe queue this fix on top of ps/platform-compat-fixes, which currently sits in next.

Noted, thanks, and thanks also for hopping on the fix.

I was wondering how to handle this situation with Junio's scripts, but applying the patch on top of ps/platform-compat-fixes and then re-building 'next' lands the patch in the expected fashion, without any additional cruft.

I am not sure whether I will do a push-out on Tuesday (as has been my schedule since Junio went offline), or Wednesday morning. I suspect that the latter is more likely, but either way, that integration round will have the new patch applied down onto 'next'.

Thanks,
Taylor