Offer to run CI/PR builds in Visual Studio Team Services

Hi Junio,

This here test matrix does not rely on it. But I do hope to get the BusyBox support (where we ship a subset of Git for Windows that is intended to be used by 3rd-party applications only, called "MinGit", with BusyBox instead of the MSYS2 Bash and all of those shell utilities required by Git's scripts) closer to production. And to test *that*, I *need* GIT_TEST_INSTALLED to work properly.

Ciao,
Dscho

From: Johannes Schindelin <[email protected]>

When building Git with RUNTIME_PREFIX and starting a test helper from t/helper/, it fails to detect the system prefix correctly.

This is the reason that the warning

RUNTIME_PREFIX requested, but prefix computation failed. [...]

to be printed.

In t0061, we did not expect that to happen, and it actually did not happen in the normal case, because bin-wrappers/test-tool specifically sets GIT_TEXTDOMAINDIR (and as a consequence, nothing in test-tool wants to know about the runtime prefix).

However, with --with-dashes, bin-wrappers/test-tool is no longer called, but t/helper/test-tool is called directly.

So let's just ignore the RUNTIME_PREFIX warning.

---
 t/t0061-run-command.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

index 99a614bc7c..5a2d087bf0 100755
@@ -166,7 +166,8 @@ test_trace () {
 	expect="$1"
 	shift
 	GIT_TRACE=1 test-tool run-command "$@" run-command true 2>&1 >/dev/null | \
-		sed -e 's/.* run_command: //' -e '/trace: .*/d' >actual &&
+		sed -e 's/.* run_command: //' -e '/trace: .*/d' \
+			-e '/RUNTIME_PREFIX requested/d' >actual &&
 	echo "$expect true" >expect &&
 	test_cmp expect actual
 }

gitgitgadget

"Johannes Schindelin via GitGitGadget" <[email protected]>

Two questions that would come to the readers' minds are

- Why "it fails to detect the system prefix correctly"? Is that a bug waiting to hurt end users?

- Why is it better not to bother fixing that failure? Is it because this happens only in the test helper and won't hurt end users?

I do not mind this particular "sweeping it under the rug" if the rationale is "it only is the strange set-up of test-tool that causes it, and we shouldn't burden the code shared with the actual runtime to compute runtime prefix just to fix this bug". I do think it is not productive to bend backwards if it is only done in order to work around unusual setup in the test helper binary and I do agree that ignoring the warning is the right solution.

I just do not like to see that our commits do not explain why we chose to ignore, instead of "fix", the issue.

Hi Junio,

I recall explaining that already in a different thread (one that actually *was* about the RUNTIME_PREFIX feature, where you asked why we cannot test for it in the test suite): to verify that we are in a valid Git installation location, we verify that the git executable is either in <PREFIX>/bin or in <PREFIX>/libexec/git-core (we need to check that, as the prefix is the parent directory in one case, and the grand parent directory in the other).

And for Git's test suite, we simply do not control the name of the directory in which the Git executable lives (think: git clone <git.git-url> junios-git).

And even if we could control that name: it simply is not a valid installation location. For example, it is unlikely that the translation live in ../share/locale/de/LC_MESSAGES/git.mo relative to the Git executable.

But that was clarified already to your satisfaction in the RUNTIME_PREFIX patches. Or do you want to reopen that case now?

Ciao,
Dscho

This is not about me. It is about those who read _this_ commit --- if the change is justified to them clearly enough, then we'd be happy. I did not think it is with the proposed log message, and that is where my comment came from.

Hi Junio,

You know what, I finally start to see your point.

Will adjust the commit message accordingly,
Dscho

From: Johannes Schindelin <[email protected]>

This speeds up the tests by a bit on Windows, where running Unix shell scripts (and spawning processes) is not exactly a cheap operation.

---
 t/README      |  9 +++++++++
 t/test-lib.sh | 19 +++++++++++++------
 2 files changed, 22 insertions(+), 6 deletions(-)

index 11ce7675e3..063530234f 100644
@@ -170,6 +170,15 @@ appropriately before running "make".
 	implied by other options like --valgrind and
 	GIT_TEST_INSTALLED.
 
+--no-bin-wrappers::
+	By default, the test suite uses the wrappers in
+	`../bin-wrappers/` to execute `git` and friends. With this option,
+	`../git` and friends are run directly. This is not recommended
+	in general, as the wrappers contain safeguards to ensure that no
+	files from an installed Git are used, but can speed up test runs
+	especially on platforms where running shell scripts is expensive
+	(most notably, Windows).
+
 --root=<directory>::
 	Create "trash" directories used to store all temporary data during
 	testing under <directory>, instead of the t/ directory.

index c790e98fd2..25e649c997 100644
@@ -111,6 +111,8 @@ do
 		test -z "$HARNESS_ACTIVE" && quiet=t ;;
 	--with-dashes)
 		with_dashes=t ;;
+	--no-bin-wrappers)
+		no_bin_wrappers=t ;;
 	--no-color)
 		color= ;;
 	--va|--val|--valg|--valgr|--valgri|--valgrin|--valgrind)

@@ -1214,16 +1216,21 @@ then
 	PATH=$GIT_TEST_INSTALLED:$GIT_BUILD_DIR/t/helper:$PATH
 	GIT_EXEC_PATH=${GIT_TEST_EXEC_PATH:-$GIT_EXEC_PATH}
 else # normal case, use ../bin-wrappers only unless $with_dashes:
-	git_bin_dir="$GIT_BUILD_DIR/bin-wrappers"
-	if ! test -x "$git_bin_dir/git"
+	if test -n "$no_bin_wrappers"
 	then
-		if test -z "$with_dashes"
+		with_dashes=t
+	else
+		git_bin_dir="$GIT_BUILD_DIR/bin-wrappers"
+		if ! test -x "$git_bin_dir/git"
 		then
-			say "$git_bin_dir/git is not executable; using GIT_EXEC_PATH"
+			if test -z "$with_dashes"
+			then
+				say "$git_bin_dir/git is not executable; using GIT_EXEC_PATH"
+			fi
+			with_dashes=t
 		fi
-		with_dashes=t
+		PATH="$git_bin_dir:$PATH"
 	fi
-	PATH="$git_bin_dir:$PATH"
 	GIT_EXEC_PATH=$GIT_BUILD_DIR
 	if test -n "$with_dashes"
 	then

gitgitgadget

From: Johannes Schindelin <[email protected]>

As Unix shell scripting comes at a hefty price on Windows, we have to see where we can save some time to run the test suite.

Let's skip the chain linting and the bin-wrappers/ redirection on Windows; this seems to shave of anywhere between 10-30% from the overall runtime.

---
 ci/lib.sh | 2 ++
 1 file changed, 2 insertions(+)

index 91cf1402bf..e1d5222ceb 100755
@@ -43,6 +43,8 @@ then
 	BREW_INSTALL_PACKAGES=
 	export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
 	export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml"
+	test windows_nt != "$CI_OS_NAME" ||
+	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 fi
 
 skip_branch_tip_with_tag () {

gitgitgadget

"Johannes Schindelin via GitGitGadget" <[email protected]>

From: Johannes Schindelin <[email protected]>

The fact that Git's test suite is implemented in Unix shell script that is as portable as we can muster, combined with the fact that Unix shell scripting is foreign to Windows (and therefore has to be emulated), results in pretty abysmal speed of the test suite on that platform, for pretty much no other reason than that language choice.

For comparison: while the Linux build & test is typically done within about 8 minutes, the Windows build & test typically lasts about 80 minutes in Azure Pipelines.

To help with that, let's use the Azure Pipeline feature where you can parallelize jobs, make jobs depend on each other, and pass artifacts between them.

The tests are distributed using the following heuristic: listing all test scripts ordered by size in descending order (as a cheap way to estimate the overall run time), every Nth script is run (where N is the total number of parallel jobs), starting at the index corresponding to the parallel job. This slicing is performed by a new function that is added to the test-tool.

To optimize the overall runtime of the entire Pipeline, we need to move the Windows jobs to the beginning (otherwise there would be a very decent chance for the Pipeline to be run only the Windows build, while all the parallel Windows test jobs wait for this single one).

We use Azure Pipelines Artifacts for both the minimal Git for Windows SDK as well as the built executables, as deduplication and caching close to the agents makes that really fast. For comparison: while downloading and unpacking the minimal Git for Windows SDK via PowerShell takes only one minute (down from anywhere between 2.5 to 7 when using a shallow clone), uploading it as Pipeline Artifact takes less than 30s and downloading and unpacking less than 20s (sometimes even as little as only twelve seconds).

---
 Makefile                   | 10 +++++
 azure-pipelines.yml        | 82 +++++++++++++++++++++++++++++++++-----
 ci/make-test-artifacts.sh  | 12 ++++++
 ci/run-test-slice.sh       | 17 ++++++++
 t/helper/test-path-utils.c | 31 ++++++++++++++
 5 files changed, 143 insertions(+), 9 deletions(-)
 create mode 100755 ci/make-test-artifacts.sh
 create mode 100755 ci/run-test-slice.sh

index 044b4f77bd..daa318fe17 100644
@@ -2927,6 +2927,16 @@ rpm::
 	@false
 .PHONY: rpm
 
+artifacts-tar:: $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) $(OTHER_PROGRAMS) \
+		GIT-BUILD-OPTIONS $(TEST_PROGRAMS) $(test_bindir_programs) \
+		$(NO_INSTALL) $(MOFILES)
+	$(QUIET_SUBDIR0)templates $(QUIET_SUBDIR1) \
+		SHELL_PATH='$(SHELL_PATH_SQ)' PERL_PATH='$(PERL_PATH_SQ)'
+	test -n "$(ARTIFACTS_DIRECTORY)"
+	mkdir -p "$(ARTIFACTS_DIRECTORY)"
+	$(TAR) czf "$(ARTIFACTS_DIRECTORY)/artifacts.tar.gz" $^ templates/blt/
+.PHONY: artifacts-tar
+
 htmldocs = git-htmldocs-$(GIT_VERSION)
 manpages = git-manpages-$(GIT_VERSION)
 .PHONY: dist-doc distclean

new file mode 100755
index 0000000000..646967481f
@@ -0,0 +1,12 @@
+#!/bin/sh
+#
+# Build Git and store artifacts for testing
+#
+
+mkdir -p "$1" # in case ci/lib.sh decides to quit early
+
+. ${0%/*}/lib.sh
+
+make artifacts-tar ARTIFACTS_DIRECTORY="$1"
+
+check_unignored_build_artifacts

new file mode 100755
index 0000000000..f8c2c3106a
@@ -0,0 +1,17 @@
+#!/bin/sh
+#
+# Test Git in parallel
+#
+
+. ${0%/*}/lib.sh
+
+case "$CI_OS_NAME" in
+windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";;
+*) ln -s "$cache_dir/.prove" t/.prove;;
+esac
+
+make --quiet -C t T="$(cd t &&
+	./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh |
+	tr '\n' ' ')"
+
+check_unignored_build_artifacts

index 6efde6f5ba..5d543ad21f 100644
@@ -177,6 +177,14 @@ static int is_dotgitmodules(const char *path)
 	return is_hfs_dotgitmodules(path) || is_ntfs_dotgitmodules(path);
 }
 
+static int cmp_by_st_size(const void *a, const void *b)
+{
+	intptr_t x = (intptr_t)((struct string_list_item *)a)->util;
+	intptr_t y = (intptr_t)((struct string_list_item *)b)->util;
+
+	return x > y ? -1 : (x < y ? +1 : 0);
+}
+
 int cmd__path_utils(int argc, const char **argv)
 {
 	if (argc == 3 && !strcmp(argv[1], "normalize_path_copy")) {

@@ -324,6 +332,29 @@ int cmd__path_utils(int argc, const char **argv)
 		return 0;
 	}
 
+	if (argc > 5 && !strcmp(argv[1], "slice-tests")) {
+		int res = 0;
+		long offset, stride, i;
+		struct string_list list = STRING_LIST_INIT_NODUP;
+		struct stat st;
+
+		offset = strtol(argv[2], NULL, 10);
+		stride = strtol(argv[3], NULL, 10);
+		if (stride < 1)
+			stride = 1;
+		for (i = 4; i < argc; i++)
+			if (stat(argv[i], &st))
+				res = error_errno("Cannot stat '%s'", argv[i]);
+			else
+				string_list_append(&list, argv[i])->util =
+					(void *)(intptr_t)st.st_size;
+		QSORT(list.items, list.nr, cmp_by_st_size);
+		for (i = offset; i < list.nr; i+= stride)
+			printf("%s\n", list.items[i].string);
+
+		return !!res;
+	}
+
 	fprintf(stderr, "%s: unknown function name: %s\n", argv[0],
 		argv[1] ? argv[1] : "(there was none)");
 	return 1;

gitgitgadget

Hi all,

This improvement in run time is something I was very, very excited about, because it finally lets me use the cloud as my build & test machine, effectively.

And not only me! You can use it, too, because I ported part of this Azure Pipeline to GitGitGadget's Git fork, it is now the default PR build. It takes about 20 minutes to complete if nothing else is building at the time, and if it is all green, you can be relatively sure that your code is good on Windows, macOS and Linux.

If you want to test your code that way, just open a PR on
https://github.com/gititgadget/git and wait for what GitHub calls "Check" to complete.

Since the Pipeline is now so fast, and since I have to mirror the branches in gitster/git to gitgitgadget/git anyway (so that you can contribute patch series on top of other patch series that are in-flight), I also decided to turn on the same PR build as CI build: whenever Junio pushes a new branch or a branch update (and when that change is picked up by the Pipeline that mirrors it into gitgitgadget/git), the Pipeline will run, so I will never have to bisect pu again (which is a pain due to the many, many, many merge bases) but instead the respective broken branch (because I can now see which branch is broken directly).

You can even see those build outcomes yourself at

https://github.com/gitgitgadget/git/branches/active

Each of those checkmarks (or red X marks) represents a build, and you can click on it to see the Checks, click on the Details of "CI for GitGitGadget's Git fork" and finally follow the "View more details on Azure Pipelines" link to see the full build log.

For example, the build log of the ph/pack-objects-mutex-fix branch is
here: https://dev.azure.com/gitgitgadget/git/_build/results?buildId=248

Enjoy,
Johannes

"Johannes Schindelin via GitGitGadget" <[email protected]>

I left comments here and there on individual steps, but overall it was a pleasant read.

Thanks.

For a long time already, we have tested Git's source code continuously via Travis CI, see e.g. https://travis-ci.org/git/git/builds/421738884. It has served us well, and more and more developers actually pay attention and benefit from the testing this gives us.

It is also an invaluable tool for contributors who can validate their code contributions via PRs on GitHub, e.g. to verify that their tests do actually run on macOS (i.e. with the BSD family of Unix tools instead of the GNU one).

The one sad part about this is the Windows support. Travis used to lack it (Travis announced some early support for Windows, hot off the press
[https://blog.travis-ci.com/2018-10-11-windows-early-release]), and we work around that by using Azure Pipelines (the CI part of Azure DevOps, formerly known as Visual Studio Team Services) indirectly: one phase in Travis would trigger a build, wait for its log, and then paste that log.

As Git's Windows builds (and tests!) take quite a bit of time, Travis often timed out, or somehow the trigger did not work, and for security reasons (the Windows builds are performed in a private pool of containers), the Windows builds are completely disabled for Pull Requests on GitHub.

One might ask why we did not use Azure Pipelines directly. There were a couple of reasons for that:

* most notably, Azure Pipelines' build logs could not be viewed anonymously,
* while Azure Pipelines had Linux and Windows agents, it lacked macOS agents,
* etc

The main two reasons no longer apply: macOS agents are available now
[https://docs.microsoft.com/en-us/azure/devops/release-notes/2018/jul-10-vsts]
, and are public projects
[https://docs.microsoft.com/en-us/azure/devops/organizations/public/about-public-projects]
now, i.e. it is possible to configure a Azure Pipelines project so that anybody can view the logs. Since I offered v1, Azure Pipelines has been made available via the GitHub Marketplace, free of cost for open source projects.

I had secured such a public project for Git for Windows already, and I also got one for Git. For now, the latter is hooked up with my personal git.git fork on GitHub, but it is my hope that I convince y'all that these Azure Pipelines builds are a good idea, and then hook it up with
https://github.com/git/git.

As a special treat, this patch series adds the ability to present the outcome of Git's test suite as JUnit-style .xml files. This allows the Azure Pipelines build to present fun diagrams, trends, and makes it a lot easier to drill down to test failures than before. See for example
https://dev.azure.com/git/git/_build/results?buildId=113&view=ms.vss-test-web.test-result-details
[https://dev.azure.com/git/git/_build/results?buildId=113&view=ms.vss-test-web.test-result-details]
(you can click on the label of the failed test, and then see the detailed output in the right pane).

But maybe you're not interested as much in the presentation of test failures as in the speed? Well, I got you covered with that, too. As of v3, the test suite is run in parallel on Windows, cutting the overall run time to ~33 minutes (see one of the latest builds
[https://dev.azure.com/git/git/_build/results?buildId=311&view=logs], for example).

This patch series took waaaaaaaay more time than I had originally anticipated, but I think that in particular the advanced display of the test results and the reduction of the overall run time was worth it. Please let me know what you think about this.

Changes since v4:

* Clarified in the commit message why BREW_INSTALL_PACKAGES is needed.
* Added a clause to error out if we could not detect the CI type.
* Introduced a CI_TYPE variable to guard CI type-specific code.
* MAKEFLAGS is now actually configured in the CI type-specific section.
* Moved the inadvertent fixup for GIT_TEST_OPTS from "Add a build definition for Azure DevOps" to "ci/lib.sh: add support for Azure Pipelines".
* Backed out the Windows support from the commit adding the initial Azure Pipelines support, instead adding it later to help the flow of the patch series.
* Clarified in "test-date: add a subcommand to measure times in shell scripts"'s commit message that one of the goals is portability.
* Moved all of those CI type-specific definitions in ci/lib.sh after the set -ex statement (as suggested by Gábor a loooong time ago already).
* Guarded the Travis-specific way to "publish" trash directories of failed tests via base64-encoded tar files in the log; For Azure Pipelines, these are published as build artifacts instead.

Changes since v3:

* Fixed the incorrect usage of cut -c that resulted in an empty trace when failed tests were published (except if the very first test case in a test script failed, then it was correct by mistake).
* Excluded the previous test case's "ok" (or "not ok") line from the trace published with failed tests.
* Renamed TRAVIS_BRANCH to CI_BRANCH already in the first commit, as we should not override TRAVIS_BRANCH with something that it is not.
* Rebased onto current master to avoid merge conflicts with the recently-merged sg/stress-test branch (no, Junio, I really trust myself more than you to resolve those merge conflicts).

Changes since v2:

* Removed left-over debugging code that would skip a good chunk of t0000-init.sh.
* Fixed the URL of the build badge.
* Removed a trailing empty line from, and added a missing closing pointy bracket to, ci/mount-fileshare.sh.
* Moved the "travis: fix skipping tagged releases" commit up to the beginning of the patch series.
* The commit message of "ci/lib.sh: add support for Azure Pipelines" now mentions explicitly that the Homebrew packages that need to be installed on Travis' macOS agents are already installed on Azure Pipelines'.
* Some commands were not guarded by || exit 1, i.e. if they would fail, the build would not have failed.
* We now install gcc-8 for the linux-gcc job.
* We no longer try to re-run failed tests with verbose log. Instead, we simply use the verbose log to begin with. Tests showed that it had a negligible impact on the overall run time.
* The test_atexit_handler function was scratched; It would be the right thing to do, but is actually an independent topic (it was only implemented in v2 to accommodate the "re-run with verbose log on failure" trick)
* We now use a new YAML schema (most notably, "phase" is now known as "job")
* The Windows job contained PowerShell sections that were indented with 3 spaces instead of 2.
* The Windows job is now structured better, by separating different concerns into different "tasks" so that it is easier to see what exactly failed (was it the build? was it the test?)
* The Windows job was split into a job to build Git and 10 parallel jobs to run the test suite with the artifacts built by the first job. This reduces the overall run time from ~1h20 (which was the run time by the Windows job) to ~35 minutes (which is the run time of the linux-gcc job).
* The JUnit XML is now written using a test helper rather than a fragile and probably not even portable sed call.
* Since we needed to determine the file size of the verbose log (to cut out individual test cases' log output), we now introduce a test helper to do that, and use it throughout the test suite (where Perl was used before).
* It would appear that a recent change either in Cygwin or in the Azure VMs causes problems sporadically where the trash directories cannot be removed, but a subsequent rm will succeed. We now simply do that, because it won't harm the common case (where the first rm succeeds already) and because it helps the Windows job succeed pretty reliably.

Changes since v1:

* Removed a superfluous eval.
* Added the commit that fixes the Travis PR builds targeting master that just happens to be tagged (see
https://travis-ci.org/git/git/jobs/424276413 for an incorrectly-skipped build).
* The commit messages and the cover letter now reflect the name change from Visual Studio Team Services to Azure DevOps (and in particular, Azure Pipelines for the automated builds).
* Now we're using test_atexit (which we introduced for that purpose) instead of hard-coding kill_p4d and stop_git_daemon.
* The build should now also succeed for Pull Requests (where secret variables are not available, for security reasons, and as a consequence the file share cannot be mounted).
* The shell scripted parts now use proper && chains.

Johannes Schindelin (21):
travis: fix skipping tagged releases
ci: rename the library of common functions
ci/lib.sh: encapsulate Travis-specific things
ci: inherit --jobs via MAKEFLAGS in run-build-and-tests
ci: use a junction on Windows instead of a symlink
test-date: add a subcommand to measure times in shell scripts
tests: optionally write results as JUnit-style .xml ci/lib.sh: add support for Azure Pipelines
Add a build definition for Azure DevOps
ci: add a Windows job to the Azure Pipelines definition
ci: use git-sdk-64-minimal build artifact
mingw: be more generous when wrapping up the setitimer() emulation
README: add a build badge (status of the Azure Pipelines build)
tests: avoid calling Perl just to determine file sizes
tests: include detailed trace logs with --write-junit-xml upon failure
mingw: try to work around issues with the test cleanup
tests: add t/helper/ to the PATH with --with-dashes
t0061: fix with --with-dashes and RUNTIME_PREFIX
tests: optionally skip bin-wrappers/
ci: speed up Windows phase
ci: parallelize testing on Windows

 Makefile                            |  11 +
 README.md                           |   2 +
 azure-pipelines.yml                 | 387 ++++++++++++++++++++++++++++
 ci/install-dependencies.sh          |   5 +-
 ci/{lib-travisci.sh => lib.sh}      |  92 +++++--
 ci/make-test-artifacts.sh           |  12 +
 ci/mount-fileshare.sh               |  25 ++
 ci/print-test-failures.sh           |  15 +-
 ci/run-build-and-tests.sh           |   9 +-
 ci/run-linux32-build.sh             |   2 +-
 ci/run-linux32-docker.sh            |   2 +-
 ci/run-static-analysis.sh           |   4 +-
 ci/run-test-slice.sh                |  17 ++
 ci/run-windows-build.sh             |   2 +-
 ci/test-documentation.sh            |   7 +-
 compat/mingw.c                      |   2 +-
 t/.gitignore                        |   1 +
 t/README                            |   9 +
 t/helper/test-date.c                |  12 +
 t/helper/test-path-utils.c          |  64 +++++
 t/helper/test-tool.c                |   1 +
 t/helper/test-tool.h                |   1 +
 t/helper/test-xml-encode.c          |  80 ++++++
 t/t0021-conversion.sh               |   2 +-
 t/t0061-run-command.sh              |   3 +-
 t/t1050-large.sh                    |   2 +-
 t/t5315-pack-objects-compression.sh |   2 +-
 t/t9303-fast-import-compression.sh  |   2 +-
 t/test-lib.sh                       | 138 +++++++++-
 29 files changed, 862 insertions(+), 49 deletions(-)
 create mode 100644 azure-pipelines.yml
 rename ci/{lib-travisci.sh => lib.sh} (50%)
 create mode 100755 ci/make-test-artifacts.sh
 create mode 100755 ci/mount-fileshare.sh
 create mode 100755 ci/run-test-slice.sh
 create mode 100644 t/helper/test-xml-encode.c

base-commit: 16a465bc018d09e9d7bbbdc5f40a7fb99c21f8ef
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-31%2Fdscho%2Fvsts-ci-v5
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-31/dscho/vsts-ci-v5
Pull-Request: https://github.com/gitgitgadget/git/pull/31

Range-diff vs v4:

1: 6ed2f2a35c ! 1: f553fd4fb3 travis: fix skipping tagged releases

     @@ -21,14 +21,6 @@
       --- a/ci/lib-travisci.sh
       +++ b/ci/lib-travisci.sh
      @@
     - # Library of functions shared by all CI scripts
     - 
     -+# When building a PR, TRAVIS_BRANCH refers to the *target* branch. Not what we
     -+# want here. We want the source branch instead.
     -+CI_BRANCH="${TRAVIS_PULL_REQUEST_BRANCH:-$TRAVIS_BRANCH}"
     -+
     - skip_branch_tip_with_tag () {
     - 	# Sometimes, a branch is pushed at the same time the tag that points
       	# at the same commit as the tip of the branch is pushed, and building
       	# both at the same time is a waste.
       	#
     @@ -57,3 +49,14 @@
       		exit 0
       	fi
       }
     +@@
     + # and installing dependencies.
     + set -ex
     + 
     ++# When building a PR, TRAVIS_BRANCH refers to the *target* branch. Not what we
     ++# want here. We want the source branch instead.
     ++CI_BRANCH="${TRAVIS_PULL_REQUEST_BRANCH:-$TRAVIS_BRANCH}"
     ++
     + cache_dir="$HOME/travis-cache"
     + good_trees_file="$cache_dir/good-trees"
     + 
  2:  b086f36660 =  2:  f0852de8ab ci: rename the library of common functions
  3:  2d0b62f186 !  3:  7c16d31b46 ci/lib.sh: encapsulate Travis-specific things
     @@ -6,6 +6,18 @@
          (i.e. Azure DevOps' Continuous Integration), where variable names and
          URLs look a bit different than in Travis CI.
      
     +    Also, the configurations of the available agents are different. For
     +    example, Travis' and Azure Pipelines' macOS agents are set up
     +    differently, so that on Travis, we have to install the git-lfs and
     +    gettext Homebrew packages, and on Azure Pipelines we do not need to.
     +    Likewise, Azure Pipelines' Ubuntu agents already have asciidoctor
     +    installed.
     +
     +    Finally, on Azure Pipelines the natural way is not to base64-encode tar
     +    files of the trash directories of failed tests, but to publish build
     +    artifacts instead. Therefore, that code to log those base64-encoded tar
     +    files is guarded to be Travis-specific.
     +
          Signed-off-by: Johannes Schindelin <[email protected]>
      
       diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
     @@ -25,36 +37,6 @@
       diff --git a/ci/lib.sh b/ci/lib.sh
       --- a/ci/lib.sh
       +++ b/ci/lib.sh
     -@@
     - # Library of functions shared by all CI scripts
     - 
     --# When building a PR, TRAVIS_BRANCH refers to the *target* branch. Not what we
     --# want here. We want the source branch instead.
     --CI_BRANCH="${TRAVIS_PULL_REQUEST_BRANCH:-$TRAVIS_BRANCH}"
     -+if test true = "$TRAVIS"
     -+then
     -+	# When building a PR, TRAVIS_BRANCH refers to the *target* branch. Not
     -+	# what we want here. We want the source branch instead.
     -+	CI_BRANCH="${TRAVIS_PULL_REQUEST_BRANCH:-$TRAVIS_BRANCH}"
     -+	CI_COMMIT="$TRAVIS_COMMIT"
     -+	CI_JOB_ID="$TRAVIS_JOB_ID"
     -+	CI_JOB_NUMBER="$TRAVIS_JOB_NUMBER"
     -+	CI_OS_NAME="$TRAVIS_OS_NAME"
     -+	CI_REPO_SLUG="$TRAVIS_REPO_SLUG"
     -+
     -+	cache_dir="$HOME/travis-cache"
     -+
     -+	url_for_job_id () {
     -+		echo "https://travis-ci.org/$CI_REPO_SLUG/jobs/$1"
     -+	}
     -+
     -+	BREW_INSTALL_PACKAGES="git-lfs gettext"
     -+	export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
     -+	export GIT_TEST_OPTS="--verbose-log -x --immediate"
     -+fi
     - 
     - skip_branch_tip_with_tag () {
     - 	# Sometimes, a branch is pushed at the same time the tag that points
      @@
       # job if we encounter the same tree again and can provide a useful info
       # message.
     @@ -100,6 +82,35 @@
       # and installing dependencies.
       set -ex
       
     +-# When building a PR, TRAVIS_BRANCH refers to the *target* branch. Not what we
     +-# want here. We want the source branch instead.
     +-CI_BRANCH="${TRAVIS_PULL_REQUEST_BRANCH:-$TRAVIS_BRANCH}"
     ++if test true = "$TRAVIS"
     ++then
     ++	CI_TYPE=travis
     ++	# When building a PR, TRAVIS_BRANCH refers to the *target* branch. Not
     ++	# what we want here. We want the source branch instead.
     ++	CI_BRANCH="${TRAVIS_PULL_REQUEST_BRANCH:-$TRAVIS_BRANCH}"
     ++	CI_COMMIT="$TRAVIS_COMMIT"
     ++	CI_JOB_ID="$TRAVIS_JOB_ID"
     ++	CI_JOB_NUMBER="$TRAVIS_JOB_NUMBER"
     ++	CI_OS_NAME="$TRAVIS_OS_NAME"
     ++	CI_REPO_SLUG="$TRAVIS_REPO_SLUG"
     ++
     ++	cache_dir="$HOME/travis-cache"
     ++
     ++	url_for_job_id () {
     ++		echo "https://travis-ci.org/$CI_REPO_SLUG/jobs/$1"
     ++	}
     ++
     ++	BREW_INSTALL_PACKAGES="git-lfs gettext"
     ++	export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
     ++	export GIT_TEST_OPTS="--verbose-log -x --immediate"
     ++else
     ++	echo "Could not identify CI type" >&2
     ++	exit 1
     ++fi
     + 
      -cache_dir="$HOME/travis-cache"
       good_trees_file="$cache_dir/good-trees"
       
     @@ -124,14 +135,20 @@
       --- a/ci/print-test-failures.sh
       +++ b/ci/print-test-failures.sh
      @@
     - 	fi
     - done
     - 
     --if [ $combined_trash_size -gt 0 ]
     -+if [ -n "$TRAVIS_JOB_ID" -a $combined_trash_size -gt 0 ]
     - then
     - 	echo "------------------------------------------------------------------------"
     - 	echo "Trash directories embedded in this log can be extracted by running:"
     + 		test_name="${TEST_EXIT%.exit}"
     + 		test_name="${test_name##*/}"
     + 		trash_dir="trash directory.$test_name"
     ++		case "$CI_TYPE" in
     ++		travis)
     ++			;;
     ++		*)
     ++			echo "Unhandled CI type: $CI_TYPE" >&2
     ++			exit 1
     ++			;;
     ++		esac
     + 		trash_tgz_b64="trash.$test_name.base64"
     + 		if [ -d "$trash_dir" ]
     + 		then
      
       diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
       --- a/ci/test-documentation.sh
  4:  83b92a87e7 <  -:  ---------- ci: inherit --jobs via MAKEFLAGS in run-build-and-tests
  -:  ---------- >  4:  bf72fb1004 ci: inherit --jobs via MAKEFLAGS in run-build-and-tests
  5:  6c9eb4e33e =  5:  681f8e65b4 ci: use a junction on Windows instead of a symlink
  6:  aa053ed993 !  6:  ccf8bf53d7 test-date: add a subcommand to measure times in shell scripts

@@ -14,7 +14,10 @@
be able to obtain precise timings).

So let's introduce test-tool date getnanos, with an optional start
- time, that outputs preciser values.
+ time, that outputs preciser values. Note that this might not actually
+ give us nanosecond precision on some platforms, but it will give us as
+ precise information as possible, without the portability issues of shell
+ commands.

Granted, it is a bit pointless to try measuring times accurately in shell scripts, certainly to nanosecond precision. But it is better than

  7:  07fb37e0a6 =  7:  31fbe11425 tests: optionally write results as JUnit-style .xml
  8:  178dc9b789 !  8:  47fe8aa84a ci/lib.sh: add support for Azure Pipelines

@@ -6,9 +6,12 @@
variables and a function that displays the URL given the job id (to identify previous runs for known-good trees).

- For example, we do not have to install the git-lfs and gettext packages
- on Azure Pipelines' macOS agents: they are already installed, and trying
- to install them again would result in an error.
+ Because Azure Pipeline's macOS agents already have git-lfs and gettext
+ installed, we can leave BREW_INSTALL_PACKAGES empty (unlike in
+ Travis' case).
+
+ Note: this patch does not introduce an Azure Pipelines definition yet;
+ That is left for the next patch.

@@ -16,11 +19,12 @@
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@
- BREW_INSTALL_PACKAGES="git-lfs gettext"
export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save" export GIT_TEST_OPTS="--verbose-log -x --immediate"
+ export MAKEFLAGS="--jobs=2"
+elif test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI" +then
++ CI_TYPE=azure-pipelines
+ # We are running in Azure Pipelines
+ CI_BRANCH="$BUILD_SOURCEBRANCH"
+ CI_COMMIT="$BUILD_SOURCEVERSION"
@@ -41,7 +45,24 @@
+
+ BREW_INSTALL_PACKAGES=
+ export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
-+ export GIT_TEST_OPTS="--quiet --write-junit-xml"
- fi
-
- skip_branch_tip_with_tag () {
++ export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml"
++ export MAKEFLAGS="--jobs=10"
+ else
+ echo "Could not identify CI type" >&2
+ exit 1
+
+ diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh
+ --- a/ci/print-test-failures.sh
+ +++ b/ci/print-test-failures.sh
+@@
+ case "$CI_TYPE" in
+ travis)
+ ;;
++ azure-pipelines)
++ mkdir -p failed-test-artifacts
++ mv "$trash_dir" failed-test-artifacts
++ continue
++ ;;
+ *)
+ echo "Unhandled CI type: $CI_TYPE" >&2
+ exit 1
9: eaba471b89 ! 9: 6044b919b0 Add a build definition for Azure DevOps
@@ -5,6 +5,19 @@
This commit adds an azure-pipelines.yml file which is Azure DevOps' equivalent to Travis CI's .travis.yml.

+ The main idea is to replicate the Travis configuration as faithfully as
+ possible, to make it easy to compare the Azure Pipeline builds to the
+ Travis ones (spoiler: some parts, especially the macOS jobs, are way
+ faster in Azure Pileines). Meaning: the number and the order of the jobs
+ added in this commit faithfully replicates what we have in .travis.yml.
+
+ Note: Our .travis.yml configuration has a Windows part that is *not*
+ replicated in the Azure Pipelines definition. The reason is easy to see:
+ As Travis cannot support our Windws needs (even with the preliminary
+ Windows support that was recently added to Travis after waiting for
+ *years* for that feature, our test suite would simply hit Travis'
+ timeout every single time).
+
To make things a bit easier to understand, we refrain from using the matrix feature here because (while it is powerful) it can be a bit confusing to users who are not familiar with CI setups. Therefore, we
@@ -15,7 +28,9 @@
test suite can output JUnit-style .xml files. This information is made available in a nice UI that allows the viewer to filter by phase and/or test number, and to see trends such as: number of (failing) tests, time
- spent running the test suite, etc.
+ spent running the test suite, etc. (While this seemingly contradicts the
+ intention to replicate the Travis configuration as faithfully as
+ possible, it is just too nice to show off that capability here already.)

@@ -60,6 +75,12 @@
+ platform: Linux
+ publishRunAttachments: false
+ condition: succeededOrFailed()
++ - task: PublishBuildArtifacts@1
++ displayName: 'Publish trash directories of failed tests'
++ condition: failed()
++ inputs:
++ PathtoPublish: t/failed-test-artifacts
++ ArtifactName: failed-test-artifacts
+
+- job: linux_gcc
+ displayName: linux-gcc
@@ -91,6 +112,12 @@
+ platform: Linux
+ publishRunAttachments: false
+ condition: succeededOrFailed()
++ - task: PublishBuildArtifacts@1
++ displayName: 'Publish trash directories of failed tests'
++ condition: failed()
++ inputs:
++ PathtoPublish: t/failed-test-artifacts
++ ArtifactName: failed-test-artifacts
+
+- job: osx_clang
+ displayName: osx-clang
@@ -120,6 +147,12 @@
+ platform: macOS
+ publishRunAttachments: false
+ condition: succeededOrFailed()
++ - task: PublishBuildArtifacts@1
++ displayName: 'Publish trash directories of failed tests'
++ condition: failed()
++ inputs:
++ PathtoPublish: t/failed-test-artifacts
++ ArtifactName: failed-test-artifacts
+
+- job: osx_gcc
+ displayName: osx-gcc
@@ -147,6 +180,12 @@
+ platform: macOS
+ publishRunAttachments: false
+ condition: succeededOrFailed()
++ - task: PublishBuildArtifacts@1
++ displayName: 'Publish trash directories of failed tests'
++ condition: failed()
++ inputs:
++ PathtoPublish: t/failed-test-artifacts
++ ArtifactName: failed-test-artifacts
+
+- job: gettext_poison
+ displayName: GETTEXT_POISON
@@ -178,92 +217,12 @@
+ platform: Linux
+ publishRunAttachments: false
+ condition: succeededOrFailed()
-+
-+- job: windows
-+ displayName: Windows
-+ condition: succeeded()
-+ pool: Hosted
-+ timeoutInMinutes: 240
-+ steps:
-+ - powershell: |
-+ if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "$(gitfileshare.pwd)") {
-+ net use s: \\gitfileshare.file.core.windows.net\test-cache "$GITFILESHAREPWD" /user:AZURE\gitfileshare /persistent:no
-+ cmd /c mklink /d "$(Build.SourcesDirectory)\test-cache" S:\
-+ }
-+ displayName: 'Mount test-cache'
-+ env:
-+ GITFILESHAREPWD: $(gitfileshare.pwd)
-+ - powershell: |
-+ # Helper to check the error level of the latest command (exit with error when appropriate)
-+ function c() { if (!$?) { exit(1) } }
-+
-+ # Add build agent's MinGit to PATH
-+ $env:PATH = $env:AGENT_HOMEDIRECTORY +"\externals\\git\cmd;" +$env:PATH
-+
-+ # Helper to initialize (or update) a Git worktree
-+ function init ($path, $url, $set_origin) {
-+ if (Test-Path $path) {
-+ cd $path; c
-+ if (Test-Path .git) {
-+ & git init; c
-+ } else {
-+ & git status
-+ }
-+ } else {
-+ & git init $path; c
-+ cd $path; c
-+ }
-+ & git config core.autocrlf false; c
-+ & git config core.untrackedCache true; c
-+ if (($set_origin -ne 0) -and !(git config remote.origin.url)) {
-+ & git remote add origin $url; c
-+ }
-+ & git fetch --depth=1 $url master; c
-+ & git reset --hard FETCH_HEAD; c
-+ & git clean -df; c
-+ }
-+
-+ # Initialize Git for Windows' SDK
-+ $sdk_path = "$(Build.SourcesDirectory)\git-sdk-64"
-+ init "$sdk_path" "https://dev.azure.com/git-for-windows/git-sdk-64/_git/git-sdk-64" 0
-+
-+ # Let Git ignore the SDK and the test-cache
-+ "/git-sdk-64/n/test-cache/n" | Out-File -NoNewLine -Encoding ascii -Append "$(Build.SourcesDirectory)\.git\info\exclude"
-+ displayName: 'Initialize the Git for Windows SDK'
-+ - powershell: |
-+ & "git-sdk-64\git-cmd.exe" --command=usr\\bin\\bash.exe -lc @"
-+ export MAKEFLAGS=-j10
-+ export DEVELOPER=1
-+ export NO_PERL=1
-+ export NO_SVN_TESTS=1
-+ export GIT_TEST_SKIP_REBASE_P=1
-+
-+ ci/run-build-and-tests.sh || {
-+ ci/print-test-failures.sh
-+ exit 1
-+ }
-+ "@
-+ if (!$?) { exit(1) }
-+ displayName: 'Build & Test'
-+ env:
-+ HOME: $(Build.SourcesDirectory)
-+ MSYSTEM: MINGW64
-+ - powershell: |
-+ if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "$(gitfileshare.pwd)") {
-+ cmd /c rmdir "$(Build.SourcesDirectory)\test-cache"
-+ }
-+ displayName: 'Unmount test-cache'
-+ condition: true
-+ env:
-+ GITFILESHAREPWD: $(gitfileshare.pwd)
-+ - task: PublishTestResults@2
-+ displayName: 'Publish Test Results **/TEST-*.xml'
++ - task: PublishBuildArtifacts@1
++ displayName: 'Publish trash directories of failed tests'
++ condition: failed()
+ inputs:
-+ mergeTestResults: true
-+ testRunTitle: 'windows'
-+ platform: Windows
-+ publishRunAttachments: false
-+ condition: succeededOrFailed()
++ PathtoPublish: t/failed-test-artifacts
++ ArtifactName: failed-test-artifacts
+
+- job: linux32
+ displayName: Linux32
@@ -273,11 +232,14 @@
+ - bash: |
+ test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
+
-+ sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS=-j3 bash -lxc ci/run-linux32-docker.sh || exit 1
++ res=0
++ sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" bash -lxc ci/run-linux32-docker.sh || res=1
+
+ sudo chmod a+r t/out/TEST-*.xml
++ test ! -d t/failed-test-artifacts || sudo chmod a+r t/failed-test-artifacts
+
-+ test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
++ test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || res=1
++ exit $res
+ displayName: 'ci/run-linux32-docker.sh'
+ env:
+ GITFILESHAREPWD: $(gitfileshare.pwd)
@@ -289,6 +251,12 @@
+ platform: Linux
+ publishRunAttachments: false
+ condition: succeededOrFailed()
++ - task: PublishBuildArtifacts@1
++ displayName: 'Publish trash directories of failed tests'
++ condition: failed()
++ inputs:
++ PathtoPublish: t/failed-test-artifacts
++ ArtifactName: failed-test-artifacts
+
+- job: static_analysis
+ displayName: StaticAnalysis
@@ -331,19 +299,6 @@
+ env:
+ GITFILESHAREPWD: $(gitfileshare.pwd)

- diff --git a/ci/lib.sh b/ci/lib.sh
- --- a/ci/lib.sh
- +++ b/ci/lib.sh
-@@
-
- BREW_INSTALL_PACKAGES=
- export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
-- export GIT_TEST_OPTS="--quiet --write-junit-xml"
-+ export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml"
- fi
-
- skip_branch_tip_with_tag () {
-
diff --git a/ci/mount-fileshare.sh b/ci/mount-fileshare.sh new file mode 100755
--- /dev/null
10: 5bdc6a08a8 ! 10: 3a2993c826 ci: move the Windows job to the top
@@ -1,17 +1,30 @@
Author: Johannes Schindelin <[email protected]>

- ci: move the Windows job to the top
+ ci: add a Windows job to the Azure Pipelines definition

- The Windows job currently takes a whopping ~1h20m to complete. Which is
- *far* longer than the next-longest job takes (linux-gcc, ~35m). As such,
- it makes sense to start the Windows job first, to minimize the overall
- run time (which is now pretty safely the run time of the Windows job).
+ Previously, we did not have robust support for Windows in our CI
+ definition, simply because Travis cannot accommodate our needs (even
+ after Travis added experimental Windows support very recently, it takes
+ longer than Travis' 50 minute timeout to build Git and run the test
+ suite on Windows). Instead, we used a hack that started a dedicated
+ Azure Pipeline from Travis and waited for the output, often timing out
+ (which is quite fragile, as we found out).

- This affects only the Azure Pipelines configuration, not the Travis one,
- of course, as Travis cannot run our Windows job: 1h20m is distinctly
- longer than the 50 minute timeout of Travis' free tier.
+ With this commit, we finally have first-class support for Windows in our
+ CI definition (in the Azure Pipelines one, that is).

- This commit is best viewed with --color-moved.
+ Due to our reliance on Unix shell scripting in the test suite, combined
+ with the challenges on executing such scripts on Windows, the Windows
+ job currently takes a whopping ~1h20m to complete. Which is *far* longer
+ than the next-longest job takes (linux-gcc, ~35m).
+
+ Now, Azure Pipelines's free tier for open source projects (such as Git)
+ offers up to 10 concurrent jobs for free, meaning that the overall run
+ time will be dominated by the slowest job(s).
+
+ Therefore, it makes sense to start the Windows job first, to minimize
+ the time the entire build takes from start to end (which is now pretty
+ safely the run time of the Windows job).

@@ -75,7 +88,6 @@
+ displayName: 'Initialize the Git for Windows SDK'
+ - powershell: |
+ & "git-sdk-64\git-cmd.exe" --command=usr\\bin\\bash.exe -lc @"
-+ export MAKEFLAGS=-j10
+ export DEVELOPER=1
+ export NO_PERL=1
+ export NO_SVN_TESTS=1
@@ -107,100 +119,13 @@
+ platform: Windows
+ publishRunAttachments: false
+ condition: succeededOrFailed()
++ - task: PublishBuildArtifacts@1
++ displayName: 'Publish trash directories of failed tests'
++ condition: failed()
++ inputs:
++ PathtoPublish: t/failed-test-artifacts
++ ArtifactName: failed-test-artifacts
+
- job: linux_clang
displayName: linux-clang
condition: succeeded()
-@@
- publishRunAttachments: false
- condition: succeededOrFailed()
-
--- job: windows
-- displayName: Windows
-- condition: succeeded()
-- pool: Hosted
-- timeoutInMinutes: 240
-- steps:
-- - powershell: |
-- if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "$(gitfileshare.pwd)") {
-- net use s: \\gitfileshare.file.core.windows.net\test-cache "$GITFILESHAREPWD" /user:AZURE\gitfileshare /persistent:no
-- cmd /c mklink /d "$(Build.SourcesDirectory)\test-cache" S:\
-- }
-- displayName: 'Mount test-cache'
-- env:
-- GITFILESHAREPWD: $(gitfileshare.pwd)
-- - powershell: |
-- # Helper to check the error level of the latest command (exit with error when appropriate)
-- function c() { if (!$?) { exit(1) } }
--
-- # Add build agent's MinGit to PATH
-- $env:PATH = $env:AGENT_HOMEDIRECTORY +"\externals\\git\cmd;" +$env:PATH
--
-- # Helper to initialize (or update) a Git worktree
-- function init ($path, $url, $set_origin) {
-- if (Test-Path $path) {
-- cd $path; c
-- if (Test-Path .git) {
-- & git init; c
-- } else {
-- & git status
-- }
-- } else {
-- & git init $path; c
-- cd $path; c
-- }
-- & git config core.autocrlf false; c
-- & git config core.untrackedCache true; c
-- if (($set_origin -ne 0) -and !(git config remote.origin.url)) {
-- & git remote add origin $url; c
-- }
-- & git fetch --depth=1 $url master; c
-- & git reset --hard FETCH_HEAD; c
-- & git clean -df; c
-- }
--
-- # Initialize Git for Windows' SDK
-- $sdk_path = "$(Build.SourcesDirectory)\git-sdk-64"
-- init "$sdk_path" "https://dev.azure.com/git-for-windows/git-sdk-64/_git/git-sdk-64" 0
--
-- # Let Git ignore the SDK and the test-cache
-- "/git-sdk-64/n/test-cache/n" | Out-File -NoNewLine -Encoding ascii -Append "$(Build.SourcesDirectory)\.git\info\exclude"
-- displayName: 'Initialize the Git for Windows SDK'
-- - powershell: |
-- & "git-sdk-64\git-cmd.exe" --command=usr\\bin\\bash.exe -lc @"
-- export MAKEFLAGS=-j10
-- export DEVELOPER=1
-- export NO_PERL=1
-- export NO_SVN_TESTS=1
-- export GIT_TEST_SKIP_REBASE_P=1
--
-- ci/run-build-and-tests.sh || {
-- ci/print-test-failures.sh
-- exit 1
-- }
-- "@
-- if (!$?) { exit(1) }
-- displayName: 'Build & Test'
-- env:
-- HOME: $(Build.SourcesDirectory)
-- MSYSTEM: MINGW64
-- - powershell: |
-- if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "$(gitfileshare.pwd)") {
-- cmd /c rmdir "$(Build.SourcesDirectory)\test-cache"
-- }
-- displayName: 'Unmount test-cache'
-- condition: true
-- env:
-- GITFILESHAREPWD: $(gitfileshare.pwd)
-- - task: PublishTestResults@2
-- displayName: 'Publish Test Results **/TEST-*.xml'
-- inputs:
-- mergeTestResults: true
-- testRunTitle: 'windows'
-- platform: Windows
-- publishRunAttachments: false
-- condition: succeededOrFailed()
--
- - job: linux32
- displayName: Linux32
- condition: succeeded()
11: 93eebb74ce ! 11: 52d63018c9 ci: use git-sdk-64-minimal build artifact
@@ -78,6 +78,6 @@
- powershell: |
- & "git-sdk-64\git-cmd.exe" --command=usr\\bin\\bash.exe -lc @"
+ & git-sdk-64-minimal\usr\bin\bash.exe -lc @"
- export MAKEFLAGS=-j10
export DEVELOPER=1
export NO_PERL=1
+ export NO_SVN_TESTS=1

 12:  18d76823e5 = 12:  1cef14e4c0 mingw: be more generous when wrapping up the setitimer() emulation
 13:  5d593acee5 = 13:  eda46863e8 README: add a build badge (status of the Azure Pipelines build)
 14:  3083041e33 = 14:  4303bbd4e8 tests: avoid calling Perl just to determine file sizes
 15:  1c44d3b8b8 = 15:  dfd3516f39 tests: include detailed trace logs with --write-junit-xml upon failure
 16:  59c1194ae2 = 16:  9984c6d17b mingw: try to work around issues with the test cleanup
 17:  966c412f03 = 17:  a8101ae3e1 tests: add t/helper/ to the PATH with --with-dashes
 18:  d613c79aff = 18:  7518266a19 t0061: fix with --with-dashes and RUNTIME_PREFIX
 19:  a198885b76 = 19:  6a8adbe8ba tests: optionally skip bin-wrappers/
 20:  6520f4603c <  -:  ---------- ci: speed up Windows phase
  -:  ---------- > 20:  b39e16527d ci: speed up Windows phase
 21:  8bdd9804a1 ! 21:  6ff87d1772 ci: parallelize testing on Windows

@@ -78,7 +78,6 @@ $^
displayName: 'Download git-sdk-64-minimal'
- powershell: |
& git-sdk-64-minimal\usr\bin\bash.exe -lc @"
-- export MAKEFLAGS=-j10
- export DEVELOPER=1
- export NO_PERL=1
- export NO_SVN_TESTS=1
@@ -90,7 +89,6 @@ $^
+ env:
+ HOME: $(Build.SourcesDirectory)
+ MSYSTEM: MINGW64
-+ MAKEFLAGS: -j10
+ DEVELOPER: 1
+ NO_PERL: 1
+ - task: PublishPipelineArtifact@0
@@ -162,7 +160,6 @@ $^
env:
HOME: $(Build.SourcesDirectory)
MSYSTEM: MINGW64
-+ MAKEFLAGS: -j10
+ NO_SVN_TESTS: 1
+ GIT_TEST_SKIP_REBASE_P: 1
- powershell: |

gitgitgadget

From: Johannes Schindelin <[email protected]>

When building a PR, TRAVIS_BRANCH refers to the *target branch*. Therefore, if a PR targets master, and master happened to be tagged, we skipped the build by mistake.

Fix this by using TRAVIS_PULL_REQUEST_BRANCH (i.e. the *source branch*) when available, falling back to TRAVIS_BRANCH (i.e. for CI builds, also known as "push builds").

Let's give it a new variable name, too: CI_BRANCH (as it is different from TRAVIS_BRANCH). This also prepares for the upcoming patches which will make our ci/* code a bit more independent from Travis and open it to other CI systems (in particular to Azure Pipelines).

---
 ci/lib-travisci.sh | 25 ++++++++++++++-----------
 1 file changed, 14 insertions(+), 11 deletions(-)

index 69dff4d1ec..9c6ddeb374 100755
@@ -5,18 +5,17 @@ skip_branch_tip_with_tag () {
 	# at the same commit as the tip of the branch is pushed, and building
 	# both at the same time is a waste.
 	#
-	# Travis gives a tagname e.g. v2.14.0 in $TRAVIS_BRANCH when
-	# the build is triggered by a push to a tag.  Let's see if
-	# $TRAVIS_BRANCH is exactly at a tag, and if so, if it is
-	# different from $TRAVIS_BRANCH.  That way, we can tell if
-	# we are building the tip of a branch that is tagged and
-	# we can skip the build because we won't be skipping a build
-	# of a tag.
-
-	if TAG=$(git describe --exact-match "$TRAVIS_BRANCH" 2>/dev/null) &&
-		test "$TAG" != "$TRAVIS_BRANCH"
+	# When the build is triggered by a push to a tag, $CI_BRANCH will
+	# have that tagname, e.g. v2.14.0.  Let's see if $CI_BRANCH is
+	# exactly at a tag, and if so, if it is different from $CI_BRANCH.
+	# That way, we can tell if we are building the tip of a branch that
+	# is tagged and we can skip the build because we won't be skipping a
+	# build of a tag.
+
+	if TAG=$(git describe --exact-match "$CI_BRANCH" 2>/dev/null) &&
+		test "$TAG" != "$CI_BRANCH"
 	then
-		echo "$(tput setaf 2)Tip of $TRAVIS_BRANCH is exactly at $TAG$(tput sgr0)"
+		echo "$(tput setaf 2)Tip of $CI_BRANCH is exactly at $TAG$(tput sgr0)"
 		exit 0
 	fi
 }

@@ -81,6 +80,10 @@ check_unignored_build_artifacts ()
 # and installing dependencies.
 set -ex
 
+# When building a PR, TRAVIS_BRANCH refers to the *target* branch. Not what we
+# want here. We want the source branch instead.
+CI_BRANCH="${TRAVIS_PULL_REQUEST_BRANCH:-$TRAVIS_BRANCH}"
+
 cache_dir="$HOME/travis-cache"
 good_trees_file="$cache_dir/good-trees"
 

gitgitgadget

From: Johannes Schindelin <[email protected]>

The name is hard-coded to reflect that we use Travis CI for continuous testing.

In the next commits, we will extend this to be able use Azure DevOps, too.

So let's adjust the name to make it more generic.

---
 ci/install-dependencies.sh     | 2 +-
 ci/{lib-travisci.sh => lib.sh} | 0
 ci/print-test-failures.sh      | 2 +-
 ci/run-build-and-tests.sh      | 2 +-
 ci/run-linux32-docker.sh       | 2 +-
 ci/run-static-analysis.sh      | 2 +-
 ci/run-windows-build.sh        | 2 +-
 ci/test-documentation.sh       | 2 +-
 8 files changed, 7 insertions(+), 7 deletions(-)
 rename ci/{lib-travisci.sh => lib.sh} (100%)

index 06c3546e1e..fe65144152 100755
@@ -3,7 +3,7 @@
 # Install dependencies required to build and test Git on Linux and macOS
 #
 
-. ${0%/*}/lib-travisci.sh
+. ${0%/*}/lib.sh
 
 P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION
 LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION

similarity index 100%
rename from ci/lib-travisci.sh
rename to ci/lib.sh

index d55460a212..7aef39a2fd 100755
@@ -3,7 +3,7 @@
 # Print output of failing tests
 #
 
-. ${0%/*}/lib-travisci.sh
+. ${0%/*}/lib.sh
 
 # Tracing executed commands would produce too much noise in the loop below.
 set +x

index cda170d5c2..db342bb6a8 100755
@@ -3,7 +3,7 @@
 # Build and test Git
 #
 
-. ${0%/*}/lib-travisci.sh
+. ${0%/*}/lib.sh
 
 ln -s "$cache_dir/.prove" t/.prove
 

index 21637903ce..751acfcf8a 100755
@@ -3,7 +3,7 @@
 # Download and run Docker image to build and test 32-bit Git
 #
 
-. ${0%/*}/lib-travisci.sh
+. ${0%/*}/lib.sh
 
 docker pull daald/ubuntu32:xenial
 

index 5688f261d0..dc189c7456 100755
@@ -3,7 +3,7 @@
 # Perform various static code analysis checks
 #
 
-. ${0%/*}/lib-travisci.sh
+. ${0%/*}/lib.sh
 
 make --jobs=2 coccicheck
 

index d99a180e52..a73a4eca0a 100755
@@ -6,7 +6,7 @@
 # supported) and a commit hash.
 #
 
-. ${0%/*}/lib-travisci.sh
+. ${0%/*}/lib.sh
 
 test $# -ne 2 && echo "Unexpected number of parameters" && exit 1
 test -z "$GFW_CI_TOKEN" && echo "GFW_CI_TOKEN not defined" && exit

index a20de9ca12..d3cdbac73f 100755
@@ -3,7 +3,7 @@
 # Perform sanity checks on documentation and build it.
 #
 
-. ${0%/*}/lib-travisci.sh
+. ${0%/*}/lib.sh
 
 gem install asciidoctor
 

gitgitgadget

From: Johannes Schindelin <[email protected]>

The upcoming patches will allow building git.git via Azure Pipelines (i.e. Azure DevOps' Continuous Integration), where variable names and URLs look a bit different than in Travis CI.

Also, the configurations of the available agents are different. For example, Travis' and Azure Pipelines' macOS agents are set up differently, so that on Travis, we have to install the git-lfs and gettext Homebrew packages, and on Azure Pipelines we do not need to. Likewise, Azure Pipelines' Ubuntu agents already have asciidoctor installed.

Finally, on Azure Pipelines the natural way is not to base64-encode tar files of the trash directories of failed tests, but to publish build artifacts instead. Therefore, that code to log those base64-encoded tar files is guarded to be Travis-specific.

---
 ci/install-dependencies.sh |  3 ++-
 ci/lib.sh                  | 45 +++++++++++++++++++++++++++-----------
 ci/print-test-failures.sh  |  8 +++++++
 ci/test-documentation.sh   |  1 +
 4 files changed, 43 insertions(+), 14 deletions(-)

index fe65144152..bcdcc71592 100755
@@ -37,7 +37,8 @@ osx-clang|osx-gcc)
 	brew update --quiet
 	# Uncomment this if you want to run perf tests:
 	# brew install gnu-time
-	brew install git-lfs gettext
+	test -z "$BREW_INSTALL_PACKAGES" ||
+	brew install $BREW_INSTALL_PACKAGES
 	brew link --force gettext
 	brew install caskroom/cask/perforce
 	;;

index 7aef39a2fd..cf321b474d 100755
@@ -38,6 +38,14 @@ do
 		test_name="${TEST_EXIT%.exit}"
 		test_name="${test_name##*/}"
 		trash_dir="trash directory.$test_name"
+		case "$CI_TYPE" in
+		travis)
+			;;
+		*)
+			echo "Unhandled CI type: $CI_TYPE" >&2
+			exit 1
+			;;
+		esac
 		trash_tgz_b64="trash.$test_name.base64"
 		if [ -d "$trash_dir" ]
 		then

index d3cdbac73f..7d0beb2832 100755
@@ -5,6 +5,7 @@
 
 . ${0%/*}/lib.sh
 
+test -n "$ALREADY_HAVE_ASCIIDOCTOR" ||
 gem install asciidoctor
 
 make check-builtins

gitgitgadget

From: Johannes Schindelin <[email protected]>

Let's not decide in the generic ci/ part how many jobs to run in parallel; different CI configurations would favor a different number of parallel jobs, and it is easy enough to hand that information down via the MAKEFLAGS variable.

---
 ci/lib.sh                 | 1 +
 ci/run-build-and-tests.sh | 2 +-
 ci/run-linux32-build.sh   | 2 +-
 ci/run-static-analysis.sh | 2 +-
 ci/test-documentation.sh  | 4 ++--
 5 files changed, 6 insertions(+), 5 deletions(-)

index 3f286d86a6..32a28fd209 100755
@@ -101,6 +101,7 @@ then
 	BREW_INSTALL_PACKAGES="git-lfs gettext"
 	export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
 	export GIT_TEST_OPTS="--verbose-log -x --immediate"
+	export MAKEFLAGS="--jobs=2"
 else
 	echo "Could not identify CI type" >&2
 	exit 1

index db342bb6a8..80d72d120f 100755
@@ -7,7 +7,7 @@
 
 ln -s "$cache_dir/.prove" t/.prove
 
-make --jobs=2
+make
 make --quiet test
 if test "$jobname" = "linux-gcc"
 then

index 2c60d2e70a..09e9276e12 100755
@@ -55,6 +55,6 @@ linux32 --32bit i386 su -m -l $CI_USER -c '
 	set -ex
 	cd /usr/src/git
 	test -n "$cache_dir" && ln -s "$cache_dir/.prove" t/.prove
-	make --jobs=2
+	make
 	make --quiet test
 '

index dc189c7456..a19aa7ebbc 100755
@@ -5,7 +5,7 @@
 
 . ${0%/*}/lib.sh
 
-make --jobs=2 coccicheck
+make coccicheck
 
 set +x
 

index 7d0beb2832..be3b7d376a 100755
@@ -12,7 +12,7 @@ make check-builtins
 make check-docs
 
 # Build docs with AsciiDoc
-make --jobs=2 doc > >(tee stdout.log) 2> >(tee stderr.log >&2)
+make doc > >(tee stdout.log) 2> >(tee stderr.log >&2)
 ! test -s stderr.log
 test -s Documentation/git.html
 test -s Documentation/git.xml

@@ -24,7 +24,7 @@ check_unignored_build_artifacts
 
 # Build docs with AsciiDoctor
 make clean
-make --jobs=2 USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.log >&2)
+make USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.log >&2)
 sed '/^GIT_VERSION = / d' stderr.log
 ! test -s stderr.log
 test -s Documentation/git.html

gitgitgadget

From: Johannes Schindelin <[email protected]>

Symbolic links are still not quite as easy to use on Windows as on Linux (for example, on versions older than Windows 10, only administrators can create symlinks, and on Windows 10 you still need to be in developer mode for regular users to have permission), but NTFS junctions can give us a way out.

---
 ci/run-build-and-tests.sh | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

index 80d72d120f..74d838ea01 100755
@@ -5,7 +5,10 @@
 
 . ${0%/*}/lib.sh
 
-ln -s "$cache_dir/.prove" t/.prove
+case "$CI_OS_NAME" in
+windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";;
+*) ln -s "$cache_dir/.prove" t/.prove;;
+esac
 
 make
 make --quiet test

gitgitgadget

From: Johannes Schindelin <[email protected]>

In the next commit, we want to teach Git's test suite to optionally output test results in JUnit-style .xml files. These files contain information about the time spent. So we need a way to measure time.

While we could use date +%s for that, this will give us only seconds, i.e. very coarse-grained timings.

GNU date supports date +%s.%N (i.e. nanosecond-precision output), but there is no equivalent in BSD date (read: on macOS, we would not be able to obtain precise timings).

So let's introduce test-tool date getnanos, with an optional start time, that outputs preciser values. Note that this might not actually give us nanosecond precision on some platforms, but it will give us as precise information as possible, without the portability issues of shell commands.

Granted, it is a bit pointless to try measuring times accurately in shell scripts, certainly to nanosecond precision. But it is better than second-granularity.

---
 t/helper/test-date.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

index a0837371ab..792a805374 100644
@@ -7,6 +7,7 @@ static const char *usage_msg = "\n"
 "  test-tool date parse [date]...\n"
 "  test-tool date approxidate [date]...\n"
 "  test-tool date timestamp [date]...\n"
+"  test-tool date getnanos [start-nanos]\n"
 "  test-tool date is64bit\n"
 "  test-tool date time_t-is64bit\n";
 

@@ -82,6 +83,15 @@ static void parse_approx_timestamp(const char **argv, struct timeval *now)
 	}
 }
 
+static void getnanos(const char **argv, struct timeval *now)
+{
+	double seconds = getnanotime() / 1.0e9;
+
+	if (*argv)
+		seconds -= strtod(*argv, NULL);
+	printf("%lf\n", seconds);
+}
+
 int cmd__date(int argc, const char **argv)
 {
 	struct timeval now;

@@ -108,6 +118,8 @@ int cmd__date(int argc, const char **argv)
 		parse_approxidate(argv+1, &now);
 	else if (!strcmp(*argv, "timestamp"))
 		parse_approx_timestamp(argv+1, &now);
+	else if (!strcmp(*argv, "getnanos"))
+		getnanos(argv+1, &now);
 	else if (!strcmp(*argv, "is64bit"))
 		return sizeof(timestamp_t) == 8 ? 0 : 1;
 	else if (!strcmp(*argv, "time_t-is64bit"))

gitgitgadget

From: Johannes Schindelin <[email protected]>

This will come in handy when publishing the results of Git's test suite during an automated Azure DevOps run.

Note: we need to make extra sure that invalid UTF-8 encoding is turned into valid UTF-8 (using the Replacement Character, \uFFFD) because t9902's trace contains such invalid byte sequences, and the task in the Azure Pipeline that uploads the test results would refuse to do anything if it was asked to parse an .xml file with invalid UTF-8 in it.

---
 Makefile                   |  1 +
 t/.gitignore               |  1 +
 t/helper/test-tool.c       |  1 +
 t/helper/test-tool.h       |  1 +
 t/helper/test-xml-encode.c | 80 +++++++++++++++++++++++++++++++++
 t/test-lib.sh              | 91 ++++++++++++++++++++++++++++++++++++++
 6 files changed, 175 insertions(+)
 create mode 100644 t/helper/test-xml-encode.c

index 1a44c811aa..044b4f77bd 100644
@@ -754,6 +754,7 @@ TEST_BUILTINS_OBJS += test-submodule-config.o
 TEST_BUILTINS_OBJS += test-submodule-nested-repo-config.o
 TEST_BUILTINS_OBJS += test-subprocess.o
 TEST_BUILTINS_OBJS += test-urlmatch-normalization.o
+TEST_BUILTINS_OBJS += test-xml-encode.o
 TEST_BUILTINS_OBJS += test-wildmatch.o
 TEST_BUILTINS_OBJS += test-windows-named-pipe.o
 TEST_BUILTINS_OBJS += test-write-cache.o

index 348715f0e4..91cf5772fe 100644
@@ -2,3 +2,4 @@
 /test-results
 /.prove
 /chainlinttmp
+/out/

index bfb195b1a8..4b4b397d93 100644
@@ -49,6 +49,7 @@ static struct test_cmd cmds[] = {
 	{ "submodule-nested-repo-config", cmd__submodule_nested_repo_config },
 	{ "subprocess", cmd__subprocess },
 	{ "urlmatch-normalization", cmd__urlmatch_normalization },
+	{ "xml-encode", cmd__xml_encode },
 	{ "wildmatch", cmd__wildmatch },
 #ifdef GIT_WINDOWS_NATIVE
 	{ "windows-named-pipe", cmd__windows_named_pipe },

index 042f12464b..c0ab65e370 100644
@@ -45,6 +45,7 @@ int cmd__submodule_config(int argc, const char **argv);
 int cmd__submodule_nested_repo_config(int argc, const char **argv);
 int cmd__subprocess(int argc, const char **argv);
 int cmd__urlmatch_normalization(int argc, const char **argv);
+int cmd__xml_encode(int argc, const char **argv);
 int cmd__wildmatch(int argc, const char **argv);
 #ifdef GIT_WINDOWS_NATIVE
 int cmd__windows_named_pipe(int argc, const char **argv);

gitgitgadget

From: Johannes Schindelin <[email protected]>

This patch introduces a conditional arm that defines some environment variables and a function that displays the URL given the job id (to identify previous runs for known-good trees).

Because Azure Pipeline's macOS agents already have git-lfs and gettext installed, we can leave BREW_INSTALL_PACKAGES empty (unlike in Travis' case).

Note: this patch does not introduce an Azure Pipelines definition yet; That is left for the next patch.

---
 ci/lib.sh                 | 25 +++++++++++++++++++++++++
 ci/print-test-failures.sh |  5 +++++
 2 files changed, 30 insertions(+)

index 32a28fd209..5505776876 100755
@@ -102,6 +102,31 @@ then
 	export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
 	export GIT_TEST_OPTS="--verbose-log -x --immediate"
 	export MAKEFLAGS="--jobs=2"
+elif test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI"
+then
+	CI_TYPE=azure-pipelines
+	# We are running in Azure Pipelines
+	CI_BRANCH="$BUILD_SOURCEBRANCH"
+	CI_COMMIT="$BUILD_SOURCEVERSION"
+	CI_JOB_ID="$BUILD_BUILDID"
+	CI_JOB_NUMBER="$BUILD_BUILDNUMBER"
+	CI_OS_NAME="$(echo "$AGENT_OS" | tr A-Z a-z)"
+	test darwin != "$CI_OS_NAME" || CI_OS_NAME=osx
+	CI_REPO_SLUG="$(expr "$BUILD_REPOSITORY_URI" : '.*/\([^/]*/[^/]*\)$')"
+	CC="${CC:-gcc}"
+
+	# use a subdirectory of the cache dir (because the file share is shared
+	# among *all* phases)
+	cache_dir="$HOME/test-cache/$SYSTEM_PHASENAME"
+
+	url_for_job_id () {
+		echo "$SYSTEM_TASKDEFINITIONSURI$SYSTEM_TEAMPROJECT/_build/results?buildId=$1"
+	}
+
+	BREW_INSTALL_PACKAGES=
+	export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
+	export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml"
+	export MAKEFLAGS="--jobs=10"
 else
 	echo "Could not identify CI type" >&2
 	exit 1

index cf321b474d..e688a26f0d 100755
@@ -41,6 +41,11 @@ do
 		case "$CI_TYPE" in
 		travis)
 			;;
+		azure-pipelines)
+			mkdir -p failed-test-artifacts
+			mv "$trash_dir" failed-test-artifacts
+			continue
+			;;
 		*)
 			echo "Unhandled CI type: $CI_TYPE" >&2
 			exit 1

gitgitgadget

From: Johannes Schindelin <[email protected]>

This commit adds an azure-pipelines.yml file which is Azure DevOps' equivalent to Travis CI's .travis.yml.

The main idea is to replicate the Travis configuration as faithfully as possible, to make it easy to compare the Azure Pipeline builds to the Travis ones (spoiler: some parts, especially the macOS jobs, are way faster in Azure Pileines). Meaning: the number and the order of the jobs added in this commit faithfully replicates what we have in .travis.yml.

Note: Our .travis.yml configuration has a Windows part that is *not* replicated in the Azure Pipelines definition. The reason is easy to see:
As Travis cannot support our Windws needs (even with the preliminary Windows support that was recently added to Travis after waiting for *years* for that feature, our test suite would simply hit Travis' timeout every single time).

To make things a bit easier to understand, we refrain from using the matrix feature here because (while it is powerful) it can be a bit confusing to users who are not familiar with CI setups. Therefore, we use a separate phase even for similar configurations (such as GCC vs Clang on Linux, GCC vs Clang on macOS).

Also, we make use of the shiny new feature we just introduced where the test suite can output JUnit-style .xml files. This information is made available in a nice UI that allows the viewer to filter by phase and/or test number, and to see trends such as: number of (failing) tests, time spent running the test suite, etc. (While this seemingly contradicts the intention to replicate the Travis configuration as faithfully as possible, it is just too nice to show off that capability here already.)

---
 azure-pipelines.yml   | 259 ++++++++++++++++++++++++++++++++++++++++++
 ci/mount-fileshare.sh |  25 ++++
 2 files changed, 284 insertions(+)
 create mode 100644 azure-pipelines.yml
 create mode 100755 ci/mount-fileshare.sh

new file mode 100644
index 0000000000..8cdef105c6
@@ -0,0 +1,259 @@
+resources:
+- repo: self
+  fetchDepth: 1
+
+jobs:
+- job: linux_clang
+  displayName: linux-clang
+  condition: succeeded()
+  pool: Hosted Ubuntu 1604
+  steps:
+  - bash: |
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
+
+       sudo apt-get update &&
+       sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev apache2-bin &&
+
+       export CC=clang || exit 1
+
+       ci/install-dependencies.sh || exit 1
+       ci/run-build-and-tests.sh || {
+           ci/print-test-failures.sh
+           exit 1
+       }
+
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
+    displayName: 'ci/run-build-and-tests.sh'
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)
+  - task: PublishTestResults@2
+    displayName: 'Publish Test Results **/TEST-*.xml'
+    inputs:
+      mergeTestResults: true
+      testRunTitle: 'linux-clang'
+      platform: Linux
+      publishRunAttachments: false
+    condition: succeededOrFailed()
+  - task: PublishBuildArtifacts@1
+    displayName: 'Publish trash directories of failed tests'
+    condition: failed()
+    inputs:
+      PathtoPublish: t/failed-test-artifacts
+      ArtifactName: failed-test-artifacts
+
+- job: linux_gcc
+  displayName: linux-gcc
+  condition: succeeded()
+  pool: Hosted Ubuntu 1604
+  steps:
+  - bash: |
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
+
+       sudo add-apt-repository ppa:ubuntu-toolchain-r/test &&
+       sudo apt-get update &&
+       sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev apache2 language-pack-is git-svn gcc-8 || exit 1
+
+       ci/install-dependencies.sh || exit 1
+       ci/run-build-and-tests.sh || {
+           ci/print-test-failures.sh
+           exit 1
+       }
+
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
+    displayName: 'ci/run-build-and-tests.sh'
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)
+  - task: PublishTestResults@2
+    displayName: 'Publish Test Results **/TEST-*.xml'
+    inputs:
+      mergeTestResults: true
+      testRunTitle: 'linux-gcc'
+      platform: Linux
+      publishRunAttachments: false
+    condition: succeededOrFailed()
+  - task: PublishBuildArtifacts@1
+    displayName: 'Publish trash directories of failed tests'
+    condition: failed()
+    inputs:
+      PathtoPublish: t/failed-test-artifacts
+      ArtifactName: failed-test-artifacts
+
+- job: osx_clang
+  displayName: osx-clang
+  condition: succeeded()
+  pool: Hosted macOS
+  steps:
+  - bash: |
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
+
+       export CC=clang
+
+       ci/install-dependencies.sh || exit 1
+       ci/run-build-and-tests.sh || {
+           ci/print-test-failures.sh
+           exit 1
+       }
+
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || umount "$HOME/test-cache" || exit 1
+    displayName: 'ci/run-build-and-tests.sh'
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)
+  - task: PublishTestResults@2
+    displayName: 'Publish Test Results **/TEST-*.xml'
+    inputs:
+      mergeTestResults: true
+      testRunTitle: 'osx-clang'
+      platform: macOS
+      publishRunAttachments: false
+    condition: succeededOrFailed()
+  - task: PublishBuildArtifacts@1
+    displayName: 'Publish trash directories of failed tests'
+    condition: failed()
+    inputs:
+      PathtoPublish: t/failed-test-artifacts
+      ArtifactName: failed-test-artifacts
+
+- job: osx_gcc
+  displayName: osx-gcc
+  condition: succeeded()
+  pool: Hosted macOS
+  steps:
+  - bash: |
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
+
+       ci/install-dependencies.sh || exit 1
+       ci/run-build-and-tests.sh || {
+           ci/print-test-failures.sh
+           exit 1
+       }
+
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || umount "$HOME/test-cache" || exit 1
+    displayName: 'ci/run-build-and-tests.sh'
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)
+  - task: PublishTestResults@2
+    displayName: 'Publish Test Results **/TEST-*.xml'
+    inputs:
+      mergeTestResults: true
+      testRunTitle: 'osx-gcc'
+      platform: macOS
+      publishRunAttachments: false
+    condition: succeededOrFailed()
+  - task: PublishBuildArtifacts@1
+    displayName: 'Publish trash directories of failed tests'
+    condition: failed()
+    inputs:
+      PathtoPublish: t/failed-test-artifacts
+      ArtifactName: failed-test-artifacts
+
+- job: gettext_poison
+  displayName: GETTEXT_POISON
+  condition: succeeded()
+  pool: Hosted Ubuntu 1604
+  steps:
+  - bash: |
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
+
+       sudo apt-get update &&
+       sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev &&
+
+       export jobname=GETTEXT_POISON || exit 1
+
+       ci/run-build-and-tests.sh || {
+           ci/print-test-failures.sh
+           exit 1
+       }
+
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
+    displayName: 'ci/run-build-and-tests.sh'
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)
+  - task: PublishTestResults@2
+    displayName: 'Publish Test Results **/TEST-*.xml'
+    inputs:
+      mergeTestResults: true
+      testRunTitle: 'gettext-poison'
+      platform: Linux
+      publishRunAttachments: false
+    condition: succeededOrFailed()
+  - task: PublishBuildArtifacts@1
+    displayName: 'Publish trash directories of failed tests'
+    condition: failed()
+    inputs:
+      PathtoPublish: t/failed-test-artifacts
+      ArtifactName: failed-test-artifacts
+
+- job: linux32
+  displayName: Linux32
+  condition: succeeded()
+  pool: Hosted Ubuntu 1604
+  steps:
+  - bash: |
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
+
+       res=0
+       sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" bash -lxc ci/run-linux32-docker.sh || res=1
+
+       sudo chmod a+r t/out/TEST-*.xml
+       test ! -d t/failed-test-artifacts || sudo chmod a+r t/failed-test-artifacts
+
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || res=1
+       exit $res
+    displayName: 'ci/run-linux32-docker.sh'
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)
+  - task: PublishTestResults@2
+    displayName: 'Publish Test Results **/TEST-*.xml'
+    inputs:
+      mergeTestResults: true
+      testRunTitle: 'linux32'
+      platform: Linux
+      publishRunAttachments: false
+    condition: succeededOrFailed()
+  - task: PublishBuildArtifacts@1
+    displayName: 'Publish trash directories of failed tests'
+    condition: failed()
+    inputs:
+      PathtoPublish: t/failed-test-artifacts
+      ArtifactName: failed-test-artifacts
+
+- job: static_analysis
+  displayName: StaticAnalysis
+  condition: succeeded()
+  pool: Hosted Ubuntu 1604
+  steps:
+  - bash: |
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
+
+       sudo apt-get update &&
+       sudo apt-get install -y coccinelle &&
+
+       export jobname=StaticAnalysis &&
+
+       ci/run-static-analysis.sh || exit 1
+
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
+    displayName: 'ci/run-static-analysis.sh'
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)
+
+- job: documentation
+  displayName: Documentation
+  condition: succeeded()
+  pool: Hosted Ubuntu 1604
+  steps:
+  - bash: |
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
+
+       sudo apt-get update &&
+       sudo apt-get install -y asciidoc xmlto asciidoctor &&
+
+       export ALREADY_HAVE_ASCIIDOCTOR=yes. &&
+       export jobname=Documentation &&
+
+       ci/test-documentation.sh || exit 1
+
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
+    displayName: 'ci/test-documentation.sh'
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)

new file mode 100755
index 0000000000..26b58a8096
@@ -0,0 +1,25 @@
+#!/bin/sh
+
+die () {
+	echo "$*" >&2
+	exit 1
+}
+
+test $# = 4 ||
+die "Usage: $0 <share> <username> <password> <mountpoint>"
+
+mkdir -p "$4" || die "Could not create $4"
+
+case "$(uname -s)" in
+Linux)
+	sudo mount -t cifs -o vers=3.0,username="$2",password="$3",dir_mode=0777,file_mode=0777,serverino "$1" "$4"
+	;;
+Darwin)
+	pass="$(echo "$3" | sed -e 's/\//%2F/g' -e 's/+/%2B/g')" &&
+	mount -t smbfs,soft "smb://$2:$pass@${1#//}" "$4"
+	;;
+*)
+	die "No support for $(uname -s)"
+	;;
+esac ||
+die "Could not mount $4"

gitgitgadget

From: Johannes Schindelin <[email protected]>

Previously, we did not have robust support for Windows in our CI definition, simply because Travis cannot accommodate our needs (even after Travis added experimental Windows support very recently, it takes longer than Travis' 50 minute timeout to build Git and run the test suite on Windows). Instead, we used a hack that started a dedicated Azure Pipeline from Travis and waited for the output, often timing out (which is quite fragile, as we found out).

With this commit, we finally have first-class support for Windows in our CI definition (in the Azure Pipelines one, that is).

Due to our reliance on Unix shell scripting in the test suite, combined with the challenges on executing such scripts on Windows, the Windows job currently takes a whopping ~1h20m to complete. Which is *far* longer than the next-longest job takes (linux-gcc, ~35m).

Now, Azure Pipelines's free tier for open source projects (such as Git) offers up to 10 concurrent jobs for free, meaning that the overall run time will be dominated by the slowest job(s).

Therefore, it makes sense to start the Windows job first, to minimize the time the entire build takes from start to end (which is now pretty safely the run time of the Windows job).

---
 azure-pipelines.yml | 91 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 91 insertions(+)

index 8cdef105c6..0f7b2125a1 100644
@@ -3,6 +3,97 @@ resources:
   fetchDepth: 1
 
 jobs:
+- job: windows
+  displayName: Windows
+  condition: succeeded()
+  pool: Hosted
+  timeoutInMinutes: 240
+  steps:
+  - powershell: |
+      if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") {
+        net use s: \\gitfileshare.file.core.windows.net\test-cache "$GITFILESHAREPWD" /user:AZURE\gitfileshare /persistent:no
+        cmd /c mklink /d "$(Build.SourcesDirectory)\test-cache" S:\
+      }
+    displayName: 'Mount test-cache'
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)
+  - powershell: |
+      # Helper to check the error level of the latest command (exit with error when appropriate)
+      function c() { if (!$?) { exit(1) } }
+
+      # Add build agent's MinGit to PATH
+      $env:PATH = $env:AGENT_HOMEDIRECTORY +"\externals\\git\cmd;" +$env:PATH
+
+      # Helper to initialize (or update) a Git worktree
+      function init ($path, $url, $set_origin) {
+        if (Test-Path $path) {
+          cd $path; c
+          if (Test-Path .git) {
+            & git init; c
+          } else {
+            & git status
+          }
+        } else {
+          & git init $path; c
+          cd $path; c
+        }
+        & git config core.autocrlf false; c
+        & git config core.untrackedCache true; c
+        if (($set_origin -ne 0) -and !(git config remote.origin.url)) {
+          & git remote add origin $url; c
+        }
+        & git fetch --depth=1 $url master; c
+        & git reset --hard FETCH_HEAD; c
+        & git clean -df; c
+      }
+
+      # Initialize Git for Windows' SDK
+      $sdk_path = "$(Build.SourcesDirectory)\git-sdk-64"
+      init "$sdk_path" "https://dev.azure.com/git-for-windows/git-sdk-64/_git/git-sdk-64" 0
+
+      # Let Git ignore the SDK and the test-cache
+      "/git-sdk-64/`n/test-cache/`n" | Out-File -NoNewLine -Encoding ascii -Append "$(Build.SourcesDirectory)\.git\info\exclude"
+    displayName: 'Initialize the Git for Windows SDK'
+  - powershell: |
+      & "git-sdk-64\git-cmd.exe" --command=usr\\bin\\bash.exe -lc @"
+        export DEVELOPER=1
+        export NO_PERL=1
+        export NO_SVN_TESTS=1
+        export GIT_TEST_SKIP_REBASE_P=1
+
+        ci/run-build-and-tests.sh || {
+          ci/print-test-failures.sh
+          exit 1
+        }
+      "@
+      if (!$?) { exit(1) }
+    displayName: 'Build & Test'
+    env:
+      HOME: $(Build.SourcesDirectory)
+      MSYSTEM: MINGW64
+  - powershell: |
+      if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") {
+        cmd /c rmdir "$(Build.SourcesDirectory)\test-cache"
+      }
+    displayName: 'Unmount test-cache'
+    condition: true
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)
+  - task: PublishTestResults@2
+    displayName: 'Publish Test Results **/TEST-*.xml'
+    inputs:
+      mergeTestResults: true
+      testRunTitle: 'windows'
+      platform: Windows
+      publishRunAttachments: false
+    condition: succeededOrFailed()
+  - task: PublishBuildArtifacts@1
+    displayName: 'Publish trash directories of failed tests'
+    condition: failed()
+    inputs:
+      PathtoPublish: t/failed-test-artifacts
+      ArtifactName: failed-test-artifacts
+
 - job: linux_clang
   displayName: linux-clang
   condition: succeeded()

gitgitgadget

From: Johannes Schindelin <[email protected]>

Instead of a shallow fetch followed by a sparse checkout, we are better off by using a separate, dedicated Pipeline that bundles the SDK as a build artifact, and then consuming that build artifact here.

In fact, since this artifact will be used a lot, we spent substantial time on figuring out a minimal subset of the Git for Windows SDK, just enough to build and test Git. The result is a size reduction from around 1GB (compressed) to around 55MB (compressed). This also comes with the change where we now call usr\bin\bash.exe directly, as git-cmd.exe is not included in the minimal SDK.

That reduces the time to initialize Git for Windows' SDK from anywhere between 2m30s-7m to a little over 1m.

Note: in theory, we could also use the DownloadBuildArtifacts@0 task here. However, restricted permissions that are in effect when building from forks would let this fail for PR builds, defeating the whole purpose of the Azure Pipelines support for git.git.

---
 azure-pipelines.yml | 44 +++++++++-----------------------------------
 1 file changed, 9 insertions(+), 35 deletions(-)

index 0f7b2125a1..480e841a85 100644
@@ -18,44 +18,18 @@ jobs:
     env:
       GITFILESHAREPWD: $(gitfileshare.pwd)
   - powershell: |
-      # Helper to check the error level of the latest command (exit with error when appropriate)
-      function c() { if (!$?) { exit(1) } }
-
-      # Add build agent's MinGit to PATH
-      $env:PATH = $env:AGENT_HOMEDIRECTORY +"\externals\\git\cmd;" +$env:PATH
-
-      # Helper to initialize (or update) a Git worktree
-      function init ($path, $url, $set_origin) {
-        if (Test-Path $path) {
-          cd $path; c
-          if (Test-Path .git) {
-            & git init; c
-          } else {
-            & git status
-          }
-        } else {
-          & git init $path; c
-          cd $path; c
-        }
-        & git config core.autocrlf false; c
-        & git config core.untrackedCache true; c
-        if (($set_origin -ne 0) -and !(git config remote.origin.url)) {
-          & git remote add origin $url; c
-        }
-        & git fetch --depth=1 $url master; c
-        & git reset --hard FETCH_HEAD; c
-        & git clean -df; c
-      }
-
-      # Initialize Git for Windows' SDK
-      $sdk_path = "$(Build.SourcesDirectory)\git-sdk-64"
-      init "$sdk_path" "https://dev.azure.com/git-for-windows/git-sdk-64/_git/git-sdk-64" 0
+      $urlbase = "https://dev.azure.com/git-for-windows/git/_apis/build/builds"
+      $id = ((Invoke-WebRequest -UseBasicParsing "${urlbase}?definitions=22&statusFilter=completed&resultFilter=succeeded&`$top=1").content | ConvertFrom-JSON).value[0].id
+      $downloadUrl = ((Invoke-WebRequest -UseBasicParsing "${urlbase}/$id/artifacts").content | ConvertFrom-JSON).value[1].resource.downloadUrl
+      (New-Object Net.WebClient).DownloadFile($downloadUrl,"git-sdk-64-minimal.zip")
+      Expand-Archive git-sdk-64-minimal.zip -DestinationPath . -Force
+      Remove-Item git-sdk-64-minimal.zip
 
       # Let Git ignore the SDK and the test-cache
-      "/git-sdk-64/`n/test-cache/`n" | Out-File -NoNewLine -Encoding ascii -Append "$(Build.SourcesDirectory)\.git\info\exclude"
-    displayName: 'Initialize the Git for Windows SDK'
+      "/git-sdk-64-minimal/`n/test-cache/`n" | Out-File -NoNewLine -Encoding ascii -Append "$(Build.SourcesDirectory)\.git\info\exclude"
+    displayName: 'Download git-sdk-64-minimal'
   - powershell: |
-      & "git-sdk-64\git-cmd.exe" --command=usr\\bin\\bash.exe -lc @"
+      & git-sdk-64-minimal\usr\bin\bash.exe -lc @"
         export DEVELOPER=1
         export NO_PERL=1
         export NO_SVN_TESTS=1

gitgitgadget

From: Johannes Schindelin <[email protected]>

Every once in a while, the Azure Pipeline fails with some semi-random

error: timer thread did not terminate timely

This error message means that the thread that is used to emulate the setitimer() function did not terminate within 1,000 milliseconds.

The most likely explanation (and therefore the one we should assume to be true, according to Occam's Razor) is that the timeout of one second is simply not enough because we try to run so many tasks in parallel.

So let's give it ten seconds instead of only one. That should be enough.

---
 compat/mingw.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

index b459e1a291..e0dfe8844d 100644
@@ -2100,7 +2100,7 @@ static void stop_timer_thread(void)
 	if (timer_event)
 		SetEvent(timer_event);	/* tell thread to terminate */
 	if (timer_thread) {
-		int rc = WaitForSingleObject(timer_thread, 1000);
+		int rc = WaitForSingleObject(timer_thread, 10000);
 		if (rc == WAIT_TIMEOUT)
 			error("timer thread did not terminate timely");
 		else if (rc != WAIT_OBJECT_0)

gitgitgadget

From: Johannes Schindelin <[email protected]>

Just like so many other OSS projects, we now also have a build badge.

---
 README.md | 2 ++
 1 file changed, 2 insertions(+)

index f920a42fad..764c480c66 100644
@@ -1,3 +1,5 @@
+[![Build Status](https://dev.azure.com/git/git/_apis/build/status/test-git.git)](https://dev.azure.com/git/git/_build/latest?definitionId=2)
+
 Git - fast, scalable, distributed revision control system
 =========================================================
 

gitgitgadget

From: Johannes Schindelin <[email protected]>

It is a bit ridiculous to spin up a full-blown Perl instance (especially on Windows, where that means spinning up a full POSIX emulation layer, AKA the MSYS2 runtime) just to tell how large a given file is.

So let's just use the test-tool to do that job instead.

This command will also be used over the next commits, to allow for cutting out individual test cases' verbose log from the file generated via --verbose-log.

---
 t/helper/test-path-utils.c          | 12 ++++++++++++
 t/t0021-conversion.sh               |  2 +-
 t/t1050-large.sh                    |  2 +-
 t/t5315-pack-objects-compression.sh |  2 +-
 t/t9303-fast-import-compression.sh  |  2 +-
 5 files changed, 16 insertions(+), 4 deletions(-)

index ae091d9b3e..30211d6d64 100644
@@ -291,6 +291,18 @@ int cmd__path_utils(int argc, const char **argv)
 		return !!res;
 	}
 
+	if (argc > 2 && !strcmp(argv[1], "file-size")) {
+		int res = 0, i;
+		struct stat st;
+
+		for (i = 2; i < argc; i++)
+			if (stat(argv[i], &st))
+				res = error_errno("Cannot stat '%s'", argv[i]);
+			else
+				printf("%"PRIuMAX"\n", (uintmax_t)st.st_size);
+		return !!res;
+	}
+
 	fprintf(stderr, "%s: unknown function name: %s\n", argv[0],
 		argv[1] ? argv[1] : "(there was none)");
 	return 1;

index fd5f1ac649..e10f5f787f 100755
@@ -24,7 +24,7 @@ generate_random_characters () {
 }
 
 file_size () {
-	perl -e 'print -s $ARGV[0]' "$1"
+	test-tool path-utils file-size "$1"
 }
 
 filter_git () {

index 1a9b21b293..dcb4dbba67 100755
@@ -8,7 +8,7 @@ test_description='adding and checking out large blobs'
 # This should be moved to test-lib.sh together with the
 # copy in t0021 after both topics have graduated to 'master'.
 file_size () {
-	perl -e 'print -s $ARGV[0]' "$1"
+	test-tool path-utils file-size "$1"
 }
 
 test_expect_success setup '

index 34c47dae09..df970d7584 100755
@@ -7,7 +7,7 @@ test_description='pack-object compression configuration'
 # This should be moved to test-lib.sh together with the
 # copy in t0021 after both topics have graduated to 'master'.
 file_size () {
-	perl -e 'print -s $ARGV[0]' "$1"
+	test-tool path-utils file-size "$1"
 }
 
 test_expect_success setup '

index 856219f46a..5045f02a53 100755
@@ -6,7 +6,7 @@ test_description='compression setting of fast-import utility'
 # This should be moved to test-lib.sh together with the
 # copy in t0021 after both topics have graduated to 'master'.
 file_size () {
-	perl -e 'print -s $ARGV[0]' "$1"
+	test-tool path-utils file-size "$1"
 }
 
 import_large () {

gitgitgadget

From: Johannes Schindelin <[email protected]>

The JUnit XML format lends itself to be presented in a powerful UI, where you can drill down to the information you are interested in very quickly.

For test failures, this usually means that you want to see the detailed trace of the failing tests.

With Travis CI, we passed the --verbose-log option to get those traces. However, that seems excessive, as we do not need/use the logs in almost all of those cases: only when a test fails do we have a way to include the trace.

So let's do something different when using Azure DevOps: let's run all the tests with --quiet first, and only if a failure is encountered, try to trace the commands as they are executed.

Of course, we cannot turn on --verbose-log after the fact. So let's just re-run the test with all the same options, adding --verbose-log. And then munging the output file into the JUnit XML on the fly.

Note: there is an off chance that re-running the test in verbose mode "fixes" the failures (and this does happen from time to time!). That is a possibility we should be able to live with. Ideally, we would label this as "Passed upon rerun", and Azure Pipelines even know about that outcome, but it is not available when using the JUnit XML format for now:
https://github.com/Microsoft/azure-pipelines-agent/blob/master/src/Agent.Worker/TestResults/JunitResultReader.cs

---
 t/helper/test-path-utils.c | 21 +++++++++++++++++++++
 t/test-lib.sh              | 22 +++++++++++++++++++++-
 2 files changed, 42 insertions(+), 1 deletion(-)

index 30211d6d64..6efde6f5ba 100644
@@ -303,6 +303,27 @@ int cmd__path_utils(int argc, const char **argv)
 		return !!res;
 	}
 
+	if (argc == 4 && !strcmp(argv[1], "skip-n-bytes")) {
+		int fd = open(argv[2], O_RDONLY), offset = atoi(argv[3]);
+		char buffer[65536];
+
+		if (fd < 0)
+			die_errno("could not open '%s'", argv[2]);
+		if (lseek(fd, offset, SEEK_SET) < 0)
+			die_errno("could not skip %d bytes", offset);
+		for (;;) {
+			ssize_t count = read(fd, buffer, sizeof(buffer));
+			if (count < 0)
+				die_errno("could not read '%s'", argv[2]);
+			if (!count)
+				break;
+			if (write(1, buffer, count) < 0)
+				die_errno("could not write to stdout");
+		}
+		close(fd);
+		return 0;
+	}
+
 	fprintf(stderr, "%s: unknown function name: %s\n", argv[0],
 		argv[1] ? argv[1] : "(there was none)");
 	return 1;

index a3b2166cb5..f31a1c8f79 100644
@@ -639,8 +639,19 @@ test_failure_ () {
 		junit_insert="<failure message=\"not ok $test_count -"
 		junit_insert="$junit_insert $(xml_attr_encode "$1")\">"
 		junit_insert="$junit_insert $(xml_attr_encode \
-			"$(printf '%s\n' "$@" | sed 1d)")"
+			"$(if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
+			   then
+				test-tool path-utils skip-n-bytes \
+					"$GIT_TEST_TEE_OUTPUT_FILE" $GIT_TEST_TEE_OFFSET
+			   else
+				printf '%s\n' "$@" | sed 1d
+			   fi)")"
 		junit_insert="$junit_insert</failure>"
+		if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
+		then
+			junit_insert="$junit_insert<system-err>$(xml_attr_encode \
+				"$(cat "$GIT_TEST_TEE_OUTPUT_FILE")")</system-err>"
+		fi
 		write_junit_xml_testcase "$1" "      $junit_insert"
 	fi
 	test_failure=$(($test_failure + 1))

@@ -931,6 +942,11 @@ test_finish_ () {
 	echo >&3 ""
 	maybe_teardown_valgrind
 	maybe_teardown_verbose
+	if test -n "$GIT_TEST_TEE_OFFSET"
+	then
+		GIT_TEST_TEE_OFFSET=$(test-tool path-utils file-size \
+			"$GIT_TEST_TEE_OUTPUT_FILE")
+	fi
 }
 
 test_skip () {

@@ -1280,6 +1296,10 @@ then
 		date +%Y-%m-%dT%H:%M:%S)\""
 	write_junit_xml --truncate "<testsuites>" "  <testsuite $junit_attrs>"
 	junit_suite_start=$(test-tool date getnanos)
+	if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
+	then
+		GIT_TEST_TEE_OFFSET=0
+	fi
 fi
 
 # Provide an implementation of the 'yes' utility

gitgitgadget

From: Johannes Schindelin <[email protected]>

It seems that every once in a while in the Git for Windows SDK, there are some transient file locking issues preventing the test clean up to delete the trash directory. Let's be gentle and try again five seconds later, and only error out if it still fails the second time.

This change helps Windows, and does not hurt any other platform (normally, it is highly unlikely that said deletion fails, and if it does, normally it will fail again even 5 seconds later).

---
 t/test-lib.sh | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

index f31a1c8f79..9c0ca5effb 100644
@@ -1104,7 +1104,11 @@ test_done () {
 			error "Tests passed but trash directory already removed before test cleanup; aborting"
 
 			cd "$TRASH_DIRECTORY/.." &&
-			rm -fr "$TRASH_DIRECTORY" ||
+			rm -fr "$TRASH_DIRECTORY" || {
+				# try again in a bit
+				sleep 5;
+				rm -fr "$TRASH_DIRECTORY"
+			} ||
 			error "Tests passed but test cleanup failed; aborting"
 		fi
 		test_at_end_hook_

gitgitgadget

From: Johannes Schindelin <[email protected]>

We really need to be able to find the test helpers... Really. This change was forgotten when we moved the test helpers into t/helper/

---
 t/test-lib.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

index 9c0ca5effb..c790e98fd2 100644
@@ -1227,7 +1227,7 @@ else # normal case, use ../bin-wrappers only unless $with_dashes:
 	GIT_EXEC_PATH=$GIT_BUILD_DIR
 	if test -n "$with_dashes"
 	then
-		PATH="$GIT_BUILD_DIR:$PATH"
+		PATH="$GIT_BUILD_DIR:$GIT_BUILD_DIR/t/helper:$PATH"
 	fi
 fi
 GIT_TEMPLATE_DIR="$GIT_BUILD_DIR"/templates/blt

gitgitgadget

From: Johannes Schindelin <[email protected]>

When building Git with RUNTIME_PREFIX and starting a test helper from t/helper/, it fails to detect the system prefix correctly.

This is the reason that the warning

RUNTIME_PREFIX requested, but prefix computation failed. [...]

to be printed.

In t0061, we did not expect that to happen, and it actually did not happen in the normal case, because bin-wrappers/test-tool specifically sets GIT_TEXTDOMAINDIR (and as a consequence, nothing in test-tool wants to know about the runtime prefix).

However, with --with-dashes, bin-wrappers/test-tool is no longer called, but t/helper/test-tool is called directly.

So let's just ignore the RUNTIME_PREFIX warning.

---
 t/t0061-run-command.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

index 99a614bc7c..5a2d087bf0 100755
@@ -166,7 +166,8 @@ test_trace () {
 	expect="$1"
 	shift
 	GIT_TRACE=1 test-tool run-command "$@" run-command true 2>&1 >/dev/null | \
-		sed -e 's/.* run_command: //' -e '/trace: .*/d' >actual &&
+		sed -e 's/.* run_command: //' -e '/trace: .*/d' \
+			-e '/RUNTIME_PREFIX requested/d' >actual &&
 	echo "$expect true" >expect &&
 	test_cmp expect actual
 }

gitgitgadget

From: Johannes Schindelin <[email protected]>

This speeds up the tests by a bit on Windows, where running Unix shell scripts (and spawning processes) is not exactly a cheap operation.

---
 t/README      |  9 +++++++++
 t/test-lib.sh | 19 +++++++++++++------
 2 files changed, 22 insertions(+), 6 deletions(-)

index 11ce7675e3..063530234f 100644
@@ -170,6 +170,15 @@ appropriately before running "make".
 	implied by other options like --valgrind and
 	GIT_TEST_INSTALLED.
 
+--no-bin-wrappers::
+	By default, the test suite uses the wrappers in
+	`../bin-wrappers/` to execute `git` and friends. With this option,
+	`../git` and friends are run directly. This is not recommended
+	in general, as the wrappers contain safeguards to ensure that no
+	files from an installed Git are used, but can speed up test runs
+	especially on platforms where running shell scripts is expensive
+	(most notably, Windows).
+
 --root=<directory>::
 	Create "trash" directories used to store all temporary data during
 	testing under <directory>, instead of the t/ directory.

index c790e98fd2..25e649c997 100644
@@ -111,6 +111,8 @@ do
 		test -z "$HARNESS_ACTIVE" && quiet=t ;;
 	--with-dashes)
 		with_dashes=t ;;
+	--no-bin-wrappers)
+		no_bin_wrappers=t ;;
 	--no-color)
 		color= ;;
 	--va|--val|--valg|--valgr|--valgri|--valgrin|--valgrind)

@@ -1214,16 +1216,21 @@ then
 	PATH=$GIT_TEST_INSTALLED:$GIT_BUILD_DIR/t/helper:$PATH
 	GIT_EXEC_PATH=${GIT_TEST_EXEC_PATH:-$GIT_EXEC_PATH}
 else # normal case, use ../bin-wrappers only unless $with_dashes:
-	git_bin_dir="$GIT_BUILD_DIR/bin-wrappers"
-	if ! test -x "$git_bin_dir/git"
+	if test -n "$no_bin_wrappers"
 	then
-		if test -z "$with_dashes"
+		with_dashes=t
+	else
+		git_bin_dir="$GIT_BUILD_DIR/bin-wrappers"
+		if ! test -x "$git_bin_dir/git"
 		then
-			say "$git_bin_dir/git is not executable; using GIT_EXEC_PATH"
+			if test -z "$with_dashes"
+			then
+				say "$git_bin_dir/git is not executable; using GIT_EXEC_PATH"
+			fi
+			with_dashes=t
 		fi
-		with_dashes=t
+		PATH="$git_bin_dir:$PATH"
 	fi
-	PATH="$git_bin_dir:$PATH"
 	GIT_EXEC_PATH=$GIT_BUILD_DIR
 	if test -n "$with_dashes"
 	then

gitgitgadget

From: Johannes Schindelin <[email protected]>

As Unix shell scripting comes at a hefty price on Windows, we have to see where we can save some time to run the test suite.

Let's skip the chain linting and the bin-wrappers/ redirection on Windows; this seems to shave of anywhere between 10-30% from the overall runtime.

---
 ci/lib.sh | 2 ++
 1 file changed, 2 insertions(+)

index 5505776876..c2bc6c68b9 100755
@@ -127,6 +127,8 @@ then
 	export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
 	export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml"
 	export MAKEFLAGS="--jobs=10"
+	test windows_nt != "$CI_OS_NAME" ||
+	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 else
 	echo "Could not identify CI type" >&2
 	exit 1

gitgitgadget

From: Johannes Schindelin <[email protected]>

The fact that Git's test suite is implemented in Unix shell script that is as portable as we can muster, combined with the fact that Unix shell scripting is foreign to Windows (and therefore has to be emulated), results in pretty abysmal speed of the test suite on that platform, for pretty much no other reason than that language choice.

For comparison: while the Linux build & test is typically done within about 8 minutes, the Windows build & test typically lasts about 80 minutes in Azure Pipelines.

To help with that, let's use the Azure Pipeline feature where you can parallelize jobs, make jobs depend on each other, and pass artifacts between them.

The tests are distributed using the following heuristic: listing all test scripts ordered by size in descending order (as a cheap way to estimate the overall run time), every Nth script is run (where N is the total number of parallel jobs), starting at the index corresponding to the parallel job. This slicing is performed by a new function that is added to the test-tool.

To optimize the overall runtime of the entire Pipeline, we need to move the Windows jobs to the beginning (otherwise there would be a very decent chance for the Pipeline to be run only the Windows build, while all the parallel Windows test jobs wait for this single one).

We use Azure Pipelines Artifacts for both the minimal Git for Windows SDK as well as the built executables, as deduplication and caching close to the agents makes that really fast. For comparison: while downloading and unpacking the minimal Git for Windows SDK via PowerShell takes only one minute (down from anywhere between 2.5 to 7 when using a shallow clone), uploading it as Pipeline Artifact takes less than 30s and downloading and unpacking less than 20s (sometimes even as little as only twelve seconds).

---
 Makefile                   | 10 +++++
 azure-pipelines.yml        | 79 ++++++++++++++++++++++++++++++++++----
 ci/make-test-artifacts.sh  | 12 ++++++
 ci/run-test-slice.sh       | 17 ++++++++
 t/helper/test-path-utils.c | 31 +++++++++++++++
 5 files changed, 141 insertions(+), 8 deletions(-)
 create mode 100755 ci/make-test-artifacts.sh
 create mode 100755 ci/run-test-slice.sh

index 044b4f77bd..daa318fe17 100644
@@ -2927,6 +2927,16 @@ rpm::
 	@false
 .PHONY: rpm
 
+artifacts-tar:: $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) $(OTHER_PROGRAMS) \
+		GIT-BUILD-OPTIONS $(TEST_PROGRAMS) $(test_bindir_programs) \
+		$(NO_INSTALL) $(MOFILES)
+	$(QUIET_SUBDIR0)templates $(QUIET_SUBDIR1) \
+		SHELL_PATH='$(SHELL_PATH_SQ)' PERL_PATH='$(PERL_PATH_SQ)'
+	test -n "$(ARTIFACTS_DIRECTORY)"
+	mkdir -p "$(ARTIFACTS_DIRECTORY)"
+	$(TAR) czf "$(ARTIFACTS_DIRECTORY)/artifacts.tar.gz" $^ templates/blt/
+.PHONY: artifacts-tar
+
 htmldocs = git-htmldocs-$(GIT_VERSION)
 manpages = git-manpages-$(GIT_VERSION)
 .PHONY: dist-doc distclean

new file mode 100755
index 0000000000..646967481f
@@ -0,0 +1,12 @@
+#!/bin/sh
+#
+# Build Git and store artifacts for testing
+#
+
+mkdir -p "$1" # in case ci/lib.sh decides to quit early
+
+. ${0%/*}/lib.sh
+
+make artifacts-tar ARTIFACTS_DIRECTORY="$1"
+
+check_unignored_build_artifacts

new file mode 100755
index 0000000000..f8c2c3106a
@@ -0,0 +1,17 @@
+#!/bin/sh
+#
+# Test Git in parallel
+#
+
+. ${0%/*}/lib.sh
+
+case "$CI_OS_NAME" in
+windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";;
+*) ln -s "$cache_dir/.prove" t/.prove;;
+esac
+
+make --quiet -C t T="$(cd t &&
+	./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh |
+	tr '\n' ' ')"
+
+check_unignored_build_artifacts

index 6efde6f5ba..5d543ad21f 100644
@@ -177,6 +177,14 @@ static int is_dotgitmodules(const char *path)
 	return is_hfs_dotgitmodules(path) || is_ntfs_dotgitmodules(path);
 }
 
+static int cmp_by_st_size(const void *a, const void *b)
+{
+	intptr_t x = (intptr_t)((struct string_list_item *)a)->util;
+	intptr_t y = (intptr_t)((struct string_list_item *)b)->util;
+
+	return x > y ? -1 : (x < y ? +1 : 0);
+}
+
 int cmd__path_utils(int argc, const char **argv)
 {
 	if (argc == 3 && !strcmp(argv[1], "normalize_path_copy")) {

@@ -324,6 +332,29 @@ int cmd__path_utils(int argc, const char **argv)
 		return 0;
 	}
 
+	if (argc > 5 && !strcmp(argv[1], "slice-tests")) {
+		int res = 0;
+		long offset, stride, i;
+		struct string_list list = STRING_LIST_INIT_NODUP;
+		struct stat st;
+
+		offset = strtol(argv[2], NULL, 10);
+		stride = strtol(argv[3], NULL, 10);
+		if (stride < 1)
+			stride = 1;
+		for (i = 4; i < argc; i++)
+			if (stat(argv[i], &st))
+				res = error_errno("Cannot stat '%s'", argv[i]);
+			else
+				string_list_append(&list, argv[i])->util =
+					(void *)(intptr_t)st.st_size;
+		QSORT(list.items, list.nr, cmp_by_st_size);
+		for (i = offset; i < list.nr; i+= stride)
+			printf("%s\n", list.items[i].string);
+
+		return !!res;
+	}
+
 	fprintf(stderr, "%s: unknown function name: %s\n", argv[0],
 		argv[1] ? argv[1] : "(there was none)");
 	return 1;

gitgitgadget

For a long time already, we have tested Git's source code continuously via Travis CI, see e.g. https://travis-ci.org/git/git/builds/421738884. It has served us well, and more and more developers actually pay attention and benefit from the testing this gives us.

It is also an invaluable tool for contributors who can validate their code contributions via PRs on GitHub, e.g. to verify that their tests do actually run on macOS (i.e. with the BSD family of Unix tools instead of the GNU one).

The one sad part about this is the Windows support. Travis used to lack it (Travis announced some early support for Windows, hot off the press
[https://blog.travis-ci.com/2018-10-11-windows-early-release]), and we work around that by using Azure Pipelines (the CI part of Azure DevOps, formerly known as Visual Studio Team Services) indirectly: one phase in Travis would trigger a build, wait for its log, and then paste that log.

As Git's Windows builds (and tests!) take quite a bit of time, Travis often timed out, or somehow the trigger did not work, and for security reasons (the Windows builds are performed in a private pool of containers), the Windows builds are completely disabled for Pull Requests on GitHub.

One might ask why we did not use Azure Pipelines directly. There were a couple of reasons for that:

* most notably, Azure Pipelines' build logs could not be viewed anonymously,
* while Azure Pipelines had Linux and Windows agents, it lacked macOS agents,
* etc

The main two reasons no longer apply: macOS agents are available now
[https://docs.microsoft.com/en-us/azure/devops/release-notes/2018/jul-10-vsts]
, as are public projects
[https://docs.microsoft.com/en-us/azure/devops/organizations/public/about-public-projects]
, i.e. it is possible to configure a Azure Pipelines project so that anybody can view the logs. Since I offered v1, Azure Pipelines has been made available via the GitHub Marketplace, free of cost for open source projects.

I had secured such a public project for Git for Windows already, and I also got one for Git. For now, the latter is hooked up with my personal git.git fork on GitHub, but it is my hope that I convince y'all that these Azure Pipelines builds are a good idea, and then hook it up with
https://github.com/git/git.

As a special treat, this patch series adds the ability to present the outcome of Git's test suite as JUnit-style .xml files. This allows the Azure Pipelines build to present fun diagrams, trends, and makes it a lot easier to drill down to test failures than before. See for example
https://dev.azure.com/git/git/_build/results?buildId=113&view=ms.vss-test-web.test-result-details
[https://dev.azure.com/git/git/_build/results?buildId=113&view=ms.vss-test-web.test-result-details]
(you can click on the label of the failed test, and then see the detailed output in the right pane).

But maybe you're not interested as much in the presentation of test failures as in the speed? Well, I got you covered with that, too. As of v3, the test suite is run in parallel on Windows, cutting the overall run time to ~33 minutes (see one of the latest builds
[https://dev.azure.com/git/git/_build/results?buildId=311&view=logs], for example).

This patch series took waaaaaaaay more time than I had originally anticipated, but I think that in particular the advanced display of the test results and the reduction of the overall run time was worth it. Please let me know what you think about this.

Changes since v5:

* Fixed sparse warnings.
* Reworded the commit message of "t0061: fix for --with-dashes and RUN TIME_PREFIX" to clarify why there is a warning message in the first place, why this cannot be fixed, and that we introduce a simple workaround here.

Changes since v4:

* Clarified in the commit message why BREW_INSTALL_PACKAGES is needed.
* Added a clause to error out if we could not detect the CI type.
* Introduced a CI_TYPE variable to guard CI type-specific code.
* MAKEFLAGS is now actually configured in the CI type-specific section.
* Moved the inadvertent fixup for GIT_TEST_OPTS from "Add a build definition for Azure DevOps" to "ci/lib.sh: add support for Azure Pipelines".
* Backed out the Windows support from the commit adding the initial Azure Pipelines support, instead adding it later to help the flow of the patch series.
* Clarified in "test-date: add a subcommand to measure times in shell scripts"'s commit message that one of the goals is portability.
* Moved all of those CI type-specific definitions in ci/lib.sh after the set -ex statement (as suggested by Gábor a loooong time ago already).
* Guarded the Travis-specific way to "publish" trash directories of failed tests via base64-encoded tar files in the log; For Azure Pipelines, these are published as build artifacts instead.

Changes since v3:

* Fixed the incorrect usage of cut -c that resulted in an empty trace when failed tests were published (except if the very first test case in a test script failed, then it was correct by mistake).
* Excluded the previous test case's "ok" (or "not ok") line from the trace published with failed tests.
* Renamed TRAVIS_BRANCH to CI_BRANCH already in the first commit, as we should not override TRAVIS_BRANCH with something that it is not.
* Rebased onto current master to avoid merge conflicts with the recently-merged sg/stress-test branch (no, Junio, I really trust myself more than you to resolve those merge conflicts).

Changes since v2:

* Removed left-over debugging code that would skip a good chunk of t0000-init.sh.
* Fixed the URL of the build badge.
* Removed a trailing empty line from, and added a missing closing pointy bracket to, ci/mount-fileshare.sh.
* Moved the "travis: fix skipping tagged releases" commit up to the beginning of the patch series.
* The commit message of "ci/lib.sh: add support for Azure Pipelines" now mentions explicitly that the Homebrew packages that need to be installed on Travis' macOS agents are already installed on Azure Pipelines'.
* Some commands were not guarded by || exit 1, i.e. if they would fail, the build would not have failed.
* We now install gcc-8 for the linux-gcc job.
* We no longer try to re-run failed tests with verbose log. Instead, we simply use the verbose log to begin with. Tests showed that it had a negligible impact on the overall run time.
* The test_atexit_handler function was scratched; It would be the right thing to do, but is actually an independent topic (it was only implemented in v2 to accommodate the "re-run with verbose log on failure" trick)
* We now use a new YAML schema (most notably, "phase" is now known as "job")
* The Windows job contained PowerShell sections that were indented with 3 spaces instead of 2.
* The Windows job is now structured better, by separating different concerns into different "tasks" so that it is easier to see what exactly failed (was it the build? was it the test?)
* The Windows job was split into a job to build Git and 10 parallel jobs to run the test suite with the artifacts built by the first job. This reduces the overall run time from ~1h20 (which was the run time by the Windows job) to ~35 minutes (which is the run time of the linux-gcc job).
* The JUnit XML is now written using a test helper rather than a fragile and probably not even portable sed call.
* Since we needed to determine the file size of the verbose log (to cut out individual test cases' log output), we now introduce a test helper to do that, and use it throughout the test suite (where Perl was used before).
* It would appear that a recent change either in Cygwin or in the Azure VMs causes problems sporadically where the trash directories cannot be removed, but a subsequent rm will succeed. We now simply do that, because it won't harm the common case (where the first rm succeeds already) and because it helps the Windows job succeed pretty reliably.

Changes since v1:

* Removed a superfluous eval.
* Added the commit that fixes the Travis PR builds targeting master that just happens to be tagged (see
https://travis-ci.org/git/git/jobs/424276413 for an incorrectly-skipped build).
* The commit messages and the cover letter now reflect the name change from Visual Studio Team Services to Azure DevOps (and in particular, Azure Pipelines for the automated builds).
* Now we're using test_atexit (which we introduced for that purpose) instead of hard-coding kill_p4d and stop_git_daemon.
* The build should now also succeed for Pull Requests (where secret variables are not available, for security reasons, and as a consequence the file share cannot be mounted).
* The shell scripted parts now use proper && chains.

Johannes Schindelin (21):
travis: fix skipping tagged releases
ci: rename the library of common functions
ci/lib.sh: encapsulate Travis-specific things
ci: inherit --jobs via MAKEFLAGS in run-build-and-tests
ci: use a junction on Windows instead of a symlink
test-date: add a subcommand to measure times in shell scripts
tests: optionally write results as JUnit-style .xml ci/lib.sh: add support for Azure Pipelines
Add a build definition for Azure DevOps
ci: add a Windows job to the Azure Pipelines definition
ci: use git-sdk-64-minimal build artifact
mingw: be more generous when wrapping up the setitimer() emulation
README: add a build badge (status of the Azure Pipelines build)
tests: avoid calling Perl just to determine file sizes
tests: include detailed trace logs with --write-junit-xml upon failure
mingw: try to work around issues with the test cleanup
tests: add t/helper/ to the PATH with --with-dashes
t0061: workaround issues with --with-dashes and RUNTIME_PREFIX
tests: optionally skip bin-wrappers/
ci: speed up Windows phase
ci: parallelize testing on Windows

 Makefile                            |  11 +
 README.md                           |   2 +
 azure-pipelines.yml                 | 387 ++++++++++++++++++++++++++++
 ci/install-dependencies.sh          |   5 +-
 ci/{lib-travisci.sh => lib.sh}      |  92 +++++--
 ci/make-test-artifacts.sh           |  12 +
 ci/mount-fileshare.sh               |  25 ++
 ci/print-test-failures.sh           |  15 +-
 ci/run-build-and-tests.sh           |   9 +-
 ci/run-linux32-build.sh             |   2 +-
 ci/run-linux32-docker.sh            |   2 +-
 ci/run-static-analysis.sh           |   4 +-
 ci/run-test-slice.sh                |  17 ++
 ci/run-windows-build.sh             |   2 +-
 ci/test-documentation.sh            |   7 +-
 compat/mingw.c                      |   2 +-
 t/.gitignore                        |   1 +
 t/README                            |   9 +
 t/helper/test-date.c                |  12 +
 t/helper/test-path-utils.c          |  64 +++++
 t/helper/test-tool.c                |   1 +
 t/helper/test-tool.h                |   1 +
 t/helper/test-xml-encode.c          |  80 ++++++
 t/t0021-conversion.sh               |   2 +-
 t/t0061-run-command.sh              |   3 +-
 t/t1050-large.sh                    |   2 +-
 t/t5315-pack-objects-compression.sh |   2 +-
 t/t9303-fast-import-compression.sh  |   2 +-
 t/test-lib.sh                       | 138 +++++++++-
 29 files changed, 862 insertions(+), 49 deletions(-)
 create mode 100644 azure-pipelines.yml
 rename ci/{lib-travisci.sh => lib.sh} (50%)
 create mode 100755 ci/make-test-artifacts.sh
 create mode 100755 ci/mount-fileshare.sh
 create mode 100755 ci/run-test-slice.sh
 create mode 100644 t/helper/test-xml-encode.c

base-commit: 16a465bc018d09e9d7bbbdc5f40a7fb99c21f8ef
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-31%2Fdscho%2Fvsts-ci-v6
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-31/dscho/vsts-ci-v6
Pull-Request: https://github.com/gitgitgadget/git/pull/31

Range-diff vs v5:

  1:  f553fd4fb3 =  1:  f553fd4fb3 travis: fix skipping tagged releases
  2:  f0852de8ab =  2:  f0852de8ab ci: rename the library of common functions
  3:  7c16d31b46 =  3:  7c16d31b46 ci/lib.sh: encapsulate Travis-specific things
  4:  bf72fb1004 =  4:  bf72fb1004 ci: inherit --jobs via MAKEFLAGS in run-build-and-tests
  5:  681f8e65b4 =  5:  681f8e65b4 ci: use a junction on Windows instead of a symlink
  6:  ccf8bf53d7 =  6:  ccf8bf53d7 test-date: add a subcommand to measure times in shell scripts
  7:  31fbe11425 !  7:  e9a869df57 tests: optionally write results as JUnit-style .xml
     @@ -91,7 +91,7 @@
      +		if (tmp2) {
      +			if ((ch & 0xc0) != 0x80) {
      +				fputs(utf8_replace_character, stdout);
     -+				tmp2 = 0;
     ++				tmp2 = NULL;
      +				cur--;
      +				continue;
      +			}
     @@ -99,7 +99,7 @@
      +			tmp2++;
      +			if (--remaining == 0) {
      +				fwrite(tmp, tmp2 - tmp, 1, stdout);
     -+				tmp2 = 0;
     ++				tmp2 = NULL;
      +			}
      +			continue;
      +		}
  8:  47fe8aa84a =  8:  4c78085af7 ci/lib.sh: add support for Azure Pipelines
  9:  6044b919b0 =  9:  d34812bdf6 Add a build definition for Azure DevOps
 10:  3a2993c826 = 10:  d565131781 ci: add a Windows job to the Azure Pipelines definition
 11:  52d63018c9 = 11:  1ab1d14e06 ci: use git-sdk-64-minimal build artifact
 12:  1cef14e4c0 = 12:  c1ab8df677 mingw: be more generous when wrapping up the setitimer() emulation
 13:  eda46863e8 = 13:  b6316e1f7d README: add a build badge (status of the Azure Pipelines build)
 14:  4303bbd4e8 = 14:  7a5caa2e0f tests: avoid calling Perl just to determine file sizes
 15:  dfd3516f39 = 15:  2593b9ba7e tests: include detailed trace logs with --write-junit-xml upon failure
 16:  9984c6d17b = 16:  991b41afa4 mingw: try to work around issues with the test cleanup
 17:  a8101ae3e1 = 17:  77896b2f8e tests: add t/helper/ to the PATH with --with-dashes
 18:  7518266a19 ! 18:  4ec6cc83ef t0061: fix with --with-dashes and RUNTIME_PREFIX

@@ -1,23 +1,29 @@
Author: Johannes Schindelin <[email protected]>

- t0061: fix with --with-dashes and RUNTIME_PREFIX
+ t0061: workaround issues with --with-dashes and RUNTIME_PREFIX

When building Git with RUNTIME_PREFIX and starting a test helper from
- t/helper/, it fails to detect the system prefix correctly.
-
- This is the reason that the warning
+ t/helper/, it fails to detect a system prefix. The reason is that the
+ RUNTIME_PREFIX feature wants to use the location of the Git executable
+ to determine where the support files can be found, e.g. system-wide Git
+ config or the translations. This does not make any sense for the test
+ helpers, though, as they are distinctly not in a directory structure
+ resembling the final installation location of Git.
+
+ That is the reason why the test helpers rely on environment variables to
+ indicate the location of the needed support files, e.g.
+ GIT_TEXTDOMAINDIR. If this information is missing, the output will
+ contain warnings like this one:

RUNTIME_PREFIX requested, but prefix computation failed. [...]

- to be printed.
-
- In t0061, we did not expect that to happen, and it actually did not
- happen in the normal case, because bin-wrappers/test-tool specifically
- sets GIT_TEXTDOMAINDIR (and as a consequence, nothing in test-tool wants
- to know about the runtime prefix).
+ In t0061, we did not expect that to happen, and it actually does not
+ happen in the regular case, because bin-wrappers/test-tool specifically
+ sets GIT_TEXTDOMAINDIR (and as a consequence, nothing in test-tool needs
+ to know anything about any runtime prefix).

However, with --with-dashes, bin-wrappers/test-tool is no longer called,
- but t/helper/test-tool is called directly.
+ but t/helper/test-tool is called directly instead.

So let's just ignore the RUNTIME_PREFIX warning.

 19:  6a8adbe8ba = 19:  248473d9fa tests: optionally skip bin-wrappers/
 20:  b39e16527d = 20:  3532811a49 ci: speed up Windows phase
 21:  6ff87d1772 = 21:  1572444361 ci: parallelize testing on Windows

gitgitgadget

From: Johannes Schindelin <[email protected]>

When building a PR, TRAVIS_BRANCH refers to the *target branch*. Therefore, if a PR targets master, and master happened to be tagged, we skipped the build by mistake.

Fix this by using TRAVIS_PULL_REQUEST_BRANCH (i.e. the *source branch*) when available, falling back to TRAVIS_BRANCH (i.e. for CI builds, also known as "push builds").

Let's give it a new variable name, too: CI_BRANCH (as it is different from TRAVIS_BRANCH). This also prepares for the upcoming patches which will make our ci/* code a bit more independent from Travis and open it to other CI systems (in particular to Azure Pipelines).

---
 ci/lib-travisci.sh | 25 ++++++++++++++-----------
 1 file changed, 14 insertions(+), 11 deletions(-)

index 69dff4d1ec..9c6ddeb374 100755
@@ -5,18 +5,17 @@ skip_branch_tip_with_tag () {
 	# at the same commit as the tip of the branch is pushed, and building
 	# both at the same time is a waste.
 	#
-	# Travis gives a tagname e.g. v2.14.0 in $TRAVIS_BRANCH when
-	# the build is triggered by a push to a tag.  Let's see if
-	# $TRAVIS_BRANCH is exactly at a tag, and if so, if it is
-	# different from $TRAVIS_BRANCH.  That way, we can tell if
-	# we are building the tip of a branch that is tagged and
-	# we can skip the build because we won't be skipping a build
-	# of a tag.
-
-	if TAG=$(git describe --exact-match "$TRAVIS_BRANCH" 2>/dev/null) &&
-		test "$TAG" != "$TRAVIS_BRANCH"
+	# When the build is triggered by a push to a tag, $CI_BRANCH will
+	# have that tagname, e.g. v2.14.0.  Let's see if $CI_BRANCH is
+	# exactly at a tag, and if so, if it is different from $CI_BRANCH.
+	# That way, we can tell if we are building the tip of a branch that
+	# is tagged and we can skip the build because we won't be skipping a
+	# build of a tag.
+
+	if TAG=$(git describe --exact-match "$CI_BRANCH" 2>/dev/null) &&
+		test "$TAG" != "$CI_BRANCH"
 	then
-		echo "$(tput setaf 2)Tip of $TRAVIS_BRANCH is exactly at $TAG$(tput sgr0)"
+		echo "$(tput setaf 2)Tip of $CI_BRANCH is exactly at $TAG$(tput sgr0)"
 		exit 0
 	fi
 }

@@ -81,6 +80,10 @@ check_unignored_build_artifacts ()
 # and installing dependencies.
 set -ex
 
+# When building a PR, TRAVIS_BRANCH refers to the *target* branch. Not what we
+# want here. We want the source branch instead.
+CI_BRANCH="${TRAVIS_PULL_REQUEST_BRANCH:-$TRAVIS_BRANCH}"
+
 cache_dir="$HOME/travis-cache"
 good_trees_file="$cache_dir/good-trees"
 

gitgitgadget

From: Johannes Schindelin <[email protected]>

The name is hard-coded to reflect that we use Travis CI for continuous testing.

In the next commits, we will extend this to be able use Azure DevOps, too.

So let's adjust the name to make it more generic.

---
 ci/install-dependencies.sh     | 2 +-
 ci/{lib-travisci.sh => lib.sh} | 0
 ci/print-test-failures.sh      | 2 +-
 ci/run-build-and-tests.sh      | 2 +-
 ci/run-linux32-docker.sh       | 2 +-
 ci/run-static-analysis.sh      | 2 +-
 ci/run-windows-build.sh        | 2 +-
 ci/test-documentation.sh       | 2 +-
 8 files changed, 7 insertions(+), 7 deletions(-)
 rename ci/{lib-travisci.sh => lib.sh} (100%)

index 06c3546e1e..fe65144152 100755
@@ -3,7 +3,7 @@
 # Install dependencies required to build and test Git on Linux and macOS
 #
 
-. ${0%/*}/lib-travisci.sh
+. ${0%/*}/lib.sh
 
 P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION
 LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION

similarity index 100%
rename from ci/lib-travisci.sh
rename to ci/lib.sh

index d55460a212..7aef39a2fd 100755
@@ -3,7 +3,7 @@
 # Print output of failing tests
 #
 
-. ${0%/*}/lib-travisci.sh
+. ${0%/*}/lib.sh
 
 # Tracing executed commands would produce too much noise in the loop below.
 set +x

index cda170d5c2..db342bb6a8 100755
@@ -3,7 +3,7 @@
 # Build and test Git
 #
 
-. ${0%/*}/lib-travisci.sh
+. ${0%/*}/lib.sh
 
 ln -s "$cache_dir/.prove" t/.prove
 

index 21637903ce..751acfcf8a 100755
@@ -3,7 +3,7 @@
 # Download and run Docker image to build and test 32-bit Git
 #
 
-. ${0%/*}/lib-travisci.sh
+. ${0%/*}/lib.sh
 
 docker pull daald/ubuntu32:xenial
 

index 5688f261d0..dc189c7456 100755
@@ -3,7 +3,7 @@
 # Perform various static code analysis checks
 #
 
-. ${0%/*}/lib-travisci.sh
+. ${0%/*}/lib.sh
 
 make --jobs=2 coccicheck
 

index d99a180e52..a73a4eca0a 100755
@@ -6,7 +6,7 @@
 # supported) and a commit hash.
 #
 
-. ${0%/*}/lib-travisci.sh
+. ${0%/*}/lib.sh
 
 test $# -ne 2 && echo "Unexpected number of parameters" && exit 1
 test -z "$GFW_CI_TOKEN" && echo "GFW_CI_TOKEN not defined" && exit

index a20de9ca12..d3cdbac73f 100755
@@ -3,7 +3,7 @@
 # Perform sanity checks on documentation and build it.
 #
 
-. ${0%/*}/lib-travisci.sh
+. ${0%/*}/lib.sh
 
 gem install asciidoctor
 

gitgitgadget

From: Johannes Schindelin <[email protected]>

The upcoming patches will allow building git.git via Azure Pipelines (i.e. Azure DevOps' Continuous Integration), where variable names and URLs look a bit different than in Travis CI.

Also, the configurations of the available agents are different. For example, Travis' and Azure Pipelines' macOS agents are set up differently, so that on Travis, we have to install the git-lfs and gettext Homebrew packages, and on Azure Pipelines we do not need to. Likewise, Azure Pipelines' Ubuntu agents already have asciidoctor installed.

Finally, on Azure Pipelines the natural way is not to base64-encode tar files of the trash directories of failed tests, but to publish build artifacts instead. Therefore, that code to log those base64-encoded tar files is guarded to be Travis-specific.

---
 ci/install-dependencies.sh |  3 ++-
 ci/lib.sh                  | 45 +++++++++++++++++++++++++++-----------
 ci/print-test-failures.sh  |  8 +++++++
 ci/test-documentation.sh   |  1 +
 4 files changed, 43 insertions(+), 14 deletions(-)

index fe65144152..bcdcc71592 100755
@@ -37,7 +37,8 @@ osx-clang|osx-gcc)
 	brew update --quiet
 	# Uncomment this if you want to run perf tests:
 	# brew install gnu-time
-	brew install git-lfs gettext
+	test -z "$BREW_INSTALL_PACKAGES" ||
+	brew install $BREW_INSTALL_PACKAGES
 	brew link --force gettext
 	brew install caskroom/cask/perforce
 	;;

index 7aef39a2fd..cf321b474d 100755
@@ -38,6 +38,14 @@ do
 		test_name="${TEST_EXIT%.exit}"
 		test_name="${test_name##*/}"
 		trash_dir="trash directory.$test_name"
+		case "$CI_TYPE" in
+		travis)
+			;;
+		*)
+			echo "Unhandled CI type: $CI_TYPE" >&2
+			exit 1
+			;;
+		esac
 		trash_tgz_b64="trash.$test_name.base64"
 		if [ -d "$trash_dir" ]
 		then

index d3cdbac73f..7d0beb2832 100755
@@ -5,6 +5,7 @@
 
 . ${0%/*}/lib.sh
 
+test -n "$ALREADY_HAVE_ASCIIDOCTOR" ||
 gem install asciidoctor
 
 make check-builtins

gitgitgadget

From: Johannes Schindelin <[email protected]>

Let's not decide in the generic ci/ part how many jobs to run in parallel; different CI configurations would favor a different number of parallel jobs, and it is easy enough to hand that information down via the MAKEFLAGS variable.

---
 ci/lib.sh                 | 1 +
 ci/run-build-and-tests.sh | 2 +-
 ci/run-linux32-build.sh   | 2 +-
 ci/run-static-analysis.sh | 2 +-
 ci/test-documentation.sh  | 4 ++--
 5 files changed, 6 insertions(+), 5 deletions(-)

index 3f286d86a6..32a28fd209 100755
@@ -101,6 +101,7 @@ then
 	BREW_INSTALL_PACKAGES="git-lfs gettext"
 	export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
 	export GIT_TEST_OPTS="--verbose-log -x --immediate"
+	export MAKEFLAGS="--jobs=2"
 else
 	echo "Could not identify CI type" >&2
 	exit 1

index db342bb6a8..80d72d120f 100755
@@ -7,7 +7,7 @@
 
 ln -s "$cache_dir/.prove" t/.prove
 
-make --jobs=2
+make
 make --quiet test
 if test "$jobname" = "linux-gcc"
 then

index 2c60d2e70a..09e9276e12 100755
@@ -55,6 +55,6 @@ linux32 --32bit i386 su -m -l $CI_USER -c '
 	set -ex
 	cd /usr/src/git
 	test -n "$cache_dir" && ln -s "$cache_dir/.prove" t/.prove
-	make --jobs=2
+	make
 	make --quiet test
 '

index dc189c7456..a19aa7ebbc 100755
@@ -5,7 +5,7 @@
 
 . ${0%/*}/lib.sh
 
-make --jobs=2 coccicheck
+make coccicheck
 
 set +x
 

index 7d0beb2832..be3b7d376a 100755
@@ -12,7 +12,7 @@ make check-builtins
 make check-docs
 
 # Build docs with AsciiDoc
-make --jobs=2 doc > >(tee stdout.log) 2> >(tee stderr.log >&2)
+make doc > >(tee stdout.log) 2> >(tee stderr.log >&2)
 ! test -s stderr.log
 test -s Documentation/git.html
 test -s Documentation/git.xml

@@ -24,7 +24,7 @@ check_unignored_build_artifacts
 
 # Build docs with AsciiDoctor
 make clean
-make --jobs=2 USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.log >&2)
+make USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.log >&2)
 sed '/^GIT_VERSION = / d' stderr.log
 ! test -s stderr.log
 test -s Documentation/git.html

gitgitgadget

From: Johannes Schindelin <[email protected]>

Symbolic links are still not quite as easy to use on Windows as on Linux (for example, on versions older than Windows 10, only administrators can create symlinks, and on Windows 10 you still need to be in developer mode for regular users to have permission), but NTFS junctions can give us a way out.

---
 ci/run-build-and-tests.sh | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

index 80d72d120f..74d838ea01 100755
@@ -5,7 +5,10 @@
 
 . ${0%/*}/lib.sh
 
-ln -s "$cache_dir/.prove" t/.prove
+case "$CI_OS_NAME" in
+windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";;
+*) ln -s "$cache_dir/.prove" t/.prove;;
+esac
 
 make
 make --quiet test

gitgitgadget

From: Johannes Schindelin <[email protected]>

In the next commit, we want to teach Git's test suite to optionally output test results in JUnit-style .xml files. These files contain information about the time spent. So we need a way to measure time.

While we could use date +%s for that, this will give us only seconds, i.e. very coarse-grained timings.

GNU date supports date +%s.%N (i.e. nanosecond-precision output), but there is no equivalent in BSD date (read: on macOS, we would not be able to obtain precise timings).

So let's introduce test-tool date getnanos, with an optional start time, that outputs preciser values. Note that this might not actually give us nanosecond precision on some platforms, but it will give us as precise information as possible, without the portability issues of shell commands.

Granted, it is a bit pointless to try measuring times accurately in shell scripts, certainly to nanosecond precision. But it is better than second-granularity.

---
 t/helper/test-date.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

index a0837371ab..792a805374 100644
@@ -7,6 +7,7 @@ static const char *usage_msg = "\n"
 "  test-tool date parse [date]...\n"
 "  test-tool date approxidate [date]...\n"
 "  test-tool date timestamp [date]...\n"
+"  test-tool date getnanos [start-nanos]\n"
 "  test-tool date is64bit\n"
 "  test-tool date time_t-is64bit\n";
 

@@ -82,6 +83,15 @@ static void parse_approx_timestamp(const char **argv, struct timeval *now)
 	}
 }
 
+static void getnanos(const char **argv, struct timeval *now)
+{
+	double seconds = getnanotime() / 1.0e9;
+
+	if (*argv)
+		seconds -= strtod(*argv, NULL);
+	printf("%lf\n", seconds);
+}
+
 int cmd__date(int argc, const char **argv)
 {
 	struct timeval now;

@@ -108,6 +118,8 @@ int cmd__date(int argc, const char **argv)
 		parse_approxidate(argv+1, &now);
 	else if (!strcmp(*argv, "timestamp"))
 		parse_approx_timestamp(argv+1, &now);
+	else if (!strcmp(*argv, "getnanos"))
+		getnanos(argv+1, &now);
 	else if (!strcmp(*argv, "is64bit"))
 		return sizeof(timestamp_t) == 8 ? 0 : 1;
 	else if (!strcmp(*argv, "time_t-is64bit"))

gitgitgadget

From: Johannes Schindelin <[email protected]>

This will come in handy when publishing the results of Git's test suite during an automated Azure DevOps run.

Note: we need to make extra sure that invalid UTF-8 encoding is turned into valid UTF-8 (using the Replacement Character, \uFFFD) because t9902's trace contains such invalid byte sequences, and the task in the Azure Pipeline that uploads the test results would refuse to do anything if it was asked to parse an .xml file with invalid UTF-8 in it.

---
 Makefile                   |  1 +
 t/.gitignore               |  1 +
 t/helper/test-tool.c       |  1 +
 t/helper/test-tool.h       |  1 +
 t/helper/test-xml-encode.c | 80 +++++++++++++++++++++++++++++++++
 t/test-lib.sh              | 91 ++++++++++++++++++++++++++++++++++++++
 6 files changed, 175 insertions(+)
 create mode 100644 t/helper/test-xml-encode.c

index 1a44c811aa..044b4f77bd 100644
@@ -754,6 +754,7 @@ TEST_BUILTINS_OBJS += test-submodule-config.o
 TEST_BUILTINS_OBJS += test-submodule-nested-repo-config.o
 TEST_BUILTINS_OBJS += test-subprocess.o
 TEST_BUILTINS_OBJS += test-urlmatch-normalization.o
+TEST_BUILTINS_OBJS += test-xml-encode.o
 TEST_BUILTINS_OBJS += test-wildmatch.o
 TEST_BUILTINS_OBJS += test-windows-named-pipe.o
 TEST_BUILTINS_OBJS += test-write-cache.o

index 348715f0e4..91cf5772fe 100644
@@ -2,3 +2,4 @@
 /test-results
 /.prove
 /chainlinttmp
+/out/

index bfb195b1a8..4b4b397d93 100644
@@ -49,6 +49,7 @@ static struct test_cmd cmds[] = {
 	{ "submodule-nested-repo-config", cmd__submodule_nested_repo_config },
 	{ "subprocess", cmd__subprocess },
 	{ "urlmatch-normalization", cmd__urlmatch_normalization },
+	{ "xml-encode", cmd__xml_encode },
 	{ "wildmatch", cmd__wildmatch },
 #ifdef GIT_WINDOWS_NATIVE
 	{ "windows-named-pipe", cmd__windows_named_pipe },

index 042f12464b..c0ab65e370 100644
@@ -45,6 +45,7 @@ int cmd__submodule_config(int argc, const char **argv);
 int cmd__submodule_nested_repo_config(int argc, const char **argv);
 int cmd__subprocess(int argc, const char **argv);
 int cmd__urlmatch_normalization(int argc, const char **argv);
+int cmd__xml_encode(int argc, const char **argv);
 int cmd__wildmatch(int argc, const char **argv);
 #ifdef GIT_WINDOWS_NATIVE
 int cmd__windows_named_pipe(int argc, const char **argv);

gitgitgadget

From: Johannes Schindelin <[email protected]>

This patch introduces a conditional arm that defines some environment variables and a function that displays the URL given the job id (to identify previous runs for known-good trees).

Because Azure Pipeline's macOS agents already have git-lfs and gettext installed, we can leave BREW_INSTALL_PACKAGES empty (unlike in Travis' case).

Note: this patch does not introduce an Azure Pipelines definition yet; That is left for the next patch.

---
 ci/lib.sh                 | 25 +++++++++++++++++++++++++
 ci/print-test-failures.sh |  5 +++++
 2 files changed, 30 insertions(+)

index 32a28fd209..5505776876 100755
@@ -102,6 +102,31 @@ then
 	export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
 	export GIT_TEST_OPTS="--verbose-log -x --immediate"
 	export MAKEFLAGS="--jobs=2"
+elif test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI"
+then
+	CI_TYPE=azure-pipelines
+	# We are running in Azure Pipelines
+	CI_BRANCH="$BUILD_SOURCEBRANCH"
+	CI_COMMIT="$BUILD_SOURCEVERSION"
+	CI_JOB_ID="$BUILD_BUILDID"
+	CI_JOB_NUMBER="$BUILD_BUILDNUMBER"
+	CI_OS_NAME="$(echo "$AGENT_OS" | tr A-Z a-z)"
+	test darwin != "$CI_OS_NAME" || CI_OS_NAME=osx
+	CI_REPO_SLUG="$(expr "$BUILD_REPOSITORY_URI" : '.*/\([^/]*/[^/]*\)$')"
+	CC="${CC:-gcc}"
+
+	# use a subdirectory of the cache dir (because the file share is shared
+	# among *all* phases)
+	cache_dir="$HOME/test-cache/$SYSTEM_PHASENAME"
+
+	url_for_job_id () {
+		echo "$SYSTEM_TASKDEFINITIONSURI$SYSTEM_TEAMPROJECT/_build/results?buildId=$1"
+	}
+
+	BREW_INSTALL_PACKAGES=
+	export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
+	export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml"
+	export MAKEFLAGS="--jobs=10"
 else
 	echo "Could not identify CI type" >&2
 	exit 1

index cf321b474d..e688a26f0d 100755
@@ -41,6 +41,11 @@ do
 		case "$CI_TYPE" in
 		travis)
 			;;
+		azure-pipelines)
+			mkdir -p failed-test-artifacts
+			mv "$trash_dir" failed-test-artifacts
+			continue
+			;;
 		*)
 			echo "Unhandled CI type: $CI_TYPE" >&2
 			exit 1

gitgitgadget

From: Johannes Schindelin <[email protected]>

This commit adds an azure-pipelines.yml file which is Azure DevOps' equivalent to Travis CI's .travis.yml.

The main idea is to replicate the Travis configuration as faithfully as possible, to make it easy to compare the Azure Pipeline builds to the Travis ones (spoiler: some parts, especially the macOS jobs, are way faster in Azure Pileines). Meaning: the number and the order of the jobs added in this commit faithfully replicates what we have in .travis.yml.

Note: Our .travis.yml configuration has a Windows part that is *not* replicated in the Azure Pipelines definition. The reason is easy to see:
As Travis cannot support our Windws needs (even with the preliminary Windows support that was recently added to Travis after waiting for *years* for that feature, our test suite would simply hit Travis' timeout every single time).

To make things a bit easier to understand, we refrain from using the matrix feature here because (while it is powerful) it can be a bit confusing to users who are not familiar with CI setups. Therefore, we use a separate phase even for similar configurations (such as GCC vs Clang on Linux, GCC vs Clang on macOS).

Also, we make use of the shiny new feature we just introduced where the test suite can output JUnit-style .xml files. This information is made available in a nice UI that allows the viewer to filter by phase and/or test number, and to see trends such as: number of (failing) tests, time spent running the test suite, etc. (While this seemingly contradicts the intention to replicate the Travis configuration as faithfully as possible, it is just too nice to show off that capability here already.)

---
 azure-pipelines.yml   | 259 ++++++++++++++++++++++++++++++++++++++++++
 ci/mount-fileshare.sh |  25 ++++
 2 files changed, 284 insertions(+)
 create mode 100644 azure-pipelines.yml
 create mode 100755 ci/mount-fileshare.sh

new file mode 100644
index 0000000000..8cdef105c6
@@ -0,0 +1,259 @@
+resources:
+- repo: self
+  fetchDepth: 1
+
+jobs:
+- job: linux_clang
+  displayName: linux-clang
+  condition: succeeded()
+  pool: Hosted Ubuntu 1604
+  steps:
+  - bash: |
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
+
+       sudo apt-get update &&
+       sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev apache2-bin &&
+
+       export CC=clang || exit 1
+
+       ci/install-dependencies.sh || exit 1
+       ci/run-build-and-tests.sh || {
+           ci/print-test-failures.sh
+           exit 1
+       }
+
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
+    displayName: 'ci/run-build-and-tests.sh'
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)
+  - task: PublishTestResults@2
+    displayName: 'Publish Test Results **/TEST-*.xml'
+    inputs:
+      mergeTestResults: true
+      testRunTitle: 'linux-clang'
+      platform: Linux
+      publishRunAttachments: false
+    condition: succeededOrFailed()
+  - task: PublishBuildArtifacts@1
+    displayName: 'Publish trash directories of failed tests'
+    condition: failed()
+    inputs:
+      PathtoPublish: t/failed-test-artifacts
+      ArtifactName: failed-test-artifacts
+
+- job: linux_gcc
+  displayName: linux-gcc
+  condition: succeeded()
+  pool: Hosted Ubuntu 1604
+  steps:
+  - bash: |
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
+
+       sudo add-apt-repository ppa:ubuntu-toolchain-r/test &&
+       sudo apt-get update &&
+       sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev apache2 language-pack-is git-svn gcc-8 || exit 1
+
+       ci/install-dependencies.sh || exit 1
+       ci/run-build-and-tests.sh || {
+           ci/print-test-failures.sh
+           exit 1
+       }
+
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
+    displayName: 'ci/run-build-and-tests.sh'
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)
+  - task: PublishTestResults@2
+    displayName: 'Publish Test Results **/TEST-*.xml'
+    inputs:
+      mergeTestResults: true
+      testRunTitle: 'linux-gcc'
+      platform: Linux
+      publishRunAttachments: false
+    condition: succeededOrFailed()
+  - task: PublishBuildArtifacts@1
+    displayName: 'Publish trash directories of failed tests'
+    condition: failed()
+    inputs:
+      PathtoPublish: t/failed-test-artifacts
+      ArtifactName: failed-test-artifacts
+
+- job: osx_clang
+  displayName: osx-clang
+  condition: succeeded()
+  pool: Hosted macOS
+  steps:
+  - bash: |
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
+
+       export CC=clang
+
+       ci/install-dependencies.sh || exit 1
+       ci/run-build-and-tests.sh || {
+           ci/print-test-failures.sh
+           exit 1
+       }
+
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || umount "$HOME/test-cache" || exit 1
+    displayName: 'ci/run-build-and-tests.sh'
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)
+  - task: PublishTestResults@2
+    displayName: 'Publish Test Results **/TEST-*.xml'
+    inputs:
+      mergeTestResults: true
+      testRunTitle: 'osx-clang'
+      platform: macOS
+      publishRunAttachments: false
+    condition: succeededOrFailed()
+  - task: PublishBuildArtifacts@1
+    displayName: 'Publish trash directories of failed tests'
+    condition: failed()
+    inputs:
+      PathtoPublish: t/failed-test-artifacts
+      ArtifactName: failed-test-artifacts
+
+- job: osx_gcc
+  displayName: osx-gcc
+  condition: succeeded()
+  pool: Hosted macOS
+  steps:
+  - bash: |
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
+
+       ci/install-dependencies.sh || exit 1
+       ci/run-build-and-tests.sh || {
+           ci/print-test-failures.sh
+           exit 1
+       }
+
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || umount "$HOME/test-cache" || exit 1
+    displayName: 'ci/run-build-and-tests.sh'
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)
+  - task: PublishTestResults@2
+    displayName: 'Publish Test Results **/TEST-*.xml'
+    inputs:
+      mergeTestResults: true
+      testRunTitle: 'osx-gcc'
+      platform: macOS
+      publishRunAttachments: false
+    condition: succeededOrFailed()
+  - task: PublishBuildArtifacts@1
+    displayName: 'Publish trash directories of failed tests'
+    condition: failed()
+    inputs:
+      PathtoPublish: t/failed-test-artifacts
+      ArtifactName: failed-test-artifacts
+
+- job: gettext_poison
+  displayName: GETTEXT_POISON
+  condition: succeeded()
+  pool: Hosted Ubuntu 1604
+  steps:
+  - bash: |
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
+
+       sudo apt-get update &&
+       sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev &&
+
+       export jobname=GETTEXT_POISON || exit 1
+
+       ci/run-build-and-tests.sh || {
+           ci/print-test-failures.sh
+           exit 1
+       }
+
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
+    displayName: 'ci/run-build-and-tests.sh'
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)
+  - task: PublishTestResults@2
+    displayName: 'Publish Test Results **/TEST-*.xml'
+    inputs:
+      mergeTestResults: true
+      testRunTitle: 'gettext-poison'
+      platform: Linux
+      publishRunAttachments: false
+    condition: succeededOrFailed()
+  - task: PublishBuildArtifacts@1
+    displayName: 'Publish trash directories of failed tests'
+    condition: failed()
+    inputs:
+      PathtoPublish: t/failed-test-artifacts
+      ArtifactName: failed-test-artifacts
+
+- job: linux32
+  displayName: Linux32
+  condition: succeeded()
+  pool: Hosted Ubuntu 1604
+  steps:
+  - bash: |
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
+
+       res=0
+       sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" bash -lxc ci/run-linux32-docker.sh || res=1
+
+       sudo chmod a+r t/out/TEST-*.xml
+       test ! -d t/failed-test-artifacts || sudo chmod a+r t/failed-test-artifacts
+
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || res=1
+       exit $res
+    displayName: 'ci/run-linux32-docker.sh'
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)
+  - task: PublishTestResults@2
+    displayName: 'Publish Test Results **/TEST-*.xml'
+    inputs:
+      mergeTestResults: true
+      testRunTitle: 'linux32'
+      platform: Linux
+      publishRunAttachments: false
+    condition: succeededOrFailed()
+  - task: PublishBuildArtifacts@1
+    displayName: 'Publish trash directories of failed tests'
+    condition: failed()
+    inputs:
+      PathtoPublish: t/failed-test-artifacts
+      ArtifactName: failed-test-artifacts
+
+- job: static_analysis
+  displayName: StaticAnalysis
+  condition: succeeded()
+  pool: Hosted Ubuntu 1604
+  steps:
+  - bash: |
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
+
+       sudo apt-get update &&
+       sudo apt-get install -y coccinelle &&
+
+       export jobname=StaticAnalysis &&
+
+       ci/run-static-analysis.sh || exit 1
+
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
+    displayName: 'ci/run-static-analysis.sh'
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)
+
+- job: documentation
+  displayName: Documentation
+  condition: succeeded()
+  pool: Hosted Ubuntu 1604
+  steps:
+  - bash: |
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
+
+       sudo apt-get update &&
+       sudo apt-get install -y asciidoc xmlto asciidoctor &&
+
+       export ALREADY_HAVE_ASCIIDOCTOR=yes. &&
+       export jobname=Documentation &&
+
+       ci/test-documentation.sh || exit 1
+
+       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
+    displayName: 'ci/test-documentation.sh'
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)

new file mode 100755
index 0000000000..26b58a8096
@@ -0,0 +1,25 @@
+#!/bin/sh
+
+die () {
+	echo "$*" >&2
+	exit 1
+}
+
+test $# = 4 ||
+die "Usage: $0 <share> <username> <password> <mountpoint>"
+
+mkdir -p "$4" || die "Could not create $4"
+
+case "$(uname -s)" in
+Linux)
+	sudo mount -t cifs -o vers=3.0,username="$2",password="$3",dir_mode=0777,file_mode=0777,serverino "$1" "$4"
+	;;
+Darwin)
+	pass="$(echo "$3" | sed -e 's/\//%2F/g' -e 's/+/%2B/g')" &&
+	mount -t smbfs,soft "smb://$2:$pass@${1#//}" "$4"
+	;;
+*)
+	die "No support for $(uname -s)"
+	;;
+esac ||
+die "Could not mount $4"

gitgitgadget

From: Johannes Schindelin <[email protected]>

Previously, we did not have robust support for Windows in our CI definition, simply because Travis cannot accommodate our needs (even after Travis added experimental Windows support very recently, it takes longer than Travis' 50 minute timeout to build Git and run the test suite on Windows). Instead, we used a hack that started a dedicated Azure Pipeline from Travis and waited for the output, often timing out (which is quite fragile, as we found out).

With this commit, we finally have first-class support for Windows in our CI definition (in the Azure Pipelines one, that is).

Due to our reliance on Unix shell scripting in the test suite, combined with the challenges on executing such scripts on Windows, the Windows job currently takes a whopping ~1h20m to complete. Which is *far* longer than the next-longest job takes (linux-gcc, ~35m).

Now, Azure Pipelines's free tier for open source projects (such as Git) offers up to 10 concurrent jobs for free, meaning that the overall run time will be dominated by the slowest job(s).

Therefore, it makes sense to start the Windows job first, to minimize the time the entire build takes from start to end (which is now pretty safely the run time of the Windows job).

---
 azure-pipelines.yml | 91 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 91 insertions(+)

index 8cdef105c6..0f7b2125a1 100644
@@ -3,6 +3,97 @@ resources:
   fetchDepth: 1
 
 jobs:
+- job: windows
+  displayName: Windows
+  condition: succeeded()
+  pool: Hosted
+  timeoutInMinutes: 240
+  steps:
+  - powershell: |
+      if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") {
+        net use s: \\gitfileshare.file.core.windows.net\test-cache "$GITFILESHAREPWD" /user:AZURE\gitfileshare /persistent:no
+        cmd /c mklink /d "$(Build.SourcesDirectory)\test-cache" S:\
+      }
+    displayName: 'Mount test-cache'
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)
+  - powershell: |
+      # Helper to check the error level of the latest command (exit with error when appropriate)
+      function c() { if (!$?) { exit(1) } }
+
+      # Add build agent's MinGit to PATH
+      $env:PATH = $env:AGENT_HOMEDIRECTORY +"\externals\\git\cmd;" +$env:PATH
+
+      # Helper to initialize (or update) a Git worktree
+      function init ($path, $url, $set_origin) {
+        if (Test-Path $path) {
+          cd $path; c
+          if (Test-Path .git) {
+            & git init; c
+          } else {
+            & git status
+          }
+        } else {
+          & git init $path; c
+          cd $path; c
+        }
+        & git config core.autocrlf false; c
+        & git config core.untrackedCache true; c
+        if (($set_origin -ne 0) -and !(git config remote.origin.url)) {
+          & git remote add origin $url; c
+        }
+        & git fetch --depth=1 $url master; c
+        & git reset --hard FETCH_HEAD; c
+        & git clean -df; c
+      }
+
+      # Initialize Git for Windows' SDK
+      $sdk_path = "$(Build.SourcesDirectory)\git-sdk-64"
+      init "$sdk_path" "https://dev.azure.com/git-for-windows/git-sdk-64/_git/git-sdk-64" 0
+
+      # Let Git ignore the SDK and the test-cache
+      "/git-sdk-64/`n/test-cache/`n" | Out-File -NoNewLine -Encoding ascii -Append "$(Build.SourcesDirectory)\.git\info\exclude"
+    displayName: 'Initialize the Git for Windows SDK'
+  - powershell: |
+      & "git-sdk-64\git-cmd.exe" --command=usr\\bin\\bash.exe -lc @"
+        export DEVELOPER=1
+        export NO_PERL=1
+        export NO_SVN_TESTS=1
+        export GIT_TEST_SKIP_REBASE_P=1
+
+        ci/run-build-and-tests.sh || {
+          ci/print-test-failures.sh
+          exit 1
+        }
+      "@
+      if (!$?) { exit(1) }
+    displayName: 'Build & Test'
+    env:
+      HOME: $(Build.SourcesDirectory)
+      MSYSTEM: MINGW64
+  - powershell: |
+      if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") {
+        cmd /c rmdir "$(Build.SourcesDirectory)\test-cache"
+      }
+    displayName: 'Unmount test-cache'
+    condition: true
+    env:
+      GITFILESHAREPWD: $(gitfileshare.pwd)
+  - task: PublishTestResults@2
+    displayName: 'Publish Test Results **/TEST-*.xml'
+    inputs:
+      mergeTestResults: true
+      testRunTitle: 'windows'
+      platform: Windows
+      publishRunAttachments: false
+    condition: succeededOrFailed()
+  - task: PublishBuildArtifacts@1
+    displayName: 'Publish trash directories of failed tests'
+    condition: failed()
+    inputs:
+      PathtoPublish: t/failed-test-artifacts
+      ArtifactName: failed-test-artifacts
+
 - job: linux_clang
   displayName: linux-clang
   condition: succeeded()

gitgitgadget

From: Johannes Schindelin <[email protected]>

Instead of a shallow fetch followed by a sparse checkout, we are better off by using a separate, dedicated Pipeline that bundles the SDK as a build artifact, and then consuming that build artifact here.

In fact, since this artifact will be used a lot, we spent substantial time on figuring out a minimal subset of the Git for Windows SDK, just enough to build and test Git. The result is a size reduction from around 1GB (compressed) to around 55MB (compressed). This also comes with the change where we now call usr\bin\bash.exe directly, as git-cmd.exe is not included in the minimal SDK.

That reduces the time to initialize Git for Windows' SDK from anywhere between 2m30s-7m to a little over 1m.

Note: in theory, we could also use the DownloadBuildArtifacts@0 task here. However, restricted permissions that are in effect when building from forks would let this fail for PR builds, defeating the whole purpose of the Azure Pipelines support for git.git.

---
 azure-pipelines.yml | 44 +++++++++-----------------------------------
 1 file changed, 9 insertions(+), 35 deletions(-)

index 0f7b2125a1..480e841a85 100644
@@ -18,44 +18,18 @@ jobs:
     env:
       GITFILESHAREPWD: $(gitfileshare.pwd)
   - powershell: |
-      # Helper to check the error level of the latest command (exit with error when appropriate)
-      function c() { if (!$?) { exit(1) } }
-
-      # Add build agent's MinGit to PATH
-      $env:PATH = $env:AGENT_HOMEDIRECTORY +"\externals\\git\cmd;" +$env:PATH
-
-      # Helper to initialize (or update) a Git worktree
-      function init ($path, $url, $set_origin) {
-        if (Test-Path $path) {
-          cd $path; c
-          if (Test-Path .git) {
-            & git init; c
-          } else {
-            & git status
-          }
-        } else {
-          & git init $path; c
-          cd $path; c
-        }
-        & git config core.autocrlf false; c
-        & git config core.untrackedCache true; c
-        if (($set_origin -ne 0) -and !(git config remote.origin.url)) {
-          & git remote add origin $url; c
-        }
-        & git fetch --depth=1 $url master; c
-        & git reset --hard FETCH_HEAD; c
-        & git clean -df; c
-      }
-
-      # Initialize Git for Windows' SDK
-      $sdk_path = "$(Build.SourcesDirectory)\git-sdk-64"
-      init "$sdk_path" "https://dev.azure.com/git-for-windows/git-sdk-64/_git/git-sdk-64" 0
+      $urlbase = "https://dev.azure.com/git-for-windows/git/_apis/build/builds"
+      $id = ((Invoke-WebRequest -UseBasicParsing "${urlbase}?definitions=22&statusFilter=completed&resultFilter=succeeded&`$top=1").content | ConvertFrom-JSON).value[0].id
+      $downloadUrl = ((Invoke-WebRequest -UseBasicParsing "${urlbase}/$id/artifacts").content | ConvertFrom-JSON).value[1].resource.downloadUrl
+      (New-Object Net.WebClient).DownloadFile($downloadUrl,"git-sdk-64-minimal.zip")
+      Expand-Archive git-sdk-64-minimal.zip -DestinationPath . -Force
+      Remove-Item git-sdk-64-minimal.zip
 
       # Let Git ignore the SDK and the test-cache
-      "/git-sdk-64/`n/test-cache/`n" | Out-File -NoNewLine -Encoding ascii -Append "$(Build.SourcesDirectory)\.git\info\exclude"
-    displayName: 'Initialize the Git for Windows SDK'
+      "/git-sdk-64-minimal/`n/test-cache/`n" | Out-File -NoNewLine -Encoding ascii -Append "$(Build.SourcesDirectory)\.git\info\exclude"
+    displayName: 'Download git-sdk-64-minimal'
   - powershell: |
-      & "git-sdk-64\git-cmd.exe" --command=usr\\bin\\bash.exe -lc @"
+      & git-sdk-64-minimal\usr\bin\bash.exe -lc @"
         export DEVELOPER=1
         export NO_PERL=1
         export NO_SVN_TESTS=1

gitgitgadget

From: Johannes Schindelin <[email protected]>

Every once in a while, the Azure Pipeline fails with some semi-random

error: timer thread did not terminate timely

This error message means that the thread that is used to emulate the setitimer() function did not terminate within 1,000 milliseconds.

The most likely explanation (and therefore the one we should assume to be true, according to Occam's Razor) is that the timeout of one second is simply not enough because we try to run so many tasks in parallel.

So let's give it ten seconds instead of only one. That should be enough.

---
 compat/mingw.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

index b459e1a291..e0dfe8844d 100644
@@ -2100,7 +2100,7 @@ static void stop_timer_thread(void)
 	if (timer_event)
 		SetEvent(timer_event);	/* tell thread to terminate */
 	if (timer_thread) {
-		int rc = WaitForSingleObject(timer_thread, 1000);
+		int rc = WaitForSingleObject(timer_thread, 10000);
 		if (rc == WAIT_TIMEOUT)
 			error("timer thread did not terminate timely");
 		else if (rc != WAIT_OBJECT_0)

gitgitgadget

From: Johannes Schindelin <[email protected]>

Just like so many other OSS projects, we now also have a build badge.

---
 README.md | 2 ++
 1 file changed, 2 insertions(+)

index f920a42fad..764c480c66 100644
@@ -1,3 +1,5 @@
+[![Build Status](https://dev.azure.com/git/git/_apis/build/status/test-git.git)](https://dev.azure.com/git/git/_build/latest?definitionId=2)
+
 Git - fast, scalable, distributed revision control system
 =========================================================
 

gitgitgadget

From: Johannes Schindelin <[email protected]>

It is a bit ridiculous to spin up a full-blown Perl instance (especially on Windows, where that means spinning up a full POSIX emulation layer, AKA the MSYS2 runtime) just to tell how large a given file is.

So let's just use the test-tool to do that job instead.

This command will also be used over the next commits, to allow for cutting out individual test cases' verbose log from the file generated via --verbose-log.

---
 t/helper/test-path-utils.c          | 12 ++++++++++++
 t/t0021-conversion.sh               |  2 +-
 t/t1050-large.sh                    |  2 +-
 t/t5315-pack-objects-compression.sh |  2 +-
 t/t9303-fast-import-compression.sh  |  2 +-
 5 files changed, 16 insertions(+), 4 deletions(-)

index ae091d9b3e..30211d6d64 100644
@@ -291,6 +291,18 @@ int cmd__path_utils(int argc, const char **argv)
 		return !!res;
 	}
 
+	if (argc > 2 && !strcmp(argv[1], "file-size")) {
+		int res = 0, i;
+		struct stat st;
+
+		for (i = 2; i < argc; i++)
+			if (stat(argv[i], &st))
+				res = error_errno("Cannot stat '%s'", argv[i]);
+			else
+				printf("%"PRIuMAX"\n", (uintmax_t)st.st_size);
+		return !!res;
+	}
+
 	fprintf(stderr, "%s: unknown function name: %s\n", argv[0],
 		argv[1] ? argv[1] : "(there was none)");
 	return 1;

index fd5f1ac649..e10f5f787f 100755
@@ -24,7 +24,7 @@ generate_random_characters () {
 }
 
 file_size () {
-	perl -e 'print -s $ARGV[0]' "$1"
+	test-tool path-utils file-size "$1"
 }
 
 filter_git () {

index 1a9b21b293..dcb4dbba67 100755
@@ -8,7 +8,7 @@ test_description='adding and checking out large blobs'
 # This should be moved to test-lib.sh together with the
 # copy in t0021 after both topics have graduated to 'master'.
 file_size () {
-	perl -e 'print -s $ARGV[0]' "$1"
+	test-tool path-utils file-size "$1"
 }
 
 test_expect_success setup '

index 34c47dae09..df970d7584 100755
@@ -7,7 +7,7 @@ test_description='pack-object compression configuration'
 # This should be moved to test-lib.sh together with the
 # copy in t0021 after both topics have graduated to 'master'.
 file_size () {
-	perl -e 'print -s $ARGV[0]' "$1"
+	test-tool path-utils file-size "$1"
 }
 
 test_expect_success setup '

index 856219f46a..5045f02a53 100755
@@ -6,7 +6,7 @@ test_description='compression setting of fast-import utility'
 # This should be moved to test-lib.sh together with the
 # copy in t0021 after both topics have graduated to 'master'.
 file_size () {
-	perl -e 'print -s $ARGV[0]' "$1"
+	test-tool path-utils file-size "$1"
 }
 
 import_large () {

gitgitgadget

From: Johannes Schindelin <[email protected]>

The JUnit XML format lends itself to be presented in a powerful UI, where you can drill down to the information you are interested in very quickly.

For test failures, this usually means that you want to see the detailed trace of the failing tests.

With Travis CI, we passed the --verbose-log option to get those traces. However, that seems excessive, as we do not need/use the logs in almost all of those cases: only when a test fails do we have a way to include the trace.

So let's do something different when using Azure DevOps: let's run all the tests with --quiet first, and only if a failure is encountered, try to trace the commands as they are executed.

Of course, we cannot turn on --verbose-log after the fact. So let's just re-run the test with all the same options, adding --verbose-log. And then munging the output file into the JUnit XML on the fly.

Note: there is an off chance that re-running the test in verbose mode "fixes" the failures (and this does happen from time to time!). That is a possibility we should be able to live with. Ideally, we would label this as "Passed upon rerun", and Azure Pipelines even know about that outcome, but it is not available when using the JUnit XML format for now:
https://github.com/Microsoft/azure-pipelines-agent/blob/master/src/Agent.Worker/TestResults/JunitResultReader.cs

---
 t/helper/test-path-utils.c | 21 +++++++++++++++++++++
 t/test-lib.sh              | 22 +++++++++++++++++++++-
 2 files changed, 42 insertions(+), 1 deletion(-)

index 30211d6d64..6efde6f5ba 100644
@@ -303,6 +303,27 @@ int cmd__path_utils(int argc, const char **argv)
 		return !!res;
 	}
 
+	if (argc == 4 && !strcmp(argv[1], "skip-n-bytes")) {
+		int fd = open(argv[2], O_RDONLY), offset = atoi(argv[3]);
+		char buffer[65536];
+
+		if (fd < 0)
+			die_errno("could not open '%s'", argv[2]);
+		if (lseek(fd, offset, SEEK_SET) < 0)
+			die_errno("could not skip %d bytes", offset);
+		for (;;) {
+			ssize_t count = read(fd, buffer, sizeof(buffer));
+			if (count < 0)
+				die_errno("could not read '%s'", argv[2]);
+			if (!count)
+				break;
+			if (write(1, buffer, count) < 0)
+				die_errno("could not write to stdout");
+		}
+		close(fd);
+		return 0;
+	}
+
 	fprintf(stderr, "%s: unknown function name: %s\n", argv[0],
 		argv[1] ? argv[1] : "(there was none)");
 	return 1;

index a3b2166cb5..f31a1c8f79 100644
@@ -639,8 +639,19 @@ test_failure_ () {
 		junit_insert="<failure message=\"not ok $test_count -"
 		junit_insert="$junit_insert $(xml_attr_encode "$1")\">"
 		junit_insert="$junit_insert $(xml_attr_encode \
-			"$(printf '%s\n' "$@" | sed 1d)")"
+			"$(if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
+			   then
+				test-tool path-utils skip-n-bytes \
+					"$GIT_TEST_TEE_OUTPUT_FILE" $GIT_TEST_TEE_OFFSET
+			   else
+				printf '%s\n' "$@" | sed 1d
+			   fi)")"
 		junit_insert="$junit_insert</failure>"
+		if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
+		then
+			junit_insert="$junit_insert<system-err>$(xml_attr_encode \
+				"$(cat "$GIT_TEST_TEE_OUTPUT_FILE")")</system-err>"
+		fi
 		write_junit_xml_testcase "$1" "      $junit_insert"
 	fi
 	test_failure=$(($test_failure + 1))

@@ -931,6 +942,11 @@ test_finish_ () {
 	echo >&3 ""
 	maybe_teardown_valgrind
 	maybe_teardown_verbose
+	if test -n "$GIT_TEST_TEE_OFFSET"
+	then
+		GIT_TEST_TEE_OFFSET=$(test-tool path-utils file-size \
+			"$GIT_TEST_TEE_OUTPUT_FILE")
+	fi
 }
 
 test_skip () {

@@ -1280,6 +1296,10 @@ then
 		date +%Y-%m-%dT%H:%M:%S)\""
 	write_junit_xml --truncate "<testsuites>" "  <testsuite $junit_attrs>"
 	junit_suite_start=$(test-tool date getnanos)
+	if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
+	then
+		GIT_TEST_TEE_OFFSET=0
+	fi
 fi
 
 # Provide an implementation of the 'yes' utility

gitgitgadget

From: Johannes Schindelin <[email protected]>

It seems that every once in a while in the Git for Windows SDK, there are some transient file locking issues preventing the test clean up to delete the trash directory. Let's be gentle and try again five seconds later, and only error out if it still fails the second time.

This change helps Windows, and does not hurt any other platform (normally, it is highly unlikely that said deletion fails, and if it does, normally it will fail again even 5 seconds later).

---
 t/test-lib.sh | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

index f31a1c8f79..9c0ca5effb 100644
@@ -1104,7 +1104,11 @@ test_done () {
 			error "Tests passed but trash directory already removed before test cleanup; aborting"
 
 			cd "$TRASH_DIRECTORY/.." &&
-			rm -fr "$TRASH_DIRECTORY" ||
+			rm -fr "$TRASH_DIRECTORY" || {
+				# try again in a bit
+				sleep 5;
+				rm -fr "$TRASH_DIRECTORY"
+			} ||
 			error "Tests passed but test cleanup failed; aborting"
 		fi
 		test_at_end_hook_

gitgitgadget

Hi Ævar,

thanks for digging out this old thread, I really could have done never thinking about it again!

It does not really break it, it just delays the inevitable failure.

If by "CL" you refer to the cover letter, then I might have made it sound as if it was only encountered in the Azure Pipelines agents. I vaguely seem to remember seeing something like this quite often on my personal machine, too, though. Most likely Microsoft Defender going a little wild.

No, I'm not under CI, and I did encounter these issues. And they abruptly stopped with the patch you apparently still want to discuss ;-)

I don't understand this statement. Or was there a question in it that you'd like me to answer?

Ciao,
Johannes

I didn't think I'd dug up something so traumatic :)

Yes, the cover letter.

If the fix was purely for Azure's CI setup I was suggesting that a better solution might be to keep the trash around until the end. But that's not the only setup as you note here, so let's discard that suggestion.

In any case, your patch clearly undoes whatever canary for gc issues df4c0d1a792 was trying to put into the test-lib, but didn't say so in its commit message.

So I figured it was something that was missed at the time, and that I should send a quick E-Mail to both authors to see if anyone cared, maybe nobody does.

It's just something I ran into while reviewing test-lib.sh for some unrelated changes I was making...

Good eyes.

I am not quite sure if we are better off catching a transitory failure, which could go away when retried in 5 seconds, to remove the test directory (in which case, the change in question is a clear regression from what df4c0d1a (test-lib: abort when can't remove trash directory, 2017-04-20) intended to catch), or we only want to catch permanent failures (in which case the patch in question is a clear improvement).

In either way, the decision and the rationale behind it should be in the log message.

Thanks.

Hi Ævar,

I still think this is the best answer to this (implicit) question:

I was not _really_ paying attention to that commit when I implemented the work-around you mentioned above. At the same time I think it does _not_ undo the canary. If the trash directory cannot be removed via rm -fr, and if that is an indicator for something fishy going on, chances are that the second rm -fr a couple seconds later will _also_ fail, and we still get that error message.

The only reason why the second rm should succeed, at least that I can think of, is that something on Windows blocked those files from being deleted, and it is no longer blocking after a couple seconds, and that usually means that an anti-malware scanned those files.

Ciao,
Dscho

Both commits referenced in df4c0d1a79's log message fixed races between 'test_done's cleanup and a still running background 'git gc', and df4c0d1a79 was meant to draw our attention to similar issues in the future. And it did:

https://public-inbox.org/git/[email protected]/

So no, the failure is not inevitable, there are other reasons why the second 'rm' might still succeed after the first failed, even just a fraction of a second later. And yes, that 'sleep' added in the patch above did unquestionably break that canary,

Having read that thread now I agree, but I also left with a "who cares?" and "so let's keep the sleep then?".

I.e. is this a problem that any of the software we're maintaining is going to care about in the wild, it's not like people are expecting gc, repack, fast-import etc. to behave well in the face of rm -rfing the directory they're operating on.

So it seems like just an issue that crops up because of how our test suite manages and removes per-test trash directories. So it seems better to:

1. Just keep that "sleep a bit" and retry hack

2. Maybe on some/most platforms we can use cgroups or whatever passes for a reliable "I started a process tree starting at this PID, kill -9 the whole thing please" before cleanup these days.

Not at all. The real problem is that some stray background git process is *still* actively writing to the test repository when the test script is already supposed to be finished.

What really seems better:

3. Keep that "sleep a bit" hack only on platforms that can't delete opened files.

From: Johannes Schindelin <[email protected]>

We really need to be able to find the test helpers... Really. This change was forgotten when we moved the test helpers into t/helper/

---
 t/test-lib.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

index 9c0ca5effb..c790e98fd2 100644
@@ -1227,7 +1227,7 @@ else # normal case, use ../bin-wrappers only unless $with_dashes:
 	GIT_EXEC_PATH=$GIT_BUILD_DIR
 	if test -n "$with_dashes"
 	then
-		PATH="$GIT_BUILD_DIR:$PATH"
+		PATH="$GIT_BUILD_DIR:$GIT_BUILD_DIR/t/helper:$PATH"
 	fi
 fi
 GIT_TEMPLATE_DIR="$GIT_BUILD_DIR"/templates/blt

gitgitgadget

From: Johannes Schindelin <[email protected]>

When building Git with RUNTIME_PREFIX and starting a test helper from t/helper/, it fails to detect a system prefix. The reason is that the RUNTIME_PREFIX feature wants to use the location of the Git executable to determine where the support files can be found, e.g. system-wide Git config or the translations. This does not make any sense for the test helpers, though, as they are distinctly not in a directory structure resembling the final installation location of Git.

That is the reason why the test helpers rely on environment variables to indicate the location of the needed support files, e.g. GIT_TEXTDOMAINDIR. If this information is missing, the output will contain warnings like this one:

RUNTIME_PREFIX requested, but prefix computation failed. [...]

In t0061, we did not expect that to happen, and it actually does not happen in the regular case, because bin-wrappers/test-tool specifically sets GIT_TEXTDOMAINDIR (and as a consequence, nothing in test-tool needs to know anything about any runtime prefix).

However, with --with-dashes, bin-wrappers/test-tool is no longer called, but t/helper/test-tool is called directly instead.

So let's just ignore the RUNTIME_PREFIX warning.

---
 t/t0061-run-command.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

index 99a614bc7c..5a2d087bf0 100755
@@ -166,7 +166,8 @@ test_trace () {
 	expect="$1"
 	shift
 	GIT_TRACE=1 test-tool run-command "$@" run-command true 2>&1 >/dev/null | \
-		sed -e 's/.* run_command: //' -e '/trace: .*/d' >actual &&
+		sed -e 's/.* run_command: //' -e '/trace: .*/d' \
+			-e '/RUNTIME_PREFIX requested/d' >actual &&
 	echo "$expect true" >expect &&
 	test_cmp expect actual
 }

gitgitgadget

From: Johannes Schindelin <[email protected]>

This speeds up the tests by a bit on Windows, where running Unix shell scripts (and spawning processes) is not exactly a cheap operation.

---
 t/README      |  9 +++++++++
 t/test-lib.sh | 19 +++++++++++++------
 2 files changed, 22 insertions(+), 6 deletions(-)

index 11ce7675e3..063530234f 100644
@@ -170,6 +170,15 @@ appropriately before running "make".
 	implied by other options like --valgrind and
 	GIT_TEST_INSTALLED.
 
+--no-bin-wrappers::
+	By default, the test suite uses the wrappers in
+	`../bin-wrappers/` to execute `git` and friends. With this option,
+	`../git` and friends are run directly. This is not recommended
+	in general, as the wrappers contain safeguards to ensure that no
+	files from an installed Git are used, but can speed up test runs
+	especially on platforms where running shell scripts is expensive
+	(most notably, Windows).
+
 --root=<directory>::
 	Create "trash" directories used to store all temporary data during
 	testing under <directory>, instead of the t/ directory.

index c790e98fd2..25e649c997 100644
@@ -111,6 +111,8 @@ do
 		test -z "$HARNESS_ACTIVE" && quiet=t ;;
 	--with-dashes)
 		with_dashes=t ;;
+	--no-bin-wrappers)
+		no_bin_wrappers=t ;;
 	--no-color)
 		color= ;;
 	--va|--val|--valg|--valgr|--valgri|--valgrin|--valgrind)

@@ -1214,16 +1216,21 @@ then
 	PATH=$GIT_TEST_INSTALLED:$GIT_BUILD_DIR/t/helper:$PATH
 	GIT_EXEC_PATH=${GIT_TEST_EXEC_PATH:-$GIT_EXEC_PATH}
 else # normal case, use ../bin-wrappers only unless $with_dashes:
-	git_bin_dir="$GIT_BUILD_DIR/bin-wrappers"
-	if ! test -x "$git_bin_dir/git"
+	if test -n "$no_bin_wrappers"
 	then
-		if test -z "$with_dashes"
+		with_dashes=t
+	else
+		git_bin_dir="$GIT_BUILD_DIR/bin-wrappers"
+		if ! test -x "$git_bin_dir/git"
 		then
-			say "$git_bin_dir/git is not executable; using GIT_EXEC_PATH"
+			if test -z "$with_dashes"
+			then
+				say "$git_bin_dir/git is not executable; using GIT_EXEC_PATH"
+			fi
+			with_dashes=t
 		fi
-		with_dashes=t
+		PATH="$git_bin_dir:$PATH"
 	fi
-	PATH="$git_bin_dir:$PATH"
 	GIT_EXEC_PATH=$GIT_BUILD_DIR
 	if test -n "$with_dashes"
 	then

gitgitgadget

From: Johannes Schindelin <[email protected]>

As Unix shell scripting comes at a hefty price on Windows, we have to see where we can save some time to run the test suite.

Let's skip the chain linting and the bin-wrappers/ redirection on Windows; this seems to shave of anywhere between 10-30% from the overall runtime.

---
 ci/lib.sh | 2 ++
 1 file changed, 2 insertions(+)

index 5505776876..c2bc6c68b9 100755
@@ -127,6 +127,8 @@ then
 	export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
 	export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml"
 	export MAKEFLAGS="--jobs=10"
+	test windows_nt != "$CI_OS_NAME" ||
+	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
 else
 	echo "Could not identify CI type" >&2
 	exit 1

gitgitgadget

From: Johannes Schindelin <[email protected]>

The fact that Git's test suite is implemented in Unix shell script that is as portable as we can muster, combined with the fact that Unix shell scripting is foreign to Windows (and therefore has to be emulated), results in pretty abysmal speed of the test suite on that platform, for pretty much no other reason than that language choice.

For comparison: while the Linux build & test is typically done within about 8 minutes, the Windows build & test typically lasts about 80 minutes in Azure Pipelines.

To help with that, let's use the Azure Pipeline feature where you can parallelize jobs, make jobs depend on each other, and pass artifacts between them.

The tests are distributed using the following heuristic: listing all test scripts ordered by size in descending order (as a cheap way to estimate the overall run time), every Nth script is run (where N is the total number of parallel jobs), starting at the index corresponding to the parallel job. This slicing is performed by a new function that is added to the test-tool.

To optimize the overall runtime of the entire Pipeline, we need to move the Windows jobs to the beginning (otherwise there would be a very decent chance for the Pipeline to be run only the Windows build, while all the parallel Windows test jobs wait for this single one).

We use Azure Pipelines Artifacts for both the minimal Git for Windows SDK as well as the built executables, as deduplication and caching close to the agents makes that really fast. For comparison: while downloading and unpacking the minimal Git for Windows SDK via PowerShell takes only one minute (down from anywhere between 2.5 to 7 when using a shallow clone), uploading it as Pipeline Artifact takes less than 30s and downloading and unpacking less than 20s (sometimes even as little as only twelve seconds).

---
 Makefile                   | 10 +++++
 azure-pipelines.yml        | 79 ++++++++++++++++++++++++++++++++++----
 ci/make-test-artifacts.sh  | 12 ++++++
 ci/run-test-slice.sh       | 17 ++++++++
 t/helper/test-path-utils.c | 31 +++++++++++++++
 5 files changed, 141 insertions(+), 8 deletions(-)
 create mode 100755 ci/make-test-artifacts.sh
 create mode 100755 ci/run-test-slice.sh

index 044b4f77bd..daa318fe17 100644
@@ -2927,6 +2927,16 @@ rpm::
 	@false
 .PHONY: rpm
 
+artifacts-tar:: $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) $(OTHER_PROGRAMS) \
+		GIT-BUILD-OPTIONS $(TEST_PROGRAMS) $(test_bindir_programs) \
+		$(NO_INSTALL) $(MOFILES)
+	$(QUIET_SUBDIR0)templates $(QUIET_SUBDIR1) \
+		SHELL_PATH='$(SHELL_PATH_SQ)' PERL_PATH='$(PERL_PATH_SQ)'
+	test -n "$(ARTIFACTS_DIRECTORY)"
+	mkdir -p "$(ARTIFACTS_DIRECTORY)"
+	$(TAR) czf "$(ARTIFACTS_DIRECTORY)/artifacts.tar.gz" $^ templates/blt/
+.PHONY: artifacts-tar
+
 htmldocs = git-htmldocs-$(GIT_VERSION)
 manpages = git-manpages-$(GIT_VERSION)
 .PHONY: dist-doc distclean

new file mode 100755
index 0000000000..646967481f
@@ -0,0 +1,12 @@
+#!/bin/sh
+#
+# Build Git and store artifacts for testing
+#
+
+mkdir -p "$1" # in case ci/lib.sh decides to quit early
+
+. ${0%/*}/lib.sh
+
+make artifacts-tar ARTIFACTS_DIRECTORY="$1"
+
+check_unignored_build_artifacts

new file mode 100755
index 0000000000..f8c2c3106a
@@ -0,0 +1,17 @@
+#!/bin/sh
+#
+# Test Git in parallel
+#
+
+. ${0%/*}/lib.sh
+
+case "$CI_OS_NAME" in
+windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";;
+*) ln -s "$cache_dir/.prove" t/.prove;;
+esac
+
+make --quiet -C t T="$(cd t &&
+	./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh |
+	tr '\n' ' ')"
+
+check_unignored_build_artifacts

index 6efde6f5ba..5d543ad21f 100644
@@ -177,6 +177,14 @@ static int is_dotgitmodules(const char *path)
 	return is_hfs_dotgitmodules(path) || is_ntfs_dotgitmodules(path);
 }
 
+static int cmp_by_st_size(const void *a, const void *b)
+{
+	intptr_t x = (intptr_t)((struct string_list_item *)a)->util;
+	intptr_t y = (intptr_t)((struct string_list_item *)b)->util;
+
+	return x > y ? -1 : (x < y ? +1 : 0);
+}
+
 int cmd__path_utils(int argc, const char **argv)
 {
 	if (argc == 3 && !strcmp(argv[1], "normalize_path_copy")) {

@@ -324,6 +332,29 @@ int cmd__path_utils(int argc, const char **argv)
 		return 0;
 	}
 
+	if (argc > 5 && !strcmp(argv[1], "slice-tests")) {
+		int res = 0;
+		long offset, stride, i;
+		struct string_list list = STRING_LIST_INIT_NODUP;
+		struct stat st;
+
+		offset = strtol(argv[2], NULL, 10);
+		stride = strtol(argv[3], NULL, 10);
+		if (stride < 1)
+			stride = 1;
+		for (i = 4; i < argc; i++)
+			if (stat(argv[i], &st))
+				res = error_errno("Cannot stat '%s'", argv[i]);
+			else
+				string_list_append(&list, argv[i])->util =
+					(void *)(intptr_t)st.st_size;
+		QSORT(list.items, list.nr, cmp_by_st_size);
+		for (i = offset; i < list.nr; i+= stride)
+			printf("%s\n", list.items[i].string);
+
+		return !!res;
+	}
+
 	fprintf(stderr, "%s: unknown function name: %s\n", argv[0],
 		argv[1] ? argv[1] : "(there was none)");
 	return 1;

gitgitgadget

I have not reviewed this in any detail, but incorporating this in some form or other seems like a no-brainer to me.

If we have "free" (from the perspective of the project) CPU being offered by various CI setups let's use it.

If people don't like it they don't have to look at those CI runs, and the burden of carrying another CI in git.git is minimal, and actually this patch series makes it much easier to incorporate other CI's later, since you took care of moving the Travis-specific bits out.

As an aside I poked Junio via private mail in late August to see if he'd be interested in pushing to gitlab.com/git/git.git too as part of his normal push-outs. One neat thing that would buy us is the ability to have a .gitlab-ci.yml in git.git and use their CI implementation.

That would be great since it allows gitlab forks to have CI support from the beginning. But gitlab ci has time execution limits if I remember correctly and I'm not sure if we'll use it all up before the end of the month (or whatever period that is), or do they offer something special to git.git?

Duy

People at GitLab also have wondered if it would be possible for GitLab to host an official Git mirror, and I think it would be nice indeed if Junio could push to gitlab.com.

Yeah, that would be nice for people who prefer to use GitLab CI.

In general I agree that it is a good idea to have support for many CI tools even if it is not always up-to-date and even if it's not used by many people.

I can ask for more time if that would help.

Hi team,

AFAIR Junio does not push to github.com/git/git, it is an automatic mirror.

GitLab could easily do the same.

Ciao,
Johannes

On https://git-blame.blogspot.com/p/git-public-repositories.html there is:

"Copies of the source code to Git live in many repositories, and this is a list of the ones I push into. Some repositories have only a subset of branches.

With maint, master, next, pu, todo:

[...]
https://github.com/git/git/
"

Yeah, GitLab could easily create an automatic mirror. I think it would be a bit better though if the git repo on GitLab could be independent. It could make things more resilient against outages or security issues.

It used to be in the early days but these days git/git and gitster/git are updated in a same for loop that pushes to various destinations. You are correct that GitLab or any other hosting sites could do the same polling and mirroring. I am just too lazy to open a new account at yet another hosting site to add that for loop, but I may choose to when I am absolutely bored and nothing else to do ;-).

Do you mind if I squat gitlab.com/git/git in the meantime (i.e. create an org etc.) and have it mirror github.com/git/git?, I'll hand the keys over to you if/when you'd like to update your for-loop :)

In the meantime sometimes GitLab's UI s better than GitHub's, or the other way around, so it would be useful to have it there as mirror for that reason alone, and then we can always experiment with a .gitlab-ci.yml in-repo.

Obviously somebody who is not even interested in obtaining an account would appreciate, not just "would not mind", if a trusted member in the community did that for the community ;-)

Thanks.

I've set this up at https://gitlab.com/git-vcs

The /git namespace was taken (and I asked GitLab support if it was stale, they said no). Also tried /git-scm and /gitscm, ditto. So I settled on /git-vcs (version control system).

I mirrored all the repos that were on github.com/git (except for cabal.git of course). This is being kept up-to-date with GitLab's own mirroring feature, so it should always be max ~15m out of date v.s. the GitHub version.

As an aside, I noticed that
https://github.com/git/sha1collisiondetection/ has never worked in combination with git.git, i.e. it's cloned at a version that pre-dates the initial introduction of the sha1collisiondetection submodule. Our other mirrors don't seem to have it at all relative to ../sha1collisiondetection.git from their git.git.

Junio: What was the plan with that? It's never been used in combination with git.git, so maybe we should just drop it?

If we'd like to keep it and if it was kept up-to-date some scripts of yours we could use a relative URL in the .gitmodules, so e.g. if you cloned from kernel.org you'd also get the submodule from there.

Also since we're fixing it, it would be good to delete the existing repo and re-make it by clicking "fork" on
https://github.com/cr-marcstevens/sha1collisiondetection so GitHub shows "forked from...", i.e. it's associated with the parent project in GitHub's UI.