ci: add support for macOS to GitLab CI

Hi,

this is the second version of my patch series that adds a macOS job to GitLab CI. Changes compared to v1:

- Added a fix for a flaky test in t7527 that caused the pipeline to fail in ~50% of all runs.

- Improved some commit messages.

- Tests now write test data into a RAMDisk. This speeds up tests and fixes some hung pipelines I was seeing.

Thanks for your reviews so far!

Patrick

Patrick Steinhardt (5):
t7527: decrease likelihood of racing with fsmonitor daemon
Makefile: detect new Homebrew location for ARM-based Macs
ci: handle TEST_OUTPUT_DIRECTORY when printing test failures
ci: make p4 setup on macOS more robust
ci: add macOS jobs to GitLab CI

 .gitlab-ci.yml               | 34 +++++++++++++++++++++++++++++++++-
 ci/install-dependencies.sh   | 10 ++++------
 ci/lib.sh                    | 12 +++++++++++-
 ci/print-test-failures.sh    |  2 +-
 config.mak.uname             | 13 +++++++++++++
 t/t7527-builtin-fsmonitor.sh |  2 +-
 6 files changed, 63 insertions(+), 10 deletions(-)

Range-diff against v1:

-:  ---------- > 1:  554b1c8546 t7527: decrease likelihood of racing with fsmonitor daemon
2:  3adb0b7ae8 = 2:  32d8bd1d78 Makefile: detect new Homebrew location for ARM-based Macs
-:  ---------- > 3:  d55da77747 ci: handle TEST_OUTPUT_DIRECTORY when printing test failures
1:  a5d725bea7 ! 4:  1ed6e68650 ci: make p4 setup on macOS more robust

@@ Commit message
into a separate directory which we then manually append to our PATH. This matches what we do on Linux-based jobs.

+ Note that it may seem like we already did append "$HOME/bin" to PATH
+ because we're actually removing the lines that adapt PATH. But we only
+ ever adapted the PATH variable in "ci/install-dependencies.sh", and
+ didn't adapt it when running "ci/run-build-and-test.sh". Consequently,
+ the required binaries wouldn't be found during the test run unless the
+ CI platform already had the "$HOME/bin" in PATH right from the start.
+

## ci/install-dependencies.sh ##
3: d196cfd9d0 ! 5: c5ed38f0a6 ci: add macOS jobs to GitLab CI
@@ Metadata
## Commit message ##
ci: add macOS jobs to GitLab CI

- Add two macOS-based jobs to GitLab CI, one for Clang and one for GCC.
- This matches equivalent jobs we have for GitHub Workflows, except that
- we use macOS 14 instead of macOS 13.
+ Add a job to GitLab CI which runs tests on macOS, which matches the
+ equivalent "osx-clang" job that we have for GitHub Workflows. One
+ significant difference though is that this new job runs on Apple M1
+ machines and thus uses the "arm64" architecture. As GCC does not yet
+ support this comparatively new architecture we cannot easily include an
+ equivalent for the "osx-gcc" job that exists in GitHub Workflows.

Note that one test marked as test_must_fail is surprisingly passing:

t7815-grep-binary.sh (Wstat: 0 Tests: 22 Failed: 0)
TODO passed: 12

- This seems to boil down to an unexpected difference in how regcomp(1)
+ This seems to boil down to an unexpected difference in how regcomp(3P)
works when matching NUL bytes. Cross-checking with the respective GitHub
- job shows though that this is not an issue unique to the GitLab CI job
- as it passes in the same way there.
-
- Further note that we do not include the equivalent for the "osx-gcc" job
- that we use with GitHub Workflows. This is because the runner for macOS
- on GitLab is running on Apple M1 machines and thus uses the "arm64"
- architecture. GCC does not support this platform yet.
+ job shows that this is not an issue unique to the GitLab CI job as it
+ passes in the same way there.

@@ .gitlab-ci.yml: test:
+ image: $image
+ tags:
+ - saas-macos-medium-m1
++ variables:
++ TEST_OUTPUT_DIRECTORY: "/Volumes/RAMDisk"
+ before_script:
++ # Create a 4GB RAM disk that we use to store test output on. This small hack
++ # significantly speeds up tests by more than a factor of 2 because the
++ # macOS runners use network-attached storage as disks, which is _really_
++ # slow with the many small writes that our tests do.
++ - sudo diskutil apfs create $(hdiutil attach -nomount ram://8192000) RAMDisk
+ - ./ci/install-dependencies.sh
+ script:
+ - ./ci/run-build-and-tests.sh
@@ .gitlab-ci.yml: test:
+ if test "$CI_JOB_STATUS" != 'success'
+ then
+ ./ci/print-test-failures.sh
++ mv "$TEST_OUTPUT_DIRECTORY"/failed-test-artifacts t/
+ fi
+ parallel:
+ matrix:

base-commit: cd69c635a1a62b0c8bfdbf221778be8a512ad048

2.43.GIT