Skip to contentSkip to conversation

ci: update freebsd 12 cirrus job

PATCH In review Carlo Marcelo Arenas Belón 3 participants 4 messages Aug 12, 2021 – Aug 14, 2021

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 ci: update freebsd 12 cirrus job

make sure it uses a supported OS branch and uses all the resources that can be allocated efficiently.

while only 1GB of memory is needed, 2GB is the minimum for a 2 CPU machine (the default), but by increasing parallelism wall time has been reduced by 35%.

---
 .cirrus.yml | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
.cirrus.yml +8 -1
index c2f5fe385a..e114ffee1a 100644
@@ -2,8 +2,15 @@ env:
   CIRRUS_CLONE_DEPTH: 1
 
 freebsd_12_task:
+  env:
+    GIT_PROVE_OPTS: "--timer --jobs 10"
+    GIT_TEST_OPTS: "--no-chain-lint --no-bin-wrappers"
+    MAKEFLAGS: "-j4"
+    DEFAULT_TEST_TARGET: prove
+    DEVELOPER: 1
   freebsd_instance:
-    image: freebsd-12-1-release-amd64
+    image_family: freebsd-12-2
+    memory: 2G
   install_script:
     pkg install -y gettext gmake perl5
   create_user_script:
Signature · 1 line
2.33.0.rc1.379.g2890ef5eb6
#1 lore raw
Quoted 17 lines · Carlo Marcelo Arenas Belón

make sure it uses a supported OS branch and uses all the resources that can be allocated efficiently.

while only 1GB of memory is needed, 2GB is the minimum for a 2 CPU machine (the default), but by increasing parallelism wall time has been reduced by 35%.

---
 .cirrus.yml | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
.cirrus.yml +0 -0
index c2f5fe385a..e114ffee1a 100644

.cirrus.yml env: · line 6

Quoted 5 lines
.cirrus.yml +2 -0
@@ -2,8 +2,15 @@ env:
   CIRRUS_CLONE_DEPTH: 1
 
 freebsd_12_task:
+  env:
+    GIT_PROVE_OPTS: "--timer --jobs 10"

Why these prove options?

On other CI systems we pass 'prove' the option '--state=failed,slow,save' as well to reduce runtime. However, this only works when there is a persistent place for prove's state files, e.g. the cache feature of Travis CI. If Cirrus CI lacks a similar feature, then we can't benefit from this option, but it'd be worth mentioning in the commit message.

.cirrus.yml env: · line 7

Quoted 1 line
.cirrus.yml +1 -0
+    GIT_TEST_OPTS: "--no-chain-lint --no-bin-wrappers"

Why these test options?

chain-linting is done by a mighty sed script; I think it's worth running it with FreeBSD's 'sed' as well.

Quoting 't/README', '--no-bin-wrappers' "can speed up test runs especially on platforms where running shell scripts is expensive". Is running shell scripts really that expensive on FreeBSD?

OTOH, why are there no options that would show us some information about test failures, i.e. why no '--verbose-log -x --immediate' like on other CI systems?

Quoted 13 lines
.cirrus.yml +5 -1
+    MAKEFLAGS: "-j4"
+    DEFAULT_TEST_TARGET: prove
+    DEVELOPER: 1
   freebsd_instance:
-    image: freebsd-12-1-release-amd64
+    image_family: freebsd-12-2
+    memory: 2G
   install_script:
     pkg install -y gettext gmake perl5
   create_user_script:
Signature · 1 line
2.33.0.rc1.379.g2890ef5eb6
#2 lore raw
Quoted 9 lines · SZEDER Gábor
Quoted 7 lines · Carlo Marcelo Arenas Belón
.cirrus.yml +0 -0
index c2f5fe385a..e114ffee1a 100644
@@ -2,8 +2,15 @@ env:
   CIRRUS_CLONE_DEPTH: 1
 
Quoted 5 lines

.cirrus.yml env: · line 6

Quoted 3 lines
.cirrus.yml +2 -0
 freebsd_12_task:
+  env:
+    GIT_PROVE_OPTS: "--timer --jobs 10"

Why these prove options?

that line came from the Github Actions part of ci/lib.sh.

--jobs is the one I was going after, but kept --timer for consistency.

Quoted 4 lines

On other CI systems we pass 'prove' the option '--state=failed,slow,save' as well to reduce runtime. However, this only works when there is a persistent place for prove's state files, e.g. the cache feature of Travis CI.

note that when GitHub Actions was added[1], it was explicitly excluded because it wasn't really that effective

Quoted 3 lines

If Cirrus CI lacks a similar
feature, then we can't benefit from this option, but it'd be worth mentioning in the commit message.

They do have the option and will likely be a small change that I could send as a follow up, but wasn't included in this change since it was "controversial" as pointed above.

note also that the other uses of it aren't really being exercised, since AFAIK neither azure or travis are running now, and indeed last time I tried to run CI in my own account it didn't end up well, with no successful run and running out of credit before I could even attempt to fix them all.

indeed, was even going to suggest (after release) that they are removed, which will also make the code cleaner for integrating cirrus (which still has a usable opensource plan).

Quoted 2 lines

.cirrus.yml env: · line 7

Quoted 1 line
.cirrus.yml +1 -0
+    GIT_TEST_OPTS: "--no-chain-lint --no-bin-wrappers"

Why these test options?

again lifted from the "GitHub Actions" code just for consistency and to avoid breaking if either of those are broken (considering the only other running CI does test with those unconditionally), but I realize now that I might have been too conservative since my focus with this commit was "speed and accuracy".

Quoted 2 lines

chain-linting is done by a mighty sed script; I think it's worth running it with FreeBSD's 'sed' as well.

indeed, even if I recall correctly it was Solaris sed the one that broke, and I would expect anyone breaking this chainlinting will notice very quickly on their local run.

Quoted 3 lines

Quoting 't/README', '--no-bin-wrappers' "can speed up test runs especially on platforms where running shell scripts is expensive". Is running shell scripts really that expensive on FreeBSD?

no, I am assuming they were added with Windows in mind, but note that as configure they also affect all the linux builds through GitHub Actions.
Both these flags might be worth adding back for all non Windows systems, but doing so will be easier when using the "ci" scripts for all environments and after it has been discussed IMHO.

Quoted 3 lines

OTOH, why are there no options that would show us some information about test failures, i.e. why no '--verbose-log -x --immediate' like on other CI systems?

I wasn't the original author, so can't answer why it wasn't done; but I sure think the next step will be to add support for it to the ci/ scripts and with that make it more consistent, which likely will include that change.

This patch was just the minimal change to make sure that it runs efficiently and is representative of current users.

Carlo

[1] https://lore.kernel.org/git/40fe4f7e2c34c6997a6824ef0dc6aad7a71a4434.1586538752.git.congdanhqx@gmail.com/

#3 lore raw
In reply to SZEDER Gábor

Hi Gábor,

Quoted 20 lines · SZEDER Gábor
Quoted 19 lines · Carlo Marcelo Arenas Belón

make sure it uses a supported OS branch and uses all the resources that can be allocated efficiently.

while only 1GB of memory is needed, 2GB is the minimum for a 2 CPU machine (the default), but by increasing parallelism wall time has been reduced by 35%.

---
 .cirrus.yml | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
.cirrus.yml +0 -0
index c2f5fe385a..e114ffee1a 100644
@@ -2,8 +2,15 @@ env:
   CIRRUS_CLONE_DEPTH: 1
 
Quoted 5 lines

.cirrus.yml env: · line 6

Quoted 3 lines
.cirrus.yml +2 -0
 freebsd_12_task:
+  env:
+    GIT_PROVE_OPTS: "--timer --jobs 10"

Why these prove options?

The only problem I see with it is that its --jobs 10 disagrees with the MAKEFLAGS: -j4 below. Either both should say 10, or both should say 4.

Quoted 6 lines

On other CI systems we pass 'prove' the option '--state=failed,slow,save' as well to reduce runtime. However, this only works when there is a persistent place for prove's state files, e.g. the cache feature of Travis CI. If Cirrus CI lacks a similar feature, then we can't benefit from this option, but it'd be worth mentioning in the commit message.

We also don't benefit from this in the GitHub workflow because there is likewise no easy way to persist the state.

Quoted 3 lines

.cirrus.yml env: · line 7

Quoted 1 line
.cirrus.yml +1 -0
+    GIT_TEST_OPTS: "--no-chain-lint --no-bin-wrappers"

Why these test options?

I guess that's for the same reason we use these options in the Windows
tests: it speeds up things for a historically pretty slow build axis. At least I seem to see FreeBSD runs lagging behind all the time.

Quoted 6 lines

chain-linting is done by a mighty sed script; I think it's worth running it with FreeBSD's 'sed' as well.

Quoting 't/README', '--no-bin-wrappers' "can speed up test runs especially on platforms where running shell scripts is expensive". Is running shell scripts really that expensive on FreeBSD?

I don't think it is FreeBSD per se, but the available VMs that make this speed-up worth our while.

Quoted 3 lines

OTOH, why are there no options that would show us some information about test failures, i.e. why no '--verbose-log -x --immediate' like on other CI systems?

That's because we don't use the ci/ scripts at all, and the failures are not actually shown.

This is a mild annoyance to me, I have to admit: there is very little in the way of actionable information whenever the FreeBSD jobs fail.

Lucky for me, I've so far only encountered breakages that _also_ affected the macOS build axes, and therefore I could diagnose the issues in the GitHub workflow run logs.

But yes, it would be nice if the FreeBSD CI runs were more helpful in case of problems.

As far as this here patch goes: I am in favor of integrating it as-is.

Ciao,
Dscho

Quoted 15 lines
Quoted 14 lines
.cirrus.yml +5 -2
+    MAKEFLAGS: "-j4"
+    DEFAULT_TEST_TARGET: prove
+    DEVELOPER: 1
   freebsd_instance:
-    image: freebsd-12-1-release-amd64
+    image_family: freebsd-12-2
+    memory: 2G
   install_script:
     pkg install -y gettext gmake perl5
   create_user_script:
--

2.33.0.rc1.379.g2890ef5eb6