cirrus.yml: Exclude some targets in the FreeBSD job to speed up the build
PATCH 1/1 cirrus.yml: Exclude some targets in the FreeBSD job to speed up the build
The FreeBSD jobs currently hit the 1h time limit in the Cirrus-CI. We've got to exclude some build targets here to make sure that the job finishes in time again. The targets that are excluded should not hurt much, since e.g. all the code from i386-softmmu is covered again by x86_64-softmmu.
---
Not sure why the FreeBSD builds suddenly got slower and hit the 1h time limit now. Looking at https://cirrus-ci.com/github/qemu/qemu
it seems like the FreeBSD jobs were already taking ca. 48 minutes since quite a while, but since the Meson build system has been merged, they now always hit the 1h limit. Could it be that Meson is slow on FreeBSD?
I hope disabling some of the "redundant" targets is ok for everybody to get this working again. Alternatively, we could also introduce a second FreeBSD job and run half of the targets in one job, and half of the targets in the other job if that's preferred?
.cirrus.yml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
.cirrus.yml +3 -1
@@ -12,7 +12,9 @@ freebsd_12_task: script: - mkdir build - cd build - - ../configure --enable-werror || { cat config.log; exit 1; } + - ../configure --enable-werror --target-list-exclude='mips64el-softmmu + mips-softmmu arm-softmmu i386-softmmu ppc-softmmu riscv32-softmmu + sh4eb-softmmu xtensaeb-softmmu' || { cat config.log; exit 1; } - gmake -j8 - gmake V=1 check
Signature · 1 line
2.18.2
Quoted 6 lines · Thomas Huth
The FreeBSD jobs currently hit the 1h time limit in the Cirrus-CI. We've got to exclude some build targets here to make sure that the job finishes in time again. The targets that are excluded should not hurt much, since e.g. all the code from i386-softmmu is covered again by x86_64-softmmu.
Quoted 8 lines
Signed-off-by Thomas Huth <[email protected]>---
Not sure why the FreeBSD builds suddenly got slower and hit the 1h time limit now. Looking at https://cirrus-ci.com/github/qemu/qemu
it seems like the FreeBSD jobs were already taking ca. 48 minutes since quite a while, but since the Meson build system has been merged, they now always hit the 1h limit. Could it be that Meson is slow on FreeBSD?
No, it appears to just be compiling a lot more than in the past.
$ grep Compiling main.log-meson | wc -l 5468
$ grep CC main.log-config | wc -l 4855
This seems to be caused my compiling lots of stuff under tests/ that we did not previously build
$ grep Compiling main.log-meson | grep tests/ | wc -l 616
not sure if this is specific to FreeBSD, or whether we're doing this on all platforms and merely noticed first on FreeBSD due to the timeout.
Mostly it seems to the libtestfloat ad libsoftfloat, but also a bit of libqtest stuff.
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Quoted 16 lines · Daniel P. Berrangé
Quoted 6 lines · Thomas Huth
The FreeBSD jobs currently hit the 1h time limit in the Cirrus-CI. We've got to exclude some build targets here to make sure that the job finishes in time again. The targets that are excluded should not hurt much, since e.g. all the code from i386-softmmu is covered again by x86_64-softmmu.
Quoted 8 lines
Signed-off-by Thomas Huth <[email protected]>---
Not sure why the FreeBSD builds suddenly got slower and hit the 1h time limit now. Looking at https://cirrus-ci.com/github/qemu/qemu
it seems like the FreeBSD jobs were already taking ca. 48 minutes since quite a while, but since the Meson build system has been merged, they now always hit the 1h limit. Could it be that Meson is slow on FreeBSD?
Quoted 7 lines
No, it appears to just be compiling a lot more than in the past.
$ grep Compiling main.log-meson | wc -l 5468$ grep CC main.log-config | wc -l 4855
Oh, that's interesting, thanks!
Quoted 9 lines
This seems to be caused my compiling lots of stuff under tests/ that we did not previously build
$ grep Compiling main.log-meson | grep tests/ | wc -l 616not sure if this is specific to FreeBSD, or whether we're doing this on all platforms and merely noticed first on FreeBSD due to the timeout.
I also did some build time tests on my x86 Linux laptop, but I did not notice any significant differences there.
Quoted 2 lines
Mostly it seems to the libtestfloat ad libsoftfloat, but also a bit of libqtest stuff.
Now that you've mentioned softfloat ... I think that matches what I've seen in the log output on cirrus-ci.com : Suddenly there is a lot of verbose softloat test output there that I never noticed before.
I assume that's ok that we now compile/test more on FreeBSD, so I'll go ahead with this patch and put it into my next "CI" pull request to fix the cirrus-ci builds again.
Thomas
Quoted 15 lines · Thomas Huth
The FreeBSD jobs currently hit the 1h time limit in the Cirrus-CI. We've got to exclude some build targets here to make sure that the job finishes in time again. The targets that are excluded should not hurt much, since e.g. all the code from i386-softmmu is covered again by x86_64-softmmu.
Signed-off-by Thomas Huth <[email protected]>---
Not sure why the FreeBSD builds suddenly got slower and hit the 1h time limit now. Looking at https://cirrus-ci.com/github/qemu/qemu
it seems like the FreeBSD jobs were already taking ca. 48 minutes since quite a while, but since the Meson build system has been merged, they now always hit the 1h limit. Could it be that Meson is slow on FreeBSD?
Quoted 4 lines
I hope disabling some of the "redundant" targets is ok for everybody to get this working again. Alternatively, we could also introduce a second FreeBSD job and run half of the targets in one job, and half of the targets in the other job if that's preferred?
I'd probably suggest we go for two jobs, as there doesn't look like any constraint on running multiple jobs that we'll hit any time soon.
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Quoted 16 lines · Daniel P. Berrangé
Quoted 15 lines · Thomas Huth
The FreeBSD jobs currently hit the 1h time limit in the Cirrus-CI. We've got to exclude some build targets here to make sure that the job finishes in time again. The targets that are excluded should not hurt much, since e.g. all the code from i386-softmmu is covered again by x86_64-softmmu.
Signed-off-by Thomas Huth <[email protected]>---
Not sure why the FreeBSD builds suddenly got slower and hit the 1h time limit now. Looking at https://cirrus-ci.com/github/qemu/qemu
it seems like the FreeBSD jobs were already taking ca. 48 minutes since quite a while, but since the Meson build system has been merged, they now always hit the 1h limit. Could it be that Meson is slow on FreeBSD?
Quoted 8 lines
Quoted 4 lines
I hope disabling some of the "redundant" targets is ok for everybody to get this working again. Alternatively, we could also introduce a second FreeBSD job and run half of the targets in one job, and half of the targets in the other job if that's preferred?
I'd probably suggest we go for two jobs, as there doesn't look like any constraint on running multiple jobs that we'll hit any time soon.
I gave it a try, but either I'm doing something wrong, or it is not possible to run multiple FreeBSD tasks on cirrus-ci for one commit - either the second task does not show up, or both FreeBSD tasks do not show up here. I assume there are some limitations in place (see
https://cirrus-ci.org/faq/#are-there-any-limits), but it even does not work for me when I limit the number of cpus in the tasks to a lower value like 4 or 2.
Could somebody else please have a try? Maybe I'm just doing something stupid here...
Thomas
Quoted 22 lines · Thomas Huth
Quoted 21 lines · Daniel P. Berrangé
Quoted 19 lines · Thomas Huth
The FreeBSD jobs currently hit the 1h time limit in the Cirrus-CI. We've got to exclude some build targets here to make sure that the job finishes in time again. The targets that are excluded should not hurt much, since e.g. all the code from i386-softmmu is covered again by x86_64-softmmu.
Signed-off-by Thomas Huth <[email protected]>---
Not sure why the FreeBSD builds suddenly got slower and hit the 1h time limit now. Looking at https://cirrus-ci.com/github/qemu/qemu
it seems like the FreeBSD jobs were already taking ca. 48 minutes since quite a while, but since the Meson build system has been merged, they now always hit the 1h limit. Could it be that Meson is slow on FreeBSD?I hope disabling some of the "redundant" targets is ok for everybody to get this working again. Alternatively, we could also introduce a second FreeBSD job and run half of the targets in one job, and half of the targets in the other job if that's preferred?
Quoted 6 lines
Quoted 3 lines
I'd probably suggest we go for two jobs, as there doesn't look like any constraint on running multiple jobs that we'll hit any time soon.
I gave it a try, but either I'm doing something wrong, or it is not possible to run multiple FreeBSD tasks on cirrus-ci for one commit
I apparently needed a weekend in between to get a clean head again. I was doing indeed something wrong... apparently you have to name your tasks on cirrus-ci with "_task" at the end. If you use "_task1" and "_task2", it does now work anymore. Sigh.
So I'll try to cook a proper patch to split the job instead.
Thomas