Skip to contentSkip to conversation

gitlab-ci: Test compilation on Windows with MSYS2

RFC PATCH In review Thomas Huth 2 participants 4 messages Jul 9, 2021

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 gitlab-ci: Test compilation on Windows with MSYS2

Gitlab also provides runners with Windows, we can use them to test compilation with MSYS2.

However, it takes quite a long time to set up the VM, so to stay in the 1h time frame, we can only compile and check one target here. And there is also still a problem with compiling the multiboot.bin in pc-bios/optionrom/, so this is currently disabled, too.

---
I'm sending this out as an RFC in case somebody else wants to play with this. Considering that the corresponding job in Cirrus-CI compiles way more targets within 1 hour, we might rather want to keep using the Cirrus-CI job instead?

 .gitlab-ci.d/qemu-project.yml |  1 +
 .gitlab-ci.d/windows.yml      | 59 +++++++++++++++++++++++++++++++++++
 2 files changed, 60 insertions(+)
 create mode 100644 .gitlab-ci.d/windows.yml
.gitlab-ci.d/qemu-project.yml +1 -0
index 64cb2ba1da..f1a34038af 100644
@@ -9,3 +9,4 @@ include:
   - local: '/.gitlab-ci.d/crossbuilds.yml'
   - local: '/.gitlab-ci.d/buildtest.yml'
   - local: '/.gitlab-ci.d/static_checks.yml'
+  - local: '/.gitlab-ci.d/windows.yml'
.gitlab-ci.d/windows.yml new file +59 -0

59 lines not on this page · raw message

Signature · 1 line
2.27.0
#1 lore raw
Quoted 8 lines · Thomas Huth

Gitlab also provides runners with Windows, we can use them to test compilation with MSYS2.

However, it takes quite a long time to set up the VM, so to stay in the 1h time frame, we can only compile and check one target here. And there is also still a problem with compiling the multiboot.bin in pc-bios/optionrom/, so this is currently disabled, too.

THe gitlab VM provisioning seems quite fast (2+1/2 minutes), so I guess you mean the time is sunk in intsalling msys packages.

In the crurent Cirrus job there's a bunch of code that is done to cache all the msys install, so that 90% of the time we're merely extracting a local zipfile.

I wonder if we can do similar caching here. Perhaps even have 2 seprate jobs running on the windows runner. The first job can just build a zip file and store it as an artifact and cache it in gitlab for speed. The second job can do the actual build ?

Quoted 6 lines

---
I'm sending this out as an RFC in case somebody else wants to play with this. Considering that the corresponding job in Cirrus-CI compiles way more targets within 1 hour, we might rather want to keep using the Cirrus-CI job instead?

Yes, unless we can do the same kind of caching tricks to speed it all up.

Quoted 5 lines
 .gitlab-ci.d/qemu-project.yml |  1 +
 .gitlab-ci.d/windows.yml      | 59 +++++++++++++++++++++++++++++++++++
 2 files changed, 60 insertions(+)
 create mode 100644 .gitlab-ci.d/windows.yml

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#2 lore raw
Quoted 14 lines · Daniel P. Berrangé
Quoted 8 lines · Thomas Huth

Gitlab also provides runners with Windows, we can use them to test compilation with MSYS2.

However, it takes quite a long time to set up the VM, so to stay in the 1h time frame, we can only compile and check one target here. And there is also still a problem with compiling the multiboot.bin in pc-bios/optionrom/, so this is currently disabled, too.

THe gitlab VM provisioning seems quite fast (2+1/2 minutes), so I guess you mean the time is sunk in intsalling msys packages.

Quoted 5 lines

In the crurent Cirrus job there's a bunch of code that is done to cache all the msys install, so that 90% of the time we're merely extracting a local zipfile.

I wonder if we can do similar caching here.

Maybe, yes. It might save 10 - 15 minutes or so, I guess.

Quoted 5 lines

Perhaps even
have 2 seprate jobs running on the windows runner. The first job can just build a zip file and store it as an artifact and cache it in gitlab for speed. The second job can do the actual build ?

I'm not sure ... the Windows builders are likely single-threaded on Gitlab, too, just like the Linux containers. So that the main problem - while we can compile with -j8 on Cirrus, it's 8 times slower on Gitlab.

Thomas

#3 lore raw
Quoted 29 lines · Thomas Huth
Quoted 14 lines · Daniel P. Berrangé
Quoted 8 lines · Thomas Huth

Gitlab also provides runners with Windows, we can use them to test compilation with MSYS2.

However, it takes quite a long time to set up the VM, so to stay in the 1h time frame, we can only compile and check one target here. And there is also still a problem with compiling the multiboot.bin in pc-bios/optionrom/, so this is currently disabled, too.

THe gitlab VM provisioning seems quite fast (2+1/2 minutes), so I guess you mean the time is sunk in intsalling msys packages.

Quoted 5 lines

In the crurent Cirrus job there's a bunch of code that is done to cache all the msys install, so that 90% of the time we're merely extracting a local zipfile.

I wonder if we can do similar caching here.

Maybe, yes. It might save 10 - 15 minutes or so, I guess.

Quoted 5 lines

Perhaps even
have 2 seprate jobs running on the windows runner. The first job can just build a zip file and store it as an artifact and cache it in gitlab for speed. The second job can do the actual build ?

Quoted 3 lines

I'm not sure ... the Windows builders are likely single-threaded on Gitlab, too, just like the Linux containers. So that the main problem - while we can compile with -j8 on Cirrus, it's 8 times slower on Gitlab.

Ah yes, good point.

Since we have cirrus integration for freebsd/macos now, we ought todo the same for Windows too. I auto-generated the freebsd/mac logic with libvirt-ci, but I ignored msys since it was somewhat special in its build commands. We can still set it up manually though to integrate

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|