Skip to contentSkip to conversation

gitlab-ci: Build MSYS2 job using multiple CPUs

PATCH In review Philippe Mathieu-Daudé 2 participants 2 messages Aug 19, 2024

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 gitlab-ci: Build MSYS2 job using multiple CPUs

---
I don't know how to use Powershell do use nproc+1 jobs to optimize jobs waiting on I/O.

---
 .gitlab-ci.d/windows.yml | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
.gitlab-ci.d/windows.yml +2 -1
index a83f23a786..35ccb74fee 100644
@@ -110,6 +110,7 @@ msys2-64bit:
       mingw-w64-x86_64-usbredir
       mingw-w64-x86_64-zstd"
   - Write-Output "Running build at $(Get-Date -Format u)"
+  - $env:JOBS = $(.\msys64\usr\bin\bash -lc nproc)
   - $env:CHERE_INVOKING = 'yes'  # Preserve the current working directory
   - $env:MSYS = 'winsymlinks:native' # Enable native Windows symlink
   - $env:CCACHE_BASEDIR = "$env:CI_PROJECT_DIR"
@@ -121,7 +122,7 @@ msys2-64bit:
   - cd build
   - ..\msys64\usr\bin\bash -lc "ccache --zero-stats"
   - ..\msys64\usr\bin\bash -lc "../configure --enable-fdt=system $CONFIGURE_ARGS"
-  - ..\msys64\usr\bin\bash -lc "make"
+  - ..\msys64\usr\bin\bash -lc "make -j$env:JOBS"
   - ..\msys64\usr\bin\bash -lc "make check MTESTARGS='$TEST_ARGS' || { cat meson-logs/testlog.txt; exit 1; } ;"
   - ..\msys64\usr\bin\bash -lc "ccache --show-stats"
   - Write-Output "Finished build at $(Get-Date -Format u)"
Signature · 1 line
2.45.2
#1 lore raw

According to https://docs.gitlab.com/ee/ci/runners/hosted_runners/windows.html the Windows shared runner should have 2 vCPUs nowadays, indeed! Maybe worth to mention it in the patch description?

Also, how much faster does the job now run for you?

Quoted 4 lines · Philippe Mathieu-Daudé

---
I don't know how to use Powershell do use nproc+1 jobs to optimize jobs waiting on I/O.

Well, we're calling into bash there, so you could do it as part of the bash statement?

Thomas

Quoted 25 lines
---
  .gitlab-ci.d/windows.yml | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)
.gitlab-ci.d/windows.yml +2 -1
index a83f23a786..35ccb74fee 100644
@@ -110,6 +110,7 @@ msys2-64bit:
        mingw-w64-x86_64-usbredir
        mingw-w64-x86_64-zstd"
    - Write-Output "Running build at $(Get-Date -Format u)"
+  - $env:JOBS = $(.\msys64\usr\bin\bash -lc nproc)
    - $env:CHERE_INVOKING = 'yes'  # Preserve the current working directory
    - $env:MSYS = 'winsymlinks:native' # Enable native Windows symlink
    - $env:CCACHE_BASEDIR = "$env:CI_PROJECT_DIR"
@@ -121,7 +122,7 @@ msys2-64bit:
    - cd build
    - ..\msys64\usr\bin\bash -lc "ccache --zero-stats"
    - ..\msys64\usr\bin\bash -lc "../configure --enable-fdt=system $CONFIGURE_ARGS"
-  - ..\msys64\usr\bin\bash -lc "make"
+  - ..\msys64\usr\bin\bash -lc "make -j$env:JOBS"
    - ..\msys64\usr\bin\bash -lc "make check MTESTARGS='$TEST_ARGS' || { cat meson-logs/testlog.txt; exit 1; } ;"
    - ..\msys64\usr\bin\bash -lc "ccache --show-stats"
    - Write-Output "Finished build at $(Get-Date -Format u)"