gitlab: speed up msys windows jobs with GCC
cover gitlab: speed up msys windows jobs with GCC
This is an alternative and/or complementary to Thomas' proposal to use CLang with msys:
https://lists.gnu.org/archive/html/qemu-devel/2023-07/msg05402.html
First of all, the current msys installer we're using is over 12 months out of date. Thus after running the install, pacman then replaces most of what we've just installed with new downloaded content. Using the most update installer cuts 3+1/2 minutes off the msys install time - 7 minutes becomes 3+1/2.
Secondly, QEMU defaults to compiling with -O2 and this is more computationally expensive for GCC. Switching to -O0 drops the build time from 60 minutes down to 45 minutes.
Thirdly, including debug symbols also has an overhead, and turning that off reduces time still further down to 38 minutes.
IOW, between all three changes, we can cut approx 25-26 minutes off the job execution time, bringing it nicely within the job timeout.
The actually phase of installing the mingw deps still accounts for about 10 minutes and has not been optimized.
Possibly the same trick of -O0 and skipping -g would also help the clang alternative Thomas' proposed. If so, that could be enough to let us enable more features / targets during the msys build.
Daniel P. Berrangé (8):
gitlab: remove duplication between msys jobs
gitlab: print timestamps during windows msys jobs
gitlab: always use updated msys installer
gitlab: drop $CI_PROJECT_DIR from cache path
gitlab: always populate cache for windows msys jobs
configure: support passthrough of -Dxxx args to meson
gitlab: disable optimization and debug symbols in msys build
gitlab: disable FF_SCRIPT_SECTIONS on msys jobs
.gitlab-ci.d/windows.yml | 174 +++++++++++++++++++-------------------- configure | 5 ++ 2 files changed, 92 insertions(+), 87 deletions(-)
Signature · 1 line
2.41.0
PATCH 1/8 gitlab: remove duplication between msys jobs
Although they share a common parent, the two msys jobs still have massive duplication in their script definitions that can easily be collapsed.
--- .gitlab-ci.d/windows.yml | 132 +++++++++++++++------------------------ 1 file changed, 49 insertions(+), 83 deletions(-)
.gitlab-ci.d/windows.yml +49 -83
@@ -35,97 +35,63 @@ - .\msys64\usr\bin\bash -lc 'pacman --noconfirm -Syuu' # Core update - .\msys64\usr\bin\bash -lc 'pacman --noconfirm -Syuu' # Normal update - taskkill /F /FI "MODULES eq msys-2.0.dll" - -msys2-64bit: - extends: .shared_msys2_builder script: - .\msys64\usr\bin\bash -lc "pacman -Sy --noconfirm --needed bison diffutils flex git grep make sed - mingw-w64-x86_64-capstone - mingw-w64-x86_64-curl - mingw-w64-x86_64-cyrus-sasl - mingw-w64-x86_64-dtc - mingw-w64-x86_64-gcc - mingw-w64-x86_64-glib2 - mingw-w64-x86_64-gnutls - mingw-w64-x86_64-gtk3 - mingw-w64-x86_64-libgcrypt - mingw-w64-x86_64-libjpeg-turbo - mingw-w64-x86_64-libnfs - mingw-w64-x86_64-libpng - mingw-w64-x86_64-libssh - mingw-w64-x86_64-libtasn1 - mingw-w64-x86_64-libusb - mingw-w64-x86_64-lzo2 - mingw-w64-x86_64-nettle - mingw-w64-x86_64-ninja - mingw-w64-x86_64-pixman - mingw-w64-x86_64-pkgconf - mingw-w64-x86_64-python - mingw-w64-x86_64-SDL2 - mingw-w64-x86_64-SDL2_image - mingw-w64-x86_64-snappy - mingw-w64-x86_64-spice - mingw-w64-x86_64-usbredir - mingw-w64-x86_64-zstd " + $MINGW_TARGET-capstone + $MINGW_TARGET-curl + $MINGW_TARGET-cyrus-sasl + $MINGW_TARGET-dtc + $MINGW_TARGET-gcc + $MINGW_TARGET-glib2 + $MINGW_TARGET-gnutls + $MINGW_TARGET-gtk3 + $MINGW_TARGET-libgcrypt + $MINGW_TARGET-libjpeg-turbo + $MINGW_TARGET-libnfs + $MINGW_TARGET-libpng + $MINGW_TARGET-libssh + $MINGW_TARGET-libtasn1 + $MINGW_TARGET-libusb + $MINGW_TARGET-lzo2 + $MINGW_TARGET-nettle + $MINGW_TARGET-ninja + $MINGW_TARGET-pixman + $MINGW_TARGET-pkgconf + $MINGW_TARGET-python + $MINGW_TARGET-SDL2 + $MINGW_TARGET-SDL2_image + $MINGW_TARGET-snappy + $MINGW_TARGET-spice + $MINGW_TARGET-usbredir + $MINGW_TARGET-zstd " - $env:CHERE_INVOKING = 'yes' # Preserve the current working directory - - $env:MSYSTEM = 'MINGW64' # Start a 64-bit MinGW environment - $env:MSYS = 'winsymlinks:native' # Enable native Windows symlink - mkdir build - cd build - # Note: do not remove "--without-default-devices"! - # commit 9f8e6cad65a6 ("gitlab-ci: Speed up the msys2-64bit job by using --without-default-devices" - # changed to compile QEMU with the --without-default-devices switch - # for the msys2 64-bit job, due to the build could not complete within - # the project timeout. - - ..\msys64\usr\bin\bash -lc '../configure --target-list=x86_64-softmmu - --without-default-devices --enable-fdt=system' - - ..\msys64\usr\bin\bash -lc 'make' - # qTests don't run successfully with "--without-default-devices", - # so let's exclude the qtests from CI for now. - - ..\msys64\usr\bin\bash -lc 'make check MTESTARGS=\"--no-suite qtest\" || { cat meson-logs/testlog.txt; exit 1; } ;' + - ..\msys64\usr\bin\bash -lc "../configure --enable-fdt=system $CONFIGURE_ARGS" + - ..\msys64\usr\bin\bash -lc "make" + - ..\msys64\usr\bin\bash -lc "make check MTESTARGS='$TEST_ARGS' || { cat meson-logs/testlog.txt; exit 1; } ;" + +msys2-64bit: + extends: .shared_msys2_builder + variables: + MINGW_TARGET: mingw-w64-x86_64 + MSYSTEM: MINGW64 + # do not remove "--without-default-devices"! + # commit 9f8e6cad65a6 ("gitlab-ci: Speed up the msys2-64bit job by using --without-default-devices" + # changed to compile QEMU with the --without-default-devices switch + # for the msys2 64-bit job, due to the build could not complete within + CONFIGURE_ARGS: --target-list=x86_64-softmmu --without-default-devices + # qTests don't run successfully with "--without-default-devices", + # so let's exclude the qtests from CI for now. + TEST_ARGS: --no-suite qtest msys2-32bit: extends: .shared_msys2_builder - script: - - .\msys64\usr\bin\bash -lc "pacman -Sy --noconfirm --needed - bison diffutils flex - git grep make sed - mingw-w64-i686-capstone - mingw-w64-i686-curl - mingw-w64-i686-cyrus-sasl - mingw-w64-i686-dtc - mingw-w64-i686-gcc - mingw-w64-i686-glib2 - mingw-w64-i686-gnutls - mingw-w64-i686-gtk3 - mingw-w64-i686-libgcrypt - mingw-w64-i686-libjpeg-turbo - mingw-w64-i686-libnfs - mingw-w64-i686-libpng - mingw-w64-i686-libssh - mingw-w64-i686-libtasn1 - mingw-w64-i686-libusb - mingw-w64-i686-lzo2 - mingw-w64-i686-nettle - mingw-w64-i686-ninja - mingw-w64-i686-pixman - mingw-w64-i686-pkgconf - mingw-w64-i686-python - mingw-w64-i686-SDL2 - mingw-w64-i686-SDL2_image - mingw-w64-i686-snappy - mingw-w64-i686-spice - mingw-w64-i686-usbredir - mingw-w64-i686-zstd " - - $env:CHERE_INVOKING = 'yes' # Preserve the current working directory - - $env:MSYSTEM = 'MINGW32' # Start a 32-bit MinGW environment - - $env:MSYS = 'winsymlinks:native' # Enable native Windows symlink - - mkdir build - - cd build - - ..\msys64\usr\bin\bash -lc '../configure --target-list=ppc64-softmmu - --enable-fdt=system' - - ..\msys64\usr\bin\bash -lc 'make' - - ..\msys64\usr\bin\bash -lc 'make check MTESTARGS=\"--no-suite qtest\" || - { cat meson-logs/testlog.txt; exit 1; }' + variables: + MINGW_TARGET: mingw-w64-i686 + MSYSTEM: MINGW32 + CONFIGURE_ARGS: --target-list=ppc64-softmmu + TEST_ARGS: --no-suite qtest
Signature · 1 line
2.41.0
Re: [PATCH 1/8] gitlab: remove duplication between msys jobs
Quoted 8 lines · Daniel P. Berrangé
Although they share a common parent, the two msys jobs still have massive duplication in their script definitions that can easily be collapsed.
Signed-off-by Daniel P. Berrangé <[email protected]>--- .gitlab-ci.d/windows.yml | 132 +++++++++++++++------------------------ 1 file changed, 49 insertions(+), 83 deletions(-)
As I discovered in the recent days, this is also quite helpful in case we ever want to switch to Clang, since we need to change the prefix of the packages there. And it's still easy to have distinct packages with some few lines of codes changes, as I tried it out in my Clang patch. So this patch now sounds fine to me:
PATCH 2/8 gitlab: print timestamps during windows msys jobs
In reply to Daniel P. BerrangéIt is hard to get visibility into where time is consumed in our Windows msys jobs. Adding a few log console messages with the timestamp will aid in our debugging.
--- .gitlab-ci.d/windows.yml | 5 +++++ 1 file changed, 5 insertions(+)
.gitlab-ci.d/windows.yml +5 -0
@@ -19,6 +19,7 @@ reports: junit: "build/meson-logs/testlog.junit.xml" before_script: + - Write-Output "Acquiring msys2.exe installer at $(Get-Date -Format u)" - If ( !(Test-Path -Path msys64\var\cache ) ) { mkdir msys64\var\cache }
@@ -27,6 +28,7 @@ "https://github.com/msys2/msys2-installer/releases/download/2022-06-03/msys2-base-x86_64-20220603.sfx.exe" -outfile "msys64\var\cache\msys2.exe" } + - Write-Output "Invoking msys2.exe installer at $(Get-Date -Format u)" - msys64\var\cache\msys2.exe -y - ((Get-Content -path .\msys64\etc\\post-install\\07-pacman-key.post -Raw) -replace '--refresh-keys', '--version') |
@@ -36,6 +38,7 @@ - .\msys64\usr\bin\bash -lc 'pacman --noconfirm -Syuu' # Normal update - taskkill /F /FI "MODULES eq msys-2.0.dll" script: + - Write-Output "Installing mingw packages at $(Get-Date -Format u)" - .\msys64\usr\bin\bash -lc "pacman -Sy --noconfirm --needed bison diffutils flex git grep make sed
@@ -66,6 +69,7 @@ $MINGW_TARGET-spice $MINGW_TARGET-usbredir $MINGW_TARGET-zstd " + - Write-Output "Running build at $(Get-Date -Format u)" - $env:CHERE_INVOKING = 'yes' # Preserve the current working directory - $env:MSYS = 'winsymlinks:native' # Enable native Windows symlink - mkdir build
@@ -73,6 +77,7 @@ - ..\msys64\usr\bin\bash -lc "../configure --enable-fdt=system $CONFIGURE_ARGS" - ..\msys64\usr\bin\bash -lc "make" - ..\msys64\usr\bin\bash -lc "make check MTESTARGS='$TEST_ARGS' || { cat meson-logs/testlog.txt; exit 1; } ;" + - Write-Output "Finished build at $(Get-Date -Format u)" msys2-64bit: extends: .shared_msys2_builder
Signature · 1 line
2.41.0
Re: [PATCH 2/8] gitlab: print timestamps during windows msys jobs
Quoted 46 lines · Daniel P. Berrangé
It is hard to get visibility into where time is consumed in our Windows msys jobs. Adding a few log console messages with the timestamp will aid in our debugging.
Signed-off-by Daniel P. Berrangé <[email protected]>--- .gitlab-ci.d/windows.yml | 5 +++++ 1 file changed, 5 insertions(+).gitlab-ci.d/windows.yml +4
-0@@ -19,6 +19,7 @@ reports: junit: "build/meson-logs/testlog.junit.xml" before_script: + - Write-Output "Acquiring msys2.exe installer at $(Get-Date -Format u)" - If ( !(Test-Path -Path msys64\var\cache ) ) { mkdir msys64\var\cache }@@ -27,6 +28,7 @@ "https://github.com/msys2/msys2-installer/releases/download/2022-06-03/msys2-base-x86_64-20220603.sfx.exe" -outfile "msys64\var\cache\msys2.exe" } + - Write-Output "Invoking msys2.exe installer at $(Get-Date -Format u)" - msys64\var\cache\msys2.exe -y - ((Get-Content -path .\msys64\etc\\post-install\\07-pacman-key.post -Raw) -replace '--refresh-keys', '--version') |@@ -36,6 +38,7 @@ - .\msys64\usr\bin\bash -lc 'pacman --noconfirm -Syuu' # Normal update - taskkill /F /FI "MODULES eq msys-2.0.dll" script: + - Write-Output "Installing mingw packages at $(Get-Date -Format u)" - .\msys64\usr\bin\bash -lc "pacman -Sy --noconfirm --needed bison diffutils flex git grep make sed@@ -66,6 +69,7 @@ $MINGW_TARGET-spice $MINGW_TARGET-usbredir $MINGW_TARGET-zstd " + - Write-Output "Running build at $(Get-Date -Format u)" - $env:CHERE_INVOKING = 'yes' # Preserve the current working directory - $env:MSYS = 'winsymlinks:native' # Enable native Windows symlink - mkdir build
.gitlab-ci.d/windows.yml line 83
Quoted 7 lines
.gitlab-ci.d/windows.yml +1
-0@@ -73,6 +77,7 @@ - ..\msys64\usr\bin\bash -lc "../configure --enable-fdt=system $CONFIGURE_ARGS" - ..\msys64\usr\bin\bash -lc "make" - ..\msys64\usr\bin\bash -lc "make check MTESTARGS='$TEST_ARGS' || { cat meson-logs/testlog.txt; exit 1; } ;" + - Write-Output "Finished build at $(Get-Date -Format u)" msys2-64bit: extends: .shared_msys2_builder
PATCH 3/8 gitlab: always use updated msys installer
In reply to Daniel P. BerrangéWe current reference an msys installer binary from mid-2022, which means after installation, it immediately has to re-download a bunch of newer content. This wastes precious CI time.
The msys project publishes an installer binary with a fixed URL that always references the latest content. We cache the downloads in gitlab though and so once downloaded we would never re-fetch the installer leading back to the same problem.
To deal with this we also fetch the pgp signature for the installer on every run, and compare that to the previously cached signature. If the signature changes, we re-download the full installer.
This ensures we always have the latest installer for msys, while also maximising use of the gitlab cache.
--- .gitlab-ci.d/windows.yml | 30 +++++++++++++++++++++++++++--- 1 file changed, 27 insertions(+), 3 deletions(-)
.gitlab-ci.d/windows.yml +27 -3
@@ -23,10 +23,34 @@ - If ( !(Test-Path -Path msys64\var\cache ) ) { mkdir msys64\var\cache } - - If ( !(Test-Path -Path msys64\var\cache\msys2.exe ) ) { + - Invoke-WebRequest + "https://repo.msys2.org/distrib/msys2-x86_64-latest.sfx.exe.sig" + -outfile "msys2.exe.sig" + - if ( Test-Path -Path msys64\var\cache\msys2.exe.sig ) { + Write-Output "Cached installer sig" ; + if ( ((Get-FileHash msys2.exe.sig).Hash -ne (Get-FileHash msys64\var\cache\msys2.exe.sig).Hash) ) { + Write-Output "Mis-matched installer sig, new installer download required" ; + Remove-Item -Path msys64\var\cache\msys2.exe.sig ; + if ( Test-Path -Path msys64\var\cache\msys2.exe ) { + Remove-Item -Path msys64\var\cache\msys2.exe + } + } else { + Write-Output "Matched installer sig, cached installer still valid" + } + } else { + Write-Output "No cached installer sig, new installer download required" ; + if ( Test-Path -Path msys64\var\cache\msys2.exe ) { + Remove-Item -Path msys64\var\cache\msys2.exe + } + } + - if ( !(Test-Path -Path msys64\var\cache\msys2.exe ) ) { + Write-Output "Fetching latest installer" ; Invoke-WebRequest - "https://github.com/msys2/msys2-installer/releases/download/2022-06-03/msys2-base-x86_64-20220603.sfx.exe" - -outfile "msys64\var\cache\msys2.exe" + "https://repo.msys2.org/distrib/msys2-x86_64-latest.sfx.exe" + -outfile "msys64\var\cache\msys2.exe" ; + Copy-Item -Path msys2.exe.sig -Destination msys64\var\cache\msys2.exe.sig + } else { + Write-Output "Using cached installer" } - Write-Output "Invoking msys2.exe installer at $(Get-Date -Format u)" - msys64\var\cache\msys2.exe -y
Signature · 1 line
2.41.0
Re: [PATCH 3/8] gitlab: always use updated msys installer
Quoted 53 lines · Daniel P. Berrangé
We current reference an msys installer binary from mid-2022, which means after installation, it immediately has to re-download a bunch of newer content. This wastes precious CI time.
The msys project publishes an installer binary with a fixed URL that always references the latest content. We cache the downloads in gitlab though and so once downloaded we would never re-fetch the installer leading back to the same problem.
To deal with this we also fetch the pgp signature for the installer on every run, and compare that to the previously cached signature. If the signature changes, we re-download the full installer.
This ensures we always have the latest installer for msys, while also maximising use of the gitlab cache.
Signed-off-by Daniel P. Berrangé <[email protected]>--- .gitlab-ci.d/windows.yml | 30 +++++++++++++++++++++++++++--- 1 file changed, 27 insertions(+), 3 deletions(-).gitlab-ci.d/windows.yml +22
-1@@ -23,10 +23,34 @@ - If ( !(Test-Path -Path msys64\var\cache ) ) { mkdir msys64\var\cache } - - If ( !(Test-Path -Path msys64\var\cache\msys2.exe ) ) { + - Invoke-WebRequest + "https://repo.msys2.org/distrib/msys2-x86_64-latest.sfx.exe.sig" + -outfile "msys2.exe.sig" + - if ( Test-Path -Path msys64\var\cache\msys2.exe.sig ) { + Write-Output "Cached installer sig" ; + if ( ((Get-FileHash msys2.exe.sig).Hash -ne (Get-FileHash msys64\var\cache\msys2.exe.sig).Hash) ) { + Write-Output "Mis-matched installer sig, new installer download required" ; + Remove-Item -Path msys64\var\cache\msys2.exe.sig ; + if ( Test-Path -Path msys64\var\cache\msys2.exe ) { + Remove-Item -Path msys64\var\cache\msys2.exe + } + } else { + Write-Output "Matched installer sig, cached installer still valid" + } + } else { + Write-Output "No cached installer sig, new installer download required" ; + if ( Test-Path -Path msys64\var\cache\msys2.exe ) { + Remove-Item -Path msys64\var\cache\msys2.exe + } + } + - if ( !(Test-Path -Path msys64\var\cache\msys2.exe ) ) { + Write-Output "Fetching latest installer" ; Invoke-WebRequest
.gitlab-ci.d/windows.yml line 56
Quoted 10 lines
.gitlab-ci.d/windows.yml +5
-2- "https://github.com/msys2/msys2-installer/releases/download/2022-06-03/msys2-base-x86_64-20220603.sfx.exe" - -outfile "msys64\var\cache\msys2.exe" + "https://repo.msys2.org/distrib/msys2-x86_64-latest.sfx.exe" + -outfile "msys64\var\cache\msys2.exe" ; + Copy-Item -Path msys2.exe.sig -Destination msys64\var\cache\msys2.exe.sig + } else { + Write-Output "Using cached installer" } - Write-Output "Invoking msys2.exe installer at $(Get-Date -Format u)" - msys64\var\cache\msys2.exe -y
... I think the original idea was to use a "tagged" version to avoid that we have some kind of "rolling release" here, but since the latest content is fetched anyway during the following update, that idea was likely not working as expected...
Re: [PATCH 3/8] gitlab: always use updated msys installer
Quoted 20 lines · Thomas Huth
Quoted 9 lines · Daniel P. Berrangé
We current reference an msys installer binary from mid-2022, which means after installation, it immediately has to re-download a bunch of newer content. This wastes precious CI time.
The msys project publishes an installer binary with a fixed URL that always references the latest content. We cache the downloads in gitlab though and so once downloaded we would never re-fetch the installer leading back to the same problem.
Quoted 8 lines
To deal with this we also fetch the pgp signature for the installer on every run, and compare that to the previously cached signature. If the signature changes, we re-download the full installer.
This ensures we always have the latest installer for msys, while also maximising use of the gitlab cache.
Signed-off-by Daniel P. Berrangé <[email protected]>
Quoted 6 lines
Reviewed-by Thomas Huth <[email protected]>... I think the original idea was to use a "tagged" version to avoid that we have some kind of "rolling release" here, but since the latest content is fetched anyway during the following update, that idea was likely not working as expected...
I think using the "latest" installer is also more in keeping with what we do for other distros, where we'll always pick up the latest content when the containers get rebuilt, or macOS/FreeBSD where we pick the latest from Ports/HomeBrew.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
PATCH 4/8 gitlab: drop $CI_PROJECT_DIR from cache path
In reply to Daniel P. BerrangéThe gitlab cache is limited to only handle content within the $CI_PROJECT_DIR hierarchy, and as such relative paths are always implicitly relative to $CI_PROJECT_DIR.
--- .gitlab-ci.d/windows.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
.gitlab-ci.d/windows.yml +1 -1
@@ -7,7 +7,7 @@ cache: key: "${CI_JOB_NAME}-cache" paths: - - ${CI_PROJECT_DIR}/msys64/var/cache + - msys64/var/cache needs: [] stage: build timeout: 80m
Signature · 1 line
2.41.0
Re: [PATCH 4/8] gitlab: drop $CI_PROJECT_DIR from cache path
Quoted 14 lines · Daniel P. Berrangé
The gitlab cache is limited to only handle content within the $CI_PROJECT_DIR hierarchy, and as such relative paths are always implicitly relative to $CI_PROJECT_DIR.
Signed-off-by Daniel P. Berrangé <[email protected]>--- .gitlab-ci.d/windows.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-).gitlab-ci.d/windows.yml +0
-0
.gitlab-ci.d/windows.yml line 13
Quoted 8 lines
.gitlab-ci.d/windows.yml +1
-1@@ -7,7 +7,7 @@ cache: key: "${CI_JOB_NAME}-cache" paths: - - ${CI_PROJECT_DIR}/msys64/var/cache + - msys64/var/cache needs: [] stage: build timeout: 80m
PATCH 5/8 gitlab: always populate cache for windows msys jobs
In reply to Daniel P. BerrangéThe cache is used to hold the msys installer. Even if the build phase fails, we should still populate the cache as the installer will be valid for next time.
--- .gitlab-ci.d/windows.yml | 1 + 1 file changed, 1 insertion(+)
.gitlab-ci.d/windows.yml +1 -0
@@ -8,6 +8,7 @@ key: "${CI_JOB_NAME}-cache" paths: - msys64/var/cache + when: always needs: [] stage: build timeout: 80m
Signature · 1 line
2.41.0
Re: [PATCH 5/8] gitlab: always populate cache for windows msys jobs
Quoted 14 lines · Daniel P. Berrangé
The cache is used to hold the msys installer. Even if the build phase fails, we should still populate the cache as the installer will be valid for next time.
Signed-off-by Daniel P. Berrangé <[email protected]>--- .gitlab-ci.d/windows.yml | 1 + 1 file changed, 1 insertion(+).gitlab-ci.d/windows.yml +0
-0
.gitlab-ci.d/windows.yml line 14
Quoted 7 lines
.gitlab-ci.d/windows.yml +1
-0@@ -8,6 +8,7 @@ key: "${CI_JOB_NAME}-cache" paths: - msys64/var/cache + when: always needs: [] stage: build timeout: 80m
PATCH 6/8 configure: support passthrough of -Dxxx args to meson
In reply to Daniel P. BerrangéThis can be useful for setting some meson global options, such as the optimization level or debug state.xs
--- configure | 5 +++++ 1 file changed, 5 insertions(+)
configure +5 -0
@@ -757,6 +757,9 @@ for opt do # everything else has the same name in configure and meson --*) meson_option_parse "$opt" "$optarg" ;; + # Pass through -Dxxxx options to meson + -D*) meson_options="$meson_options $opt" + ;; esac done
@@ -887,6 +890,8 @@ cat << EOF pie Position Independent Executables debug-tcg TCG debugging (default is disabled) + -Dmesonoptname=val passthrough option to meson unmodified + NOTE: The object files are built at the place where configure is launched EOF exit 0
Signature · 1 line
2.41.0
Re: [PATCH 6/8] configure: support passthrough of -Dxxx args to meson
Quoted 23 lines · Daniel P. Berrangé
This can be useful for setting some meson global options, such as the optimization level or debug state.xs
Signed-off-by Daniel P. Berrangé <[email protected]>--- configure | 5 +++++ 1 file changed, 5 insertions(+)configure +3
-0@@ -757,6 +757,9 @@ for opt do # everything else has the same name in configure and meson --*) meson_option_parse "$opt" "$optarg" ;; + # Pass through -Dxxxx options to meson + -D*) meson_options="$meson_options $opt" + ;; esac done
configure cat << EOF · line 893
Quoted 4 lines
configure +1
-0@@ -887,6 +890,8 @@ cat << EOF pie Position Independent Executables debug-tcg TCG debugging (default is disabled) + -Dmesonoptname=val passthrough option to meson unmodified
I'd rather place that earlier in the help text, above the "meson_options_help" line in the configure script, next to the other --option=something lines.
Thomas
Quoted 3 lines
configure +0
-0NOTE: The object files are built at the place where configure is launched EOF exit 0
Re: [PATCH 6/8] configure: support passthrough of -Dxxx args to meson
Quoted 22 lines · Thomas Huth
Quoted 21 lines · Daniel P. Berrangé
This can be useful for setting some meson global options, such as the optimization level or debug state.xs
Signed-off-by Daniel P. Berrangé <[email protected]>--- configure | 5 +++++ 1 file changed, 5 insertions(+)configure +3
-0@@ -757,6 +757,9 @@ for opt do # everything else has the same name in configure and meson --*) meson_option_parse "$opt" "$optarg" ;; + # Pass through -Dxxxx options to meson + -D*) meson_options="$meson_options $opt" + ;; esac done
Quoted 8 lines
configure cat << EOF · line 892
Quoted 4 lines
configure +1
-0@@ -887,6 +890,8 @@ cat << EOF pie Position Independent Executables debug-tcg TCG debugging (default is disabled) + -Dmesonoptname=val passthrough option to meson unmodifiedI'd rather place that earlier in the help text, above the "
meson_options_help" line in the configure script, next to the other --option=something lines.
Sure I've no preference. I was expecting Paolo to tell me to put it somewhere else anyway :-)
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
PATCH 7/8 gitlab: disable optimization and debug symbols in msys build
In reply to Daniel P. BerrangéBuilding at -O2, adds 33% to the build time, over -O2. IOW a build that takes 45 minutes at -O0, takes 60 minutes at -O2. Turning off debug symbols drops it further, down to 38 minutes.
IOW, a "-O2 -g" build is 58% slower than a "-O0" build on msys in the gitlab CI windows shared runners.
--- .gitlab-ci.d/windows.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
.gitlab-ci.d/windows.yml +2 -2
@@ -113,7 +113,7 @@ msys2-64bit: # commit 9f8e6cad65a6 ("gitlab-ci: Speed up the msys2-64bit job by using --without-default-devices" # changed to compile QEMU with the --without-default-devices switch # for the msys2 64-bit job, due to the build could not complete within - CONFIGURE_ARGS: --target-list=x86_64-softmmu --without-default-devices + CONFIGURE_ARGS: --target-list=x86_64-softmmu --without-default-devices -Ddebug=false -Doptimization=0 # qTests don't run successfully with "--without-default-devices", # so let's exclude the qtests from CI for now. TEST_ARGS: --no-suite qtest
@@ -123,5 +123,5 @@ msys2-32bit: variables: MINGW_TARGET: mingw-w64-i686 MSYSTEM: MINGW32 - CONFIGURE_ARGS: --target-list=ppc64-softmmu + CONFIGURE_ARGS: --target-list=ppc64-softmmu -Ddebug=false -Doptimization=0 TEST_ARGS: --no-suite qtest
Signature · 1 line
2.41.0
Re: [PATCH 7/8] gitlab: disable optimization and debug symbols in msys build
Quoted 26 lines · Daniel P. Berrangé
Building at -O2, adds 33% to the build time, over -O2. IOW a build that takes 45 minutes at -O0, takes 60 minutes at -O2. Turning off debug symbols drops it further, down to 38 minutes.
IOW, a "-O2 -g" build is 58% slower than a "-O0" build on msys in the gitlab CI windows shared runners.
Signed-off-by Daniel P. Berrangé <[email protected]>--- .gitlab-ci.d/windows.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-).gitlab-ci.d/windows.yml +1
-1@@ -113,7 +113,7 @@ msys2-64bit: # commit 9f8e6cad65a6 ("gitlab-ci: Speed up the msys2-64bit job by using --without-default-devices" # changed to compile QEMU with the --without-default-devices switch # for the msys2 64-bit job, due to the build could not complete within - CONFIGURE_ARGS: --target-list=x86_64-softmmu --without-default-devices + CONFIGURE_ARGS: --target-list=x86_64-softmmu --without-default-devices -Ddebug=false -Doptimization=0 # qTests don't run successfully with "--without-default-devices", # so let's exclude the qtests from CI for now. TEST_ARGS: --no-suite qtest
.gitlab-ci.d/windows.yml msys2-32bit: · line 127
Quoted 6 lines
.gitlab-ci.d/windows.yml +1
-1@@ -123,5 +123,5 @@ msys2-32bit: variables: MINGW_TARGET: mingw-w64-i686 MSYSTEM: MINGW32 - CONFIGURE_ARGS: --target-list=ppc64-softmmu + CONFIGURE_ARGS: --target-list=ppc64-softmmu -Ddebug=false -Doptimization=0 TEST_ARGS: --no-suite qtest
This is IMHO a very good idea! But I think for now it's enough if you only change the 64-bit, isn't it?
Thomas
Re: [PATCH 7/8] gitlab: disable optimization and debug symbols in msys build
Quoted 28 lines · Thomas Huth
Quoted 27 lines · Daniel P. Berrangé
Building at -O2, adds 33% to the build time, over -O2. IOW a build that takes 45 minutes at -O0, takes 60 minutes at -O2. Turning off debug symbols drops it further, down to 38 minutes.
IOW, a "-O2 -g" build is 58% slower than a "-O0" build on msys in the gitlab CI windows shared runners.
Signed-off-by Daniel P. Berrangé <[email protected]>--- .gitlab-ci.d/windows.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-).gitlab-ci.d/windows.yml +1
-1@@ -113,7 +113,7 @@ msys2-64bit: # commit 9f8e6cad65a6 ("gitlab-ci: Speed up the msys2-64bit job by using --without-default-devices" # changed to compile QEMU with the --without-default-devices switch # for the msys2 64-bit job, due to the build could not complete within - CONFIGURE_ARGS: --target-list=x86_64-softmmu --without-default-devices + CONFIGURE_ARGS: --target-list=x86_64-softmmu --without-default-devices -Ddebug=false -Doptimization=0 # qTests don't run successfully with "--without-default-devices", # so let's exclude the qtests from CI for now. TEST_ARGS: --no-suite qtest@@ -123,5 +123,5 @@ msys2-32bit: variables:
Quoted 8 lines
.gitlab-ci.d/windows.yml msys2-32bit: · line 127
Quoted 5 lines
.gitlab-ci.d/windows.yml +1
-1MINGW_TARGET: mingw-w64-i686 MSYSTEM: MINGW32 - CONFIGURE_ARGS: --target-list=ppc64-softmmu + CONFIGURE_ARGS: --target-list=ppc64-softmmu -Ddebug=false -Doptimization=0 TEST_ARGS: --no-suite qtestThis is IMHO a very good idea! But I think for now it's enough if you only change the 64-bit, isn't it?
My thought was if we do it for 32-bit too, we can enable some more targets to get a more comprehensive build.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
PATCH 8/8 gitlab: disable FF_SCRIPT_SECTIONS on msys jobs
In reply to Daniel P. BerrangéThe FF_SCRIPT_SECTIONS=1 variable should ordinarily cause output from each line of the job script to be presented in a collapsible section with execution time listed.
While it works on Linux shared runners, when used with Windows runners with PowerShell, this option does not create any sections, and actually causes echo'ing of commands to be disabled, making it even worse to debug the jobs.
--- .gitlab-ci.d/windows.yml | 4 ++++ 1 file changed, 4 insertions(+)
.gitlab-ci.d/windows.yml +4 -0
@@ -12,6 +12,10 @@ needs: [] stage: build timeout: 80m + variables: + # This feature doesn't (currently) work with PowerShell, it stops + # the echo'ing of commands being run and doesn't show any timing + FF_SCRIPT_SECTIONS: 0 artifacts: name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG" expire_in: 7 days
Signature · 1 line
2.41.0
Re: [PATCH 8/8] gitlab: disable FF_SCRIPT_SECTIONS on msys jobs
Quoted 19 lines · Daniel P. Berrangé
The FF_SCRIPT_SECTIONS=1 variable should ordinarily cause output from each line of the job script to be presented in a collapsible section with execution time listed.
While it works on Linux shared runners, when used with Windows runners with PowerShell, this option does not create any sections, and actually causes echo'ing of commands to be disabled, making it even worse to debug the jobs.
Signed-off-by Daniel P. Berrangé <[email protected]>--- .gitlab-ci.d/windows.yml | 4 ++++ 1 file changed, 4 insertions(+).gitlab-ci.d/windows.yml +0
-0
.gitlab-ci.d/windows.yml line 21
Quoted 10 lines
.gitlab-ci.d/windows.yml +4
-0@@ -12,6 +12,10 @@ needs: [] stage: build timeout: 80m + variables: + # This feature doesn't (currently) work with PowerShell, it stops + # the echo'ing of commands being run and doesn't show any timing + FF_SCRIPT_SECTIONS: 0 artifacts: name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG" expire_in: 7 days
Quoted 11 lines · Daniel P. Berrangé
This is an alternative and/or complementary to Thomas' proposal to use CLang with msys:
https://lists.gnu.org/archive/html/qemu-devel/2023-07/msg05402.html
First of all, the current msys installer we're using is over 12 months out of date. Thus after running the install, pacman then replaces most of what we've just installed with new downloaded content. Using the most update installer cuts 3+1/2 minutes off the msys install time - 7 minutes becomes 3+1/2.
Quoted 3 lines
Secondly, QEMU defaults to compiling with -O2 and this is more computationally expensive for GCC. Switching to -O0 drops the build time from 60 minutes down to 45 minutes.
From the fine manual[*]: "The effectiveness of some warnings depends on optimizations also being enabled. For example '-Wsuggest-final-types' is more effective with link-time optimization and some instances of other warnings may not be issued at all unless optimization is enabled. While optimization in general improves the efficacy of control and data flow sensitive warnings, in some cases it may also cause false positives." Do we care?
[...]
[*] https://gcc.gnu.org/onlinedocs/gcc-13.2.0/gcc/Warning-Options.html
Quoted 17 lines · Markus Armbruster
Quoted 11 lines · Daniel P. Berrangé
This is an alternative and/or complementary to Thomas' proposal to use CLang with msys:
https://lists.gnu.org/archive/html/qemu-devel/2023-07/msg05402.html
First of all, the current msys installer we're using is over 12 months out of date. Thus after running the install, pacman then replaces most of what we've just installed with new downloaded content. Using the most update installer cuts 3+1/2 minutes off the msys install time - 7 minutes becomes 3+1/2.
Quoted 3 lines
Secondly, QEMU defaults to compiling with -O2 and this is more computationally expensive for GCC. Switching to -O0 drops the build time from 60 minutes down to 45 minutes.
Quoted 7 lines
From the fine manual[*]: "The effectiveness of some warnings depends on optimizations also being enabled. For example '-Wsuggest-final-types' is more effective with link-time optimization and some instances of other warnings may not be issued at all unless optimization is enabled. While optimization in general improves the efficacy of control and data flow sensitive warnings, in some cases it may also cause false positives." Do we care?
In general, yes, we do care.
In this specific case though, we're battling to figure out the lesser of multiple evils.
Right now we configure with:
--target-list=x86_64-softmmu --without-default-devices
and so with optimization enabled, we'll get good warning coverage of a small amount of code, except we don't because people started ignoring the msys jobs as they timeout too frequently.
If we can use Thomas' clang switch or my -O0 patches, we can get within the timeout, so people can trust the job once again. If we can do both ideas and cut the time even more, then we can enable more features (perhaps drop --without-default-devices).
So the warnings might not be quite as good, but we'll have the warnings across a larger amount of code.
Alot of the warnings from the Linux/macOS builds will also apply in the Windows builds. I think on balance I'd probably prefer us to build a larger amount of code for Windows. This is in context of free shared runners at least.
As a more drastic option, we might need to consider using the Azure credits for Windows runners too. If we could have bigger VMs for Windows CI, we can build more and have better warnings at the same time.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Quoted 26 lines · Daniel P. Berrangé
Quoted 17 lines · Markus Armbruster
Quoted 11 lines · Daniel P. Berrangé
This is an alternative and/or complementary to Thomas' proposal to use CLang with msys:
https://lists.gnu.org/archive/html/qemu-devel/2023-07/msg05402.htmlFirst of all, the current msys installer we're using is over 12 months out of date. Thus after running the install, pacman then replaces most of what we've just installed with new downloaded content. Using the most update installer cuts 3+1/2 minutes off the msys install time - 7 minutes becomes 3+1/2.
Quoted 3 lines
Secondly, QEMU defaults to compiling with -O2 and this is more computationally expensive for GCC. Switching to -O0 drops the build time from 60 minutes down to 45 minutes.
Quoted 7 lines
From the fine manual[*]: "The effectiveness of some warnings depends on
optimizations also being enabled. For example '-Wsuggest-final-types' is more effective with link-time optimization and some instances of other warnings may not be issued at all unless optimization is enabled. While optimization in general improves the efficacy of control and data flow sensitive warnings, in some cases it may also cause false positives." Do we care?
Quoted 4 lines
In general, yes, we do care.
In this specific case though, we're battling to figure out the lesser of multiple evils.
I agree. Additionally, we also test compiling for Windows with the MinGW cross compiler suite in a Fedora container, and we still use the default optimization there, so we should have that covered.
Thomas
Quoted 22 lines · Daniel P. Berrangé
This is an alternative and/or complementary to Thomas' proposal to use CLang with msys:
https://lists.gnu.org/archive/html/qemu-devel/2023-07/msg05402.htmlFirst of all, the current msys installer we're using is over 12 months out of date. Thus after running the install, pacman then replaces most of what we've just installed with new downloaded content. Using the most update installer cuts 3+1/2 minutes off the msys install time - 7 minutes becomes 3+1/2.
Secondly, QEMU defaults to compiling with -O2 and this is more computationally expensive for GCC. Switching to -O0 drops the build time from 60 minutes down to 45 minutes.
Thirdly, including debug symbols also has an overhead, and turning that off reduces time still further down to 38 minutes.
IOW, between all three changes, we can cut approx 25-26 minutes off the job execution time, bringing it nicely within the job timeout.
Quoted 7 lines
The actually phase of installing the mingw deps still accounts for about 10 minutes and has not been optimized.
Possibly the same trick of -O0 and skipping -g would also help the clang alternative Thomas' proposed. If so, that could be enough to let us enable more features / targets during the msys build.
I really like the idea! And I guess my idea with Clang needs some more work 'til it is acceptable, so let's go with your idea for now to fix the timeout problem in the CI ... we can still optimize later with Clang in case we found a good solution for that ms_struct problem...
Thomas