testing: Build WHPX enabled binaries
cover testing: Build WHPX enabled binaries
Add a job to cross-build QEMU with WHPX enabled.
Use the Win10SDK headers from the Android Project, as commented in https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03842.html
Based-on: <[email protected]>
https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03844.html
Philippe Mathieu-Daudé (2):
tests/docker: Add fedora-win10sdk-cross image
.shippable.yml: Build WHPX enabled binaries
.shippable.yml | 2 ++ tests/docker/Makefile.include | 1 + .../dockerfiles/fedora-win10sdk-cross.docker | 21 +++++++++++++++++++ 3 files changed, 24 insertions(+) create mode 100644 tests/docker/dockerfiles/fedora-win10sdk-cross.docker
Signature · 1 line
2.20.1
PATCH 1/2 tests/docker: Add fedora-win10sdk-cross image
--- tests/docker/Makefile.include | 1 + .../dockerfiles/fedora-win10sdk-cross.docker | 21 +++++++++++++++++++ 2 files changed, 22 insertions(+) create mode 100644 tests/docker/dockerfiles/fedora-win10sdk-cross.docker
tests/docker/Makefile.include +1 -0
@@ -125,6 +125,7 @@ docker-image-debian-ppc64-cross: docker-image-debian10 docker-image-debian-riscv64-cross: docker-image-debian10 docker-image-debian-sh4-cross: docker-image-debian10 docker-image-debian-sparc64-cross: docker-image-debian10 +docker-image-fedora-win10sdk-cross: docker-image-fedora docker-image-travis: NOUSER=1
tests/docker/dockerfiles/fedora-win10sdk-cross.docker new file +21 -0
@@ -0,0 +1,21 @@ +# +# Docker MinGW64 cross-compiler target with WHPX header installed +# +# This docker target builds on the Fedora 30 base image. +# +# SPDX-License-Identifier: GPL-2.0-or-later +# +FROM qemu:fedora + +# Install headers extracted by the Android folks from the win10sdk.iso: +# https://developer.microsoft.com/en-us/windows/downloads/windows-10-sdk +ENV WIN10SDK_INCDIR /usr/local/include/win10sdk +RUN mkdir ${WIN10SDK_INCDIR} && \ + for hdr in WinHvEmulation.h WinHvPlatformDefs.h WinHvPlatform.h; do \ + curl -s "https://android.googlesource.com/platform/prebuilts/gcc/linux-x86/host/x86_64-w64-mingw32-4.8/+/1bde9c3b14f3a3b081ada6e32da9f2870671b46f/x86_64-w64-mingw32/include/${hdr}?format=TEXT" | base64 --decode > ${WIN10SDK_INCDIR}/${hdr}; \ + done + +ENV QEMU_CONFIGURE_OPTS ${QEMU_CONFIGURE_OPTS} \ + --cross-prefix=x86_64-w64-mingw32- \ + --extra-cflags="-I ${WIN10SDK_INCDIR}" --disable-werror \ + --enable-hax --enable-whpx
Signature · 1 line
2.20.1
Re: [Qemu-devel] [PATCH 1/2] tests/docker: Add fedora-win10sdk-cross image
Quoted 31 lines · Philippe Mathieu-Daudé
Signed-off-by Philippe Mathieu-Daudé <[email protected]>--- tests/docker/Makefile.include | 1 + .../dockerfiles/fedora-win10sdk-cross.docker | 21 +++++++++++++++++++ 2 files changed, 22 insertions(+) create mode 100644 tests/docker/dockerfiles/fedora-win10sdk-cross.dockertests/docker/Makefile.include +1
-0@@ -125,6 +125,7 @@ docker-image-debian-ppc64-cross: docker-image-debian10 docker-image-debian-riscv64-cross: docker-image-debian10 docker-image-debian-sh4-cross: docker-image-debian10 docker-image-debian-sparc64-cross: docker-image-debian10 +docker-image-fedora-win10sdk-cross: docker-image-fedora docker-image-travis: NOUSER=1tests/docker/dockerfiles/fedora-win10sdk-cross.docker new file +6
-0@@ -0,0 +1,21 @@ +# +# Docker MinGW64 cross-compiler target with WHPX header installed +# +# This docker target builds on the Fedora 30 base image. +# +# SPDX-License-Identifier: GPL-2.0-or-later
tests/docker/dockerfiles/fedora-win10sdk-cross.docker line 16
Quoted 10 lines
tests/docker/dockerfiles/fedora-win10sdk-cross.docker new file +10
-0+# +FROM qemu:fedora + +# Install headers extracted by the Android folks from the win10sdk.iso: +# https://developer.microsoft.com/en-us/windows/downloads/windows-10-sdk +ENV WIN10SDK_INCDIR /usr/local/include/win10sdk +RUN mkdir ${WIN10SDK_INCDIR} && \ + for hdr in WinHvEmulation.h WinHvPlatformDefs.h WinHvPlatform.h; do \ + curl -s "https://android.googlesource.com/platform/prebuilts/gcc/linux-x86/host/x86_64-w64-mingw32-4.8/+/1bde9c3b14f3a3b081ada6e32da9f2870671b46f/x86_64-w64-mingw32/include/${hdr}?format=TEXT" | base64 --decode > ${WIN10SDK_INCDIR}/${hdr}; \ + done
Hmm I guess this is a stable URL as Google never deletes anything?
tests/docker/dockerfiles/fedora-win10sdk-cross.docker line 21
Quoted 5 lines
tests/docker/dockerfiles/fedora-win10sdk-cross.docker new file +5
-0+ +ENV QEMU_CONFIGURE_OPTS ${QEMU_CONFIGURE_OPTS} \ + --cross-prefix=x86_64-w64-mingw32- \ + --extra-cflags="-I ${WIN10SDK_INCDIR}" --disable-werror \ + --enable-hax --enable-whpx
--
Alex Bennée
Re: [Qemu-devel] [PATCH 1/2] tests/docker: Add fedora-win10sdk-cross image
Quoted 38 lines · Alex Bennée
Quoted 35 lines · Philippe Mathieu-Daudé
Signed-off-by Philippe Mathieu-Daudé <[email protected]>--- tests/docker/Makefile.include | 1 + .../dockerfiles/fedora-win10sdk-cross.docker | 21 +++++++++++++++++++ 2 files changed, 22 insertions(+) create mode 100644 tests/docker/dockerfiles/fedora-win10sdk-cross.dockertests/docker/Makefile.include +1
-0@@ -125,6 +125,7 @@ docker-image-debian-ppc64-cross: docker-image-debian10 docker-image-debian-riscv64-cross: docker-image-debian10 docker-image-debian-sh4-cross: docker-image-debian10 docker-image-debian-sparc64-cross: docker-image-debian10 +docker-image-fedora-win10sdk-cross: docker-image-fedora docker-image-travis: NOUSER=1tests/docker/dockerfiles/fedora-win10sdk-cross.docker new file +10
-0@@ -0,0 +1,21 @@ +# +# Docker MinGW64 cross-compiler target with WHPX header installed +# +# This docker target builds on the Fedora 30 base image. +# +# SPDX-License-Identifier: GPL-2.0-or-later +# +FROM qemu:fedora + +# Install headers extracted by the Android folks from the win10sdk.iso:
Quoted 8 lines
tests/docker/dockerfiles/fedora-win10sdk-cross.docker line 16
Quoted 6 lines
tests/docker/dockerfiles/fedora-win10sdk-cross.docker new file +6
-0+# https://developer.microsoft.com/en-us/windows/downloads/windows-10-sdk +ENV WIN10SDK_INCDIR /usr/local/include/win10sdk +RUN mkdir ${WIN10SDK_INCDIR} && \ + for hdr in WinHvEmulation.h WinHvPlatformDefs.h WinHvPlatform.h; do \ + curl -s "https://android.googlesource.com/platform/prebuilts/gcc/linux-x86/host/x86_64-w64-mingw32-4.8/+/1bde9c3b14f3a3b081ada6e32da9f2870671b46f/x86_64-w64-mingw32/include/${hdr}?format=TEXT" | base64 --decode > ${WIN10SDK_INCDIR}/${hdr}; \ + doneHmm I guess this is a stable URL as Google never deletes anything?
This is not ideal indeed, but way less painful than using the msiextract tools (and much quicker, the win10sdk.iso is >800MiB).
I dunno about the license issue, this file has "Copyright (c) Microsoft Corporation. All rights reserved." but we are not storing its content.
Quoted 10 lines
tests/docker/dockerfiles/fedora-win10sdk-cross.docker line 21
Quoted 5 lines
tests/docker/dockerfiles/fedora-win10sdk-cross.docker new file +5
-0+ +ENV QEMU_CONFIGURE_OPTS ${QEMU_CONFIGURE_OPTS} \ + --cross-prefix=x86_64-w64-mingw32- \ + --extra-cflags="-I ${WIN10SDK_INCDIR}" --disable-werror \ + --enable-hax --enable-whpx--
Alex Bennée
PATCH 2/2 .shippable.yml: Build WHPX enabled binaries
In reply to Philippe Mathieu-Daudé--- .shippable.yml | 2 ++ 1 file changed, 2 insertions(+)
.shippable.yml +2 -0
@@ -25,6 +25,8 @@ env: TARGET_LIST=mips64el-softmmu,mips64el-linux-user - IMAGE=debian-ppc64el-cross TARGET_LIST=ppc64-softmmu,ppc64-linux-user,ppc64abi32-linux-user + - IMAGE=fedora-win10sdk-cross + TARGET_LIST=x86_64-softmmu,i386-softmmu build: pre_ci: # usually host ARCH is set by configure
Signature · 1 line
2.20.1
Re: [Qemu-devel] [PATCH 2/2] .shippable.yml: Build WHPX enabled binaries
Quoted 10 lines · Philippe Mathieu-Daudé
Signed-off-by Philippe Mathieu-Daudé <[email protected]>--- .shippable.yml | 2 ++ 1 file changed, 2 insertions(+).shippable.yml +0
-0
.shippable.yml env: · line 29
Quoted 5 lines
.shippable.yml +2
-0@@ -25,6 +25,8 @@ env: TARGET_LIST=mips64el-softmmu,mips64el-linux-user - IMAGE=debian-ppc64el-cross TARGET_LIST=ppc64-softmmu,ppc64-linux-user,ppc64abi32-linux-user + - IMAGE=fedora-win10sdk-cross + TARGET_LIST=x86_64-softmmu,i386-softmmu
Note for Alex, if this patch is accepted, I think we can remove x86_64-softmmu from TARGET_LIST in the debian-win64-cross job, since this one duplicates it (and even covers more).
Quoted 4 lines
.shippable.yml +0
-0build: pre_ci: # usually host ARCH is set by configure
Quoted 12 lines · Philippe Mathieu-Daudé
Add a job to cross-build QEMU with WHPX enabled.
Use the Win10SDK headers from the Android Project, as commented in https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03842.html
Based-on: <[email protected]>
https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03844.htmlPhilippe Mathieu-Daudé (2):
tests/docker: Add fedora-win10sdk-cross image
.shippable.yml: Build WHPX enabled binaries
Quoted 5 lines
.shippable.yml | 2 ++ tests/docker/Makefile.include | 1 + .../dockerfiles/fedora-win10sdk-cross.docker | 21 +++++++++++++++++++ 3 files changed, 24 insertions(+) create mode 100644 tests/docker/dockerfiles/fedora-win10sdk-cross.docker
Don't you want to include a fix for the #include "hw/boards.h" problem in your series? ... or is it not happening with your builds here?
Thomas
Quoted 13 lines · Thomas Huth
Quoted 12 lines · Philippe Mathieu-Daudé
Add a job to cross-build QEMU with WHPX enabled.
Use the Win10SDK headers from the Android Project, as commented in https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03842.html
Based-on: <[email protected]>
https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03844.htmlPhilippe Mathieu-Daudé (2):
tests/docker: Add fedora-win10sdk-cross image
.shippable.yml: Build WHPX enabled binaries
Quoted 8 lines
Quoted 5 lines
.shippable.yml | 2 ++ tests/docker/Makefile.include | 1 + .../dockerfiles/fedora-win10sdk-cross.docker | 21 +++++++++++++++++++ 3 files changed, 24 insertions(+) create mode 100644 tests/docker/dockerfiles/fedora-win10sdk-cross.dockerDon't you want to include a fix for the #include "
hw/boards.h" problem in your series? ... or is it not happening with your builds here?
Ah Based-on is for Patchew, and this series need to be tested by Shippable... you are right.
Well, this is a good opportunity to see the job catch the WHPX failure reported :)
Am 19.09.2019 um 12:59 schrieb Philippe Mathieu-Daudé:
Quoted 12 lines
Add a job to cross-build QEMU with WHPX enabled.
Use the Win10SDK headers from the Android Project, as commented in https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03842.html
Based-on: <[email protected]>
https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03844.htmlPhilippe Mathieu-Daudé (2):
tests/docker: Add fedora-win10sdk-cross image
.shippable.yml: Build WHPX enabled binaries
Quoted 6 lines
.shippable.yml | 2 ++ tests/docker/Makefile.include | 1 + .../dockerfiles/fedora-win10sdk-cross.docker | 21 +++++++++++++++++++ 3 files changed, 24 insertions(+) create mode 100644 tests/docker/dockerfiles/fedora-win10sdk-cross.docker
Please note that the required header files are part of the Win10SDK which is not published under a free license, so I am afraid that they cannot be used with QEMU code to produce free binaries.
I have addressed that some time ago, and Justin Terry is still looking for a solution on the Microsoft side.
Regards,
Stefan Weil
Quoted 20 lines · Stefan Weil
Am 19.09.2019 um 12:59 schrieb Philippe Mathieu-Daudé:
Quoted 12 lines
Add a job to cross-build QEMU with WHPX enabled.
Use the Win10SDK headers from the Android Project, as commented in https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03842.html
Based-on: <[email protected]>
https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03844.htmlPhilippe Mathieu-Daudé (2):
tests/docker: Add fedora-win10sdk-cross image
.shippable.yml: Build WHPX enabled binariesQuoted 6 lines
.shippable.yml | 2 ++ tests/docker/Makefile.include | 1 + .../dockerfiles/fedora-win10sdk-cross.docker | 21 +++++++++++++++++++ 3 files changed, 24 insertions(+) create mode 100644 tests/docker/dockerfiles/fedora-win10sdk-cross.docker
Quoted 3 lines
Please note that the required header files are part of the Win10SDK which is not published under a free license, so I am afraid that they cannot be used with QEMU code to produce free binaries.
Yes :S
Quoted 2 lines
I have addressed that some time ago, and Justin Terry is still looking for a solution on the Microsoft side.
Oh this is a good news, thanks for caring about this issue, and thanks Justin for looking for a solution!
Trying to understand how WHPX is used, I noticed there are much many Windows QEMU users than I thought, and it would be nice if we can have some upstream CI testing to not break the various projects using it.
Regards,
Phil.
+launchpad ticket
Quoted 41 lines · Philippe Mathieu-Daudé
Quoted 20 lines · Stefan Weil
Am 19.09.2019 um 12:59 schrieb Philippe Mathieu-Daudé:
Quoted 12 lines
Add a job to cross-build QEMU with WHPX enabled.
Use the Win10SDK headers from the Android Project, as commented in https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03842.html
Based-on: <[email protected]>
https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03844.htmlPhilippe Mathieu-Daudé (2):
tests/docker: Add fedora-win10sdk-cross image
.shippable.yml: Build WHPX enabled binariesQuoted 6 lines
.shippable.yml | 2 ++ tests/docker/Makefile.include | 1 + .../dockerfiles/fedora-win10sdk-cross.docker | 21 +++++++++++++++++++ 3 files changed, 24 insertions(+)create mode 100644 tests/docker/dockerfiles/fedora-win10sdk-cross.docker
Quoted 3 lines
Please note that the required header files are part of the Win10SDK which is not published under a free license, so I am afraid that they cannot be used with QEMU code to produce free binaries.
Yes :S
Quoted 2 lines
I have addressed that some time ago, and Justin Terry is still looking for a solution on the Microsoft side.
Oh this is a good news, thanks for caring about this issue, and thanks Justin for looking for a solution!
Trying to understand how WHPX is used, I noticed there are much many Windows QEMU users than I thought, and it would be nice if we can have some upstream CI testing to not break the various projects using it.
Regards,
Phil.
Quoted 20 lines · Stefan Weil
Am 19.09.2019 um 12:59 schrieb Philippe Mathieu-Daudé:
Quoted 12 lines
Add a job to cross-build QEMU with WHPX enabled.
Use the Win10SDK headers from the Android Project, as commented in https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03842.html
Based-on: <[email protected]>
https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03844.htmlPhilippe Mathieu-Daudé (2):
tests/docker: Add fedora-win10sdk-cross image
.shippable.yml: Build WHPX enabled binariesQuoted 6 lines
.shippable.yml | 2 ++ tests/docker/Makefile.include | 1 + .../dockerfiles/fedora-win10sdk-cross.docker | 21 +++++++++++++++++++ 3 files changed, 24 insertions(+) create mode 100644 tests/docker/dockerfiles/fedora-win10sdk-cross.docker
Quoted 3 lines
Please note that the required header files are part of the Win10SDK which is not published under a free license, so I am afraid that they cannot be used with QEMU code to produce free binaries.
Can you elaborate on why you think that is the case ?
Looking at the 3 header files used, all they contain are struct definitions and integer constants, fortunately no functional code in macros or inline functions. As such, it is questionable whether the headers alone can be considered copyrightable material. Any compatible implementation that follows the WHPX API/ABI will by neccessity be the same, modulo code style.
Further, from a GPL POV, these headers should fall under the system libraries exception, as WHPX is core low level operating system functionality.
There's a valid legal question as to how to we got access to the files in the first place. Assuming they are freely distributable though, given their hosting on Google Android downloads.
So the fact that the headers are not under an open source / free license doesn't look like it should impact the ability to distribute the resulting QEMU binaries.
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Quoted 38 lines · Daniel P. Berrangé
Quoted 20 lines · Stefan Weil
Am 19.09.2019 um 12:59 schrieb Philippe Mathieu-Daudé:
Quoted 12 lines
Add a job to cross-build QEMU with WHPX enabled.
Use the Win10SDK headers from the Android Project, as commented in https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03842.html
Based-on: <[email protected]>
https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03844.htmlPhilippe Mathieu-Daudé (2):
tests/docker: Add fedora-win10sdk-cross image
.shippable.yml: Build WHPX enabled binariesQuoted 6 lines
.shippable.yml | 2 ++ tests/docker/Makefile.include | 1 + .../dockerfiles/fedora-win10sdk-cross.docker | 21 +++++++++++++++++++ 3 files changed, 24 insertions(+) create mode 100644 tests/docker/dockerfiles/fedora-win10sdk-cross.dockerQuoted 3 lines
Please note that the required header files are part of the Win10SDK which is not published under a free license, so I am afraid that they cannot be used with QEMU code to produce free binaries.
Can you elaborate on why you think that is the case ?
Looking at the 3 header files used, all they contain are struct definitions and integer constants, fortunately no functional code in macros or inline functions. As such, it is questionable whether the headers alone can be considered copyrightable material. Any compatible implementation that follows the WHPX API/ABI will by neccessity be the same, modulo code style.
Further, from a GPL POV, these headers should fall under the system libraries exception, as WHPX is core low level operating system functionality.
Quoted 7 lines
There's a valid legal question as to how to we got access to the files in the first place. Assuming they are freely distributable though, given their hosting on Google Android downloads.
So the fact that the headers are not under an open source / free license doesn't look like it should impact the ability to distribute the resulting QEMU binaries.
Are we distributing the resulting binaries?
I don't think so, we only build until linking in a scratch container and discard its content, we don't have access to the container storage via Shippable (also Travis). We only use the exit code and the console output from the build.
We are not distributing those headers neither, as we currently do not distribute the docker images.
Regards,
Phil.
Quoted 47 lines · Philippe Mathieu-Daudé
Quoted 38 lines · Daniel P. Berrangé
Quoted 20 lines · Stefan Weil
Am 19.09.2019 um 12:59 schrieb Philippe Mathieu-Daudé:
Quoted 12 lines
Add a job to cross-build QEMU with WHPX enabled.
Use the Win10SDK headers from the Android Project, as commented in https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03842.html
Based-on: <[email protected]>
https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03844.htmlPhilippe Mathieu-Daudé (2):
tests/docker: Add fedora-win10sdk-cross image
.shippable.yml: Build WHPX enabled binariesQuoted 6 lines
.shippable.yml | 2 ++ tests/docker/Makefile.include | 1 + .../dockerfiles/fedora-win10sdk-cross.docker | 21 +++++++++++++++++++ 3 files changed, 24 insertions(+) create mode 100644 tests/docker/dockerfiles/fedora-win10sdk-cross.dockerQuoted 3 lines
Please note that the required header files are part of the Win10SDK which is not published under a free license, so I am afraid that they cannot be used with QEMU code to produce free binaries.
Can you elaborate on why you think that is the case ?
Looking at the 3 header files used, all they contain are struct definitions and integer constants, fortunately no functional code in macros or inline functions. As such, it is questionable whether the headers alone can be considered copyrightable material. Any compatible implementation that follows the WHPX API/ABI will by neccessity be the same, modulo code style.
Further, from a GPL POV, these headers should fall under the system libraries exception, as WHPX is core low level operating system functionality.
Quoted 7 lines
There's a valid legal question as to how to we got access to the files in the first place. Assuming they are freely distributable though, given their hosting on Google Android downloads.
So the fact that the headers are not under an open source / free license doesn't look like it should impact the ability to distribute the resulting QEMU binaries.
Quoted 8 lines
Are we distributing the resulting binaries?
I don't think so, we only build until linking in a scratch container and discard its content, we don't have access to the container storage via Shippable (also Travis). We only use the exit code and the console output from the build.We are not distributing those headers neither, as we currently do not distribute the docker images.
No, but I think its none the less useful to consider what impact is on distribution of binaries. Currently Stefan builds & distributes the QEMU Windows binaries for each release. I think the ideal would be for Windows builds to be a core deliverable of the upstram release process. So this docker CI job can be viewed as a stepping stone to that point, especially if it is made to run the full NSIS installer build, not just standalone binary build.
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|