add the BiosTablesTest UEFI app, build it with the new roms/edk2 submodule
v3 cover add the BiosTablesTest UEFI app, build it with the new roms/edk2 submodule
Previous version (v2):
http://mid.mail-archive.com/[email protected]
Updates in v3 have been noted on each patch in the series.
Thanks
Laszlo
Laszlo Ersek (5):
roms: add the edk2 project as a git submodule
roms: build the EfiRom utility from the roms/edk2 submodule
tests: introduce "uefi-test-tools" with the BiosTablesTest UEFI app tests/uefi-test-tools: add build scripts
tests/data: introduce "uefi-boot-images" with the "bios-tables-test"
ISOs
.gitmodules | 3 + Makefile | 6 +- roms/Makefile | 13 +- roms/edk2 | 1 + tests/data/uefi-boot-images/bios-tables-test.aarch64.iso.qcow2 | Bin 0 -> 11776 bytes tests/data/uefi-boot-images/bios-tables-test.arm.iso.qcow2 | Bin 0 -> 11776 bytes tests/data/uefi-boot-images/bios-tables-test.i386.iso.qcow2 | Bin 0 -> 12800 bytes tests/data/uefi-boot-images/bios-tables-test.x86_64.iso.qcow2 | Bin 0 -> 13312 bytes tests/uefi-test-tools/.gitignore | 3 + tests/uefi-test-tools/LICENSE | 25 ++++ tests/uefi-test-tools/Makefile | 106 ++++++++++++++ tests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest/BiosTablesTest.c | 130 ++++++++++++++++++ tests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest/BiosTablesTest.inf | 41 ++++++ tests/uefi-test-tools/UefiTestToolsPkg/Include/Guid/BiosTablesTest.h | 67 +++++++++ tests/uefi-test-tools/UefiTestToolsPkg/UefiTestToolsPkg.dec | 27 ++++ tests/uefi-test-tools/UefiTestToolsPkg/UefiTestToolsPkg.dsc | 69 ++++++++++ tests/uefi-test-tools/build.sh | 145 ++++++++++++++++++++ 17 files changed, 628 insertions(+), 8 deletions(-) create mode 160000 roms/edk2 create mode 100644 tests/data/uefi-boot-images/bios-tables-test.aarch64.iso.qcow2 create mode 100644 tests/data/uefi-boot-images/bios-tables-test.arm.iso.qcow2 create mode 100644 tests/data/uefi-boot-images/bios-tables-test.i386.iso.qcow2 create mode 100644 tests/data/uefi-boot-images/bios-tables-test.x86_64.iso.qcow2 create mode 100644 tests/uefi-test-tools/.gitignore create mode 100644 tests/uefi-test-tools/LICENSE create mode 100644 tests/uefi-test-tools/Makefile create mode 100644 tests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest/BiosTablesTest.c create mode 100644 tests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest/BiosTablesTest.inf create mode 100644 tests/uefi-test-tools/UefiTestToolsPkg/Include/Guid/BiosTablesTest.h create mode 100644 tests/uefi-test-tools/UefiTestToolsPkg/UefiTestToolsPkg.dec create mode 100644 tests/uefi-test-tools/UefiTestToolsPkg/UefiTestToolsPkg.dsc create mode 100755 tests/uefi-test-tools/build.sh
Signature · 1 line
2.19.1.3.g30247aa5d201
PATCH v3 1/5 roms: add the edk2 project as a git submodule
The roms/edk2 submodule can help with three goals:
- build the OVMF and ArmVirtQemu virtual UEFI firmware platforms (to be implemented later),
- build the EfiRom tool on the fly, which is used in roms/Makefile, for building the "efirom" target,
- build UEFI test applications (to be run in guests), for qtest support.
Edk2 commit 85588389222a3636baf0f9ed8227f2434af4c3f9 stands for the latest "stable tag", namely "edk2-stable201811".
The edk2 repository tracks some binary files that should not be removed by QEMU's top-level "make clean"; exempt the full pathnames from the "find" command.
---
Notes:
v3:
- no change
v2:
- pick up R-b [Gerd]
Makefile | 6 +++++- .gitmodules | 3 +++ roms/edk2 | 1 + 3 files changed, 9 insertions(+), 1 deletion(-)
Makefile +5 -1
@@ -607,7 +607,11 @@ clean: rm -f config.mak op-i386.h opc-i386.h gen-op-i386.h op-arm.h opc-arm.h gen-op-arm.h rm -f qemu-options.def rm -f *.msi - find . \( -name '*.so' -o -name '*.dll' -o -name '*.mo' -o -name '*.[oda]' \) -type f -exec rm {} + + find . \( -name '*.so' -o -name '*.dll' -o -name '*.mo' -o -name '*.[oda]' \) -type f \ + ! -path ./roms/edk2/ArmPkg/Library/GccLto/liblto-aarch64.a \ + ! -path ./roms/edk2/ArmPkg/Library/GccLto/liblto-arm.a \ + ! -path ./roms/edk2/BaseTools/Source/Python/UPT/Dll/sqlite3.dll \ + -exec rm {} + rm -f $(filter-out %.tlb,$(TOOLS)) $(HELPERS-y) qemu-ga TAGS cscope.* *.pod *~ */*~ rm -f fsdev/*.pod scsi/*.pod rm -f qemu-img-cmds.h
.gitmodules +3 -0
@@ -49,3 +49,6 @@ [submodule "tests/fp/berkeley-softfloat-3"] path = tests/fp/berkeley-softfloat-3 url = https://github.com/cota/berkeley-softfloat-3 +[submodule "roms/edk2"] + path = roms/edk2 + url = https://github.com/tianocore/edk2.git
roms/edk2 new file +1 -0
@@ -0,0 +1 @@ +Subproject commit 85588389222a3636baf0f9ed8227f2434af4c3f9
Signature · 1 line
2.19.1.3.g30247aa5d201
Re: [Qemu-devel] [PATCH v3 1/5] roms: add the edk2 project as a git submodule
Quoted 14 lines · Laszlo Ersek
The roms/edk2 submodule can help with three goals:
- build the OVMF and ArmVirtQemu virtual UEFI firmware platforms (to be implemented later),
- build the EfiRom tool on the fly, which is used in roms/Makefile, for building the "efirom" target,
- build UEFI test applications (to be run in guests), for qtest support.Edk2 commit
85588389222a3636baf0f9ed8227f2434af4c3f9stands for the latest "stable tag", namely "edk2-stable201811".The edk2 repository tracks some binary files that should not be removed by QEMU's top-level "make clean"; exempt the full pathnames from the "find" command.
Quoted 8 lines
Cc "Michael S. Tsirkin" <[email protected]>Cc Ard Biesheuvel <[email protected]>Cc Gerd Hoffmann <[email protected]>Cc Igor Mammedov <[email protected]>Cc Philippe Mathieu-Daudé <[email protected]>Cc Shannon Zhao <[email protected]>Signed-off-by Laszlo Ersek <[email protected]>Reviewed-by Gerd Hoffmann <[email protected]>
Already reviewed v2 ;)
https://www.mail-archive.com/[email protected]/msg592311.html
Quoted 50 lines
---
Notes:
v3:
- no changev2:
- pick up R-b [Gerd]Makefile | 6 +++++- .gitmodules | 3 +++ roms/edk2 | 1 + 3 files changed, 9 insertions(+), 1 deletion(-)Makefile +5
-1@@ -607,7 +607,11 @@ clean: rm -f config.mak op-i386.h opc-i386.h gen-op-i386.h op-arm.h opc-arm.h gen-op-arm.h rm -f qemu-options.def rm -f *.msi - find . \( -name '*.so' -o -name '*.dll' -o -name '*.mo' -o -name '*.[oda]' \) -type f -exec rm {} + + find . \( -name '*.so' -o -name '*.dll' -o -name '*.mo' -o -name '*.[oda]' \) -type f \ + ! -path ./roms/edk2/ArmPkg/Library/GccLto/liblto-aarch64.a \ + ! -path ./roms/edk2/ArmPkg/Library/GccLto/liblto-arm.a \ + ! -path ./roms/edk2/BaseTools/Source/Python/UPT/Dll/sqlite3.dll \ + -exec rm {} + rm -f $(filter-out %.tlb,$(TOOLS)) $(HELPERS-y) qemu-ga TAGS cscope.* *.pod *~ */*~ rm -f fsdev/*.pod scsi/*.pod rm -f qemu-img-cmds.h.gitmodules +3
-0@@ -49,3 +49,6 @@ [submodule "tests/fp/berkeley-softfloat-3"] path = tests/fp/berkeley-softfloat-3 url = https://github.com/cota/berkeley-softfloat-3 +[submodule "roms/edk2"] + path = roms/edk2 + url = https://github.com/tianocore/edk2.gitroms/edk2 new file +1
-0@@ -0,0 +1 @@ +Subproject commit 85588389222a3636baf0f9ed8227f2434af4c3f9
Re: [Qemu-devel] [PATCH v3 1/5] roms: add the edk2 project as a git submodule
Quoted 24 lines · Philippe Mathieu-Daudé
Quoted 14 lines · Laszlo Ersek
The roms/edk2 submodule can help with three goals:
- build the OVMF and ArmVirtQemu virtual UEFI firmware platforms (to be implemented later),
- build the EfiRom tool on the fly, which is used in roms/Makefile, for building the "efirom" target,
- build UEFI test applications (to be run in guests), for qtest support.Edk2 commit
85588389222a3636baf0f9ed8227f2434af4c3f9stands for the latest "stable tag", namely "edk2-stable201811".The edk2 repository tracks some binary files that should not be removed by QEMU's top-level "make clean"; exempt the full pathnames from the "find" command.
Quoted 8 lines
Cc "Michael S. Tsirkin" <[email protected]>Cc Ard Biesheuvel <[email protected]>Cc Gerd Hoffmann <[email protected]>Cc Igor Mammedov <[email protected]>Cc Philippe Mathieu-Daudé <[email protected]>Cc Shannon Zhao <[email protected]>Signed-off-by Laszlo Ersek <[email protected]>Reviewed-by Gerd Hoffmann <[email protected]>
Quoted 5 lines
Already reviewed v2 ;)
https://www.mail-archive.com/[email protected]/msg592311.htmlReviewed-by Philippe Mathieu-Daudé <[email protected]>
Ugh, my bad, for not picking up those. I was so relieved Friday (?) night about the "+" that I failed to focus on anything else. My apologies.
Thank you!
Laszlo
PATCH v3 2/5 roms: build the EfiRom utility from the roms/edk2 submodule
In reply to Laszlo ErsekBuilding the EfiRom utility from "roms/edk2/BaseTools" should make "roms/Makefile" more self-contained. Otherwise, we'd call the system-wide EfiRom for building the combined iPXE option ROMs, but call the sibling utilities from "roms/edk2/BaseTools" for building "roms/edk2" content.
---
Notes:
v3:
- no change
v2:
- pick up R-b / T-b [Gerd, Phil]
roms/Makefile | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-)
roms/Makefile +6 -7
@@ -47,10 +47,7 @@ SEABIOS_EXTRAVERSION="-prebuilt.qemu.org" # We need that to combine multiple images (legacy bios, # efi ia32, efi x64) into a single rom binary. # -# We try to find it in the path. You can also pass the location on -# the command line, i.e. "make EFIROM=/path/to/EfiRom efirom" -# -EFIROM ?= $(shell which EfiRom 2>/dev/null) +EFIROM = edk2/BaseTools/Source/C/bin/EfiRom default: @echo "nothing is build by default"
@@ -59,8 +56,7 @@ default: @echo " vgabios -- update vgabios binaries (seabios)" @echo " sgabios -- update sgabios binaries" @echo " pxerom -- update nic roms (bios only)" - @echo " efirom -- update nic roms (bios+efi, this needs" - @echo " the EfiRom utility from edk2 / tianocore)" + @echo " efirom -- update nic roms (bios+efi)" @echo " slof -- update slof.bin" @echo " skiboot -- update skiboot.lid" @echo " u-boot.e500 -- update u-boot.e500"
@@ -106,7 +102,7 @@ pxe-rom-%: build-pxe-roms efirom: $(patsubst %,efi-rom-%,$(pxerom_variants)) -efi-rom-%: build-pxe-roms build-efi-roms +efi-rom-%: build-pxe-roms build-efi-roms $(EFIROM) $(EFIROM) -f "0x$(VID)" -i "0x$(DID)" -l 0x02 \ -b ipxe/src/bin/$(VID)$(DID).rom \ -ec ipxe/src/bin-i386-efi/$(VID)$(DID).efidrv \
@@ -124,6 +120,8 @@ build-efi-roms: build-pxe-roms $(patsubst %,bin-i386-efi/%.efidrv,$(pxerom_targets)) \ $(patsubst %,bin-x86_64-efi/%.efidrv,$(pxerom_targets)) +$(EFIROM): + $(MAKE) -C edk2/BaseTools slof: $(MAKE) -C SLOF CROSS=$(powerpc64_cross_prefix) qemu
@@ -150,6 +148,7 @@ clean: $(MAKE) -C sgabios clean rm -f sgabios/.depend $(MAKE) -C ipxe/src veryclean + $(MAKE) -C edk2/BaseTools clean $(MAKE) -C SLOF clean rm -rf u-boot/build.e500 $(MAKE) -C u-boot-sam460ex distclean
Signature · 1 line
2.19.1.3.g30247aa5d201
PATCH v3 3/5 tests: introduce "uefi-test-tools" with the BiosTablesTest UEFI app
In reply to Laszlo ErsekThe "bios-tables-test" program in QEMU's test suite locates the RSD PTR ACPI table in guest RAM, and (chasing pointers to other ACPI tables) performs various sanity checks on the QEMU-generated and firmware-installed tables.
Currently this set of test cases doesn't work with UEFI guests. The ACPI spec defines distinct methods for OSPM to locate the RSD PTR on traditional BIOS vs. UEFI platforms, and the UEFI method is more difficult to implement from the hypervisor side with just raw guest memory access.
Add a UEFI application (to be booted in the UEFI guest) that populates a small, MB-aligned structure in guest RAM. The structure begins with a signature GUID. The hypervisor should loop over all MB-aligned pages in guest RAM until one matches the signature GUID at offset 0, at which point the hypervisor can fetch the RSDP address field(s) from the structure.
QEMU's test logic currently spins on a pre-determined guest address, until that address assumes a magic value. The method described in this patch is conceptually the same ("busy loop until match is found"), except there is no hard-coded address. This plays a lot more nicely with UEFI guest firmware (we'll be able to use the normal page allocation UEFI service). Given the size of EFI_GUID (16 bytes -- 128 bits), mismatches should be astronomically unlikely. In addition, given the typical guest RAM size for such tests (128 MB), there are 128 locations to check in one iteration of the "outer" loop, which shouldn't introduce an intolerable delay after the guest stores the RSDP address(es), and then the GUID.
The GUID that the hypervisor should search for is
AB87A6B1-2034-BDA0-71BD-375007757785
Expressed as a byte array:
{
0xb1, 0xa6, 0x87, 0xab,
0x34, 0x20,
0xa0, 0xbd,
0x71, 0xbd, 0x37, 0x50, 0x07, 0x75, 0x77, 0x85
}
Note that in the patch, we define "gBiosTablesTestGuid" with all bits inverted. This is a simple method to prevent the UEFI binary, which incorporates "gBiosTablesTestGuid", from matching the actual GUID in guest RAM.
The UEFI application is written against the edk2 framework, which was introduced earlier as a git submodule. The next patch will provide build scripts for maintainers.
The source code follows the edk2 coding style, and is licensed under the 2-clause BSDL (in case someone would like to include UefiTestToolsPkg content in a different edk2 platform).
The "UefiTestToolsPkg.dsc" platform description file resolves the used edk2 library classes to instances (= library implementations) such that the UEFI binaries inherit no platform dependencies. They are expected to run on any system that conforms to the UEFI-2.3.1 spec (which was released in 2012). The arch-specific build options are carried over from edk2's ArmVirtPkg and OvmfPkg platforms.
---
Notes:
v3:
- no change
v2:
- no change
v1:
- If that's necessary, I'd be glad to be designated as Maintainer or Reviewer in "MAINTAINERS" for "tests/uefi-test-tools/", I just couldn't figure out under what subsystem I should add the magic lines. "MAINTAINERS" needs a Table of Contents! :)
tests/uefi-test-tools/UefiTestToolsPkg/UefiTestToolsPkg.dec | 27 ++++ tests/uefi-test-tools/UefiTestToolsPkg/UefiTestToolsPkg.dsc | 69 +++++++++++ tests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest/BiosTablesTest.inf | 41 ++++++ tests/uefi-test-tools/UefiTestToolsPkg/Include/Guid/BiosTablesTest.h | 67 ++++++++++ tests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest/BiosTablesTest.c | 130 ++++++++++++++++++++ tests/uefi-test-tools/LICENSE | 25 ++++ 6 files changed, 359 insertions(+)
tests/uefi-test-tools/UefiTestToolsPkg/UefiTestToolsPkg.dec new file +27 -0
@@ -0,0 +1,27 @@ +## @file +# edk2 package declaration for the test helper UEFI applications that run in +# guests. +# +# Copyright (C) 2019, Red Hat, Inc. +# +# This program and the accompanying materials are licensed and made available +# under the terms and conditions of the BSD License that accompanies this +# distribution. The full text of the license may be found at +# <http://opensource.org/licenses/bsd-license.php>. +# +# THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, WITHOUT +# WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED. +## + +[Defines] + DEC_SPECIFICATION = 1.27 + PACKAGE_NAME = UefiTestToolsPkg + PACKAGE_GUID = 7b3f1794-0c85-4b27-a536-44dbf0b0669c + PACKAGE_VERSION = 0.1 + +[Includes] + Include + +[Guids] + gBiosTablesTestGuid = {0x5478594e, 0xdfcb, 0x425f, {0x8e, 0x42, 0xc8, 0xaf, 0xf8, 0x8a, 0x88, 0x7a}} +
tests/uefi-test-tools/UefiTestToolsPkg/UefiTestToolsPkg.dsc new file +69 -0
69 lines not on this page · raw message
tests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest/BiosTablesTest.inf new file +41 -0
41 lines not on this page · raw message
tests/uefi-test-tools/UefiTestToolsPkg/Include/Guid/BiosTablesTest.h new file +67 -0
67 lines not on this page · raw message
tests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest/BiosTablesTest.c new file +130 -0
130 lines not on this page · raw message
tests/uefi-test-tools/LICENSE new file +25 -0
@@ -0,0 +1,25 @@ +All the files in this directory and subdirectories are released under the +2-Clause BSD License (see header in each file). + +Copyright (C) 2019, Red Hat, Inc. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + +1. Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. + +2. Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Signature · 1 line
2.19.1.3.g30247aa5d201
Re: [Qemu-devel] [PATCH v3 3/5] tests: introduce "uefi-test-tools" with the BiosTablesTest UEFI app
Quoted 60 lines · Laszlo Ersek
The "bios-tables-test" program in QEMU's test suite locates the RSD PTR ACPI table in guest RAM, and (chasing pointers to other ACPI tables) performs various sanity checks on the QEMU-generated and firmware-installed tables.
Currently this set of test cases doesn't work with UEFI guests. The ACPI spec defines distinct methods for OSPM to locate the RSD PTR on traditional BIOS vs. UEFI platforms, and the UEFI method is more difficult to implement from the hypervisor side with just raw guest memory access.
Add a UEFI application (to be booted in the UEFI guest) that populates a small, MB-aligned structure in guest RAM. The structure begins with a signature GUID. The hypervisor should loop over all MB-aligned pages in guest RAM until one matches the signature GUID at offset 0, at which point the hypervisor can fetch the RSDP address field(s) from the structure.
QEMU's test logic currently spins on a pre-determined guest address, until that address assumes a magic value. The method described in this patch is conceptually the same ("busy loop until match is found"), except there is no hard-coded address. This plays a lot more nicely with UEFI guest firmware (we'll be able to use the normal page allocation UEFI service). Given the size of
EFI_GUID(16 bytes -- 128 bits), mismatches should be astronomically unlikely. In addition, given the typical guest RAM size for such tests (128 MB), there are 128 locations to check in one iteration of the "outer" loop, which shouldn't introduce an intolerable delay after the guest stores the RSDP address(es), and then the GUID.The GUID that the hypervisor should search for is
AB87A6B1-2034-BDA0-71BD-375007757785
Expressed as a byte array:
{ 0xb1, 0xa6, 0x87, 0xab, 0x34, 0x20, 0xa0, 0xbd, 0x71, 0xbd, 0x37, 0x50, 0x07, 0x75, 0x77, 0x85}
Note that in the patch, we define "gBiosTablesTestGuid" with all bits inverted. This is a simple method to prevent the UEFI binary, which incorporates "gBiosTablesTestGuid", from matching the actual GUID in guest RAM.
The UEFI application is written against the edk2 framework, which was introduced earlier as a git submodule. The next patch will provide build scripts for maintainers.
The source code follows the edk2 coding style, and is licensed under the 2-clause BSDL (in case someone would like to include UefiTestToolsPkg content in a different edk2 platform).
The "UefiTestToolsPkg.dsc" platform description file resolves the used edk2 library classes to instances (= library implementations) such that the UEFI binaries inherit no platform dependencies. They are expected to run on any system that conforms to the UEFI-2.3.1 spec (which was released in 2012). The arch-specific build options are carried over from edk2's ArmVirtPkg and OvmfPkg platforms.
Quoted 7 lines
Cc "Michael S. Tsirkin" <[email protected]>Cc Ard Biesheuvel <[email protected]>Cc Gerd Hoffmann <[email protected]>Cc Igor Mammedov <[email protected]>Cc Philippe Mathieu-Daudé <[email protected]>Cc Shannon Zhao <[email protected]>Signed-off-by Laszlo Ersek <[email protected]>
Reviewed here,
https://www.mail-archive.com/[email protected]/msg592330.html
So:
I simply checked it builds smoothly, no BiosTables test run:
Quoted 419 lines
---
Notes:
v3:
- no changev2:
- no changev1:
- If that's necessary, I'd be glad to be designated as Maintainer or Reviewer in "MAINTAINERS" for "tests/uefi-test-tools/", I just couldn't figure out under what subsystem I should add the magic lines. "MAINTAINERS" needs a Table of Contents! :)tests/uefi-test-tools/UefiTestToolsPkg/UefiTestToolsPkg.dec | 27 ++++ tests/uefi-test-tools/UefiTestToolsPkg/UefiTestToolsPkg.dsc | 69 +++++++++++ tests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest/BiosTablesTest.inf | 41 ++++++ tests/uefi-test-tools/UefiTestToolsPkg/Include/Guid/BiosTablesTest.h | 67 ++++++++++ tests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest/BiosTablesTest.c | 130 ++++++++++++++++++++ tests/uefi-test-tools/LICENSE | 25 ++++ 6 files changed, 359 insertions(+)tests/uefi-test-tools/UefiTestToolsPkg/UefiTestToolsPkg.dec new file +27
-0@@ -0,0 +1,27 @@ +## @file +# edk2 package declaration for the test helper UEFI applications that run in +# guests. +# +# Copyright (C) 2019, Red Hat, Inc. +# +# This program and the accompanying materials are licensed and made available +# under the terms and conditions of the BSD License that accompanies this +# distribution. The full text of the license may be found at +# <http://opensource.org/licenses/bsd-license.php>. +# +# THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, WITHOUT +# WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED. +## + +[Defines] + DEC_SPECIFICATION = 1.27 + PACKAGE_NAME = UefiTestToolsPkg + PACKAGE_GUID = 7b3f1794-0c85-4b27-a536-44dbf0b0669c + PACKAGE_VERSION = 0.1 + +[Includes] + Include + +[Guids] + gBiosTablesTestGuid = {0x5478594e, 0xdfcb, 0x425f, {0x8e, 0x42, 0xc8, 0xaf, 0xf8, 0x8a, 0x88, 0x7a}} +tests/uefi-test-tools/UefiTestToolsPkg/UefiTestToolsPkg.dsc new file +69
-0@@ -0,0 +1,69 @@ +## @file +# edk2 platform description for the test helper UEFI applications that run in +# guests. +# +# Copyright (C) 2019, Red Hat, Inc. +# +# This program and the accompanying materials are licensed and made available +# under the terms and conditions of the BSD License that accompanies this +# distribution. The full text of the license may be found at +# <http://opensource.org/licenses/bsd-license.php>. +# +# THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, WITHOUT +# WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED. +## + +[Defines] + DSC_SPECIFICATION = 1.28 + PLATFORM_GUID = 6750ccc1-8365-49f0-8437-948e516a9f55 + PLATFORM_VERSION = 0.1 + PLATFORM_NAME = UefiTestTools + SKUID_IDENTIFIER = DEFAULT + SUPPORTED_ARCHITECTURES = ARM|AARCH64|IA32|X64 + BUILD_TARGETS = DEBUG + +[BuildOptions.IA32] + GCC:*_*_IA32_CC_FLAGS = -mno-mmx -mno-sse + +[BuildOptions.X64] + GCC:*_*_X64_CC_FLAGS = -mno-mmx -mno-sse + +[BuildOptions.ARM.EDKII.UEFI_APPLICATION] + GCC:*_*_ARM_DLINK_FLAGS = -z common-page-size=0x1000 + +[BuildOptions.AARCH64.EDKII.UEFI_APPLICATION] + GCC:*_*_AARCH64_DLINK_FLAGS = -z common-page-size=0x1000 + +[BuildOptions] + GCC:*_*_*_CC_FLAGS = -D DISABLE_NEW_DEPRECATED_INTERFACES + +[SkuIds] + 0|DEFAULT + +[LibraryClasses] + BaseLib|MdePkg/Library/BaseLib/BaseLib.inf + DebugLib|MdePkg/Library/UefiDebugLibConOut/UefiDebugLibConOut.inf + DebugPrintErrorLevelLib|MdePkg/Library/BaseDebugPrintErrorLevelLib/BaseDebugPrintErrorLevelLib.inf + DevicePathLib|MdePkg/Library/UefiDevicePathLibDevicePathProtocol/UefiDevicePathLibDevicePathProtocol.inf + MemoryAllocationLib|MdePkg/Library/UefiMemoryAllocationLib/UefiMemoryAllocationLib.inf + PcdLib|MdePkg/Library/BasePcdLibNull/BasePcdLibNull.inf + PrintLib|MdePkg/Library/BasePrintLib/BasePrintLib.inf + UefiApplicationEntryPoint|MdePkg/Library/UefiApplicationEntryPoint/UefiApplicationEntryPoint.inf + UefiBootServicesTableLib|MdePkg/Library/UefiBootServicesTableLib/UefiBootServicesTableLib.inf + UefiLib|MdePkg/Library/UefiLib/UefiLib.inf + UefiRuntimeServicesTableLib|MdePkg/Library/UefiRuntimeServicesTableLib/UefiRuntimeServicesTableLib.inf + +[LibraryClasses.ARM, LibraryClasses.AARCH64] + BaseMemoryLib|MdePkg/Library/BaseMemoryLibOptDxe/BaseMemoryLibOptDxe.inf + NULL|ArmPkg/Library/CompilerIntrinsicsLib/CompilerIntrinsicsLib.inf + NULL|MdePkg/Library/BaseStackCheckLib/BaseStackCheckLib.inf + +[LibraryClasses.IA32, LibraryClasses.X64] + BaseMemoryLib|MdePkg/Library/BaseMemoryLibRepStr/BaseMemoryLibRepStr.inf + +[PcdsFixedAtBuild] + gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel|0x8040004F + gEfiMdePkgTokenSpaceGuid.PcdDebugPropertyMask|0x2F + +[Components] + UefiTestToolsPkg/BiosTablesTest/BiosTablesTest.inftests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest/BiosTablesTest.inf new file +41
-0@@ -0,0 +1,41 @@ +## @file +# Populate the BIOS_TABLES_TEST structure. +# +# Copyright (C) 2019, Red Hat, Inc. +# +# This program and the accompanying materials are licensed and made available +# under the terms and conditions of the BSD License that accompanies this +# distribution. The full text of the license may be found at +# <http://opensource.org/licenses/bsd-license.php>. +# +# THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, WITHOUT +# WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED. +## + +[Defines] + INF_VERSION = 1.27 + BASE_NAME = BiosTablesTest + UEFI_SPECIFICATION_VERSION = 2.31 + FILE_GUID = 87f00433-3b7c-45c3-ae78-a56495bd4e62 + MODULE_TYPE = UEFI_APPLICATION + ENTRY_POINT = BiosTablesTestMain + +[Sources] + BiosTablesTest.c + +[LibraryClasses] + BaseLib + BaseMemoryLib + MemoryAllocationLib + UefiApplicationEntryPoint + UefiBootServicesTableLib + UefiLib + +[Guids] + gBiosTablesTestGuid + gEfiAcpi10TableGuid + gEfiAcpi20TableGuid + +[Packages] + MdePkg/MdePkg.dec + UefiTestToolsPkg/UefiTestToolsPkg.dectests/uefi-test-tools/UefiTestToolsPkg/Include/Guid/BiosTablesTest.h new file +67
-0@@ -0,0 +1,67 @@ +/** @file +Expose the address(es) of the ACPI RSD PTR table(s) in a MB-aligned structure +to the hypervisor. + +The hypervisor locates the MB-aligned structure based on the signature GUID +that is at offset 0 in the structure. Once the RSD PTR address(es) are +retrieved, the hypervisor may perform various ACPI checks. + +This feature is a development aid, for supporting ACPI table unit tests in +hypervisors. Do not enable in production builds. + +Copyright (C) 2019, Red Hat, Inc. + +This program and the accompanying materials are licensed and made available +under the terms and conditions of the BSD License that accompanies this +distribution. The full text of the license may be found at +<http://opensource.org/licenses/bsd-license.php>. + +THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, WITHOUT +WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED. +**/ + +#ifndef __BIOS_TABLES_TEST_H__ +#define __BIOS_TABLES_TEST_H__ + +#include <Uefi/UefiBaseType.h> + +#define BIOS_TABLES_TEST_GUID \ + { \ + 0x5478594e, \ + 0xdfcb, \ + 0x425f, \ + { 0x8e, 0x42, 0xc8, 0xaf, 0xf8, 0x8a, 0x88, 0x7a } \ + } + +extern EFI_GUID gBiosTablesTestGuid; + +// +// The following structure must be allocated in Boot Services Data type memory, +// aligned at a 1MB boundary. +// +#pragma pack (1) +typedef struct { +// +// The signature GUID is written to the MB-aligned structure from +// gBiosTablesTestGuid, but with all bits inverted. That's the actual GUID +// value that the hypervisor should look for at each MB boundary, looping +// over all guest RAM pages with that alignment, until a match is found. The +// bit-flipping occurs in order not to store the actual GUID in any UEFI +// executable, which might confuse guest memory analysis. Note that EFI_GUID +// has little endian representation. +// +EFI_GUID InverseSignatureGuid; +// +// The Rsdp10 and Rsdp20 fields may be read when the signature GUID matches. +// Rsdp10 is the guest-physical address of the ACPI 1.0 specification RSD PTR +// table, in 8-byte little endian representation. Rsdp20 is the same, for the +// ACPI 2.0 or later specification RSD PTR table. Each of these fields may be +// zero (independently of the other) if the UEFI System Table does not +// provide the corresponding UEFI Configuration Table. +// +EFI_PHYSICAL_ADDRESS Rsdp10; +EFI_PHYSICAL_ADDRESS Rsdp20; +}BIOS_TABLES_TEST ; +#pragma pack () + +#endif // __BIOS_TABLES_TEST_H__ tests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest/BiosTablesTest.c new file +130
-0@@ -0,0 +1,130 @@ +/** @file +Populate the BIOS_TABLES_TEST structure. + +Copyright (C) 2019, Red Hat, Inc. + +This program and the accompanying materials are licensed and made available +under the terms and conditions of the BSD License that accompanies this +distribution. The full text of the license may be found at +<http://opensource.org/licenses/bsd-license.php>. + +THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, WITHOUT +WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED. +**/ + +#include <Guid/Acpi.h> +#include <Guid/BiosTablesTest.h> +#include <Library/BaseLib.h> +#include <Library/BaseMemoryLib.h> +#include <Library/MemoryAllocationLib.h> +#include <Library/UefiBootServicesTableLib.h> +#include <Library/UefiLib.h> + +/** +Wait for a keypress with a message that the application is about to exit. +**/ +STATIC +VOID +WaitForExitKeyPress ( +VOID + ) +{ +EFI_STATUS Status; +UINTN Idx; +EFI_INPUT_KEY Key; + +if (gST->ConIn ==NULL ) { +return ; + } +AsciiPrint ("%a: press any key to exit\n" , gEfiCallerBaseName); + Status = gBS->WaitForEvent (1 , &gST->ConIn->WaitForKey, &Idx); +if (EFI_ERROR (Status)) { +return ; + } + gST->ConIn->ReadKeyStroke (gST->ConIn, &Key); +} + +EFI_STATUS +EFIAPI +BiosTablesTestMain ( + IN EFI_HANDLE ImageHandle, + IN EFI_SYSTEM_TABLE *SystemTable + ) +{ +VOID *Pages; +volatile BIOS_TABLES_TEST *BiosTablesTest; + CONST VOID *Rsdp10; + CONST VOID *Rsdp20; + CONST EFI_CONFIGURATION_TABLE *ConfigTable; + CONST EFI_CONFIGURATION_TABLE *ConfigTablesEnd; +volatile EFI_GUID *InverseSignature; +UINTN Idx; + + Pages =AllocateAlignedPages (EFI_SIZE_TO_PAGES (sizeof *BiosTablesTest), +SIZE_1MB ); +if (Pages ==NULL ) { +AsciiErrorPrint ("%a: AllocateAlignedPages() failed\n" , + gEfiCallerBaseName); +// +// Assuming the application was launched by the boot manager as a boot +// loader, exiting with error will cause the boot manager to proceed with +// the remaining boot options. If there are no other boot options, the boot +// manager menu will be pulled up. Give the user a chance to read the error +// message. +// +WaitForExitKeyPress (); +return EFI_OUT_OF_RESOURCES ; + } + +// +// Locate both gEfiAcpi10TableGuid and gEfiAcpi20TableGuid config tables in +// one go. +// + Rsdp10 =NULL ; + Rsdp20 =NULL ; + ConfigTable = gST->ConfigurationTable; + ConfigTablesEnd = gST->ConfigurationTable + gST->NumberOfTableEntries; +while ((Rsdp10 ==NULL || Rsdp20 ==NULL ) && ConfigTable < ConfigTablesEnd) { +if (CompareGuid (&ConfigTable->VendorGuid, &gEfiAcpi10TableGuid)) { + Rsdp10 = ConfigTable->VendorTable; + }else if (CompareGuid (&ConfigTable->VendorGuid, &gEfiAcpi20TableGuid)) { + Rsdp20 = ConfigTable->VendorTable; + } + ++ConfigTable; + } + +AsciiPrint ("%a: BiosTablesTest=%p Rsdp10=%p Rsdp20=%p\n" , + gEfiCallerBaseName, Pages, Rsdp10, Rsdp20); + +// +// Store the RSD PTR address(es) first, then the signature second. +// + BiosTablesTest = Pages; + BiosTablesTest->Rsdp10 = (UINTN )Rsdp10; + BiosTablesTest->Rsdp20 = (UINTN )Rsdp20; + +MemoryFence (); + + InverseSignature = &BiosTablesTest->InverseSignatureGuid; + InverseSignature->Data1 = gBiosTablesTestGuid.Data1; + InverseSignature->Data1 ^=MAX_UINT32 ; + InverseSignature->Data2 = gBiosTablesTestGuid.Data2; + InverseSignature->Data2 ^=MAX_UINT16 ; + InverseSignature->Data3 = gBiosTablesTestGuid.Data3; + InverseSignature->Data3 ^=MAX_UINT16 ; +for (Idx =0 ; Idx <sizeof InverseSignature->Data4; ++Idx) { + InverseSignature->Data4[Idx] = gBiosTablesTestGuid.Data4[Idx]; + InverseSignature->Data4[Idx] ^=MAX_UINT8 ; + } + +// +// The wait below has dual purpose. First, it blocks the application without +// wasting VCPU cycles while the hypervisor is scanning guest RAM. Second, +// assuming the application was launched by the boot manager as a boot +// loader, exiting the app with success causes the boot manager to pull up +// the boot manager menu at once (regardless of other boot options); the wait +// gives the user a chance to read the info printed above. +// +WaitForExitKeyPress (); +return EFI_SUCCESS ; +}tests/uefi-test-tools/LICENSE new file +25
-0@@ -0,0 +1,25 @@ +All the files in this directory and subdirectories are released under the +2-Clause BSD License (see header in each file). + +Copyright (C) 2019, Red Hat, Inc. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + +1. Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. + +2. Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
PATCH v3 4/5 tests/uefi-test-tools: add build scripts
In reply to Laszlo ErsekIntroduce the following build scripts under "tests/uefi-test-tools":
* "build.sh" builds a single module (a UEFI application) from UefiTestToolsPkg, for a single QEMU emulation target.
"build.sh" relies on cross-compilers when the emulation target and the build host architecture don't match. The cross-compiler prefix is computed according to a fixed, Linux-specific pattern. No attempt is made to copy or reimplement the GNU Make magic from "qemu/roms/Makefile" for cross-compiler prefix determination. The reason is that the build host OSes that are officially supported by edk2, and those that are supported by QEMU, intersect only in Linux. (Note that the UNIXGCC toolchain is being removed from edk2,
<https://bugzilla.tianocore.org/show_bug.cgi?id=1377>.)
* "Makefile" currently builds the "UefiTestToolsPkg/BiosTablesTest" application, for arm, aarch64, i386, and x86_64, with the help of "build.sh".
"Makefile" turns each resultant UEFI executable into a UEFI-bootable, qcow2-compressed ISO image. The ISO images are output as "tests/data/uefi-boot-images/bios-tables-test.<TARGET>.iso.qcow2".
Each ISO image should be passed to QEMU as follows:
-drive id=boot-cd,if=none,readonly,format=qcow2,file=$ISO \
-device virtio-scsi-pci,id=scsi0 \
-device scsi-cd,drive=boot-cd,bus=scsi0.0,bootindex=0 \ "Makefile" assumes that "mkdosfs", "mtools", and "genisoimage" are present.
---
Notes:
v3:
- explicitly mark the "./build.sh" recipe as recursive, with the "+" indicator; document it in a comment [Phil]
- pick up R-b, T-b [Phil]
v2:
- add the .NOTPARALLEL target [Phil, help-make, edk2-devel]
tests/uefi-test-tools/Makefile | 106 ++++++++++++++ tests/uefi-test-tools/.gitignore | 3 + tests/uefi-test-tools/build.sh | 145 ++++++++++++++++++++ 3 files changed, 254 insertions(+)
tests/uefi-test-tools/Makefile new file +106 -0
@@ -0,0 +1,106 @@ +# Makefile for the test helper UEFI applications that run in guests. +# +# Copyright (C) 2019, Red Hat, Inc. +# +# This program and the accompanying materials are licensed and made available +# under the terms and conditions of the BSD License that accompanies this +# distribution. The full text of the license may be found at +# <http://opensource.org/licenses/bsd-license.php>. +# +# THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, WITHOUT +# WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED. + +edk2_dir := ../../roms/edk2 +images_dir := ../data/uefi-boot-images +emulation_targets := arm aarch64 i386 x86_64 +uefi_binaries := bios-tables-test +intermediate_suffixes := .efi .fat .iso.raw + +images: $(foreach binary,$(uefi_binaries), \ + $(foreach target,$(emulation_targets), \ + $(images_dir)/$(binary).$(target).iso.qcow2)) + +# Preserve all intermediate targets if the build succeeds. +# - Intermediate targets help with development & debugging. +# - Preserving intermediate targets also keeps spurious changes out of the +# final build products, in case the user re-runs "make" without any changes +# to the UEFI source code. Normally, the intermediate files would have been +# removed by the last "make" invocation, hence the re-run would rebuild them +# from the unchanged UEFI sources. Unfortunately, the "mkdosfs" and +# "genisoimage" utilities embed timestamp-based information in their outputs, +# which causes git to report differences for the tracked qcow2 ISO images. +.SECONDARY: $(foreach binary,$(uefi_binaries), \ + $(foreach target,$(emulation_targets), \ + $(foreach suffix,$(intermediate_suffixes), \ + Build/$(binary).$(target)$(suffix)))) + +# In the pattern rules below, the stem (%, $*) stands for +# "$(binary).$(target)". + +# Convert the raw ISO image to a qcow2 one, enabling compression, and using a +# small cluster size. This allows for small binary files under git control, +# hence for small binary patches. +$(images_dir)/%.iso.qcow2: Build/%.iso.raw + mkdir -p -- $(images_dir) + $${QTEST_QEMU_IMG:-qemu-img} convert -f raw -O qcow2 -c \ + -o cluster_size=512 -- $< $@ + +# Embed the "UEFI system partition" into an ISO9660 file system as an ElTorito +# boot image. +Build/%.iso.raw: Build/%.fat + genisoimage -input-charset ASCII -efi-boot $(notdir $<) -no-emul-boot \ + -quiet -o $@ -- $< + +# Define chained macros in order to map QEMU system emulation targets to +# *short* UEFI architecture identifiers. Periods are allowed in, and ultimately +# stripped from, the argument. +map_arm_to_uefi = $(subst arm,ARM,$(1)) +map_aarch64_to_uefi = $(subst aarch64,AA64,$(call map_arm_to_uefi,$(1))) +map_i386_to_uefi = $(subst i386,IA32,$(call map_aarch64_to_uefi,$(1))) +map_x86_64_to_uefi = $(subst x86_64,X64,$(call map_i386_to_uefi,$(1))) +map_to_uefi = $(subst .,,$(call map_x86_64_to_uefi,$(1))) + +# Format a "UEFI system partition", using the UEFI binary as the default boot +# loader. Add 10% size for filesystem metadata, round up to the next KB, and +# make sure the size is large enough for a FAT filesystem. Name the filesystem +# after the UEFI binary. (Excess characters are automatically dropped from the +# filesystem label.) +Build/%.fat: Build/%.efi + rm -f -- $@ + uefi_bin_b=$$(stat --format=%s -- $<) && \ + uefi_fat_kb=$$(( (uefi_bin_b * 11 / 10 + 1023) / 1024 )) && \ + uefi_fat_kb=$$(( uefi_fat_kb >= 64 ? uefi_fat_kb : 64 )) && \ + mkdosfs -C $@ -n $(basename $(@F)) -- $$uefi_fat_kb + MTOOLS_SKIP_CHECK=1 mmd -i $@ ::EFI + MTOOLS_SKIP_CHECK=1 mmd -i $@ ::EFI/BOOT + MTOOLS_SKIP_CHECK=1 mcopy -i $@ -- $< \ + ::EFI/BOOT/BOOT$(call map_to_uefi,$(suffix $*)).EFI + +# In the pattern rules below, the stem (%, $*) stands for "$(target)" only. The +# association between the UEFI binary (such as "bios-tables-test") and the +# component name from the edk2 platform DSC file (such as "BiosTablesTest") is +# explicit in each rule. + +# "build.sh" invokes the "build" utility of edk2 BaseTools. In any given edk2 +# workspace, at most one "build" instance may be operating at a time. Therefore +# we must serialize the rebuilding of targets in this Makefile. +.NOTPARALLEL: + +# In turn, the "build" utility of edk2 BaseTools invokes another "make". +# Although the outer "make" process advertizes its job server to all child +# processes via MAKEFLAGS in the environment, the outer "make" closes the job +# server file descriptors (exposed in MAKEFLAGS) before executing a recipe -- +# unless the recipe is recognized as a recursive "make" recipe. Recipes that +# call $(MAKE) are classified automatically as recursive; for "build.sh" below, +# we must mark the recipe manually as recursive, by using the "+" indicator. +# This way, when the inner "make" starts a parallel build of the target edk2 +# module, it can communicate with the outer "make"'s job server. +Build/bios-tables-test.%.efi: build-edk2-tools + +./build.sh $(edk2_dir) BiosTablesTest $* $@ + +build-edk2-tools: + $(MAKE) -C $(edk2_dir)/BaseTools + +clean: + rm -rf Build Conf log + $(MAKE) -C $(edk2_dir)/BaseTools clean
tests/uefi-test-tools/.gitignore new file +3 -0
@@ -0,0 +1,3 @@ +Build +Conf +log
tests/uefi-test-tools/build.sh new file +145 -0
145 lines not on this page · raw message
Signature · 1 line
2.19.1.3.g30247aa5d201
Re: [Qemu-devel] [PATCH v3 4/5] tests/uefi-test-tools: add build scripts
Quoted 150 lines · Laszlo Ersek
Introduce the following build scripts under "tests/uefi-test-tools":
* "
build.sh" builds a single module (a UEFI application) from UefiTestToolsPkg, for a single QEMU emulation target."
build.sh" relies on cross-compilers when the emulation target and the build host architecture don't match. The cross-compiler prefix is computed according to a fixed, Linux-specific pattern. No attempt is made to copy or reimplement the GNU Make magic from "qemu/roms/Makefile" for cross-compiler prefix determination. The reason is that the build host OSes that are officially supported by edk2, and those that are supported by QEMU, intersect only in Linux. (Note that the UNIXGCC toolchain is being removed from edk2,
<https://bugzilla.tianocore.org/show_bug.cgi?id=1377>.)* "
Makefile" currently builds the "UefiTestToolsPkg/BiosTablesTest" application, for arm, aarch64, i386, andx86_64, with the help of "build.sh"."
Makefile" turns each resultant UEFI executable into a UEFI-bootable, qcow2-compressed ISO image. The ISO images are output as "tests/data/uefi-boot-images/bios-tables-test.<TARGET>.iso.qcow2".Each ISO image should be passed to QEMU as follows:
-drive id=boot-cd,if=none,readonly,format=qcow2,file=$ISO \ -device virtio-scsi-pci,id=scsi0 \ -device scsi-cd,drive=boot-cd,bus=scsi0.0,bootindex=0 \"
Makefile" assumes that "mkdosfs", "mtools", and "genisoimage" are present.Cc "Michael S. Tsirkin" <[email protected]>Cc Ard Biesheuvel <[email protected]>Cc Gerd Hoffmann <[email protected]>Cc Igor Mammedov <[email protected]>Cc Philippe Mathieu-Daudé <[email protected]>Cc Shannon Zhao <[email protected]>Signed-off-by Laszlo Ersek <[email protected]>Reviewed-by Philippe Mathieu-Daudé <[email protected]>---
Notes:
v3:
- explicitly mark the "./build.sh" recipe as recursive, with the "+" indicator; document it in a comment [Phil]
- pick up R-b, T-b [Phil]v2:
- add the .NOTPARALLEL target [Phil, help-make, edk2-devel]tests/uefi-test-tools/Makefile | 106 ++++++++++++++ tests/uefi-test-tools/.gitignore | 3 + tests/uefi-test-tools/build.sh | 145 ++++++++++++++++++++ 3 files changed, 254 insertions(+)tests/uefi-test-tools/Makefile new file +87
-0@@ -0,0 +1,106 @@ +# Makefile for the test helper UEFI applications that run in guests. +# +# Copyright (C) 2019, Red Hat, Inc. +# +# This program and the accompanying materials are licensed and made available +# under the terms and conditions of the BSD License that accompanies this +# distribution. The full text of the license may be found at +# <http://opensource.org/licenses/bsd-license.php>. +# +# THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, WITHOUT +# WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED. + +edk2_dir := ../../roms/edk2 +images_dir := ../data/uefi-boot-images +emulation_targets := arm aarch64 i386 x86_64 +uefi_binaries := bios-tables-test +intermediate_suffixes := .efi .fat .iso.raw + +images: $(foreach binary,$(uefi_binaries), \ + $(foreach target,$(emulation_targets), \ + $(images_dir)/$(binary).$(target).iso.qcow2)) + +# Preserve all intermediate targets if the build succeeds. +# - Intermediate targets help with development & debugging. +# - Preserving intermediate targets also keeps spurious changes out of the +# final build products, in case the user re-runs "make" without any changes +# to the UEFI source code. Normally, the intermediate files would have been +# removed by the last "make" invocation, hence the re-run would rebuild them +# from the unchanged UEFI sources. Unfortunately, the "mkdosfs" and +# "genisoimage" utilities embed timestamp-based information in their outputs, +# which causes git to report differences for the tracked qcow2 ISO images. +.SECONDARY: $(foreach binary,$(uefi_binaries), \ + $(foreach target,$(emulation_targets), \ + $(foreach suffix,$(intermediate_suffixes), \ + Build/$(binary).$(target)$(suffix)))) + +# In the pattern rules below, the stem (%, $*) stands for +# "$(binary).$(target)". + +# Convert the raw ISO image to a qcow2 one, enabling compression, and using a +# small cluster size. This allows for small binary files under git control, +# hence for small binary patches. +$(images_dir)/%.iso.qcow2: Build/%.iso.raw + mkdir -p -- $(images_dir) + $${QTEST_QEMU_IMG:-qemu-img} convert -f raw -O qcow2 -c \ + -o cluster_size=512 -- $< $@ + +# Embed the "UEFI system partition" into an ISO9660 file system as an ElTorito +# boot image. +Build/%.iso.raw: Build/%.fat + genisoimage -input-charset ASCII -efi-boot $(notdir $<) -no-emul-boot \ + -quiet -o $@ -- $< + +# Define chained macros in order to map QEMU system emulation targets to +# *short* UEFI architecture identifiers. Periods are allowed in, and ultimately +# stripped from, the argument. +map_arm_to_uefi = $(subst arm,ARM,$(1)) +map_aarch64_to_uefi = $(subst aarch64,AA64,$(call map_arm_to_uefi,$(1))) +map_i386_to_uefi = $(subst i386,IA32,$(call map_aarch64_to_uefi,$(1))) +map_x86_64_to_uefi = $(subst x86_64,X64,$(call map_i386_to_uefi,$(1))) +map_to_uefi = $(subst .,,$(call map_x86_64_to_uefi,$(1))) + +# Format a "UEFI system partition", using the UEFI binary as the default boot +# loader. Add 10% size for filesystem metadata, round up to the next KB, and +# make sure the size is large enough for a FAT filesystem. Name the filesystem +# after the UEFI binary. (Excess characters are automatically dropped from the +# filesystem label.) +Build/%.fat: Build/%.efi + rm -f -- $@ + uefi_bin_b=$$(stat --format=%s -- $<) && \ + uefi_fat_kb=$$(( (uefi_bin_b * 11 / 10 + 1023) / 1024 )) && \ + uefi_fat_kb=$$(( uefi_fat_kb >= 64 ? uefi_fat_kb : 64 )) && \ + mkdosfs -C $@ -n $(basename $(@F)) -- $$uefi_fat_kb + MTOOLS_SKIP_CHECK=1 mmd -i $@ ::EFI + MTOOLS_SKIP_CHECK=1 mmd -i $@ ::EFI/BOOT + MTOOLS_SKIP_CHECK=1 mcopy -i $@ -- $< \ + ::EFI/BOOT/BOOT$(call map_to_uefi,$(suffix $*)).EFI + +# In the pattern rules below, the stem (%, $*) stands for "$(target)" only. The +# association between the UEFI binary (such as "bios-tables-test") and the +# component name from the edk2 platform DSC file (such as "BiosTablesTest") is +# explicit in each rule. + +# "build.sh" invokes the "build" utility of edk2 BaseTools. In any given edk2 +# workspace, at most one "build" instance may be operating at a time. Therefore +# we must serialize the rebuilding of targets in this Makefile. +.NOTPARALLEL:
tests/uefi-test-tools/Makefile line 97
Quoted 10 lines
tests/uefi-test-tools/Makefile new file +10
-0+ +# In turn, the "build" utility of edk2 BaseTools invokes another "make". +# Although the outer "make" process advertizes its job server to all child +# processes via MAKEFLAGS in the environment, the outer "make" closes the job +# server file descriptors (exposed in MAKEFLAGS) before executing a recipe -- +# unless the recipe is recognized as a recursive "make" recipe. Recipes that +# call $(MAKE) are classified automatically as recursive; for "build.sh" below, +# we must mark the recipe manually as recursive, by using the "+" indicator. +# This way, when the inner "make" starts a parallel build of the target edk2 +# module, it can communicate with the outer "make"'s job server.
Clearer than the make 'POSIX jobserver interaction' man page :)
Quoted 170 lines
tests/uefi-test-tools/Makefile new file +9
-0+Build/bios-tables-test.%.efi: build-edk2-tools + +./build.sh $(edk2_dir) BiosTablesTest $* $@ + +build-edk2-tools: + $(MAKE) -C $(edk2_dir)/BaseTools + +clean: + rm -rf Build Conf log + $(MAKE) -C $(edk2_dir)/BaseTools cleantests/uefi-test-tools/.gitignore new file +3
-0@@ -0,0 +1,3 @@ +Build +Conf +logtests/uefi-test-tools/build.sh new file +145
-0@@ -0,0 +1,145 @@ +#!/bin/bash + +# Build script that determines the edk2 toolchain to use, invokes the edk2 +# "build" utility, and copies the built UEFI binary to the requested location. +# +# Copyright (C) 2019, Red Hat, Inc. +# +# This program and the accompanying materials are licensed and made available +# under the terms and conditions of the BSD License that accompanies this +# distribution. The full text of the license may be found at +# <http://opensource.org/licenses/bsd-license.php>. +# +# THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, WITHOUT +# WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED. + +set -e -u -C + +# Save the command line arguments. We need to reset $# to 0 before sourcing +# "edksetup.sh", as it will inherit $@. +program_name=$(basename -- "$0") +edk2_dir=$1 +dsc_component=$2 +emulation_target=$3 +uefi_binary=$4 +shift 4 + +# Set up the environment for edk2 building. +export PACKAGES_PATH=$(realpath -- "$edk2_dir") +export WORKSPACE=$PWD +mkdir -p Conf + +# Source "edksetup.sh" carefully. +set +e +u +C +source "$PACKAGES_PATH/edksetup.sh" +ret=$? +set -e -u -C +if [ $ret -ne 0 ]; then + exit $ret +fi + +# Map the QEMU system emulation target to the following types of architecture +# identifiers: +# - edk2, +# - gcc cross-compilation. +# Cover only those targets that are supported by the UEFI spec and edk2. +case "$emulation_target" in + (arm) + edk2_arch=ARM + gcc_arch=arm + ;; + (aarch64) + edk2_arch=AARCH64 + gcc_arch=aarch64 + ;; + (i386) + edk2_arch=IA32 + gcc_arch=i686 + ;; + (x86_64) + edk2_arch=X64 + gcc_arch=x86_64 + ;; + (*) + printf '%s: unknown/unsupported QEMU system emulation target "%s"\n' \ + "$program_name" "$emulation_target" >&2 + exit 1 + ;; +esac + +# Check if cross-compilation is needed. +host_arch=$(uname -m) +if [ "$gcc_arch" == "$host_arch" ] || + ( [ "$gcc_arch" == i686 ] && [ "$host_arch" == x86_64 ] ); then + cross_prefix= +else + cross_prefix=${gcc_arch}-linux-gnu- +fi + +# Expose cross_prefix (which is possibly empty) to the edk2 tools. While at it, +# determine the suitable edk2 toolchain as well. +# - For ARM and AARCH64, edk2 only offers the GCC5 toolchain tag, which covers +# the gcc-5+ releases. +# - For IA32 and X64, edk2 offers the GCC44 through GCC49 toolchain tags, in +# addition to GCC5. Unfortunately, the mapping between the toolchain tags and +# the actual gcc releases isn't entirely trivial. Run "git-blame" on +# "OvmfPkg/build.sh" in edk2 for more information. +# And, because the above is too simple, we have to assign cross_prefix to an +# edk2 build variable that is specific to both the toolchain tag and the target +# architecture. +case "$edk2_arch" in + (ARM) + edk2_toolchain=GCC5 + export GCC5_ARM_PREFIX=$cross_prefix + ;; + (AARCH64) + edk2_toolchain=GCC5 + export GCC5_AARCH64_PREFIX=$cross_prefix + ;; + (IA32|X64) + gcc_version=$("${cross_prefix}gcc" -v 2>&1 | tail -1 | awk '{print $3}') + case "$gcc_version" in + ([1-3].*|4.[0-3].*) + printf '%s: unsupported gcc version "%s"\n' \ + "$program_name" "$gcc_version" >&2 + exit 1 + ;; + (4.4.*) + edk2_toolchain=GCC44 + ;; + (4.5.*) + edk2_toolchain=GCC45 + ;; + (4.6.*) + edk2_toolchain=GCC46 + ;; + (4.7.*) + edk2_toolchain=GCC47 + ;; + (4.8.*) + edk2_toolchain=GCC48 + ;; + (4.9.*|6.[0-2].*) + edk2_toolchain=GCC49 + ;; + (*) + edk2_toolchain=GCC5 + ;; + esac + eval "export ${edk2_toolchain}_BIN=\$cross_prefix" + ;; +esac + +# Build the UEFI binary +mkdir -p log +build \ + --arch="$edk2_arch" \ + --buildtarget=DEBUG \ + --platform=UefiTestToolsPkg/UefiTestToolsPkg.dsc \ + --tagname="$edk2_toolchain" \ + --module="UefiTestToolsPkg/$dsc_component/$dsc_component.inf" \ + --log="log/$dsc_component.$edk2_arch.log" \ + --report-file="log/$dsc_component.$edk2_arch.report" +cp -a -- \ + "Build/UefiTestTools/DEBUG_${edk2_toolchain}/$edk2_arch/$dsc_component.efi" \ + "$uefi_binary"
Re: [Qemu-devel] [PATCH v3 4/5] tests/uefi-test-tools: add build scripts
In reply to Laszlo ErsekQuoted 152 lines · Laszlo Ersek
Introduce the following build scripts under "tests/uefi-test-tools":
* "
build.sh" builds a single module (a UEFI application) from UefiTestToolsPkg, for a single QEMU emulation target."
build.sh" relies on cross-compilers when the emulation target and the build host architecture don't match. The cross-compiler prefix is computed according to a fixed, Linux-specific pattern. No attempt is made to copy or reimplement the GNU Make magic from "qemu/roms/Makefile" for cross-compiler prefix determination. The reason is that the build host OSes that are officially supported by edk2, and those that are supported by QEMU, intersect only in Linux. (Note that the UNIXGCC toolchain is being removed from edk2,
<https://bugzilla.tianocore.org/show_bug.cgi?id=1377>.)* "
Makefile" currently builds the "UefiTestToolsPkg/BiosTablesTest" application, for arm, aarch64, i386, andx86_64, with the help of "build.sh"."
Makefile" turns each resultant UEFI executable into a UEFI-bootable, qcow2-compressed ISO image. The ISO images are output as "tests/data/uefi-boot-images/bios-tables-test.<TARGET>.iso.qcow2".Each ISO image should be passed to QEMU as follows:
-drive id=boot-cd,if=none,readonly,format=qcow2,file=$ISO \ -device virtio-scsi-pci,id=scsi0 \ -device scsi-cd,drive=boot-cd,bus=scsi0.0,bootindex=0 \"
Makefile" assumes that "mkdosfs", "mtools", and "genisoimage" are present.Cc "Michael S. Tsirkin" <[email protected]>Cc Ard Biesheuvel <[email protected]>Cc Gerd Hoffmann <[email protected]>Cc Igor Mammedov <[email protected]>Cc Philippe Mathieu-Daudé <[email protected]>Cc Shannon Zhao <[email protected]>Signed-off-by Laszlo Ersek <[email protected]>Reviewed-by Philippe Mathieu-Daudé <[email protected]>---
Notes:
v3:
- explicitly mark the "./build.sh" recipe as recursive, with the "+" indicator; document it in a comment [Phil]
- pick up R-b, T-b [Phil]v2:
- add the .NOTPARALLEL target [Phil, help-make, edk2-devel]tests/uefi-test-tools/Makefile | 106 ++++++++++++++ tests/uefi-test-tools/.gitignore | 3 + tests/uefi-test-tools/build.sh | 145 ++++++++++++++++++++ 3 files changed, 254 insertions(+)tests/uefi-test-tools/Makefile new file +89
-0@@ -0,0 +1,106 @@ +# Makefile for the test helper UEFI applications that run in guests. +# +# Copyright (C) 2019, Red Hat, Inc. +# +# This program and the accompanying materials are licensed and made available +# under the terms and conditions of the BSD License that accompanies this +# distribution. The full text of the license may be found at +# <http://opensource.org/licenses/bsd-license.php>. +# +# THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, WITHOUT +# WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED. + +edk2_dir := ../../roms/edk2 +images_dir := ../data/uefi-boot-images +emulation_targets := arm aarch64 i386 x86_64 +uefi_binaries := bios-tables-test +intermediate_suffixes := .efi .fat .iso.raw + +images: $(foreach binary,$(uefi_binaries), \ + $(foreach target,$(emulation_targets), \ + $(images_dir)/$(binary).$(target).iso.qcow2)) + +# Preserve all intermediate targets if the build succeeds. +# - Intermediate targets help with development & debugging. +# - Preserving intermediate targets also keeps spurious changes out of the +# final build products, in case the user re-runs "make" without any changes +# to the UEFI source code. Normally, the intermediate files would have been +# removed by the last "make" invocation, hence the re-run would rebuild them +# from the unchanged UEFI sources. Unfortunately, the "mkdosfs" and +# "genisoimage" utilities embed timestamp-based information in their outputs, +# which causes git to report differences for the tracked qcow2 ISO images. +.SECONDARY: $(foreach binary,$(uefi_binaries), \ + $(foreach target,$(emulation_targets), \ + $(foreach suffix,$(intermediate_suffixes), \ + Build/$(binary).$(target)$(suffix)))) + +# In the pattern rules below, the stem (%, $*) stands for +# "$(binary).$(target)". + +# Convert the raw ISO image to a qcow2 one, enabling compression, and using a +# small cluster size. This allows for small binary files under git control, +# hence for small binary patches. +$(images_dir)/%.iso.qcow2: Build/%.iso.raw + mkdir -p -- $(images_dir) + $${QTEST_QEMU_IMG:-qemu-img} convert -f raw -O qcow2 -c \ + -o cluster_size=512 -- $< $@ + +# Embed the "UEFI system partition" into an ISO9660 file system as an ElTorito +# boot image. +Build/%.iso.raw: Build/%.fat + genisoimage -input-charset ASCII -efi-boot $(notdir $<) -no-emul-boot \ + -quiet -o $@ -- $< + +# Define chained macros in order to map QEMU system emulation targets to +# *short* UEFI architecture identifiers. Periods are allowed in, and ultimately +# stripped from, the argument. +map_arm_to_uefi = $(subst arm,ARM,$(1)) +map_aarch64_to_uefi = $(subst aarch64,AA64,$(call map_arm_to_uefi,$(1))) +map_i386_to_uefi = $(subst i386,IA32,$(call map_aarch64_to_uefi,$(1))) +map_x86_64_to_uefi = $(subst x86_64,X64,$(call map_i386_to_uefi,$(1))) +map_to_uefi = $(subst .,,$(call map_x86_64_to_uefi,$(1))) + +# Format a "UEFI system partition", using the UEFI binary as the default boot +# loader. Add 10% size for filesystem metadata, round up to the next KB, and +# make sure the size is large enough for a FAT filesystem. Name the filesystem +# after the UEFI binary. (Excess characters are automatically dropped from the +# filesystem label.) +Build/%.fat: Build/%.efi + rm -f -- $@ + uefi_bin_b=$$(stat --format=%s -- $<) && \ + uefi_fat_kb=$$(( (uefi_bin_b * 11 / 10 + 1023) / 1024 )) && \ + uefi_fat_kb=$$(( uefi_fat_kb >= 64 ? uefi_fat_kb : 64 )) && \ + mkdosfs -C $@ -n $(basename $(@F)) -- $$uefi_fat_kb + MTOOLS_SKIP_CHECK=1 mmd -i $@ ::EFI + MTOOLS_SKIP_CHECK=1 mmd -i $@ ::EFI/BOOT + MTOOLS_SKIP_CHECK=1 mcopy -i $@ -- $< \ + ::EFI/BOOT/BOOT$(call map_to_uefi,$(suffix $*)).EFI + +# In the pattern rules below, the stem (%, $*) stands for "$(target)" only. The +# association between the UEFI binary (such as "bios-tables-test") and the +# component name from the edk2 platform DSC file (such as "BiosTablesTest") is +# explicit in each rule. + +# "build.sh" invokes the "build" utility of edk2 BaseTools. In any given edk2 +# workspace, at most one "build" instance may be operating at a time. Therefore +# we must serialize the rebuilding of targets in this Makefile. +.NOTPARALLEL: + +# In turn, the "build" utility of edk2 BaseTools invokes another "make".
tests/uefi-test-tools/Makefile line 99
Quoted 10 lines
tests/uefi-test-tools/Makefile new file +10
-0+# Although the outer "make" process advertizes its job server to all child +# processes via MAKEFLAGS in the environment, the outer "make" closes the job +# server file descriptors (exposed in MAKEFLAGS) before executing a recipe -- +# unless the recipe is recognized as a recursive "make" recipe. Recipes that +# call $(MAKE) are classified automatically as recursive; for "build.sh" below, +# we must mark the recipe manually as recursive, by using the "+" indicator. +# This way, when the inner "make" starts a parallel build of the target edk2 +# module, it can communicate with the outer "make"'s job server. +Build/bios-tables-test.%.efi: build-edk2-tools + +./build.sh $(edk2_dir) BiosTablesTest $* $@
Does this actually work with an out of tree build? Shouldn't this be SRC_PATH/tests/uefi-test-tools/ ?
Quoted 170 lines
tests/uefi-test-tools/Makefile new file +7
-0+ +build-edk2-tools: + $(MAKE) -C $(edk2_dir)/BaseTools + +clean: + rm -rf Build Conf log + $(MAKE) -C $(edk2_dir)/BaseTools cleantests/uefi-test-tools/.gitignore new file +3
-0@@ -0,0 +1,3 @@ +Build +Conf +logtests/uefi-test-tools/build.sh new file +145
-0@@ -0,0 +1,145 @@ +#!/bin/bash + +# Build script that determines the edk2 toolchain to use, invokes the edk2 +# "build" utility, and copies the built UEFI binary to the requested location. +# +# Copyright (C) 2019, Red Hat, Inc. +# +# This program and the accompanying materials are licensed and made available +# under the terms and conditions of the BSD License that accompanies this +# distribution. The full text of the license may be found at +# <http://opensource.org/licenses/bsd-license.php>. +# +# THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, WITHOUT +# WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED. + +set -e -u -C + +# Save the command line arguments. We need to reset $# to 0 before sourcing +# "edksetup.sh", as it will inherit $@. +program_name=$(basename -- "$0") +edk2_dir=$1 +dsc_component=$2 +emulation_target=$3 +uefi_binary=$4 +shift 4 + +# Set up the environment for edk2 building. +export PACKAGES_PATH=$(realpath -- "$edk2_dir") +export WORKSPACE=$PWD +mkdir -p Conf + +# Source "edksetup.sh" carefully. +set +e +u +C +source "$PACKAGES_PATH/edksetup.sh" +ret=$? +set -e -u -C +if [ $ret -ne 0 ]; then + exit $ret +fi + +# Map the QEMU system emulation target to the following types of architecture +# identifiers: +# - edk2, +# - gcc cross-compilation. +# Cover only those targets that are supported by the UEFI spec and edk2. +case "$emulation_target" in + (arm) + edk2_arch=ARM + gcc_arch=arm + ;; + (aarch64) + edk2_arch=AARCH64 + gcc_arch=aarch64 + ;; + (i386) + edk2_arch=IA32 + gcc_arch=i686 + ;; + (x86_64) + edk2_arch=X64 + gcc_arch=x86_64 + ;; + (*) + printf '%s: unknown/unsupported QEMU system emulation target "%s"\n' \ + "$program_name" "$emulation_target" >&2 + exit 1 + ;; +esac + +# Check if cross-compilation is needed. +host_arch=$(uname -m) +if [ "$gcc_arch" == "$host_arch" ] || + ( [ "$gcc_arch" == i686 ] && [ "$host_arch" == x86_64 ] ); then + cross_prefix= +else + cross_prefix=${gcc_arch}-linux-gnu- +fi + +# Expose cross_prefix (which is possibly empty) to the edk2 tools. While at it, +# determine the suitable edk2 toolchain as well. +# - For ARM and AARCH64, edk2 only offers the GCC5 toolchain tag, which covers +# the gcc-5+ releases. +# - For IA32 and X64, edk2 offers the GCC44 through GCC49 toolchain tags, in +# addition to GCC5. Unfortunately, the mapping between the toolchain tags and +# the actual gcc releases isn't entirely trivial. Run "git-blame" on +# "OvmfPkg/build.sh" in edk2 for more information. +# And, because the above is too simple, we have to assign cross_prefix to an +# edk2 build variable that is specific to both the toolchain tag and the target +# architecture. +case "$edk2_arch" in + (ARM) + edk2_toolchain=GCC5 + export GCC5_ARM_PREFIX=$cross_prefix + ;; + (AARCH64) + edk2_toolchain=GCC5 + export GCC5_AARCH64_PREFIX=$cross_prefix + ;; + (IA32|X64) + gcc_version=$("${cross_prefix}gcc" -v 2>&1 | tail -1 | awk '{print $3}') + case "$gcc_version" in + ([1-3].*|4.[0-3].*) + printf '%s: unsupported gcc version "%s"\n' \ + "$program_name" "$gcc_version" >&2 + exit 1 + ;; + (4.4.*) + edk2_toolchain=GCC44 + ;; + (4.5.*) + edk2_toolchain=GCC45 + ;; + (4.6.*) + edk2_toolchain=GCC46 + ;; + (4.7.*) + edk2_toolchain=GCC47 + ;; + (4.8.*) + edk2_toolchain=GCC48 + ;; + (4.9.*|6.[0-2].*) + edk2_toolchain=GCC49 + ;; + (*) + edk2_toolchain=GCC5 + ;; + esac + eval "export ${edk2_toolchain}_BIN=\$cross_prefix" + ;; +esac + +# Build the UEFI binary +mkdir -p log +build \ + --arch="$edk2_arch" \ + --buildtarget=DEBUG \ + --platform=UefiTestToolsPkg/UefiTestToolsPkg.dsc \ + --tagname="$edk2_toolchain" \ + --module="UefiTestToolsPkg/$dsc_component/$dsc_component.inf" \ + --log="log/$dsc_component.$edk2_arch.log" \ + --report-file="log/$dsc_component.$edk2_arch.report" +cp -a -- \ + "Build/UefiTestTools/DEBUG_${edk2_toolchain}/$edk2_arch/$dsc_component.efi" \ + "$uefi_binary"Signature · 1 line
2.19.1.3.g30247aa5d201
Re: [Qemu-devel] [PATCH v3 4/5] tests/uefi-test-tools: add build scripts
Quoted 157 lines · Michael S. Tsirkin
Quoted 156 lines · Laszlo Ersek
Introduce the following build scripts under "tests/uefi-test-tools":
* "
build.sh" builds a single module (a UEFI application) from UefiTestToolsPkg, for a single QEMU emulation target."
build.sh" relies on cross-compilers when the emulation target and the build host architecture don't match. The cross-compiler prefix is computed according to a fixed, Linux-specific pattern. No attempt is made to copy or reimplement the GNU Make magic from "qemu/roms/Makefile" for cross-compiler prefix determination. The reason is that the build host OSes that are officially supported by edk2, and those that are supported by QEMU, intersect only in Linux. (Note that the UNIXGCC toolchain is being removed from edk2,
<https://bugzilla.tianocore.org/show_bug.cgi?id=1377>.)* "
Makefile" currently builds the "UefiTestToolsPkg/BiosTablesTest" application, for arm, aarch64, i386, andx86_64, with the help of "build.sh"."
Makefile" turns each resultant UEFI executable into a UEFI-bootable, qcow2-compressed ISO image. The ISO images are output as "tests/data/uefi-boot-images/bios-tables-test.<TARGET>.iso.qcow2".Each ISO image should be passed to QEMU as follows:
-drive id=boot-cd,if=none,readonly,format=qcow2,file=$ISO \ -device virtio-scsi-pci,id=scsi0 \ -device scsi-cd,drive=boot-cd,bus=scsi0.0,bootindex=0 \"
Makefile" assumes that "mkdosfs", "mtools", and "genisoimage" are present.Cc "Michael S. Tsirkin" <[email protected]>Cc Ard Biesheuvel <[email protected]>Cc Gerd Hoffmann <[email protected]>Cc Igor Mammedov <[email protected]>Cc Philippe Mathieu-Daudé <[email protected]>Cc Shannon Zhao <[email protected]>Signed-off-by Laszlo Ersek <[email protected]>Reviewed-by Philippe Mathieu-Daudé <[email protected]>---
Notes:
v3:
- explicitly mark the "./build.sh" recipe as recursive, with the "+" indicator; document it in a comment [Phil]
- pick up R-b, T-b [Phil]v2:
- add the .NOTPARALLEL target [Phil, help-make, edk2-devel]tests/uefi-test-tools/Makefile | 106 ++++++++++++++ tests/uefi-test-tools/.gitignore | 3 + tests/uefi-test-tools/build.sh | 145 ++++++++++++++++++++ 3 files changed, 254 insertions(+)tests/uefi-test-tools/Makefile new file +93
-0@@ -0,0 +1,106 @@ +# Makefile for the test helper UEFI applications that run in guests. +# +# Copyright (C) 2019, Red Hat, Inc. +# +# This program and the accompanying materials are licensed and made available +# under the terms and conditions of the BSD License that accompanies this +# distribution. The full text of the license may be found at +# <http://opensource.org/licenses/bsd-license.php>. +# +# THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, WITHOUT +# WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED. + +edk2_dir := ../../roms/edk2 +images_dir := ../data/uefi-boot-images +emulation_targets := arm aarch64 i386 x86_64 +uefi_binaries := bios-tables-test +intermediate_suffixes := .efi .fat .iso.raw + +images: $(foreach binary,$(uefi_binaries), \ + $(foreach target,$(emulation_targets), \ + $(images_dir)/$(binary).$(target).iso.qcow2)) + +# Preserve all intermediate targets if the build succeeds. +# - Intermediate targets help with development & debugging. +# - Preserving intermediate targets also keeps spurious changes out of the +# final build products, in case the user re-runs "make" without any changes +# to the UEFI source code. Normally, the intermediate files would have been +# removed by the last "make" invocation, hence the re-run would rebuild them +# from the unchanged UEFI sources. Unfortunately, the "mkdosfs" and +# "genisoimage" utilities embed timestamp-based information in their outputs, +# which causes git to report differences for the tracked qcow2 ISO images. +.SECONDARY: $(foreach binary,$(uefi_binaries), \ + $(foreach target,$(emulation_targets), \ + $(foreach suffix,$(intermediate_suffixes), \ + Build/$(binary).$(target)$(suffix)))) + +# In the pattern rules below, the stem (%, $*) stands for +# "$(binary).$(target)". + +# Convert the raw ISO image to a qcow2 one, enabling compression, and using a +# small cluster size. This allows for small binary files under git control, +# hence for small binary patches. +$(images_dir)/%.iso.qcow2: Build/%.iso.raw + mkdir -p -- $(images_dir) + $${QTEST_QEMU_IMG:-qemu-img} convert -f raw -O qcow2 -c \ + -o cluster_size=512 -- $< $@ + +# Embed the "UEFI system partition" into an ISO9660 file system as an ElTorito +# boot image. +Build/%.iso.raw: Build/%.fat + genisoimage -input-charset ASCII -efi-boot $(notdir $<) -no-emul-boot \ + -quiet -o $@ -- $< + +# Define chained macros in order to map QEMU system emulation targets to +# *short* UEFI architecture identifiers. Periods are allowed in, and ultimately +# stripped from, the argument. +map_arm_to_uefi = $(subst arm,ARM,$(1)) +map_aarch64_to_uefi = $(subst aarch64,AA64,$(call map_arm_to_uefi,$(1))) +map_i386_to_uefi = $(subst i386,IA32,$(call map_aarch64_to_uefi,$(1))) +map_x86_64_to_uefi = $(subst x86_64,X64,$(call map_i386_to_uefi,$(1))) +map_to_uefi = $(subst .,,$(call map_x86_64_to_uefi,$(1))) + +# Format a "UEFI system partition", using the UEFI binary as the default boot +# loader. Add 10% size for filesystem metadata, round up to the next KB, and +# make sure the size is large enough for a FAT filesystem. Name the filesystem +# after the UEFI binary. (Excess characters are automatically dropped from the +# filesystem label.) +Build/%.fat: Build/%.efi + rm -f -- $@ + uefi_bin_b=$$(stat --format=%s -- $<) && \ + uefi_fat_kb=$$(( (uefi_bin_b * 11 / 10 + 1023) / 1024 )) && \ + uefi_fat_kb=$$(( uefi_fat_kb >= 64 ? uefi_fat_kb : 64 )) && \ + mkdosfs -C $@ -n $(basename $(@F)) -- $$uefi_fat_kb + MTOOLS_SKIP_CHECK=1 mmd -i $@ ::EFI + MTOOLS_SKIP_CHECK=1 mmd -i $@ ::EFI/BOOT + MTOOLS_SKIP_CHECK=1 mcopy -i $@ -- $< \ + ::EFI/BOOT/BOOT$(call map_to_uefi,$(suffix $*)).EFI + +# In the pattern rules below, the stem (%, $*) stands for "$(target)" only. The +# association between the UEFI binary (such as "bios-tables-test") and the +# component name from the edk2 platform DSC file (such as "BiosTablesTest") is +# explicit in each rule. + +# "build.sh" invokes the "build" utility of edk2 BaseTools. In any given edk2 +# workspace, at most one "build" instance may be operating at a time. Therefore +# we must serialize the rebuilding of targets in this Makefile. +.NOTPARALLEL: + +# In turn, the "build" utility of edk2 BaseTools invokes another "make". +# Although the outer "make" process advertizes its job server to all child +# processes via MAKEFLAGS in the environment, the outer "make" closes the job +# server file descriptors (exposed in MAKEFLAGS) before executing a recipe -- +# unless the recipe is recognized as a recursive "make" recipe. Recipes that
Quoted 8 lines
tests/uefi-test-tools/Makefile line 99
Quoted 6 lines
tests/uefi-test-tools/Makefile new file +6
-0+# call $(MAKE) are classified automatically as recursive; for "build.sh" below, +# we must mark the recipe manually as recursive, by using the "+" indicator. +# This way, when the inner "make" starts a parallel build of the target edk2 +# module, it can communicate with the outer "make"'s job server. +Build/bios-tables-test.%.efi: build-edk2-tools + +./build.sh $(edk2_dir) BiosTablesTest $* $@Does this actually work with an out of tree build?
It's not supposed to.
Again, it's not something that a normal QEMU build includes. It is only for maintainers to rebuild when there is a reason to do so. The output binaries are tracked by git, and will be used as-is (in binary form) by the ACPI test suite. If there are updates to the UEFI source code, the binaries will have to be rebuilt by a maintainer (or by me, if I submit the UEFI code changes), and the refreshed blobs are to be checked into git. Think iPXE oproms for an analogy.
Quoted 1 line
Shouldn't this be SRC_PATH/tests/uefi-test-tools/ ?
No; nothing under roms/ is built like that, and the same applies to this patch as well.
*Conceptually*, this patch is for roms/. However, in earlier discussion, it was suggested that roms/ be kept dedicated to external git submodules only, and that we not add such ROM source to roms/ whose master repo is genuinely the QEMU repo. Please see the sub-thread at:
Re: [PATCH 10/14] tests: acpi: ignore SMBIOS tests when UEFI firmware is used
http://mid.mail-archive.com/[email protected]
The last idea was that the UEFI source code should be kept in a direct subdirectory of tests/ (rather than in roms/). And the binaries should go under tests/data/uefi-boot-images/ (rather than pc-bios/).
Thanks
Laszlo
Re: [Qemu-devel] [PATCH v3 4/5] tests/uefi-test-tools: add build scripts
Quoted 190 lines · Laszlo Ersek
Quoted 157 lines · Michael S. Tsirkin
Quoted 156 lines · Laszlo Ersek
Introduce the following build scripts under "tests/uefi-test-tools":
* "
build.sh" builds a single module (a UEFI application) from UefiTestToolsPkg, for a single QEMU emulation target."
build.sh" relies on cross-compilers when the emulation target and the build host architecture don't match. The cross-compiler prefix is computed according to a fixed, Linux-specific pattern. No attempt is made to copy or reimplement the GNU Make magic from "qemu/roms/Makefile" for cross-compiler prefix determination. The reason is that the build host OSes that are officially supported by edk2, and those that are supported by QEMU, intersect only in Linux. (Note that the UNIXGCC toolchain is being removed from edk2,
<https://bugzilla.tianocore.org/show_bug.cgi?id=1377>.)* "
Makefile" currently builds the "UefiTestToolsPkg/BiosTablesTest" application, for arm, aarch64, i386, andx86_64, with the help of "build.sh"."
Makefile" turns each resultant UEFI executable into a UEFI-bootable, qcow2-compressed ISO image. The ISO images are output as "tests/data/uefi-boot-images/bios-tables-test.<TARGET>.iso.qcow2".Each ISO image should be passed to QEMU as follows:
-drive id=boot-cd,if=none,readonly,format=qcow2,file=$ISO \ -device virtio-scsi-pci,id=scsi0 \ -device scsi-cd,drive=boot-cd,bus=scsi0.0,bootindex=0 \"
Makefile" assumes that "mkdosfs", "mtools", and "genisoimage" are present.Cc "Michael S. Tsirkin" <[email protected]>Cc Ard Biesheuvel <[email protected]>Cc Gerd Hoffmann <[email protected]>Cc Igor Mammedov <[email protected]>Cc Philippe Mathieu-Daudé <[email protected]>Cc Shannon Zhao <[email protected]>Signed-off-by Laszlo Ersek <[email protected]>Reviewed-by Philippe Mathieu-Daudé <[email protected]>---
Notes:
v3:
- explicitly mark the "./build.sh" recipe as recursive, with the "+" indicator; document it in a comment [Phil]
- pick up R-b, T-b [Phil]v2:
- add the .NOTPARALLEL target [Phil, help-make, edk2-devel]tests/uefi-test-tools/Makefile | 106 ++++++++++++++ tests/uefi-test-tools/.gitignore | 3 + tests/uefi-test-tools/build.sh | 145 ++++++++++++++++++++ 3 files changed, 254 insertions(+)tests/uefi-test-tools/Makefile new file +93
-0@@ -0,0 +1,106 @@ +# Makefile for the test helper UEFI applications that run in guests. +# +# Copyright (C) 2019, Red Hat, Inc. +# +# This program and the accompanying materials are licensed and made available +# under the terms and conditions of the BSD License that accompanies this +# distribution. The full text of the license may be found at +# <http://opensource.org/licenses/bsd-license.php>. +# +# THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, WITHOUT +# WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED. + +edk2_dir := ../../roms/edk2 +images_dir := ../data/uefi-boot-images +emulation_targets := arm aarch64 i386 x86_64 +uefi_binaries := bios-tables-test +intermediate_suffixes := .efi .fat .iso.raw + +images: $(foreach binary,$(uefi_binaries), \ + $(foreach target,$(emulation_targets), \ + $(images_dir)/$(binary).$(target).iso.qcow2)) + +# Preserve all intermediate targets if the build succeeds. +# - Intermediate targets help with development & debugging. +# - Preserving intermediate targets also keeps spurious changes out of the +# final build products, in case the user re-runs "make" without any changes +# to the UEFI source code. Normally, the intermediate files would have been +# removed by the last "make" invocation, hence the re-run would rebuild them +# from the unchanged UEFI sources. Unfortunately, the "mkdosfs" and +# "genisoimage" utilities embed timestamp-based information in their outputs, +# which causes git to report differences for the tracked qcow2 ISO images. +.SECONDARY: $(foreach binary,$(uefi_binaries), \ + $(foreach target,$(emulation_targets), \ + $(foreach suffix,$(intermediate_suffixes), \ + Build/$(binary).$(target)$(suffix)))) + +# In the pattern rules below, the stem (%, $*) stands for +# "$(binary).$(target)". + +# Convert the raw ISO image to a qcow2 one, enabling compression, and using a +# small cluster size. This allows for small binary files under git control, +# hence for small binary patches. +$(images_dir)/%.iso.qcow2: Build/%.iso.raw + mkdir -p -- $(images_dir) + $${QTEST_QEMU_IMG:-qemu-img} convert -f raw -O qcow2 -c \ + -o cluster_size=512 -- $< $@ + +# Embed the "UEFI system partition" into an ISO9660 file system as an ElTorito +# boot image. +Build/%.iso.raw: Build/%.fat + genisoimage -input-charset ASCII -efi-boot $(notdir $<) -no-emul-boot \ + -quiet -o $@ -- $< + +# Define chained macros in order to map QEMU system emulation targets to +# *short* UEFI architecture identifiers. Periods are allowed in, and ultimately +# stripped from, the argument. +map_arm_to_uefi = $(subst arm,ARM,$(1)) +map_aarch64_to_uefi = $(subst aarch64,AA64,$(call map_arm_to_uefi,$(1))) +map_i386_to_uefi = $(subst i386,IA32,$(call map_aarch64_to_uefi,$(1))) +map_x86_64_to_uefi = $(subst x86_64,X64,$(call map_i386_to_uefi,$(1))) +map_to_uefi = $(subst .,,$(call map_x86_64_to_uefi,$(1))) + +# Format a "UEFI system partition", using the UEFI binary as the default boot +# loader. Add 10% size for filesystem metadata, round up to the next KB, and +# make sure the size is large enough for a FAT filesystem. Name the filesystem +# after the UEFI binary. (Excess characters are automatically dropped from the +# filesystem label.) +Build/%.fat: Build/%.efi + rm -f -- $@ + uefi_bin_b=$$(stat --format=%s -- $<) && \ + uefi_fat_kb=$$(( (uefi_bin_b * 11 / 10 + 1023) / 1024 )) && \ + uefi_fat_kb=$$(( uefi_fat_kb >= 64 ? uefi_fat_kb : 64 )) && \ + mkdosfs -C $@ -n $(basename $(@F)) -- $$uefi_fat_kb + MTOOLS_SKIP_CHECK=1 mmd -i $@ ::EFI + MTOOLS_SKIP_CHECK=1 mmd -i $@ ::EFI/BOOT + MTOOLS_SKIP_CHECK=1 mcopy -i $@ -- $< \ + ::EFI/BOOT/BOOT$(call map_to_uefi,$(suffix $*)).EFI + +# In the pattern rules below, the stem (%, $*) stands for "$(target)" only. The +# association between the UEFI binary (such as "bios-tables-test") and the +# component name from the edk2 platform DSC file (such as "BiosTablesTest") is +# explicit in each rule. + +# "build.sh" invokes the "build" utility of edk2 BaseTools. In any given edk2 +# workspace, at most one "build" instance may be operating at a time. Therefore +# we must serialize the rebuilding of targets in this Makefile. +.NOTPARALLEL: + +# In turn, the "build" utility of edk2 BaseTools invokes another "make". +# Although the outer "make" process advertizes its job server to all child +# processes via MAKEFLAGS in the environment, the outer "make" closes the job +# server file descriptors (exposed in MAKEFLAGS) before executing a recipe -- +# unless the recipe is recognized as a recursive "make" recipe. Recipes thatQuoted 8 lines
tests/uefi-test-tools/Makefile line 99
Quoted 6 lines
tests/uefi-test-tools/Makefile new file +6
-0+# call $(MAKE) are classified automatically as recursive; for "build.sh" below, +# we must mark the recipe manually as recursive, by using the "+" indicator. +# This way, when the inner "make" starts a parallel build of the target edk2 +# module, it can communicate with the outer "make"'s job server. +Build/bios-tables-test.%.efi: build-edk2-tools + +./build.sh $(edk2_dir) BiosTablesTest $* $@Does this actually work with an out of tree build?
It's not supposed to.
Again, it's not something that a normal QEMU build includes. It is only for maintainers to rebuild when there is a reason to do so. The output binaries are tracked by git, and will be used as-is (in binary form) by the ACPI test suite. If there are updates to the UEFI source code, the binaries will have to be rebuilt by a maintainer (or by me, if I submit the UEFI code changes), and the refreshed blobs are to be checked into git. Think iPXE oproms for an analogy.
Quoted 1 line
Shouldn't this be SRC_PATH/tests/uefi-test-tools/ ?
No; nothing under roms/ is built like that, and the same applies to this patch as well.
*Conceptually*, this patch is for roms/. However, in earlier discussion, it was suggested that roms/ be kept dedicated to external git submodules only, and that we not add such ROM source to roms/ whose master repo is genuinely the QEMU repo. Please see the sub-thread at:
Re: [PATCH 10/14] tests: acpi: ignore SMBIOS tests when UEFI firmware is used
http://mid.mail-archive.com/[email protected]
Quoted 6 lines
The last idea was that the UEFI source code should be kept in a direct subdirectory of tests/ (rather than in roms/). And the binaries should go under tests/data/uefi-boot-images/ (rather than pc-bios/).
Thanks
Laszlo
Hmm I see. You see rebuild-expected-aml.sh does not work like this at all. It works fine with an out of tree build:
check it out.
So I try to keep it all out of tree.
And question would be what if someone wanted a reproducible build of QEMU, what would be the right way to do it?
Yes right now roms seems to be broken for an out of tree build but is that by design and should we add more examples of this?
Signature · 1 line
MST
Re: [Qemu-devel] [PATCH v3 4/5] tests/uefi-test-tools: add build scripts
Hi Michael,
Quoted 195 lines · Michael S. Tsirkin
Quoted 194 lines · Laszlo Ersek
Quoted 157 lines · Michael S. Tsirkin
Quoted 156 lines · Laszlo Ersek
Introduce the following build scripts under "tests/uefi-test-tools":
* "
build.sh" builds a single module (a UEFI application) from UefiTestToolsPkg, for a single QEMU emulation target."
build.sh" relies on cross-compilers when the emulation target and the build host architecture don't match. The cross-compiler prefix is computed according to a fixed, Linux-specific pattern. No attempt is made to copy or reimplement the GNU Make magic from "qemu/roms/Makefile" for cross-compiler prefix determination. The reason is that the build host OSes that are officially supported by edk2, and those that are supported by QEMU, intersect only in Linux. (Note that the UNIXGCC toolchain is being removed from edk2,
<https://bugzilla.tianocore.org/show_bug.cgi?id=1377>.)* "
Makefile" currently builds the "UefiTestToolsPkg/BiosTablesTest" application, for arm, aarch64, i386, andx86_64, with the help of "build.sh"."
Makefile" turns each resultant UEFI executable into a UEFI-bootable, qcow2-compressed ISO image. The ISO images are output as "tests/data/uefi-boot-images/bios-tables-test.<TARGET>.iso.qcow2".Each ISO image should be passed to QEMU as follows:
-drive id=boot-cd,if=none,readonly,format=qcow2,file=$ISO \ -device virtio-scsi-pci,id=scsi0 \ -device scsi-cd,drive=boot-cd,bus=scsi0.0,bootindex=0 \"
Makefile" assumes that "mkdosfs", "mtools", and "genisoimage" are present.Cc "Michael S. Tsirkin" <[email protected]>Cc Ard Biesheuvel <[email protected]>Cc Gerd Hoffmann <[email protected]>Cc Igor Mammedov <[email protected]>Cc Philippe Mathieu-Daudé <[email protected]>Cc Shannon Zhao <[email protected]>Signed-off-by Laszlo Ersek <[email protected]>Reviewed-by Philippe Mathieu-Daudé <[email protected]>---
Notes:
v3:
- explicitly mark the "./build.sh" recipe as recursive, with the "+" indicator; document it in a comment [Phil]
- pick up R-b, T-b [Phil]v2:
- add the .NOTPARALLEL target [Phil, help-make, edk2-devel]tests/uefi-test-tools/Makefile | 106 ++++++++++++++ tests/uefi-test-tools/.gitignore | 3 + tests/uefi-test-tools/build.sh | 145 ++++++++++++++++++++ 3 files changed, 254 insertions(+)tests/uefi-test-tools/Makefile new file +93
-0@@ -0,0 +1,106 @@ +# Makefile for the test helper UEFI applications that run in guests. +# +# Copyright (C) 2019, Red Hat, Inc. +# +# This program and the accompanying materials are licensed and made available +# under the terms and conditions of the BSD License that accompanies this +# distribution. The full text of the license may be found at +# <http://opensource.org/licenses/bsd-license.php>. +# +# THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, WITHOUT +# WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED. + +edk2_dir := ../../roms/edk2 +images_dir := ../data/uefi-boot-images +emulation_targets := arm aarch64 i386 x86_64 +uefi_binaries := bios-tables-test +intermediate_suffixes := .efi .fat .iso.raw + +images: $(foreach binary,$(uefi_binaries), \ + $(foreach target,$(emulation_targets), \ + $(images_dir)/$(binary).$(target).iso.qcow2)) + +# Preserve all intermediate targets if the build succeeds. +# - Intermediate targets help with development & debugging. +# - Preserving intermediate targets also keeps spurious changes out of the +# final build products, in case the user re-runs "make" without any changes +# to the UEFI source code. Normally, the intermediate files would have been +# removed by the last "make" invocation, hence the re-run would rebuild them +# from the unchanged UEFI sources. Unfortunately, the "mkdosfs" and +# "genisoimage" utilities embed timestamp-based information in their outputs, +# which causes git to report differences for the tracked qcow2 ISO images. +.SECONDARY: $(foreach binary,$(uefi_binaries), \ + $(foreach target,$(emulation_targets), \ + $(foreach suffix,$(intermediate_suffixes), \ + Build/$(binary).$(target)$(suffix)))) + +# In the pattern rules below, the stem (%, $*) stands for +# "$(binary).$(target)". + +# Convert the raw ISO image to a qcow2 one, enabling compression, and using a +# small cluster size. This allows for small binary files under git control, +# hence for small binary patches. +$(images_dir)/%.iso.qcow2: Build/%.iso.raw + mkdir -p -- $(images_dir) + $${QTEST_QEMU_IMG:-qemu-img} convert -f raw -O qcow2 -c \ + -o cluster_size=512 -- $< $@ + +# Embed the "UEFI system partition" into an ISO9660 file system as an ElTorito +# boot image. +Build/%.iso.raw: Build/%.fat + genisoimage -input-charset ASCII -efi-boot $(notdir $<) -no-emul-boot \ + -quiet -o $@ -- $< + +# Define chained macros in order to map QEMU system emulation targets to +# *short* UEFI architecture identifiers. Periods are allowed in, and ultimately +# stripped from, the argument. +map_arm_to_uefi = $(subst arm,ARM,$(1)) +map_aarch64_to_uefi = $(subst aarch64,AA64,$(call map_arm_to_uefi,$(1))) +map_i386_to_uefi = $(subst i386,IA32,$(call map_aarch64_to_uefi,$(1))) +map_x86_64_to_uefi = $(subst x86_64,X64,$(call map_i386_to_uefi,$(1))) +map_to_uefi = $(subst .,,$(call map_x86_64_to_uefi,$(1))) + +# Format a "UEFI system partition", using the UEFI binary as the default boot +# loader. Add 10% size for filesystem metadata, round up to the next KB, and +# make sure the size is large enough for a FAT filesystem. Name the filesystem +# after the UEFI binary. (Excess characters are automatically dropped from the +# filesystem label.) +Build/%.fat: Build/%.efi + rm -f -- $@ + uefi_bin_b=$$(stat --format=%s -- $<) && \ + uefi_fat_kb=$$(( (uefi_bin_b * 11 / 10 + 1023) / 1024 )) && \ + uefi_fat_kb=$$(( uefi_fat_kb >= 64 ? uefi_fat_kb : 64 )) && \ + mkdosfs -C $@ -n $(basename $(@F)) -- $$uefi_fat_kb + MTOOLS_SKIP_CHECK=1 mmd -i $@ ::EFI + MTOOLS_SKIP_CHECK=1 mmd -i $@ ::EFI/BOOT + MTOOLS_SKIP_CHECK=1 mcopy -i $@ -- $< \ + ::EFI/BOOT/BOOT$(call map_to_uefi,$(suffix $*)).EFI + +# In the pattern rules below, the stem (%, $*) stands for "$(target)" only. The +# association between the UEFI binary (such as "bios-tables-test") and the +# component name from the edk2 platform DSC file (such as "BiosTablesTest") is +# explicit in each rule. + +# "build.sh" invokes the "build" utility of edk2 BaseTools. In any given edk2 +# workspace, at most one "build" instance may be operating at a time. Therefore +# we must serialize the rebuilding of targets in this Makefile. +.NOTPARALLEL: + +# In turn, the "build" utility of edk2 BaseTools invokes another "make". +# Although the outer "make" process advertizes its job server to all child +# processes via MAKEFLAGS in the environment, the outer "make" closes the job +# server file descriptors (exposed in MAKEFLAGS) before executing a recipe -- +# unless the recipe is recognized as a recursive "make" recipe. Recipes thatQuoted 8 lines
tests/uefi-test-tools/Makefile line 99
Quoted 6 lines
tests/uefi-test-tools/Makefile new file +6
-0+# call $(MAKE) are classified automatically as recursive; for "build.sh" below, +# we must mark the recipe manually as recursive, by using the "+" indicator. +# This way, when the inner "make" starts a parallel build of the target edk2 +# module, it can communicate with the outer "make"'s job server. +Build/bios-tables-test.%.efi: build-edk2-tools + +./build.sh $(edk2_dir) BiosTablesTest $* $@Does this actually work with an out of tree build?
It's not supposed to.
Again, it's not something that a normal QEMU build includes. It is only for maintainers to rebuild when there is a reason to do so. The output binaries are tracked by git, and will be used as-is (in binary form) by the ACPI test suite. If there are updates to the UEFI source code, the binaries will have to be rebuilt by a maintainer (or by me, if I submit the UEFI code changes), and the refreshed blobs are to be checked into git. Think iPXE oproms for an analogy.
Quoted 1 line
Shouldn't this be SRC_PATH/tests/uefi-test-tools/ ?
No; nothing under roms/ is built like that, and the same applies to this patch as well.
*Conceptually*, this patch is for roms/. However, in earlier discussion, it was suggested that roms/ be kept dedicated to external git submodules only, and that we not add such ROM source to roms/ whose master repo is genuinely the QEMU repo. Please see the sub-thread at:
Re: [PATCH 10/14] tests: acpi: ignore SMBIOS tests when UEFI firmware is used
http://mid.mail-archive.com/[email protected]The last idea was that the UEFI source code should be kept in a direct subdirectory of tests/ (rather than in roms/). And the binaries should go under tests/data/uefi-boot-images/ (rather than pc-bios/).
Quoted 6 lines
Quoted 2 lines
Thanks
LaszloHmm I see. You see
rebuild-expected-aml.shdoes not work like this at all. It works fine with an out of tree build:
check it out.
But rebuild-expected-aml.sh does not depend of roms/.
Quoted 2 lines
So I try to keep it all out of tree.
Patch 5/5 add the bios-tables-test blobs under tests/data/ (as other submodules in roms/ do, adding blobs in pc-bios/).
Quoted 3 lines
And question would be what if someone wanted a reproducible build of QEMU, what would be the right way to do it?
This question deserves his own thread :)
Quoted 3 lines
Yes right now roms seems to be broken for an out of tree build but is that by design and should we add more examples of this?
IMO having these tests build out-of-tree is easier than trying to build various of the projects in roms/ out-of-tree.
This would be a good effort, but I'm not sure it is worth it with this series.
Eventually once we have a qtest using the bios-tables, we could spend some time to make this script work out-of-tree.
Regards,
Phil.
Re: [Qemu-devel] [PATCH v3 4/5] tests/uefi-test-tools: add build scripts
Quoted 198 lines · Philippe Mathieu-Daudé
Hi Michael,
Quoted 195 lines · Michael S. Tsirkin
Quoted 194 lines · Laszlo Ersek
Quoted 157 lines · Michael S. Tsirkin
Quoted 156 lines · Laszlo Ersek
Introduce the following build scripts under "tests/uefi-test-tools":
* "
build.sh" builds a single module (a UEFI application) from UefiTestToolsPkg, for a single QEMU emulation target."
build.sh" relies on cross-compilers when the emulation target and the build host architecture don't match. The cross-compiler prefix is computed according to a fixed, Linux-specific pattern. No attempt is made to copy or reimplement the GNU Make magic from "qemu/roms/Makefile" for cross-compiler prefix determination. The reason is that the build host OSes that are officially supported by edk2, and those that are supported by QEMU, intersect only in Linux. (Note that the UNIXGCC toolchain is being removed from edk2,
<https://bugzilla.tianocore.org/show_bug.cgi?id=1377>.)* "
Makefile" currently builds the "UefiTestToolsPkg/BiosTablesTest" application, for arm, aarch64, i386, andx86_64, with the help of "build.sh"."
Makefile" turns each resultant UEFI executable into a UEFI-bootable, qcow2-compressed ISO image. The ISO images are output as "tests/data/uefi-boot-images/bios-tables-test.<TARGET>.iso.qcow2".Each ISO image should be passed to QEMU as follows:
-drive id=boot-cd,if=none,readonly,format=qcow2,file=$ISO \ -device virtio-scsi-pci,id=scsi0 \ -device scsi-cd,drive=boot-cd,bus=scsi0.0,bootindex=0 \"
Makefile" assumes that "mkdosfs", "mtools", and "genisoimage" are present.Cc "Michael S. Tsirkin" <[email protected]>Cc Ard Biesheuvel <[email protected]>Cc Gerd Hoffmann <[email protected]>Cc Igor Mammedov <[email protected]>Cc Philippe Mathieu-Daudé <[email protected]>Cc Shannon Zhao <[email protected]>Signed-off-by Laszlo Ersek <[email protected]>Reviewed-by Philippe Mathieu-Daudé <[email protected]>---
Notes:
v3:
- explicitly mark the "./build.sh" recipe as recursive, with the "+" indicator; document it in a comment [Phil]
- pick up R-b, T-b [Phil]v2:
- add the .NOTPARALLEL target [Phil, help-make, edk2-devel]tests/uefi-test-tools/Makefile | 106 ++++++++++++++ tests/uefi-test-tools/.gitignore | 3 + tests/uefi-test-tools/build.sh | 145 ++++++++++++++++++++ 3 files changed, 254 insertions(+)tests/uefi-test-tools/Makefile new file +93
-0@@ -0,0 +1,106 @@ +# Makefile for the test helper UEFI applications that run in guests. +# +# Copyright (C) 2019, Red Hat, Inc. +# +# This program and the accompanying materials are licensed and made available +# under the terms and conditions of the BSD License that accompanies this +# distribution. The full text of the license may be found at +# <http://opensource.org/licenses/bsd-license.php>. +# +# THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, WITHOUT +# WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED. + +edk2_dir := ../../roms/edk2 +images_dir := ../data/uefi-boot-images +emulation_targets := arm aarch64 i386 x86_64 +uefi_binaries := bios-tables-test +intermediate_suffixes := .efi .fat .iso.raw + +images: $(foreach binary,$(uefi_binaries), \ + $(foreach target,$(emulation_targets), \ + $(images_dir)/$(binary).$(target).iso.qcow2)) + +# Preserve all intermediate targets if the build succeeds. +# - Intermediate targets help with development & debugging. +# - Preserving intermediate targets also keeps spurious changes out of the +# final build products, in case the user re-runs "make" without any changes +# to the UEFI source code. Normally, the intermediate files would have been +# removed by the last "make" invocation, hence the re-run would rebuild them +# from the unchanged UEFI sources. Unfortunately, the "mkdosfs" and +# "genisoimage" utilities embed timestamp-based information in their outputs, +# which causes git to report differences for the tracked qcow2 ISO images. +.SECONDARY: $(foreach binary,$(uefi_binaries), \ + $(foreach target,$(emulation_targets), \ + $(foreach suffix,$(intermediate_suffixes), \ + Build/$(binary).$(target)$(suffix)))) + +# In the pattern rules below, the stem (%, $*) stands for +# "$(binary).$(target)". + +# Convert the raw ISO image to a qcow2 one, enabling compression, and using a +# small cluster size. This allows for small binary files under git control, +# hence for small binary patches. +$(images_dir)/%.iso.qcow2: Build/%.iso.raw + mkdir -p -- $(images_dir) + $${QTEST_QEMU_IMG:-qemu-img} convert -f raw -O qcow2 -c \ + -o cluster_size=512 -- $< $@ + +# Embed the "UEFI system partition" into an ISO9660 file system as an ElTorito +# boot image. +Build/%.iso.raw: Build/%.fat + genisoimage -input-charset ASCII -efi-boot $(notdir $<) -no-emul-boot \ + -quiet -o $@ -- $< + +# Define chained macros in order to map QEMU system emulation targets to +# *short* UEFI architecture identifiers. Periods are allowed in, and ultimately +# stripped from, the argument. +map_arm_to_uefi = $(subst arm,ARM,$(1)) +map_aarch64_to_uefi = $(subst aarch64,AA64,$(call map_arm_to_uefi,$(1))) +map_i386_to_uefi = $(subst i386,IA32,$(call map_aarch64_to_uefi,$(1))) +map_x86_64_to_uefi = $(subst x86_64,X64,$(call map_i386_to_uefi,$(1))) +map_to_uefi = $(subst .,,$(call map_x86_64_to_uefi,$(1))) + +# Format a "UEFI system partition", using the UEFI binary as the default boot +# loader. Add 10% size for filesystem metadata, round up to the next KB, and +# make sure the size is large enough for a FAT filesystem. Name the filesystem +# after the UEFI binary. (Excess characters are automatically dropped from the +# filesystem label.) +Build/%.fat: Build/%.efi + rm -f -- $@ + uefi_bin_b=$$(stat --format=%s -- $<) && \ + uefi_fat_kb=$$(( (uefi_bin_b * 11 / 10 + 1023) / 1024 )) && \ + uefi_fat_kb=$$(( uefi_fat_kb >= 64 ? uefi_fat_kb : 64 )) && \ + mkdosfs -C $@ -n $(basename $(@F)) -- $$uefi_fat_kb + MTOOLS_SKIP_CHECK=1 mmd -i $@ ::EFI + MTOOLS_SKIP_CHECK=1 mmd -i $@ ::EFI/BOOT + MTOOLS_SKIP_CHECK=1 mcopy -i $@ -- $< \ + ::EFI/BOOT/BOOT$(call map_to_uefi,$(suffix $*)).EFI + +# In the pattern rules below, the stem (%, $*) stands for "$(target)" only. The +# association between the UEFI binary (such as "bios-tables-test") and the +# component name from the edk2 platform DSC file (such as "BiosTablesTest") is +# explicit in each rule. + +# "build.sh" invokes the "build" utility of edk2 BaseTools. In any given edk2 +# workspace, at most one "build" instance may be operating at a time. Therefore +# we must serialize the rebuilding of targets in this Makefile. +.NOTPARALLEL: + +# In turn, the "build" utility of edk2 BaseTools invokes another "make". +# Although the outer "make" process advertizes its job server to all child +# processes via MAKEFLAGS in the environment, the outer "make" closes the job +# server file descriptors (exposed in MAKEFLAGS) before executing a recipe -- +# unless the recipe is recognized as a recursive "make" recipe. Recipes thatQuoted 8 lines
tests/uefi-test-tools/Makefile line 99
Quoted 6 lines
tests/uefi-test-tools/Makefile new file +6
-0+# call $(MAKE) are classified automatically as recursive; for "build.sh" below, +# we must mark the recipe manually as recursive, by using the "+" indicator. +# This way, when the inner "make" starts a parallel build of the target edk2 +# module, it can communicate with the outer "make"'s job server. +Build/bios-tables-test.%.efi: build-edk2-tools + +./build.sh $(edk2_dir) BiosTablesTest $* $@Does this actually work with an out of tree build?
It's not supposed to.
Again, it's not something that a normal QEMU build includes. It is only for maintainers to rebuild when there is a reason to do so. The output binaries are tracked by git, and will be used as-is (in binary form) by the ACPI test suite. If there are updates to the UEFI source code, the binaries will have to be rebuilt by a maintainer (or by me, if I submit the UEFI code changes), and the refreshed blobs are to be checked into git. Think iPXE oproms for an analogy.
Quoted 1 line
Shouldn't this be SRC_PATH/tests/uefi-test-tools/ ?
No; nothing under roms/ is built like that, and the same applies to this patch as well.
*Conceptually*, this patch is for roms/. However, in earlier discussion, it was suggested that roms/ be kept dedicated to external git submodules only, and that we not add such ROM source to roms/ whose master repo is genuinely the QEMU repo. Please see the sub-thread at:
Re: [PATCH 10/14] tests: acpi: ignore SMBIOS tests when UEFI firmware is used
http://mid.mail-archive.com/[email protected]The last idea was that the UEFI source code should be kept in a direct subdirectory of tests/ (rather than in roms/). And the binaries should go under tests/data/uefi-boot-images/ (rather than pc-bios/).
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
Thanks
LaszloHmm I see. You see
rebuild-expected-aml.shdoes not work like this at all. It works fine with an out of tree build:
check it out.But
rebuild-expected-aml.shdoes not depend of roms/.
It absolutely depends on pc-bios/bios.bin
Quoted 11 lines
Quoted 2 lines
So I try to keep it all out of tree.
Patch 5/5 add the bios-tables-test blobs under tests/data/ (as other submodules in roms/ do, adding blobs in pc-bios/).
Quoted 3 lines
And question would be what if someone wanted a reproducible build of QEMU, what would be the right way to do it?
This question deserves his own thread :)
Sure.
Quoted 11 lines
Quoted 3 lines
Yes right now roms seems to be broken for an out of tree build but is that by design and should we add more examples of this?
IMO having these tests build out-of-tree is easier than trying to build various of the projects in roms/ out-of-tree.
This would be a good effort, but I'm not sure it is worth it with this series.
Eventually once we have a qtest using the bios-tables, we could spend some time to make this script work out-of-tree.
Quoted 3 lines
Regards,
Phil.
I'm not saying it's a blocker.
Signature · 1 line
MST
Re: [Qemu-devel] [PATCH v3 4/5] tests/uefi-test-tools: add build scripts
Quoted 6 lines · Michael S. Tsirkin
Quoted 5 lines · Philippe Mathieu-Daudé
Quoted 4 lines · Michael S. Tsirkin
Quoted 3 lines · Laszlo Ersek
Quoted 2 lines · Michael S. Tsirkin
Quoted 1 line · Laszlo Ersek
Introduce the following build scripts under "tests/uefi-test-tools":
[...]>>>>>> +# In turn, the "build" utility of edk2 BaseTools invokes another "make".
Quoted 43 lines
Quoted 43 lines
Quoted 43 lines
Quoted 42 lines
Quoted 4 lines
Quoted 4 lines
tests/uefi-test-tools/Makefile new file +4
-0+# Although the outer "make" process advertizes its job server to all child +# processes via MAKEFLAGS in the environment, the outer "make" closes the job +# server file descriptors (exposed in MAKEFLAGS) before executing a recipe -- +# unless the recipe is recognized as a recursive "make" recipe. Recipes thatQuoted 8 lines
tests/uefi-test-tools/Makefile line 99
Quoted 6 lines
tests/uefi-test-tools/Makefile new file +6
-0+# call $(MAKE) are classified automatically as recursive; for "build.sh" below, +# we must mark the recipe manually as recursive, by using the "+" indicator. +# This way, when the inner "make" starts a parallel build of the target edk2 +# module, it can communicate with the outer "make"'s job server. +Build/bios-tables-test.%.efi: build-edk2-tools + +./build.sh $(edk2_dir) BiosTablesTest $* $@Does this actually work with an out of tree build?
It's not supposed to.
Again, it's not something that a normal QEMU build includes. It is only for maintainers to rebuild when there is a reason to do so. The output binaries are tracked by git, and will be used as-is (in binary form) by the ACPI test suite. If there are updates to the UEFI source code, the binaries will have to be rebuilt by a maintainer (or by me, if I submit the UEFI code changes), and the refreshed blobs are to be checked into git. Think iPXE oproms for an analogy.
Quoted 1 line
Shouldn't this be SRC_PATH/tests/uefi-test-tools/ ?
No; nothing under roms/ is built like that, and the same applies to this patch as well.
*Conceptually*, this patch is for roms/. However, in earlier discussion, it was suggested that roms/ be kept dedicated to external git submodules only, and that we not add such ROM source to roms/ whose master repo is genuinely the QEMU repo. Please see the sub-thread at:
Re: [PATCH 10/14] tests: acpi: ignore SMBIOS tests when UEFI firmware is used
http://mid.mail-archive.com/[email protected]The last idea was that the UEFI source code should be kept in a direct subdirectory of tests/ (rather than in roms/). And the binaries should go under tests/data/uefi-boot-images/ (rather than pc-bios/).
Thanks
Laszlo
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Hmm I see. You see
rebuild-expected-aml.shdoes not work like this at all. It works fine with an out of tree build:
check it out.But
rebuild-expected-aml.shdoes not depend of roms/.It absolutely depends on pc-bios/bios.bin
Oh you are right!
I'm taking notes for an eventual future cleanup.
Regards,
Phil.
Re: [Qemu-devel] [PATCH v3 4/5] tests/uefi-test-tools: add build scripts
In reply to Michael S. TsirkinQuoted 4 lines · Michael S. Tsirkin
Quoted 2 lines · Philippe Mathieu-Daudé
Hi Michael,
Quoted 8 lines
Quoted 6 lines · Michael S. Tsirkin
Quoted 3 lines
And question would be what if someone wanted a reproducible build of QEMU, what would be the right way to do it?
This question deserves his own thread :)
Sure.
(I don't know enough to comment on this sensibly; so I'm only following up to show that I'm not willfully ignoring the topic.)
What constitutes a "QEMU build" in this context?
If it means the sum of artifacts that get installed by "make install", then the current patch set doesn't factor into that at all.
If it means something else, then "it depends". For example, "genisoimage" and "mkdosfs" place pseudo-random / timestamp-based identifiers into the images that they generate, and there's no way to prevent at least "genisoimage" from doing that. Thus, if we consider UEFI-bootable ISO images, rebuilt from zero, a part of a "reproducible build", then the answer is "there's no right way until someone extends genisoimage with some new cmdline options".
Quoted 10 lines
Quoted 8 lines
Quoted 2 lines
Yes right now roms seems to be broken for an out of tree build but is that by design and should we add more examples of this?
IMO having these tests build out-of-tree is easier than trying to build various of the projects in roms/ out-of-tree. This would be a good effort, but I'm not sure it is worth it with this series. Eventually once we have a qtest using the bios-tables, we could spend some time to make this script work out-of-tree.
I'm not saying it's a blocker.
Regarding roms/edk2, we have to distinguish two sets of source code, wrt. building out-of-tree.
- The firmware modules build just fine outside of the tree, and this patch set already puts that feature to use. (Namely, all the dependencies that the
tests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest/BiosTablesTest.inf
application pulls in from roms/edk2 are compiled outside of roms/edk2.)
- However, the native build utilities (in the
roms/edk2/BaseTools
subdirectory) that first have to be built natively, so that they can help produce the firmware executables from the firmware source, don't themselves build outside of that subdirectory. This is a genuine limitation of upstream edk2, as far as I can tell.
Thanks,
Laszlo
Re: [Qemu-devel] [PATCH v3 4/5] tests/uefi-test-tools: add build scripts
Quoted 10 lines · Laszlo Ersek
Quoted 9 lines · Michael S. Tsirkin
Quoted 7 lines · Philippe Mathieu-Daudé
Hi Michael,
Quoted 2 lines · Michael S. Tsirkin
And question would be what if someone wanted a reproducible build of QEMU, what would be the right way to do it?
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
This question deserves his own thread :)
Sure.
(I don't know enough to comment on this sensibly; so I'm only following up to show that I'm not willfully ignoring the topic.)
Thanks!
Quoted 4 lines
What constitutes a "QEMU build" in this context?
If it means the sum of artifacts that get installed by "make install", then the current patch set doesn't factor into that at all.
That's true in that it's just a test.
Quoted 7 lines
If it means something else, then "it depends". For example, "genisoimage" and "mkdosfs" place pseudo-random / timestamp-based identifiers into the images that they generate, and there's no way to prevent at least "genisoimage" from doing that. Thus, if we consider UEFI-bootable ISO images, rebuilt from zero, a part of a "reproducible build", then the answer is "there's no right way until someone extends genisoimage with some new cmdline options".
Yep that's true too.
Quoted 27 lines
Quoted 10 lines
Quoted 8 lines
Quoted 2 lines
Yes right now roms seems to be broken for an out of tree build but is that by design and should we add more examples of this?
IMO having these tests build out-of-tree is easier than trying to build various of the projects in roms/ out-of-tree. This would be a good effort, but I'm not sure it is worth it with this series. Eventually once we have a qtest using the bios-tables, we could spend some time to make this script work out-of-tree.
I'm not saying it's a blocker.
Regarding roms/edk2, we have to distinguish two sets of source code, wrt. building out-of-tree.
- The firmware modules build just fine outside of the tree, and this patch set already puts that feature to use. (Namely, all the dependencies that the
tests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest/BiosTablesTest.inf
application pulls in from roms/edk2 are compiled outside of roms/edk2.)
- However, the native build utilities (in the
roms/edk2/BaseTools
Quoted 7 lines
subdirectory) that first have to be built natively, so that they can help produce the firmware executables from the firmware source, don't themselves build outside of that subdirectory. This is a genuine limitation of upstream edk2, as far as I can tell.
Thanks,
Laszlo
Interesting, thanks for sharing.
Signature · 1 line
MST
PATCH v3 5/5 tests/data: introduce "uefi-boot-images" with the "bios-tables-test" ISOs
In reply to Laszlo ErsekAdd UEFI-bootable qcow2-compressed ISO images built from:
tests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest
---
Notes:
v3:
- refresh the images (should only differ in internal timestamps etc)
v2:
- refresh the images (should only differ in internal timestamps etc)
v1:
- Again, if needed, I'd be happy to be designated as Maintainer for the files being added in this patch; please let me know the right spot in "MAINTAINERS".
tests/data/uefi-boot-images/bios-tables-test.aarch64.iso.qcow2 | Bin 0 -> 11776 bytes tests/data/uefi-boot-images/bios-tables-test.arm.iso.qcow2 | Bin 0 -> 11776 bytes tests/data/uefi-boot-images/bios-tables-test.i386.iso.qcow2 | Bin 0 -> 12800 bytes tests/data/uefi-boot-images/bios-tables-test.x86_64.iso.qcow2 | Bin 0 -> 13312 bytes 4 files changed, 0 insertions(+), 0 deletions(-)
tests/data/uefi-boot-images/bios-tables-test.aarch64.iso.qcow2 new file binary +0 -0
tests/data/uefi-boot-images/bios-tables-test.arm.iso.qcow2 new file binary +0 -0
tests/data/uefi-boot-images/bios-tables-test.i386.iso.qcow2 new file binary +0 -0
tests/data/uefi-boot-images/bios-tables-test.x86_64.iso.qcow2 new file binary +0 -0
Signature · 1 line
2.19.1.3.g30247aa5d201
On Mon, 4 Feb 2019 17:03:20 +0100
Laszlo Ersek <[email protected]> wrote:
Am I to greedy to ask for prebuilt AVMF image in QEMU repo on top of this App (while you still remember all Makefile magic foo) so test won't have to fish out distro provided firmware or skip testing if firmware is not available?
Quoted 8 lines · Igor Mammedov
On Mon, 4 Feb 2019 17:03:20 +0100
Laszlo Ersek <[email protected]> wrote:Am I to greedy to ask for prebuilt AVMF image in QEMU repo on top of this App (while you still remember all
Makefilemagic foo) so test won't have to fish out distro provided firmware or skip testing if firmware is not available?
No, not too greedy :) I think that's a reasonable idea; we intended bundling "reasonably-built" edk2 firmware binaries with QEMU anyway. I can work on that once this series is merged. (I wouldn't like to work on these assignments simultaneously.)
Thanks,
Laszlo
Quoted 4 lines · On Tue, 5 Feb 2019 16:19:50 +0100 Laszlo Ersek <[email protected]> wrote:
Quoted 3 lines · Igor Mammedov
On Mon, 4 Feb 2019 17:03:20 +0100
Laszlo Ersek <[email protected]> wrote:
Quoted 8 lines
Quoted 5 lines
Am I to greedy to ask for prebuilt AVMF image in QEMU repo on top of this App (while you still remember all
Makefilemagic foo) so test won't have to fish out distro provided firmware or skip testing if firmware is not available?No, not too greedy :) I think that's a reasonable idea; we intended bundling "reasonably-built" edk2 firmware binaries with QEMU anyway.
Thanks, I'll postpone ARM's acpi tests respin till we have AVMF blobs in QEMU tree.
Quoted 2 lines
can work on that once this series is merged. (I wouldn't like to work on these assignments simultaneously.)
I think series is practically ready to be merged
(I can live without out of tree builds for blobs that's hardly ever would be rebuilt), Michael do you concur?
Quoted 3 lines
Thanks,
Laszlo
Quoted 15 lines · Igor Mammedov
Quoted 4 lines · On Tue, 5 Feb 2019 16:19:50 +0100 Laszlo Ersek <[email protected]> wrote:
Quoted 3 lines · Igor Mammedov
On Mon, 4 Feb 2019 17:03:20 +0100
Laszlo Ersek <[email protected]> wrote:Quoted 8 lines
Quoted 5 lines
Am I to greedy to ask for prebuilt AVMF image in QEMU repo on top of this App (while you still remember all
Makefilemagic foo) so test won't have to fish out distro provided firmware or skip testing if firmware is not available?No, not too greedy :) I think that's a reasonable idea; we intended bundling "reasonably-built" edk2 firmware binaries with QEMU anyway.
Quoted 8 lines
Thanks, I'll postpone ARM's acpi tests respin till we have AVMF blobs in QEMU tree.
Quoted 2 lines
can work on that once this series is merged. (I wouldn't like to work on these assignments simultaneously.)
I think series is practically ready to be merged
(I can live without out of tree builds for blobs that's hardly ever would be rebuilt), Michael do you concur?
Yes I have it tagged for the next pull.
Quoted 3 lines
Quoted 3 lines
Thanks,
Laszlo
Quoted 11 lines · On Mon, 4 Feb 2019 17:03:20 +0100 Laszlo Ersek <[email protected]> wrote:
Previous version (v2):
http://mid.mail-archive.com/[email protected]Updates in v3 have been noted on each patch in the series.
Cc "Michael S. Tsirkin" <[email protected]>Cc Ard Biesheuvel <[email protected]>Cc Gerd Hoffmann <[email protected]>Cc Igor Mammedov <[email protected]>Cc Philippe Mathieu-Daudé <[email protected]>Cc Shannon Zhao <[email protected]>
For series:
Quoted 45 lines
Thanks
LaszloLaszlo Ersek (5):
roms: add the edk2 project as a git submodule
roms: build the EfiRom utility from the roms/edk2 submodule
tests: introduce "uefi-test-tools" with the BiosTablesTest UEFI app tests/uefi-test-tools: add build scripts
tests/data: introduce "uefi-boot-images" with the "bios-tables-test"
ISOs.gitmodules | 3 + Makefile | 6 +- roms/Makefile | 13 +- roms/edk2 | 1 + tests/data/uefi-boot-images/bios-tables-test.aarch64.iso.qcow2 | Bin 0 -> 11776 bytes tests/data/uefi-boot-images/bios-tables-test.arm.iso.qcow2 | Bin 0 -> 11776 bytes tests/data/uefi-boot-images/bios-tables-test.i386.iso.qcow2 | Bin 0 -> 12800 bytes tests/data/uefi-boot-images/bios-tables-test.x86_64.iso.qcow2 | Bin 0 -> 13312 bytes tests/uefi-test-tools/.gitignore | 3 + tests/uefi-test-tools/LICENSE | 25 ++++ tests/uefi-test-tools/Makefile | 106 ++++++++++++++ tests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest/BiosTablesTest.c | 130 ++++++++++++++++++ tests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest/BiosTablesTest.inf | 41 ++++++ tests/uefi-test-tools/UefiTestToolsPkg/Include/Guid/BiosTablesTest.h | 67 +++++++++ tests/uefi-test-tools/UefiTestToolsPkg/UefiTestToolsPkg.dec | 27 ++++ tests/uefi-test-tools/UefiTestToolsPkg/UefiTestToolsPkg.dsc | 69 ++++++++++ tests/uefi-test-tools/build.sh | 145 ++++++++++++++++++++ 17 files changed, 628 insertions(+), 8 deletions(-) create mode 160000 roms/edk2 create mode 100644 tests/data/uefi-boot-images/bios-tables-test.aarch64.iso.qcow2 create mode 100644 tests/data/uefi-boot-images/bios-tables-test.arm.iso.qcow2 create mode 100644 tests/data/uefi-boot-images/bios-tables-test.i386.iso.qcow2 create mode 100644 tests/data/uefi-boot-images/bios-tables-test.x86_64.iso.qcow2 create mode 100644 tests/uefi-test-tools/.gitignore create mode 100644 tests/uefi-test-tools/LICENSE create mode 100644 tests/uefi-test-tools/Makefile create mode 100644 tests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest/BiosTablesTest.c create mode 100644 tests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest/BiosTablesTest.inf create mode 100644 tests/uefi-test-tools/UefiTestToolsPkg/Include/Guid/BiosTablesTest.h create mode 100644 tests/uefi-test-tools/UefiTestToolsPkg/UefiTestToolsPkg.dec create mode 100644 tests/uefi-test-tools/UefiTestToolsPkg/UefiTestToolsPkg.dsc create mode 100755 tests/uefi-test-tools/build.sh