Skip to contentSkip to conversation

add the BiosTablesTest UEFI app, build it with the new roms/edk2 submodule

PATCH Superseded Laszlo Ersek 4 participants 22 messages Jan 18, 2019 – Jan 24, 2019 Kbuild

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

cover add the BiosTablesTest UEFI app, build it with the new roms/edk2 submodule

Igor's series

[Qemu-devel] [PATCH 00/14] tests: acpi: add UEFI (ARM) testing support
http://mid.mail-archive.com/[email protected]

worked with my edk2 (guest firmware) series

[edk2] [PATCH 0/4] OvmfPkg, ArmVirtPkg: add ACPI Test Support
http://mid.mail-archive.com/[email protected]

as long as the guest firmware was built with the necessary extra build flag.

However, in the discussion under Igor's series, we seemed to reconsider a standalone UEFI application for the QEMU tree to carry. This series seeks to implement that option.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Ard Biesheuvel <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Philippe Mathieu-Daudé <[email protected]>
Cc Shannon Zhao <[email protected]>

Thanks,
Laszlo

Laszlo Ersek (5):
roms: add the edk2 project as a git submodule
roms: build the EfiRom utility from the roms/edk2 submodule
tests: introduce "uefi-test-tools" with the BiosTablesTest UEFI app tests/uefi-test-tools: add build scripts
tests/data: introduce "uefi-boot-images" with the "bios-tables-test"
ISOs

 .gitmodules                                                              |   3 +
 Makefile                                                                 |   6 +-
 roms/Makefile                                                            |  13 +-
 roms/edk2                                                                |   1 +
 tests/data/uefi-boot-images/bios-tables-test.aarch64.iso.qcow2           | Bin 0 -> 11776 bytes
 tests/data/uefi-boot-images/bios-tables-test.arm.iso.qcow2               | Bin 0 -> 11776 bytes
 tests/data/uefi-boot-images/bios-tables-test.i386.iso.qcow2              | Bin 0 -> 12800 bytes
 tests/data/uefi-boot-images/bios-tables-test.x86_64.iso.qcow2            | Bin 0 -> 13312 bytes
 tests/uefi-test-tools/.gitignore                                         |   3 +
 tests/uefi-test-tools/LICENSE                                            |  25 ++++
 tests/uefi-test-tools/Makefile                                           |  92 +++++++++++++
 tests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest/BiosTablesTest.c   | 130 ++++++++++++++++++
 tests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest/BiosTablesTest.inf |  41 ++++++
 tests/uefi-test-tools/UefiTestToolsPkg/Include/Guid/BiosTablesTest.h     |  67 +++++++++
 tests/uefi-test-tools/UefiTestToolsPkg/UefiTestToolsPkg.dec              |  27 ++++
 tests/uefi-test-tools/UefiTestToolsPkg/UefiTestToolsPkg.dsc              |  69 ++++++++++
 tests/uefi-test-tools/build.sh                                           | 145 ++++++++++++++++++++
 17 files changed, 614 insertions(+), 8 deletions(-)
 create mode 160000 roms/edk2
 create mode 100644 tests/data/uefi-boot-images/bios-tables-test.aarch64.iso.qcow2
 create mode 100644 tests/data/uefi-boot-images/bios-tables-test.arm.iso.qcow2
 create mode 100644 tests/data/uefi-boot-images/bios-tables-test.i386.iso.qcow2
 create mode 100644 tests/data/uefi-boot-images/bios-tables-test.x86_64.iso.qcow2
 create mode 100644 tests/uefi-test-tools/.gitignore
 create mode 100644 tests/uefi-test-tools/LICENSE
 create mode 100644 tests/uefi-test-tools/Makefile
 create mode 100644 tests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest/BiosTablesTest.c
 create mode 100644 tests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest/BiosTablesTest.inf
 create mode 100644 tests/uefi-test-tools/UefiTestToolsPkg/Include/Guid/BiosTablesTest.h
 create mode 100644 tests/uefi-test-tools/UefiTestToolsPkg/UefiTestToolsPkg.dec
 create mode 100644 tests/uefi-test-tools/UefiTestToolsPkg/UefiTestToolsPkg.dsc
 create mode 100755 tests/uefi-test-tools/build.sh
Signature · 1 line
2.19.1.3.g30247aa5d201
#1 lore raw

PATCH 1/5 roms: add the edk2 project as a git submodule

The roms/edk2 submodule can help with three goals:
- build the OVMF and ArmVirtQemu virtual UEFI firmware platforms (to be implemented later),
- build the EfiRom tool on the fly, which is used in roms/Makefile, for building the "efirom" target,
- build UEFI test applications (to be run in guests), for qtest support.

Edk2 commit 85588389222a3636baf0f9ed8227f2434af4c3f9 stands for the latest "stable tag", namely "edk2-stable201811".

The edk2 repository tracks some binary files that should not be removed by QEMU's top-level "make clean"; exempt the full pathnames from the "find" command.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Ard Biesheuvel <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Philippe Mathieu-Daudé <[email protected]>
Cc Shannon Zhao <[email protected]>
---
 Makefile    | 6 +++++-
 .gitmodules | 3 +++
 roms/edk2   | 1 +
 3 files changed, 9 insertions(+), 1 deletion(-)
Makefile +5 -1
index dccba1dca27f..1f768e2bcf8f 100644
@@ -602,7 +602,11 @@ clean:
 	rm -f config.mak op-i386.h opc-i386.h gen-op-i386.h op-arm.h opc-arm.h gen-op-arm.h
 	rm -f qemu-options.def
 	rm -f *.msi
-	find . \( -name '*.so' -o -name '*.dll' -o -name '*.mo' -o -name '*.[oda]' \) -type f -exec rm {} +
+	find . \( -name '*.so' -o -name '*.dll' -o -name '*.mo' -o -name '*.[oda]' \) -type f \
+		! -path ./roms/edk2/ArmPkg/Library/GccLto/liblto-aarch64.a \
+		! -path ./roms/edk2/ArmPkg/Library/GccLto/liblto-arm.a \
+		! -path ./roms/edk2/BaseTools/Source/Python/UPT/Dll/sqlite3.dll \
+		-exec rm {} +
 	rm -f $(filter-out %.tlb,$(TOOLS)) $(HELPERS-y) qemu-ga TAGS cscope.* *.pod *~ */*~
 	rm -f fsdev/*.pod scsi/*.pod
 	rm -f qemu-img-cmds.h
.gitmodules +3 -0
index 6b91176098c8..ceafb0ee29a0 100644
@@ -49,3 +49,6 @@
 [submodule "tests/fp/berkeley-softfloat-3"]
 	path = tests/fp/berkeley-softfloat-3
 	url = https://github.com/cota/berkeley-softfloat-3
+[submodule "roms/edk2"]
+	path = roms/edk2
+	url = https://github.com/tianocore/edk2.git
roms/edk2 new file +1 -0
new file mode 160000
index 000000000000..85588389222a
@@ -0,0 +1 @@
+Subproject commit 85588389222a3636baf0f9ed8227f2434af4c3f9
Signature · 1 line
2.19.1.3.g30247aa5d201
Reviewed-by on v1 1/5
#2 lore raw

Re: [Qemu-devel] [PATCH 1/5] roms: add the edk2 project as a git submodule

Quoted 7 lines · Laszlo Ersek

The roms/edk2 submodule can help with three goals:
- build the OVMF and ArmVirtQemu virtual UEFI firmware platforms (to be implemented later),
- build the EfiRom tool on the fly, which is used in roms/Makefile, for building the "efirom" target,
- build UEFI test applications (to be run in guests), for qtest support.

Quoted 6 lines

Edk2 commit 85588389222a3636baf0f9ed8227f2434af4c3f9 stands for the latest "stable tag", namely "edk2-stable201811".

The edk2 repository tracks some binary files that should not be removed by QEMU's top-level "make clean"; exempt the full pathnames from the "find" command.

Quoted 53 lines
Cc "Michael S. Tsirkin" <[email protected]>
Cc Ard Biesheuvel <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Philippe Mathieu-Daudé <[email protected]>
Cc Shannon Zhao <[email protected]>
---
 Makefile    | 6 +++++-
 .gitmodules | 3 +++
 roms/edk2   | 1 +
 3 files changed, 9 insertions(+), 1 deletion(-)
Makefile +5 -1
index dccba1dca27f..1f768e2bcf8f 100644
@@ -602,7 +602,11 @@ clean:
 	rm -f config.mak op-i386.h opc-i386.h gen-op-i386.h op-arm.h opc-arm.h gen-op-arm.h
 	rm -f qemu-options.def
 	rm -f *.msi
-	find . \( -name '*.so' -o -name '*.dll' -o -name '*.mo' -o -name '*.[oda]' \) -type f -exec rm {} +
+	find . \( -name '*.so' -o -name '*.dll' -o -name '*.mo' -o -name '*.[oda]' \) -type f \
+		! -path ./roms/edk2/ArmPkg/Library/GccLto/liblto-aarch64.a \
+		! -path ./roms/edk2/ArmPkg/Library/GccLto/liblto-arm.a \
+		! -path ./roms/edk2/BaseTools/Source/Python/UPT/Dll/sqlite3.dll \
+		-exec rm {} +
 	rm -f $(filter-out %.tlb,$(TOOLS)) $(HELPERS-y) qemu-ga TAGS cscope.* *.pod *~ */*~
 	rm -f fsdev/*.pod scsi/*.pod
 	rm -f qemu-img-cmds.h
.gitmodules +3 -0
index 6b91176098c8..ceafb0ee29a0 100644
@@ -49,3 +49,6 @@
 [submodule "tests/fp/berkeley-softfloat-3"]
 	path = tests/fp/berkeley-softfloat-3
 	url = https://github.com/cota/berkeley-softfloat-3
+[submodule "roms/edk2"]
+	path = roms/edk2
+	url = https://github.com/tianocore/edk2.git
roms/edk2 new file +1 -0
new file mode 160000
index 000000000000..85588389222a
@@ -0,0 +1 @@
+Subproject commit 85588389222a3636baf0f9ed8227f2434af4c3f9
Signature · 1 line
2.19.1.3.g30247aa5d201
#3 lore raw

Re: [Qemu-devel] [PATCH 1/5] roms: add the edk2 project as a git submodule

In reply to Laszlo Ersek

Cc'ing Daniel & Alex.

Quoted 32 lines · Laszlo Ersek

The roms/edk2 submodule can help with three goals:
- build the OVMF and ArmVirtQemu virtual UEFI firmware platforms (to be implemented later),
- build the EfiRom tool on the fly, which is used in roms/Makefile, for building the "efirom" target,
- build UEFI test applications (to be run in guests), for qtest support.

Edk2 commit 85588389222a3636baf0f9ed8227f2434af4c3f9 stands for the latest "stable tag", namely "edk2-stable201811".

The edk2 repository tracks some binary files that should not be removed by QEMU's top-level "make clean"; exempt the full pathnames from the "find" command.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Ard Biesheuvel <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Philippe Mathieu-Daudé <[email protected]>
Cc Shannon Zhao <[email protected]>
---
 Makefile    | 6 +++++-
 .gitmodules | 3 +++
 roms/edk2   | 1 +
 3 files changed, 9 insertions(+), 1 deletion(-)
Makefile +0 -0
index dccba1dca27f..1f768e2bcf8f 100644

Makefile clean: · line 608

Quoted 8 lines
Makefile +4 -1
@@ -602,7 +602,11 @@ clean:
 	rm -f config.mak op-i386.h opc-i386.h gen-op-i386.h op-arm.h opc-arm.h gen-op-arm.h
 	rm -f qemu-options.def
 	rm -f *.msi
-	find . \( -name '*.so' -o -name '*.dll' -o -name '*.mo' -o -name '*.[oda]' \) -type f -exec rm {} +
+	find . \( -name '*.so' -o -name '*.dll' -o -name '*.mo' -o -name '*.[oda]' \) -type f \
+		! -path ./roms/edk2/ArmPkg/Library/GccLto/liblto-aarch64.a \
+		! -path ./roms/edk2/ArmPkg/Library/GccLto/liblto-arm.a \
+		! -path ./roms/edk2/BaseTools/Source/Python/UPT/Dll/sqlite3.dll \

Hmm I never try in-tree builds and thought this rule was for the archived release out of git, so this wouldn't matter.

Quoted 9 lines
Makefile +1 -0
+		-exec rm {} +
 	rm -f $(filter-out %.tlb,$(TOOLS)) $(HELPERS-y) qemu-ga TAGS cscope.* *.pod *~ */*~
 	rm -f fsdev/*.pod scsi/*.pod
 	rm -f qemu-img-cmds.h
.gitmodules +0 -0
index 6b91176098c8..ceafb0ee29a0 100644

.gitmodules line 54

Quoted 6 lines
.gitmodules +3 -0
@@ -49,3 +49,6 @@
 [submodule "tests/fp/berkeley-softfloat-3"]
 	path = tests/fp/berkeley-softfloat-3
 	url = https://github.com/cota/berkeley-softfloat-3
+[submodule "roms/edk2"]
+	path = roms/edk2
+	url = https://github.com/tianocore/edk2.git

Indeed this is enough to build EfiRom.
However if you want to build firmwares (your patch 4/5), the submodule is incomplete, until you manually run "submodule update --init --recursive".

I suggest the following change, Daniel/Alex what do you think?

-- >8 --

scripts/git-submodule.sh +1 -1
index 98ca0f2737..5441fe3385 100755
@@ -71,7 +71,7 @@ update)
         exit 0
     fi
 
-    $GIT submodule update --init $modules 1>/dev/null
+    $GIT submodule update --init --recursive $modules 1>/dev/null
     test $? -ne 0 && error "failed to update modules"
 
     $GIT submodule status $modules > "${substat}"

---

Quoted 8 lines
roms/edk2 new file +1 -0
new file mode 160000
index 000000000000..85588389222a
@@ -0,0 +1 @@
+Subproject commit 85588389222a3636baf0f9ed8227f2434af4c3f9
#4 lore raw

Re: [Qemu-devel] [PATCH 1/5] roms: add the edk2 project as a git submodule

Quoted 30 lines · Philippe Mathieu-Daudé

Cc'ing Daniel & Alex.

Quoted 27 lines · Laszlo Ersek

The roms/edk2 submodule can help with three goals:
- build the OVMF and ArmVirtQemu virtual UEFI firmware platforms (to be implemented later),
- build the EfiRom tool on the fly, which is used in roms/Makefile, for building the "efirom" target,
- build UEFI test applications (to be run in guests), for qtest support.

Edk2 commit 85588389222a3636baf0f9ed8227f2434af4c3f9 stands for the latest "stable tag", namely "edk2-stable201811".

The edk2 repository tracks some binary files that should not be removed by QEMU's top-level "make clean"; exempt the full pathnames from the "find" command.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Ard Biesheuvel <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Philippe Mathieu-Daudé <[email protected]>
Cc Shannon Zhao <[email protected]>
---
 Makefile    | 6 +++++-
 .gitmodules | 3 +++
 roms/edk2   | 1 +
 3 files changed, 9 insertions(+), 1 deletion(-)
Quoted 16 lines

Makefile clean: · line 608

Quoted 13 lines
Makefile +4 -1
index dccba1dca27f..1f768e2bcf8f 100644
@@ -602,7 +602,11 @@ clean:
 	rm -f config.mak op-i386.h opc-i386.h gen-op-i386.h op-arm.h opc-arm.h gen-op-arm.h
 	rm -f qemu-options.def
 	rm -f *.msi
-	find . \( -name '*.so' -o -name '*.dll' -o -name '*.mo' -o -name '*.[oda]' \) -type f -exec rm {} +
+	find . \( -name '*.so' -o -name '*.dll' -o -name '*.mo' -o -name '*.[oda]' \) -type f \
+		! -path ./roms/edk2/ArmPkg/Library/GccLto/liblto-aarch64.a \
+		! -path ./roms/edk2/ArmPkg/Library/GccLto/liblto-arm.a \
+		! -path ./roms/edk2/BaseTools/Source/Python/UPT/Dll/sqlite3.dll \

Hmm I never try in-tree builds and thought this rule was for the archived release out of git, so this wouldn't matter.

That's not correct; before I posted the series, I downloaded the then-latest tarball release, and I saw that the roms/ submoule trees were flattened into it. Try:

wget -O - -q https://download.qemu.org/qemu-3.1.0.tar.xz \
| tar -t --xz \
| grep roms/

Quoted 12 lines
Quoted 12 lines
Makefile +1 -0
+		-exec rm {} +
 	rm -f $(filter-out %.tlb,$(TOOLS)) $(HELPERS-y) qemu-ga TAGS cscope.* *.pod *~ */*~
 	rm -f fsdev/*.pod scsi/*.pod
 	rm -f qemu-img-cmds.h
.gitmodules +0 -0
index 6b91176098c8..ceafb0ee29a0 100644
@@ -49,3 +49,6 @@
 [submodule "tests/fp/berkeley-softfloat-3"]
 	path = tests/fp/berkeley-softfloat-3
Quoted 8 lines

.gitmodules line 54

Quoted 4 lines
.gitmodules +3 -0
 	url = https://github.com/cota/berkeley-softfloat-3
+[submodule "roms/edk2"]
+	path = roms/edk2
+	url = https://github.com/tianocore/edk2.git

Indeed this is enough to build EfiRom.
However if you want to build firmwares (your patch 4/5), the submodule is incomplete, until you manually run "submodule update --init --recursive".

That applies equally to the other submodules (firmware or otherwise) that we have under roms/. Edk2 is not being added as a nested submodule, it is a sibling; so whatever tooling covers e.g. roms/seabios and roms/ipxe, the same tooling should cover roms/edk2 too.

Quoted 9 lines

I suggest the following change, Daniel/Alex what do you think?

-- >8 --

scripts/git-submodule.sh +0 -0
index 98ca0f2737..5441fe3385 100755

scripts/git-submodule.sh update)

Quoted 9 lines
scripts/git-submodule.sh +1 -2
@@ -71,7 +71,7 @@ update)
         exit 0
     fi
 
-    $GIT submodule update --init $modules 1>/dev/null
+    $GIT submodule update --init --recursive $modules 1>/dev/null
     test $? -ne 0 && error "failed to update modules"
 
     $GIT submodule status $modules > "${substat}"
---

This shouldn't make a difference, as edk2 is not a nested submodule of another submodule; it is a submodule directly under the main superproject, i.e. QEMU.

Thanks,
Laszlo

Quoted 9 lines
Quoted 9 lines
roms/edk2 new file +1 -0
new file mode 160000
index 000000000000..85588389222a
@@ -0,0 +1 @@
+Subproject commit 85588389222a3636baf0f9ed8227f2434af4c3f9
#5 lore raw

Re: [Qemu-devel] [PATCH 1/5] roms: add the edk2 project as a git submodule

Quoted 48 lines · Laszlo Ersek
Quoted 30 lines · Philippe Mathieu-Daudé

Cc'ing Daniel & Alex.

Quoted 27 lines · Laszlo Ersek

The roms/edk2 submodule can help with three goals:
- build the OVMF and ArmVirtQemu virtual UEFI firmware platforms (to be implemented later),
- build the EfiRom tool on the fly, which is used in roms/Makefile, for building the "efirom" target,
- build UEFI test applications (to be run in guests), for qtest support.

Edk2 commit 85588389222a3636baf0f9ed8227f2434af4c3f9 stands for the latest "stable tag", namely "edk2-stable201811".

The edk2 repository tracks some binary files that should not be removed by QEMU's top-level "make clean"; exempt the full pathnames from the "find" command.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Ard Biesheuvel <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Philippe Mathieu-Daudé <[email protected]>
Cc Shannon Zhao <[email protected]>
---
 Makefile    | 6 +++++-
 .gitmodules | 3 +++
 roms/edk2   | 1 +
 3 files changed, 9 insertions(+), 1 deletion(-)
Quoted 16 lines

Makefile clean: · line 608

Quoted 13 lines
Makefile +4 -1
index dccba1dca27f..1f768e2bcf8f 100644
@@ -602,7 +602,11 @@ clean:
 	rm -f config.mak op-i386.h opc-i386.h gen-op-i386.h op-arm.h opc-arm.h gen-op-arm.h
 	rm -f qemu-options.def
 	rm -f *.msi
-	find . \( -name '*.so' -o -name '*.dll' -o -name '*.mo' -o -name '*.[oda]' \) -type f -exec rm {} +
+	find . \( -name '*.so' -o -name '*.dll' -o -name '*.mo' -o -name '*.[oda]' \) -type f \
+		! -path ./roms/edk2/ArmPkg/Library/GccLto/liblto-aarch64.a \
+		! -path ./roms/edk2/ArmPkg/Library/GccLto/liblto-arm.a \
+		! -path ./roms/edk2/BaseTools/Source/Python/UPT/Dll/sqlite3.dll \

Hmm I never try in-tree builds and thought this rule was for the archived release out of git, so this wouldn't matter.

Quoted 3 lines

That's not correct; before I posted the series, I downloaded the then-latest tarball release, and I saw that the roms/ submoule trees were flattened into it. Try:

My previous sentence is probably incorrect in english, let me reword as:
"I thought (incorrectly) this rule wouldn't matter for in-tree builds". So I see it does matter, and your change do make sens.

Quoted 26 lines

wget -O - -q https://download.qemu.org/qemu-3.1.0.tar.xz \
| tar -t --xz \
| grep roms/

Quoted 12 lines
Quoted 12 lines
Makefile +1 -0
+		-exec rm {} +
 	rm -f $(filter-out %.tlb,$(TOOLS)) $(HELPERS-y) qemu-ga TAGS cscope.* *.pod *~ */*~
 	rm -f fsdev/*.pod scsi/*.pod
 	rm -f qemu-img-cmds.h
.gitmodules +0 -0
index 6b91176098c8..ceafb0ee29a0 100644
@@ -49,3 +49,6 @@
 [submodule "tests/fp/berkeley-softfloat-3"]
 	path = tests/fp/berkeley-softfloat-3
Quoted 8 lines

.gitmodules line 54

Quoted 4 lines
.gitmodules +3 -0
 	url = https://github.com/cota/berkeley-softfloat-3
+[submodule "roms/edk2"]
+	path = roms/edk2
+	url = https://github.com/tianocore/edk2.git

Indeed this is enough to build EfiRom.
However if you want to build firmwares (your patch 4/5), the submodule is incomplete, until you manually run "submodule update --init --recursive".

Quoted 4 lines

That applies equally to the other submodules (firmware or otherwise) that we have under roms/. Edk2 is not being added as a nested submodule, it is a sibling; so whatever tooling covers e.g. roms/seabios and roms/ipxe, the same tooling should cover roms/edk2 too.

OK.

Quoted 16 lines
Quoted 16 lines

I suggest the following change, Daniel/Alex what do you think?

-- >8 --

scripts/git-submodule.sh +1 -1
index 98ca0f2737..5441fe3385 100755
@@ -71,7 +71,7 @@ update)
         exit 0
     fi
 
-    $GIT submodule update --init $modules 1>/dev/null
+    $GIT submodule update --init --recursive $modules 1>/dev/null
     test $? -ne 0 && error "failed to update modules"
 
Quoted 6 lines

scripts/git-submodule.sh update)

Quoted 2 lines
scripts/git-submodule.sh +0 -1
     $GIT submodule status $modules > "${substat}"
---

This shouldn't make a difference, as edk2 is not a nested submodule of another submodule; it is a submodule directly under the main superproject, i.e. QEMU.

OK (as long as we don't want to build OVMF within QEMU to run QEMU tests).

Quoted 14 lines

Thanks,
Laszlo

Quoted 9 lines
Quoted 9 lines
roms/edk2 new file +1 -0
new file mode 160000
index 000000000000..85588389222a
@@ -0,0 +1 @@
+Subproject commit 85588389222a3636baf0f9ed8227f2434af4c3f9
#6 lore raw

Re: [Qemu-devel] [PATCH 1/5] roms: add the edk2 project as a git submodule

Quoted 18 lines · Philippe Mathieu-Daudé
Quoted 17 lines · Laszlo Ersek
Quoted 15 lines · Philippe Mathieu-Daudé

I suggest the following change, Daniel/Alex what do you think?

-- >8 --

scripts/git-submodule.sh +1 -1
index 98ca0f2737..5441fe3385 100755
@@ -71,7 +71,7 @@ update)
         exit 0
     fi
 
-    $GIT submodule update --init $modules 1>/dev/null
+    $GIT submodule update --init --recursive $modules 1>/dev/null
     test $? -ne 0 && error "failed to update modules"
 
Quoted 8 lines
Quoted 6 lines

scripts/git-submodule.sh update)

Quoted 2 lines
scripts/git-submodule.sh +0 -1
     $GIT submodule status $modules > "${substat}"
---

This shouldn't make a difference, as edk2 is not a nested submodule of another submodule; it is a submodule directly under the main superproject, i.e. QEMU.

OK (as long as we don't want to build OVMF within QEMU to run QEMU tests).

I think if / whenever we bundle OVMF binaries with QEMU, there'll be a two step process (like with other roms); maintainers will (re)build the binaries and commit/merge them, and tests will use the bundled binaries as they are.

Thanks
Laszlo

#7 lore raw

PATCH 2/5 roms: build the EfiRom utility from the roms/edk2 submodule

In reply to Laszlo Ersek

Building the EfiRom utility from "roms/edk2/BaseTools" should make "roms/Makefile" more self-contained. Otherwise, we'd call the system-wide EfiRom for building the combined iPXE option ROMs, but call the sibling utilities from "roms/edk2/BaseTools" for building "roms/edk2" content.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Ard Biesheuvel <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Philippe Mathieu-Daudé <[email protected]>
Cc Shannon Zhao <[email protected]>
---
 roms/Makefile | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)
roms/Makefile +6 -7
index a6043eff37e9..78d5dd18c301 100644
@@ -47,10 +47,7 @@ SEABIOS_EXTRAVERSION="-prebuilt.qemu.org"
 # We need that to combine multiple images (legacy bios,
 # efi ia32, efi x64) into a single rom binary.
 #
-# We try to find it in the path.  You can also pass the location on
-# the command line, i.e. "make EFIROM=/path/to/EfiRom efirom"
-#
-EFIROM ?= $(shell which EfiRom 2>/dev/null)
+EFIROM = edk2/BaseTools/Source/C/bin/EfiRom
 
 default:
 	@echo "nothing is build by default"
@@ -59,8 +56,7 @@ default:
 	@echo "  vgabios        -- update vgabios binaries (seabios)"
 	@echo "  sgabios        -- update sgabios binaries"
 	@echo "  pxerom         -- update nic roms (bios only)"
-	@echo "  efirom         -- update nic roms (bios+efi, this needs"
-	@echo "                    the EfiRom utility from edk2 / tianocore)"
+	@echo "  efirom         -- update nic roms (bios+efi)"
 	@echo "  slof           -- update slof.bin"
 	@echo "  skiboot        -- update skiboot.lid"
 	@echo "  u-boot.e500    -- update u-boot.e500"
@@ -106,7 +102,7 @@ pxe-rom-%: build-pxe-roms
 
 efirom: $(patsubst %,efi-rom-%,$(pxerom_variants))
 
-efi-rom-%: build-pxe-roms build-efi-roms
+efi-rom-%: build-pxe-roms build-efi-roms $(EFIROM)
 	$(EFIROM) -f "0x$(VID)" -i "0x$(DID)" -l 0x02 \
 		-b ipxe/src/bin/$(VID)$(DID).rom \
 		-ec ipxe/src/bin-i386-efi/$(VID)$(DID).efidrv \
@@ -124,6 +120,8 @@ build-efi-roms: build-pxe-roms
 		$(patsubst %,bin-i386-efi/%.efidrv,$(pxerom_targets)) \
 		$(patsubst %,bin-x86_64-efi/%.efidrv,$(pxerom_targets))
 
+$(EFIROM):
+	$(MAKE) -C edk2/BaseTools
 
 slof:
 	$(MAKE) -C SLOF CROSS=$(powerpc64_cross_prefix) qemu
@@ -150,6 +148,7 @@ clean:
 	$(MAKE) -C sgabios clean
 	rm -f sgabios/.depend
 	$(MAKE) -C ipxe/src veryclean
+	$(MAKE) -C edk2/BaseTools clean
 	$(MAKE) -C SLOF clean
 	rm -rf u-boot/build.e500
 	$(MAKE) -C u-boot-sam460ex distclean
Signature · 1 line
2.19.1.3.g30247aa5d201
Reviewed-by on v1 2/5
#8 lore raw

Re: [Qemu-devel] [PATCH 2/5] roms: build the EfiRom utility from the roms/edk2 submodule

Quoted 5 lines · Laszlo Ersek

Building the EfiRom utility from "roms/edk2/BaseTools" should make "roms/Makefile" more self-contained. Otherwise, we'd call the system-wide EfiRom for building the combined iPXE option ROMs, but call the sibling utilities from "roms/edk2/BaseTools" for building "roms/edk2" content.

Quoted 7 lines
Cc "Michael S. Tsirkin" <[email protected]>
Cc Ard Biesheuvel <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Philippe Mathieu-Daudé <[email protected]>
Cc Shannon Zhao <[email protected]>
Quoted 60 lines
---
 roms/Makefile | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)
roms/Makefile +6 -7
index a6043eff37e9..78d5dd18c301 100644
@@ -47,10 +47,7 @@ SEABIOS_EXTRAVERSION="-prebuilt.qemu.org"
 # We need that to combine multiple images (legacy bios,
 # efi ia32, efi x64) into a single rom binary.
 #
-# We try to find it in the path.  You can also pass the location on
-# the command line, i.e. "make EFIROM=/path/to/EfiRom efirom"
-#
-EFIROM ?= $(shell which EfiRom 2>/dev/null)
+EFIROM = edk2/BaseTools/Source/C/bin/EfiRom
 
 default:
 	@echo "nothing is build by default"
@@ -59,8 +56,7 @@ default:
 	@echo "  vgabios        -- update vgabios binaries (seabios)"
 	@echo "  sgabios        -- update sgabios binaries"
 	@echo "  pxerom         -- update nic roms (bios only)"
-	@echo "  efirom         -- update nic roms (bios+efi, this needs"
-	@echo "                    the EfiRom utility from edk2 / tianocore)"
+	@echo "  efirom         -- update nic roms (bios+efi)"
 	@echo "  slof           -- update slof.bin"
 	@echo "  skiboot        -- update skiboot.lid"
 	@echo "  u-boot.e500    -- update u-boot.e500"
@@ -106,7 +102,7 @@ pxe-rom-%: build-pxe-roms
 
 efirom: $(patsubst %,efi-rom-%,$(pxerom_variants))
 
-efi-rom-%: build-pxe-roms build-efi-roms
+efi-rom-%: build-pxe-roms build-efi-roms $(EFIROM)
 	$(EFIROM) -f "0x$(VID)" -i "0x$(DID)" -l 0x02 \
 		-b ipxe/src/bin/$(VID)$(DID).rom \
 		-ec ipxe/src/bin-i386-efi/$(VID)$(DID).efidrv \
@@ -124,6 +120,8 @@ build-efi-roms: build-pxe-roms
 		$(patsubst %,bin-i386-efi/%.efidrv,$(pxerom_targets)) \
 		$(patsubst %,bin-x86_64-efi/%.efidrv,$(pxerom_targets))
 
+$(EFIROM):
+	$(MAKE) -C edk2/BaseTools
 
 slof:
 	$(MAKE) -C SLOF CROSS=$(powerpc64_cross_prefix) qemu
@@ -150,6 +148,7 @@ clean:
 	$(MAKE) -C sgabios clean
 	rm -f sgabios/.depend
 	$(MAKE) -C ipxe/src veryclean
+	$(MAKE) -C edk2/BaseTools clean
 	$(MAKE) -C SLOF clean
 	rm -rf u-boot/build.e500
 	$(MAKE) -C u-boot-sam460ex distclean
Signature · 1 line
2.19.1.3.g30247aa5d201
Reviewed-by Tested-by on v1 2/5
#9 lore raw

Re: [Qemu-devel] [PATCH 2/5] roms: build the EfiRom utility from the roms/edk2 submodule

In reply to Laszlo Ersek

Hi Laszlo,

Quoted 52 lines · Laszlo Ersek

Building the EfiRom utility from "roms/edk2/BaseTools" should make "roms/Makefile" more self-contained. Otherwise, we'd call the system-wide EfiRom for building the combined iPXE option ROMs, but call the sibling utilities from "roms/edk2/BaseTools" for building "roms/edk2" content.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Ard Biesheuvel <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Philippe Mathieu-Daudé <[email protected]>
Cc Shannon Zhao <[email protected]>
---
 roms/Makefile | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)
roms/Makefile +3 -7
index a6043eff37e9..78d5dd18c301 100644
@@ -47,10 +47,7 @@ SEABIOS_EXTRAVERSION="-prebuilt.qemu.org"
 # We need that to combine multiple images (legacy bios,
 # efi ia32, efi x64) into a single rom binary.
 #
-# We try to find it in the path.  You can also pass the location on
-# the command line, i.e. "make EFIROM=/path/to/EfiRom efirom"
-#
-EFIROM ?= $(shell which EfiRom 2>/dev/null)
+EFIROM = edk2/BaseTools/Source/C/bin/EfiRom
 
 default:
 	@echo "nothing is build by default"
@@ -59,8 +56,7 @@ default:
 	@echo "  vgabios        -- update vgabios binaries (seabios)"
 	@echo "  sgabios        -- update sgabios binaries"
 	@echo "  pxerom         -- update nic roms (bios only)"
-	@echo "  efirom         -- update nic roms (bios+efi, this needs"
-	@echo "                    the EfiRom utility from edk2 / tianocore)"
+	@echo "  efirom         -- update nic roms (bios+efi)"
 	@echo "  slof           -- update slof.bin"
 	@echo "  skiboot        -- update skiboot.lid"
 	@echo "  u-boot.e500    -- update u-boot.e500"
@@ -106,7 +102,7 @@ pxe-rom-%: build-pxe-roms
 
 efirom: $(patsubst %,efi-rom-%,$(pxerom_variants))
 
-efi-rom-%: build-pxe-roms build-efi-roms
+efi-rom-%: build-pxe-roms build-efi-roms $(EFIROM)
 	$(EFIROM) -f "0x$(VID)" -i "0x$(DID)" -l 0x02 \
 		-b ipxe/src/bin/$(VID)$(DID).rom \
 		-ec ipxe/src/bin-i386-efi/$(VID)$(DID).efidrv \

roms/Makefile build-efi-roms: build-pxe-roms · line 124

Quoted 5 lines
roms/Makefile +2 -0
@@ -124,6 +120,8 @@ build-efi-roms: build-pxe-roms
 		$(patsubst %,bin-i386-efi/%.efidrv,$(pxerom_targets)) \
 		$(patsubst %,bin-x86_64-efi/%.efidrv,$(pxerom_targets))
 
+$(EFIROM):
+	$(MAKE) -C edk2/BaseTools

Since this is a single tool/file, can we simply use:

        $(MAKE) -C edk2/BaseTools/Source/C/EfiRom

roms/Makefile clean: · line 151

Quoted 8 lines
roms/Makefile +1 -0
 slof:
 	$(MAKE) -C SLOF CROSS=$(powerpc64_cross_prefix) qemu
@@ -150,6 +148,7 @@ clean:
 	$(MAKE) -C sgabios clean
 	rm -f sgabios/.depend
 	$(MAKE) -C ipxe/src veryclean
+	$(MAKE) -C edk2/BaseTools clean

Ditto.

roms/Makefile clean: · line 154

Quoted 4 lines
roms/Makefile +0 -0
 	$(MAKE) -C SLOF clean
 	rm -rf u-boot/build.e500
 	$(MAKE) -C u-boot-sam460ex distclean

Whichever path used:

#10 lore raw

Re: [Qemu-devel] [PATCH 2/5] roms: build the EfiRom utility from the roms/edk2 submodule

Quoted 58 lines · Philippe Mathieu-Daudé

Hi Laszlo,

Quoted 55 lines · Laszlo Ersek

Building the EfiRom utility from "roms/edk2/BaseTools" should make "roms/Makefile" more self-contained. Otherwise, we'd call the system-wide EfiRom for building the combined iPXE option ROMs, but call the sibling utilities from "roms/edk2/BaseTools" for building "roms/edk2" content.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Ard Biesheuvel <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Philippe Mathieu-Daudé <[email protected]>
Cc Shannon Zhao <[email protected]>
---
 roms/Makefile | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)
roms/Makefile +3 -7
index a6043eff37e9..78d5dd18c301 100644
@@ -47,10 +47,7 @@ SEABIOS_EXTRAVERSION="-prebuilt.qemu.org"
 # We need that to combine multiple images (legacy bios,
 # efi ia32, efi x64) into a single rom binary.
 #
-# We try to find it in the path.  You can also pass the location on
-# the command line, i.e. "make EFIROM=/path/to/EfiRom efirom"
-#
-EFIROM ?= $(shell which EfiRom 2>/dev/null)
+EFIROM = edk2/BaseTools/Source/C/bin/EfiRom
 
 default:
 	@echo "nothing is build by default"
@@ -59,8 +56,7 @@ default:
 	@echo "  vgabios        -- update vgabios binaries (seabios)"
 	@echo "  sgabios        -- update sgabios binaries"
 	@echo "  pxerom         -- update nic roms (bios only)"
-	@echo "  efirom         -- update nic roms (bios+efi, this needs"
-	@echo "                    the EfiRom utility from edk2 / tianocore)"
+	@echo "  efirom         -- update nic roms (bios+efi)"
 	@echo "  slof           -- update slof.bin"
 	@echo "  skiboot        -- update skiboot.lid"
 	@echo "  u-boot.e500    -- update u-boot.e500"
@@ -106,7 +102,7 @@ pxe-rom-%: build-pxe-roms
 
 efirom: $(patsubst %,efi-rom-%,$(pxerom_variants))
 
-efi-rom-%: build-pxe-roms build-efi-roms
+efi-rom-%: build-pxe-roms build-efi-roms $(EFIROM)
 	$(EFIROM) -f "0x$(VID)" -i "0x$(DID)" -l 0x02 \
 		-b ipxe/src/bin/$(VID)$(DID).rom \
 		-ec ipxe/src/bin-i386-efi/$(VID)$(DID).efidrv \
@@ -124,6 +120,8 @@ build-efi-roms: build-pxe-roms
 		$(patsubst %,bin-i386-efi/%.efidrv,$(pxerom_targets)) \
 		$(patsubst %,bin-x86_64-efi/%.efidrv,$(pxerom_targets))
 
Quoted 6 lines

roms/Makefile build-efi-roms: build-pxe-roms · line 124

Quoted 2 lines
roms/Makefile +2 -0
+$(EFIROM):
+	$(MAKE) -C edk2/BaseTools

Since this is a single tool/file, can we simply use:

        $(MAKE) -C edk2/BaseTools/Source/C/EfiRom

Also, note that this doesn't work for out-of-tree builds, but this problem is pre-existent to your series.

Quoted 21 lines

roms/Makefile clean: · line 151

Quoted 9 lines
roms/Makefile +1 -0
 slof:
 	$(MAKE) -C SLOF CROSS=$(powerpc64_cross_prefix) qemu
@@ -150,6 +148,7 @@ clean:
 	$(MAKE) -C sgabios clean
 	rm -f sgabios/.depend
 	$(MAKE) -C ipxe/src veryclean
+	$(MAKE) -C edk2/BaseTools clean

Ditto.

roms/Makefile clean: · line 154

Quoted 4 lines
roms/Makefile +0 -0
 	$(MAKE) -C SLOF clean
 	rm -rf u-boot/build.e500
 	$(MAKE) -C u-boot-sam460ex distclean

Whichever path used:

#11 lore raw

Re: [Qemu-devel] [PATCH 2/5] roms: build the EfiRom utility from the roms/edk2 submodule

Quoted 64 lines · Philippe Mathieu-Daudé
Quoted 63 lines · Philippe Mathieu-Daudé

Hi Laszlo,

Quoted 52 lines · Laszlo Ersek

Building the EfiRom utility from "roms/edk2/BaseTools" should make "roms/Makefile" more self-contained. Otherwise, we'd call the system-wide EfiRom for building the combined iPXE option ROMs, but call the sibling utilities from "roms/edk2/BaseTools" for building "roms/edk2" content.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Ard Biesheuvel <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Philippe Mathieu-Daudé <[email protected]>
Cc Shannon Zhao <[email protected]>
---
 roms/Makefile | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)
roms/Makefile +3 -7
index a6043eff37e9..78d5dd18c301 100644
@@ -47,10 +47,7 @@ SEABIOS_EXTRAVERSION="-prebuilt.qemu.org"
 # We need that to combine multiple images (legacy bios,
 # efi ia32, efi x64) into a single rom binary.
 #
-# We try to find it in the path.  You can also pass the location on
-# the command line, i.e. "make EFIROM=/path/to/EfiRom efirom"
-#
-EFIROM ?= $(shell which EfiRom 2>/dev/null)
+EFIROM = edk2/BaseTools/Source/C/bin/EfiRom
 
 default:
 	@echo "nothing is build by default"
@@ -59,8 +56,7 @@ default:
 	@echo "  vgabios        -- update vgabios binaries (seabios)"
 	@echo "  sgabios        -- update sgabios binaries"
 	@echo "  pxerom         -- update nic roms (bios only)"
-	@echo "  efirom         -- update nic roms (bios+efi, this needs"
-	@echo "                    the EfiRom utility from edk2 / tianocore)"
+	@echo "  efirom         -- update nic roms (bios+efi)"
 	@echo "  slof           -- update slof.bin"
 	@echo "  skiboot        -- update skiboot.lid"
 	@echo "  u-boot.e500    -- update u-boot.e500"
@@ -106,7 +102,7 @@ pxe-rom-%: build-pxe-roms
 
 efirom: $(patsubst %,efi-rom-%,$(pxerom_variants))
 
-efi-rom-%: build-pxe-roms build-efi-roms
+efi-rom-%: build-pxe-roms build-efi-roms $(EFIROM)
 	$(EFIROM) -f "0x$(VID)" -i "0x$(DID)" -l 0x02 \
 		-b ipxe/src/bin/$(VID)$(DID).rom \
 		-ec ipxe/src/bin-i386-efi/$(VID)$(DID).efidrv \

roms/Makefile build-efi-roms: build-pxe-roms · line 124

Quoted 5 lines
roms/Makefile +2 -0
@@ -124,6 +120,8 @@ build-efi-roms: build-pxe-roms
 		$(patsubst %,bin-i386-efi/%.efidrv,$(pxerom_targets)) \
 		$(patsubst %,bin-x86_64-efi/%.efidrv,$(pxerom_targets))
 
+$(EFIROM):
+	$(MAKE) -C edk2/BaseTools

Since this is a single tool/file, can we simply use:

Quoted 4 lines
Quoted 1 line
        $(MAKE) -C edk2/BaseTools/Source/C/EfiRom

Also, note that this doesn't work for out-of-tree builds, but this problem is pre-existent to your series.

Right, "roms/Makefile" is unrelated to building QEMU itself (which is supposed to work outside of the source tree); it is for maintainers when they intend to refresh the bundled fw stuff.

Thanks
Laszlo

Quoted 22 lines
Quoted 22 lines

roms/Makefile clean: · line 151

Quoted 10 lines
roms/Makefile +1 -0
 slof:
 	$(MAKE) -C SLOF CROSS=$(powerpc64_cross_prefix) qemu
@@ -150,6 +148,7 @@ clean:
 	$(MAKE) -C sgabios clean
 	rm -f sgabios/.depend
 	$(MAKE) -C ipxe/src veryclean
+	$(MAKE) -C edk2/BaseTools clean

Ditto.

roms/Makefile clean: · line 154

Quoted 4 lines
roms/Makefile +0 -0
 	$(MAKE) -C SLOF clean
 	rm -rf u-boot/build.e500
 	$(MAKE) -C u-boot-sam460ex distclean

Whichever path used:

#12 lore raw

Re: [Qemu-devel] [PATCH 2/5] roms: build the EfiRom utility from the roms/edk2 submodule

In reply to Philippe Mathieu-Daudé
Quoted 58 lines · Philippe Mathieu-Daudé

Hi Laszlo,

Quoted 55 lines · Laszlo Ersek

Building the EfiRom utility from "roms/edk2/BaseTools" should make "roms/Makefile" more self-contained. Otherwise, we'd call the system-wide EfiRom for building the combined iPXE option ROMs, but call the sibling utilities from "roms/edk2/BaseTools" for building "roms/edk2" content.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Ard Biesheuvel <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Philippe Mathieu-Daudé <[email protected]>
Cc Shannon Zhao <[email protected]>
---
 roms/Makefile | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)
roms/Makefile +3 -7
index a6043eff37e9..78d5dd18c301 100644
@@ -47,10 +47,7 @@ SEABIOS_EXTRAVERSION="-prebuilt.qemu.org"
 # We need that to combine multiple images (legacy bios,
 # efi ia32, efi x64) into a single rom binary.
 #
-# We try to find it in the path.  You can also pass the location on
-# the command line, i.e. "make EFIROM=/path/to/EfiRom efirom"
-#
-EFIROM ?= $(shell which EfiRom 2>/dev/null)
+EFIROM = edk2/BaseTools/Source/C/bin/EfiRom
 
 default:
 	@echo "nothing is build by default"
@@ -59,8 +56,7 @@ default:
 	@echo "  vgabios        -- update vgabios binaries (seabios)"
 	@echo "  sgabios        -- update sgabios binaries"
 	@echo "  pxerom         -- update nic roms (bios only)"
-	@echo "  efirom         -- update nic roms (bios+efi, this needs"
-	@echo "                    the EfiRom utility from edk2 / tianocore)"
+	@echo "  efirom         -- update nic roms (bios+efi)"
 	@echo "  slof           -- update slof.bin"
 	@echo "  skiboot        -- update skiboot.lid"
 	@echo "  u-boot.e500    -- update u-boot.e500"
@@ -106,7 +102,7 @@ pxe-rom-%: build-pxe-roms
 
 efirom: $(patsubst %,efi-rom-%,$(pxerom_variants))
 
-efi-rom-%: build-pxe-roms build-efi-roms
+efi-rom-%: build-pxe-roms build-efi-roms $(EFIROM)
 	$(EFIROM) -f "0x$(VID)" -i "0x$(DID)" -l 0x02 \
 		-b ipxe/src/bin/$(VID)$(DID).rom \
 		-ec ipxe/src/bin-i386-efi/$(VID)$(DID).efidrv \
@@ -124,6 +120,8 @@ build-efi-roms: build-pxe-roms
 		$(patsubst %,bin-i386-efi/%.efidrv,$(pxerom_targets)) \
 		$(patsubst %,bin-x86_64-efi/%.efidrv,$(pxerom_targets))
 
Quoted 6 lines

roms/Makefile build-efi-roms: build-pxe-roms · line 124

Quoted 2 lines
roms/Makefile +2 -0
+$(EFIROM):
+	$(MAKE) -C edk2/BaseTools

Since this is a single tool/file, can we simply use:

        $(MAKE) -C edk2/BaseTools/Source/C/EfiRom

Unfortunately, the BaseTools makefile(s) aren't granular enough for this. They don't track dependencies well enough, at the individual tool level. The above command would fail:

-------
make: Entering directory `/home/lacos/src/upstream/edk2/BaseTools/Source/C/EfiRom'
mkdir ../bin
gcc -c -I .. -I ../Include/Common -I ../Include/ -I ../Include/IndustryStandard -I ../Common/ -I .. -I . -I ../Include/X64/ -MD -fshort-wchar -fno-strict-aliasing -Wall -Werror -Wno-deprecated-declarations -Wno-stringop-truncation -Wno-restrict -Wno-unused-result -nostdlib -g -O2 EfiRom.c -o EfiRom.o
gcc -o ../bin/EfiRom EfiRom.o -L../libs -lCommon
/usr/bin/ld: cannot find -lCommon
collect2: error: ld returned 1 exit status
-------

Quoted 12 lines

roms/Makefile clean: · line 151

Quoted 9 lines
roms/Makefile +1 -0
 slof:
 	$(MAKE) -C SLOF CROSS=$(powerpc64_cross_prefix) qemu
@@ -150,6 +148,7 @@ clean:
 	$(MAKE) -C sgabios clean
 	rm -f sgabios/.depend
 	$(MAKE) -C ipxe/src veryclean
+	$(MAKE) -C edk2/BaseTools clean

Ditto.

Quoted 9 lines

roms/Makefile clean: · line 154

Quoted 4 lines
roms/Makefile +0 -0
 	$(MAKE) -C SLOF clean
 	rm -rf u-boot/build.e500
 	$(MAKE) -C u-boot-sam460ex distclean

Whichever path used:

thanks!
Laszlo

#13 lore raw

PATCH 3/5 tests: introduce "uefi-test-tools" with the BiosTablesTest UEFI app

In reply to Laszlo Ersek

The "bios-tables-test" program in QEMU's test suite locates the RSD PTR ACPI table in guest RAM, and (chasing pointers to other ACPI tables) performs various sanity checks on the QEMU-generated and firmware-installed tables.

Currently this set of test cases doesn't work with UEFI guests. The ACPI spec defines distinct methods for OSPM to locate the RSD PTR on traditional BIOS vs. UEFI platforms, and the UEFI method is more difficult to implement from the hypervisor side with just raw guest memory access.

Add a UEFI application (to be booted in the UEFI guest) that populates a small, MB-aligned structure in guest RAM. The structure begins with a signature GUID. The hypervisor should loop over all MB-aligned pages in guest RAM until one matches the signature GUID at offset 0, at which point the hypervisor can fetch the RSDP address field(s) from the structure.

QEMU's test logic currently spins on a pre-determined guest address, until that address assumes a magic value. The method described in this patch is conceptually the same ("busy loop until match is found"), except there is no hard-coded address. This plays a lot more nicely with UEFI guest firmware (we'll be able to use the normal page allocation UEFI service). Given the size of EFI_GUID (16 bytes -- 128 bits), mismatches should be astronomically unlikely. In addition, given the typical guest RAM size for such tests (128 MB), there are 128 locations to check in one iteration of the "outer" loop, which shouldn't introduce an intolerable delay after the guest stores the RSDP address(es), and then the GUID.

The GUID that the hypervisor should search for is

AB87A6B1-2034-BDA0-71BD-375007757785

Expressed as a byte array:

{
0xb1, 0xa6, 0x87, 0xab,
0x34, 0x20,
0xa0, 0xbd,
0x71, 0xbd, 0x37, 0x50, 0x07, 0x75, 0x77, 0x85
}

Note that in the patch, we define "gBiosTablesTestGuid" with all bits inverted. This is a simple method to prevent the UEFI binary, which incorporates "gBiosTablesTestGuid", from matching the actual GUID in guest RAM.

The UEFI application is written against the edk2 framework, which was introduced earlier as a git submodule. The next patch will provide build scripts for maintainers.

The source code follows the edk2 coding style, and is licensed under the 2-clause BSDL (in case someone would like to include UefiTestToolsPkg content in a different edk2 platform).

The "UefiTestToolsPkg.dsc" platform description file resolves the used edk2 library classes to instances (= library implementations) such that the UEFI binaries inherit no platform dependencies. They are expected to run on any system that conforms to the UEFI-2.3.1 spec (which was released in 2012). The arch-specific build options are carried over from edk2's ArmVirtPkg and OvmfPkg platforms.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Ard Biesheuvel <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Philippe Mathieu-Daudé <[email protected]>
Cc Shannon Zhao <[email protected]>

---

Notes:
If that's necessary, I'd be glad to be designated as Maintainer or Reviewer in "MAINTAINERS" for "tests/uefi-test-tools/", I just couldn't figure out under what subsystem I should add the magic lines. "MAINTAINERS" needs a Table of Contents! :)

 tests/uefi-test-tools/UefiTestToolsPkg/UefiTestToolsPkg.dec              |  27 ++++
 tests/uefi-test-tools/UefiTestToolsPkg/UefiTestToolsPkg.dsc              |  69 +++++++++++
 tests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest/BiosTablesTest.inf |  41 ++++++
 tests/uefi-test-tools/UefiTestToolsPkg/Include/Guid/BiosTablesTest.h     |  67 ++++++++++
 tests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest/BiosTablesTest.c   | 130 ++++++++++++++++++++
 tests/uefi-test-tools/LICENSE                                            |  25 ++++
 6 files changed, 359 insertions(+)
tests/uefi-test-tools/UefiTestToolsPkg/UefiTestToolsPkg.dec new file +27 -0
new file mode 100644
index 000000000000..ed3a2fe11084
@@ -0,0 +1,27 @@
+## @file
+# edk2 package declaration for the test helper UEFI applications that run in
+# guests.
+#
+# Copyright (C) 2019, Red Hat, Inc.
+#
+# This program and the accompanying materials are licensed and made available
+# under the terms and conditions of the BSD License that accompanies this
+# distribution. The full text of the license may be found at
+# <http://opensource.org/licenses/bsd-license.php>.
+#
+# THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, WITHOUT
+# WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
+##
+
+[Defines]
+  DEC_SPECIFICATION = 1.27
+  PACKAGE_NAME      = UefiTestToolsPkg
+  PACKAGE_GUID      = 7b3f1794-0c85-4b27-a536-44dbf0b0669c
+  PACKAGE_VERSION   = 0.1
+
+[Includes]
+  Include
+
+[Guids]
+  gBiosTablesTestGuid = {0x5478594e, 0xdfcb, 0x425f, {0x8e, 0x42, 0xc8, 0xaf, 0xf8, 0x8a, 0x88, 0x7a}}
+
tests/uefi-test-tools/UefiTestToolsPkg/UefiTestToolsPkg.dsc new file +69 -0

69 lines not on this page · raw message

tests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest/BiosTablesTest.inf new file +41 -0

41 lines not on this page · raw message

tests/uefi-test-tools/UefiTestToolsPkg/Include/Guid/BiosTablesTest.h new file +67 -0

67 lines not on this page · raw message

tests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest/BiosTablesTest.c new file +130 -0

130 lines not on this page · raw message

tests/uefi-test-tools/LICENSE new file +25 -0
new file mode 100644
index 000000000000..38b78aecdb0e
@@ -0,0 +1,25 @@
+All the files in this directory and subdirectories are released under the
+2-Clause BSD License (see header in each file).
+
+Copyright (C) 2019, Red Hat, Inc.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Signature · 1 line
2.19.1.3.g30247aa5d201
#14 lore raw

PATCH 4/5 tests/uefi-test-tools: add build scripts

In reply to Laszlo Ersek

Introduce the following build scripts under "tests/uefi-test-tools":

* "build.sh" builds a single module (a UEFI application) from UefiTestToolsPkg, for a single QEMU emulation target.

"build.sh" relies on cross-compilers when the emulation target and the build host architecture don't match. The cross-compiler prefix is computed according to a fixed, Linux-specific pattern. No attempt is made to copy or reimplement the GNU Make magic from "qemu/roms/Makefile" for cross-compiler prefix determination. The reason is that the build host OSes that are officially supported by edk2, and those that are supported by QEMU, intersect only in Linux. (Note that the UNIXGCC toolchain is being removed from edk2,
<https://bugzilla.tianocore.org/show_bug.cgi?id=1377>.)

* "Makefile" currently builds the "UefiTestToolsPkg/BiosTablesTest" application, for arm, aarch64, i386, and x86_64, with the help of "build.sh".

"Makefile" turns each resultant UEFI executable into a UEFI-bootable, qcow2-compressed ISO image. The ISO images are output as "tests/data/uefi-boot-images/bios-tables-test.<TARGET>.iso.qcow2".

Each ISO image should be passed to QEMU as follows:

    -drive id=boot-cd,if=none,readonly,format=qcow2,file=$ISO \
    -device virtio-scsi-pci,id=scsi0 \
    -device scsi-cd,drive=boot-cd,bus=scsi0.0,bootindex=0 \

"Makefile" assumes that "mkdosfs", "mtools", and "genisoimage" are present.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Ard Biesheuvel <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Philippe Mathieu-Daudé <[email protected]>
Cc Shannon Zhao <[email protected]>
---
 tests/uefi-test-tools/Makefile   |  92 +++++++++++++
 tests/uefi-test-tools/.gitignore |   3 +
 tests/uefi-test-tools/build.sh   | 145 ++++++++++++++++++++
 3 files changed, 240 insertions(+)
tests/uefi-test-tools/Makefile new file +92 -0
new file mode 100644
index 000000000000..7b6dd227e433
@@ -0,0 +1,92 @@
+# Makefile for the test helper UEFI applications that run in guests.
+#
+# Copyright (C) 2019, Red Hat, Inc.
+#
+# This program and the accompanying materials are licensed and made available
+# under the terms and conditions of the BSD License that accompanies this
+# distribution. The full text of the license may be found at
+# <http://opensource.org/licenses/bsd-license.php>.
+#
+# THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, WITHOUT
+# WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
+
+edk2_dir              := ../../roms/edk2
+images_dir            := ../data/uefi-boot-images
+emulation_targets     := arm aarch64 i386 x86_64
+uefi_binaries         := bios-tables-test
+intermediate_suffixes := .efi .fat .iso.raw
+
+images: $(foreach binary,$(uefi_binaries), \
+		$(foreach target,$(emulation_targets), \
+			$(images_dir)/$(binary).$(target).iso.qcow2))
+
+# Preserve all intermediate targets if the build succeeds.
+# - Intermediate targets help with development & debugging.
+# - Preserving intermediate targets also keeps spurious changes out of the
+#   final build products, in case the user re-runs "make" without any changes
+#   to the UEFI source code. Normally, the intermediate files would have been
+#   removed by the last "make" invocation, hence the re-run would rebuild them
+#   from the unchanged UEFI sources. Unfortunately, the "mkdosfs" and
+#   "genisoimage" utilities embed timestamp-based information in their outputs,
+#   which causes git to report differences for the tracked qcow2 ISO images.
+.SECONDARY: $(foreach binary,$(uefi_binaries), \
+		$(foreach target,$(emulation_targets), \
+			$(foreach suffix,$(intermediate_suffixes), \
+				Build/$(binary).$(target)$(suffix))))
+
+# In the pattern rules below, the stem (%, $*) stands for
+# "$(binary).$(target)".
+
+# Convert the raw ISO image to a qcow2 one, enabling compression, and using a
+# small cluster size. This allows for small binary files under git control,
+# hence for small binary patches.
+$(images_dir)/%.iso.qcow2: Build/%.iso.raw
+	mkdir -p -- $(images_dir)
+	$${QTEST_QEMU_IMG:-qemu-img} convert -f raw -O qcow2 -c \
+		-o cluster_size=512 -- $< $@
+
+# Embed the "UEFI system partition" into an ISO9660 file system as an ElTorito
+# boot image.
+Build/%.iso.raw: Build/%.fat
+	genisoimage -input-charset ASCII -efi-boot $(notdir $<) -no-emul-boot \
+		-quiet -o $@ -- $<
+
+# Define chained macros in order to map QEMU system emulation targets to
+# *short* UEFI architecture identifiers. Periods are allowed in, and ultimately
+# stripped from, the argument.
+map_arm_to_uefi     = $(subst arm,ARM,$(1))
+map_aarch64_to_uefi = $(subst aarch64,AA64,$(call map_arm_to_uefi,$(1)))
+map_i386_to_uefi    = $(subst i386,IA32,$(call map_aarch64_to_uefi,$(1)))
+map_x86_64_to_uefi  = $(subst x86_64,X64,$(call map_i386_to_uefi,$(1)))
+map_to_uefi         = $(subst .,,$(call map_x86_64_to_uefi,$(1)))
+
+# Format a "UEFI system partition", using the UEFI binary as the default boot
+# loader. Add 10% size for filesystem metadata, round up to the next KB, and
+# make sure the size is large enough for a FAT filesystem. Name the filesystem
+# after the UEFI binary. (Excess characters are automatically dropped from the
+# filesystem label.)
+Build/%.fat: Build/%.efi
+	rm -f -- $@
+	uefi_bin_b=$$(stat --format=%s -- $<) && \
+		uefi_fat_kb=$$(( (uefi_bin_b * 11 / 10 + 1023) / 1024 )) && \
+		uefi_fat_kb=$$(( uefi_fat_kb >= 64 ? uefi_fat_kb : 64 )) && \
+		mkdosfs -C $@ -n $(basename $(@F)) -- $$uefi_fat_kb
+	MTOOLS_SKIP_CHECK=1 mmd -i $@ ::EFI
+	MTOOLS_SKIP_CHECK=1 mmd -i $@ ::EFI/BOOT
+	MTOOLS_SKIP_CHECK=1 mcopy -i $@ -- $< \
+		::EFI/BOOT/BOOT$(call map_to_uefi,$(suffix $*)).EFI
+
+# In the pattern rules below, the stem (%, $*) stands for "$(target)" only. The
+# association between the UEFI binary (such as "bios-tables-test") and the
+# component name from the edk2 platform DSC file (such as "BiosTablesTest") is
+# explicit in each rule.
+
+Build/bios-tables-test.%.efi: build-edk2-tools
+	./build.sh $(edk2_dir) BiosTablesTest $* $@
+
+build-edk2-tools:
+	$(MAKE) -C $(edk2_dir)/BaseTools
+
+clean:
+	rm -rf Build Conf log
+	$(MAKE) -C $(edk2_dir)/BaseTools clean
tests/uefi-test-tools/.gitignore new file +3 -0
new file mode 100644
index 000000000000..9f246701dea1
@@ -0,0 +1,3 @@
+Build
+Conf
+log
tests/uefi-test-tools/build.sh new file +145 -0

145 lines not on this page · raw message

Signature · 1 line
2.19.1.3.g30247aa5d201
#15 lore raw

Re: [Qemu-devel] [PATCH 4/5] tests/uefi-test-tools: add build scripts

Hi Laszlo,

Quoted 130 lines · Laszlo Ersek

Introduce the following build scripts under "tests/uefi-test-tools":

* "build.sh" builds a single module (a UEFI application) from UefiTestToolsPkg, for a single QEMU emulation target.

"build.sh" relies on cross-compilers when the emulation target and the build host architecture don't match. The cross-compiler prefix is computed according to a fixed, Linux-specific pattern. No attempt is made to copy or reimplement the GNU Make magic from "qemu/roms/Makefile" for cross-compiler prefix determination. The reason is that the build host OSes that are officially supported by edk2, and those that are supported by QEMU, intersect only in Linux. (Note that the UNIXGCC toolchain is being removed from edk2,
<https://bugzilla.tianocore.org/show_bug.cgi?id=1377>.)

* "Makefile" currently builds the "UefiTestToolsPkg/BiosTablesTest" application, for arm, aarch64, i386, and x86_64, with the help of "build.sh".

"Makefile" turns each resultant UEFI executable into a UEFI-bootable, qcow2-compressed ISO image. The ISO images are output as "tests/data/uefi-boot-images/bios-tables-test.<TARGET>.iso.qcow2".

Each ISO image should be passed to QEMU as follows:

    -drive id=boot-cd,if=none,readonly,format=qcow2,file=$ISO \
    -device virtio-scsi-pci,id=scsi0 \
    -device scsi-cd,drive=boot-cd,bus=scsi0.0,bootindex=0 \

"Makefile" assumes that "mkdosfs", "mtools", and "genisoimage" are present.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Ard Biesheuvel <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Philippe Mathieu-Daudé <[email protected]>
Cc Shannon Zhao <[email protected]>
---
 tests/uefi-test-tools/Makefile   |  92 +++++++++++++
 tests/uefi-test-tools/.gitignore |   3 +
 tests/uefi-test-tools/build.sh   | 145 ++++++++++++++++++++
 3 files changed, 240 insertions(+)
tests/uefi-test-tools/Makefile new file +79 -0
new file mode 100644
index 000000000000..7b6dd227e433
@@ -0,0 +1,92 @@
+# Makefile for the test helper UEFI applications that run in guests.
+#
+# Copyright (C) 2019, Red Hat, Inc.
+#
+# This program and the accompanying materials are licensed and made available
+# under the terms and conditions of the BSD License that accompanies this
+# distribution. The full text of the license may be found at
+# <http://opensource.org/licenses/bsd-license.php>.
+#
+# THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, WITHOUT
+# WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
+
+edk2_dir              := ../../roms/edk2
+images_dir            := ../data/uefi-boot-images
+emulation_targets     := arm aarch64 i386 x86_64
+uefi_binaries         := bios-tables-test
+intermediate_suffixes := .efi .fat .iso.raw
+
+images: $(foreach binary,$(uefi_binaries), \
+		$(foreach target,$(emulation_targets), \
+			$(images_dir)/$(binary).$(target).iso.qcow2))
+
+# Preserve all intermediate targets if the build succeeds.
+# - Intermediate targets help with development & debugging.
+# - Preserving intermediate targets also keeps spurious changes out of the
+#   final build products, in case the user re-runs "make" without any changes
+#   to the UEFI source code. Normally, the intermediate files would have been
+#   removed by the last "make" invocation, hence the re-run would rebuild them
+#   from the unchanged UEFI sources. Unfortunately, the "mkdosfs" and
+#   "genisoimage" utilities embed timestamp-based information in their outputs,
+#   which causes git to report differences for the tracked qcow2 ISO images.
+.SECONDARY: $(foreach binary,$(uefi_binaries), \
+		$(foreach target,$(emulation_targets), \
+			$(foreach suffix,$(intermediate_suffixes), \
+				Build/$(binary).$(target)$(suffix))))
+
+# In the pattern rules below, the stem (%, $*) stands for
+# "$(binary).$(target)".
+
+# Convert the raw ISO image to a qcow2 one, enabling compression, and using a
+# small cluster size. This allows for small binary files under git control,
+# hence for small binary patches.
+$(images_dir)/%.iso.qcow2: Build/%.iso.raw
+	mkdir -p -- $(images_dir)
+	$${QTEST_QEMU_IMG:-qemu-img} convert -f raw -O qcow2 -c \
+		-o cluster_size=512 -- $< $@
+
+# Embed the "UEFI system partition" into an ISO9660 file system as an ElTorito
+# boot image.
+Build/%.iso.raw: Build/%.fat
+	genisoimage -input-charset ASCII -efi-boot $(notdir $<) -no-emul-boot \
+		-quiet -o $@ -- $<
+
+# Define chained macros in order to map QEMU system emulation targets to
+# *short* UEFI architecture identifiers. Periods are allowed in, and ultimately
+# stripped from, the argument.
+map_arm_to_uefi     = $(subst arm,ARM,$(1))
+map_aarch64_to_uefi = $(subst aarch64,AA64,$(call map_arm_to_uefi,$(1)))
+map_i386_to_uefi    = $(subst i386,IA32,$(call map_aarch64_to_uefi,$(1)))
+map_x86_64_to_uefi  = $(subst x86_64,X64,$(call map_i386_to_uefi,$(1)))
+map_to_uefi         = $(subst .,,$(call map_x86_64_to_uefi,$(1)))
+
+# Format a "UEFI system partition", using the UEFI binary as the default boot
+# loader. Add 10% size for filesystem metadata, round up to the next KB, and
+# make sure the size is large enough for a FAT filesystem. Name the filesystem
+# after the UEFI binary. (Excess characters are automatically dropped from the
+# filesystem label.)
+Build/%.fat: Build/%.efi
+	rm -f -- $@
+	uefi_bin_b=$$(stat --format=%s -- $<) && \
+		uefi_fat_kb=$$(( (uefi_bin_b * 11 / 10 + 1023) / 1024 )) && \
+		uefi_fat_kb=$$(( uefi_fat_kb >= 64 ? uefi_fat_kb : 64 )) && \
+		mkdosfs -C $@ -n $(basename $(@F)) -- $$uefi_fat_kb
+	MTOOLS_SKIP_CHECK=1 mmd -i $@ ::EFI
+	MTOOLS_SKIP_CHECK=1 mmd -i $@ ::EFI/BOOT
+	MTOOLS_SKIP_CHECK=1 mcopy -i $@ -- $< \
+		::EFI/BOOT/BOOT$(call map_to_uefi,$(suffix $*)).EFI
+
+# In the pattern rules below, the stem (%, $*) stands for "$(target)" only. The

tests/uefi-test-tools/Makefile line 89

Quoted 10 lines
tests/uefi-test-tools/Makefile new file +10 -0
+# association between the UEFI binary (such as "bios-tables-test") and the
+# component name from the edk2 platform DSC file (such as "BiosTablesTest") is
+# explicit in each rule.
+
+Build/bios-tables-test.%.efi: build-edk2-tools
+	./build.sh $(edk2_dir) BiosTablesTest $* $@
+
+build-edk2-tools:
+	$(MAKE) -C $(edk2_dir)/BaseTools
+

[...]

I got errors [1] and [2] I couldn't figure out while running 'make -j4'.

The following patch didn't help, any idea?

-- >8 --

tests/uefi-test-tools/Makefile +0 -0
index 7b6dd227e4..798c55c823 100644
@@ -81,6 +81,8 @@ Build/%.fat: Build/%.efi
 # component name from the edk2 platform DSC file (such as

"BiosTablesTest") is

 # explicit in each rule.

+.NOTPARALLEL: $(foreach target,$(emulation_targets),Build/bios-tables-test.$(target).efi) +
Build/bios-tables-test.%.efi: build-edk2-tools

        ./build.sh $(edk2_dir) BiosTablesTest $* $@
tests/uefi-test-tools/build.sh +1 -0
index 155cb75c4d..13891a7385 100755
@@ -133,6 +133,7 @@ esac
 # Build the UEFI binary
 mkdir -p log
 build \
+  -n 1 \
   --arch="$edk2_arch" \
   --buildtarget=DEBUG \
   --platform=UefiTestToolsPkg/UefiTestToolsPkg.dsc \

---

Anyway building using "make -j1" works.

[1]:

Building ...
qemu/roms/edk2/ArmPkg/Library/CompilerIntrinsicsLib/CompilerIntrinsicsLib.inf
[ARM]
make[1]: Entering directory
'qemu/tests/uefi-test-tools/Build/UefiTestTools/DEBUG_GCC5/ARM/ArmPkg/Library/CompilerIntrinsicsLib/CompilerIntrinsicsLib'
"arm-linux-gnu-gcc" -mthumb -march=armv7-a -E -x assembler-with-cpp
-include qemu/tests/uefi-test-tools/Build/UefiTestTools/DEBUG_GCC5/ARM/ArmPkg/Library/CompilerIntrinsicsLib/CompilerIntrinsicsLib/DEBUG/AutoGen.h -Iqemu/roms/edk2/ArmPkg/Library/CompilerIntrinsicsLib/Arm
-Iqemu/roms/edk2/ArmPkg/Library/CompilerIntrinsicsLib -Iqemu/tests/uefi-test-tools/Build/UefiTestTools/DEBUG_GCC5/ARM/ArmPkg/Library/CompilerIntrinsicsLib/CompilerIntrinsicsLib/DEBUG -Iqemu/roms/edk2/MdePkg -Iqemu/roms/edk2/MdePkg/Include
-Iqemu/roms/edk2/MdePkg/Include/Arm -Iqemu/roms/edk2/ArmPkg
-Iqemu/roms/edk2/ArmPkg/Include
qemu/roms/edk2/ArmPkg/Library/CompilerIntrinsicsLib/Arm/divsi3.S > qemu/tests/uefi-test-tools/Build/UefiTestTools/DEBUG_GCC5/ARM/ArmPkg/Library/CompilerIntrinsicsLib/CompilerIntrinsicsLib/OUTPUT/Arm/divsi3.i
make[1]: *** read jobs pipe: Bad file descriptor. Stop.
make[1]: *** Waiting for unfinished jobs....
build.py...
: error 7000: Failed to execute command
make tbuild
[qemu/tests/uefi-test-tools/Build/UefiTestTools/DEBUG_GCC5/ARM/ArmPkg/Library/CompilerIntrinsicsLib/CompilerIntrinsicsLib]

build.py...
: error F002: Failed to build module

qemu/roms/edk2/ArmPkg/Library/CompilerIntrinsicsLib/CompilerIntrinsicsLib.inf [ARM, GCC5, DEBUG]

    - Failed -
    Build end time: 13:08:01, Jan.21 2019
    Build total time: 00:00:02

[2]:

Active Platform =
qemu/tests/uefi-test-tools/UefiTestToolsPkg/UefiTestToolsPkg.dsc
Active Module =
qemu/tests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest/BiosTablesTest.inf

Processing meta-data .

build.py...
: error C0DE: Unknown fatal error when processing
[qemu/roms/edk2/MdePkg/MdePkg.dec]

(Please send email to [email protected] for help, attaching
following call stack trace!)

(Python 2.7.15 on linux2) Traceback (most recent call last):
File
"qemu/roms/edk2/BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py", line 2494, in Main
MyBuild.Launch()
File
"qemu/roms/edk2/BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py", line 2231, in Launch
self._BuildModule()
File
"qemu/roms/edk2/BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py", line 1837, in _BuildModule
self.ModuleFile
File "qemu/roms/edk2/BaseTools/Source/Python/AutoGen/AutoGen.py",
line 241, in __init__
self._InitWorker(Workspace, MetaFile, Target, Toolchain, Arch,
*args, **kwargs)
File "qemu/roms/edk2/BaseTools/Source/Python/AutoGen/AutoGen.py",
line 405, in _InitWorker
PlatformPcds = Platform.Pcds
File
"qemu/roms/edk2/BaseTools/Source/Python/Workspace/DscBuildData.py", line 1176, in Pcds
self._Pcds.update(self._GetPcd(MODEL_PCD_FIXED_AT_BUILD))
File
"qemu/roms/edk2/BaseTools/Source/Python/Workspace/DscBuildData.py", line 1601, in _GetPcd
PcdValue, DatumType, MaxDatumSize = self._ValidatePcd(PcdCName,
TokenSpaceGuid, Setting, Type, Dummy4)
File
"qemu/roms/edk2/BaseTools/Source/Python/Workspace/DscBuildData.py", line 897, in _ValidatePcd
self._DecPcds, self._GuidDict = GetDeclaredPcd(self, self._Bdb,
self._Arch, self._Target, self._Toolchain, PkgSet)
File
"qemu/roms/edk2/BaseTools/Source/Python/Workspace/WorkspaceCommon.py", line 68, in GetDeclaredPcd
Guids = Pkg.Guids
File
"qemu/roms/edk2/BaseTools/Source/Python/Workspace/DecBuildData.py", line 262, in Guids
RecordList = self._RawData[MODEL_EFI_GUID, self._Arch]
File
"qemu/roms/edk2/BaseTools/Source/Python/Workspace/MetaFileParser.py", line 249, in __getitem__
self.Start()
File
"qemu/roms/edk2/BaseTools/Source/Python/Workspace/MetaFileParser.py", line 1788, in Start

        0
      File

"qemu/roms/edk2/BaseTools/Source/Python/Workspace/MetaFileParser.py", line 195, in _Store
return self._Table.Insert(*Args)
File
"qemu/roms/edk2/BaseTools/Source/Python/Workspace/MetaFileTable.py", line 211, in Insert
Enabled
File
"qemu/roms/edk2/BaseTools/Source/Python/Workspace/MetaDataTable.py", line 79, in Insert
self.Cur.execute(SqlCommand)
IntegrityError: UNIQUE constraint failed: _1012_3.ID

- Failed -
Build end time: 12:44:27, Jan.21 2019
Build total time: 00:00:02

make: *** [Makefile:85: Build/bios-tables-test.x86_64.efi] Error 1
make: *** Waiting for unfinished jobs....

#16 lore raw

Re: [Qemu-devel] [PATCH 4/5] tests/uefi-test-tools: add build scripts

Quoted 170 lines · Philippe Mathieu-Daudé

Hi Laszlo,

Quoted 130 lines · Laszlo Ersek

Introduce the following build scripts under "tests/uefi-test-tools":

* "build.sh" builds a single module (a UEFI application) from UefiTestToolsPkg, for a single QEMU emulation target.

"build.sh" relies on cross-compilers when the emulation target and the build host architecture don't match. The cross-compiler prefix is computed according to a fixed, Linux-specific pattern. No attempt is made to copy or reimplement the GNU Make magic from "qemu/roms/Makefile" for cross-compiler prefix determination. The reason is that the build host OSes that are officially supported by edk2, and those that are supported by QEMU, intersect only in Linux. (Note that the UNIXGCC toolchain is being removed from edk2,
<https://bugzilla.tianocore.org/show_bug.cgi?id=1377>.)

* "Makefile" currently builds the "UefiTestToolsPkg/BiosTablesTest" application, for arm, aarch64, i386, and x86_64, with the help of "build.sh".

"Makefile" turns each resultant UEFI executable into a UEFI-bootable, qcow2-compressed ISO image. The ISO images are output as "tests/data/uefi-boot-images/bios-tables-test.<TARGET>.iso.qcow2".

Each ISO image should be passed to QEMU as follows:

    -drive id=boot-cd,if=none,readonly,format=qcow2,file=$ISO \
    -device virtio-scsi-pci,id=scsi0 \
    -device scsi-cd,drive=boot-cd,bus=scsi0.0,bootindex=0 \

"Makefile" assumes that "mkdosfs", "mtools", and "genisoimage" are present.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Ard Biesheuvel <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Philippe Mathieu-Daudé <[email protected]>
Cc Shannon Zhao <[email protected]>
---
 tests/uefi-test-tools/Makefile   |  92 +++++++++++++
 tests/uefi-test-tools/.gitignore |   3 +
 tests/uefi-test-tools/build.sh   | 145 ++++++++++++++++++++
 3 files changed, 240 insertions(+)
tests/uefi-test-tools/Makefile new file +79 -0
new file mode 100644
index 000000000000..7b6dd227e433
@@ -0,0 +1,92 @@
+# Makefile for the test helper UEFI applications that run in guests.
+#
+# Copyright (C) 2019, Red Hat, Inc.
+#
+# This program and the accompanying materials are licensed and made available
+# under the terms and conditions of the BSD License that accompanies this
+# distribution. The full text of the license may be found at
+# <http://opensource.org/licenses/bsd-license.php>.
+#
+# THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, WITHOUT
+# WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
+
+edk2_dir              := ../../roms/edk2
+images_dir            := ../data/uefi-boot-images
+emulation_targets     := arm aarch64 i386 x86_64
+uefi_binaries         := bios-tables-test
+intermediate_suffixes := .efi .fat .iso.raw
+
+images: $(foreach binary,$(uefi_binaries), \
+		$(foreach target,$(emulation_targets), \
+			$(images_dir)/$(binary).$(target).iso.qcow2))
+
+# Preserve all intermediate targets if the build succeeds.
+# - Intermediate targets help with development & debugging.
+# - Preserving intermediate targets also keeps spurious changes out of the
+#   final build products, in case the user re-runs "make" without any changes
+#   to the UEFI source code. Normally, the intermediate files would have been
+#   removed by the last "make" invocation, hence the re-run would rebuild them
+#   from the unchanged UEFI sources. Unfortunately, the "mkdosfs" and
+#   "genisoimage" utilities embed timestamp-based information in their outputs,
+#   which causes git to report differences for the tracked qcow2 ISO images.
+.SECONDARY: $(foreach binary,$(uefi_binaries), \
+		$(foreach target,$(emulation_targets), \
+			$(foreach suffix,$(intermediate_suffixes), \
+				Build/$(binary).$(target)$(suffix))))
+
+# In the pattern rules below, the stem (%, $*) stands for
+# "$(binary).$(target)".
+
+# Convert the raw ISO image to a qcow2 one, enabling compression, and using a
+# small cluster size. This allows for small binary files under git control,
+# hence for small binary patches.
+$(images_dir)/%.iso.qcow2: Build/%.iso.raw
+	mkdir -p -- $(images_dir)
+	$${QTEST_QEMU_IMG:-qemu-img} convert -f raw -O qcow2 -c \
+		-o cluster_size=512 -- $< $@
+
+# Embed the "UEFI system partition" into an ISO9660 file system as an ElTorito
+# boot image.
+Build/%.iso.raw: Build/%.fat
+	genisoimage -input-charset ASCII -efi-boot $(notdir $<) -no-emul-boot \
+		-quiet -o $@ -- $<
+
+# Define chained macros in order to map QEMU system emulation targets to
+# *short* UEFI architecture identifiers. Periods are allowed in, and ultimately
+# stripped from, the argument.
+map_arm_to_uefi     = $(subst arm,ARM,$(1))
+map_aarch64_to_uefi = $(subst aarch64,AA64,$(call map_arm_to_uefi,$(1)))
+map_i386_to_uefi    = $(subst i386,IA32,$(call map_aarch64_to_uefi,$(1)))
+map_x86_64_to_uefi  = $(subst x86_64,X64,$(call map_i386_to_uefi,$(1)))
+map_to_uefi         = $(subst .,,$(call map_x86_64_to_uefi,$(1)))
+
+# Format a "UEFI system partition", using the UEFI binary as the default boot
+# loader. Add 10% size for filesystem metadata, round up to the next KB, and
+# make sure the size is large enough for a FAT filesystem. Name the filesystem
+# after the UEFI binary. (Excess characters are automatically dropped from the
+# filesystem label.)
+Build/%.fat: Build/%.efi
+	rm -f -- $@
+	uefi_bin_b=$$(stat --format=%s -- $<) && \
+		uefi_fat_kb=$$(( (uefi_bin_b * 11 / 10 + 1023) / 1024 )) && \
+		uefi_fat_kb=$$(( uefi_fat_kb >= 64 ? uefi_fat_kb : 64 )) && \
+		mkdosfs -C $@ -n $(basename $(@F)) -- $$uefi_fat_kb
+	MTOOLS_SKIP_CHECK=1 mmd -i $@ ::EFI
+	MTOOLS_SKIP_CHECK=1 mmd -i $@ ::EFI/BOOT
+	MTOOLS_SKIP_CHECK=1 mcopy -i $@ -- $< \
+		::EFI/BOOT/BOOT$(call map_to_uefi,$(suffix $*)).EFI
+
+# In the pattern rules below, the stem (%, $*) stands for "$(target)" only. The

tests/uefi-test-tools/Makefile line 89

Quoted 10 lines
tests/uefi-test-tools/Makefile new file +10 -0
+# association between the UEFI binary (such as "bios-tables-test") and the
+# component name from the edk2 platform DSC file (such as "BiosTablesTest") is
+# explicit in each rule.
+
+Build/bios-tables-test.%.efi: build-edk2-tools
+	./build.sh $(edk2_dir) BiosTablesTest $* $@
+
+build-edk2-tools:
+	$(MAKE) -C $(edk2_dir)/BaseTools
+

[...]

I got errors [1] and [2] I couldn't figure out while running 'make -j4'.

The following patch didn't help, any idea?

-- >8 --

tests/uefi-test-tools/Makefile +0 -0
index 7b6dd227e4..798c55c823 100644
@@ -81,6 +81,8 @@ Build/%.fat: Build/%.efi
 # component name from the edk2 platform DSC file (such as

"BiosTablesTest") is

 # explicit in each rule.

+.NOTPARALLEL: $(foreach target,$(emulation_targets),Build/bios-tables-test.$(target).efi)

tests/uefi-test-tools/Makefile +1 -0
+
 Build/bios-tables-test.%.efi: build-edk2-tools
        ./build.sh $(edk2_dir) BiosTablesTest $* $@
tests/uefi-test-tools/build.sh +0 -0
index 155cb75c4d..13891a7385 100755

tests/uefi-test-tools/build.sh esac

Quoted 8 lines
tests/uefi-test-tools/build.sh +1 -1
@@ -133,6 +133,7 @@ esac
 # Build the UEFI binary
 mkdir -p log
 build \
+  -n 1 \
   --arch="$edk2_arch" \
   --buildtarget=DEBUG \
   --platform=UefiTestToolsPkg/UefiTestToolsPkg.dsc \
---

It wasn't clear to me whether and how multi-threaded builds were supposed to be used by maintainers, whenever they'd update "tests/data/uefi-boot-images/*".

I saw that "make" was invoked everywhere as $(MAKE), but that didn't clarify any intent around "-j". So I didn't test "-j" at all, and in fact I wouldn't have expected it to work:

- At any point in time, there shouldn't be more than one instance of the "build" base tool working.

- the "build" base tool itself contains some level of support (although not complete) for parallelizing edk2 module builds: it does not parallelize the compilation of source files between each other *within* a single edk2 module (= INF file). It parallelizes modules (INF files) between each other.

- This means if you build only one module at a time, passing the "-m" switch (plus an INF file) to the "build" base tool, then there's nothing to parallelize.

- "build.sh" does pass the "-m" switch to the "build" tool, because I wanted the build output to be granular. That is, it should be possible for someone to build just a given uefi-test-tools image for just a given architecture, if they specify the corresponding pathname (as target) when they invoke "make". This usage requires the "-m" flag, which in turn precludes the parallelism built into the "build" base tool itself. And, invoking multiple "build" instances in parallel is not supposed to work.

Regarding why ".NOTPARALLEL" doesn't work -- the GNU Make documentation writes:

[...] Any recursively invoked make command will still run recipes in parallel (unless its makefile also contains this target). [...]

The "build" base tool in edk2 implements part of the job with generated makefiles and invoking "make" itself, thus, despite .NOTPARALLEL, it likely inherits the outermost -j<N> setting -- and it doesn't expect such.

So the best I can offer here is to check $MAKEFLAGS in "build.sh", and exit with an early, explicit error if $MAKEFLAGS contains "-j", "-l", or their long variants (--jobs, --load-average).

I realize this would not be acceptable for building QEMU itself, but "tests/uefi-test-tools/Makefile" should be run by some subsystem maintainers only, interactively (or in their on custom scripts).

Thanks
Laszlo

#17 lore raw

Re: [Qemu-devel] [PATCH 4/5] tests/uefi-test-tools: add build scripts

Quoted 7 lines · Laszlo Ersek

It wasn't clear to me whether and how multi-threaded builds were supposed to be used by maintainers, whenever they'd update "tests/data/uefi-boot-images/*".

I saw that "make" was invoked everywhere as $(MAKE), but that didn't clarify any intent around "-j". So I didn't test "-j" at all, and in fact I wouldn't have expected it to work:

The usual assumption with make is that "-jN" should work and at least (if the thing being built can't actually be parallelized usefully) be no worse than if you'd not specified a -j option. We have occasionally had problems with -jN in 'make check' (usually because several test cases were trying to use the same temp filename or similar) but we've treated them as bugs and squashed them.

Quoted 7 lines

The "build" base tool in edk2 implements part of the job with generated makefiles and invoking "make" itself, thus, despite .NOTPARALLEL, it likely inherits the outermost -j<N> setting -- and it doesn't expect such.

So the best I can offer here is to check $MAKEFLAGS in "build.sh", and exit with an early, explicit error if $MAKEFLAGS contains "-j", "-l", or their long variants (--jobs, --load-average).

Could you sanitize MAKEFLAGS in build.sh instead to remove the parallelization options?

thanks
-- PMM

#18 lore raw

Re: [Qemu-devel] [PATCH 4/5] tests/uefi-test-tools: add build scripts

Quoted 21 lines · Peter Maydell
Quoted 7 lines · Laszlo Ersek

It wasn't clear to me whether and how multi-threaded builds were supposed to be used by maintainers, whenever they'd update "tests/data/uefi-boot-images/*".

I saw that "make" was invoked everywhere as $(MAKE), but that didn't clarify any intent around "-j". So I didn't test "-j" at all, and in fact I wouldn't have expected it to work:

The usual assumption with make is that "-jN" should work and at least (if the thing being built can't actually be parallelized usefully) be no worse than if you'd not specified a -j option. We have occasionally had problems with -jN in 'make check' (usually because several test cases were trying to use the same temp filename or similar) but we've treated them as bugs and squashed them.

Quoted 4 lines

The "build" base tool in edk2 implements part of the job with generated makefiles and invoking "make" itself, thus, despite .NOTPARALLEL, it likely inherits the outermost -j<N> setting -- and it doesn't expect such.

Quoted 6 lines
Quoted 3 lines

So the best I can offer here is to check $MAKEFLAGS in "build.sh", and exit with an early, explicit error if $MAKEFLAGS contains "-j", "-l", or their long variants (--jobs, --load-average).

Could you sanitize MAKEFLAGS in build.sh instead to remove the parallelization options?

I've looked into MAKEFLAGS in a bit more depth now; both the documentation and some debug prints. Manipulating MAKEFLAGS looks somewhat brittle.

So, I was about to suggest that I use the .NOTPARALLEL special target recommended by Phil, for the .efi binaries (which would ensure that no two instances of the "build" base tool run at the same time), *plus* that I submit a patch to edk2 so that the makefiles generated by the "build" tool also contain .NOTPARALLEL.

(Because, to quote the make docs again, "If .NOTPARALLEL is mentioned as a target, then this invocation of make will be run serially, even if the ‘-j’ option is given. Any recursively invoked make command will still run recipes in parallel (unless its makefile also contains this target).")

However: when I wanted to see the actual error from using .NOTPARALLEL *only* in "tests/uefi-test-tools/Makefile", and not in the build-generated makefiles, I failed to get any error. All the output images were built just fine.

Phil: when you wrote that "The following patch didn't help" -- referring to .NOTPARALLEL, added only to "tests/uefi-test-tools/Makefile"-- , did you clean your tree first (with "make clean" or "git clean -ffdx")?

Because now I'm thinking that the *individual* makefiles generated by edk2's "build" base tool might actually compatible with "-j", and your testing of .NOTPARALLEL failed only because your build tree (for example the Conf/ subdir -- list it with "-A") was in a messy state from your previous -j4 attempt (where you didn't use .NOTPARALLEL at all).

Thanks
Laszlo

#19 lore raw

Re: [Qemu-devel] [PATCH 4/5] tests/uefi-test-tools: add build scripts

Quoted 53 lines · Laszlo Ersek
Quoted 21 lines · Peter Maydell
Quoted 7 lines · Laszlo Ersek

It wasn't clear to me whether and how multi-threaded builds were supposed to be used by maintainers, whenever they'd update "tests/data/uefi-boot-images/*".

I saw that "make" was invoked everywhere as $(MAKE), but that didn't clarify any intent around "-j". So I didn't test "-j" at all, and in fact I wouldn't have expected it to work:

The usual assumption with make is that "-jN" should work and at least (if the thing being built can't actually be parallelized usefully) be no worse than if you'd not specified a -j option. We have occasionally had problems with -jN in 'make check' (usually because several test cases were trying to use the same temp filename or similar) but we've treated them as bugs and squashed them.

Quoted 4 lines

The "build" base tool in edk2 implements part of the job with generated makefiles and invoking "make" itself, thus, despite .NOTPARALLEL, it likely inherits the outermost -j<N> setting -- and it doesn't expect such.

Quoted 6 lines
Quoted 3 lines

So the best I can offer here is to check $MAKEFLAGS in "build.sh", and exit with an early, explicit error if $MAKEFLAGS contains "-j", "-l", or their long variants (--jobs, --load-average).

Could you sanitize MAKEFLAGS in build.sh instead to remove the parallelization options?

I've looked into MAKEFLAGS in a bit more depth now; both the documentation and some debug prints. Manipulating MAKEFLAGS looks somewhat brittle.

So, I was about to suggest that I use the .NOTPARALLEL special target recommended by Phil, for the .efi binaries (which would ensure that no two instances of the "build" base tool run at the same time), *plus* that I submit a patch to edk2 so that the makefiles generated by the "build" tool also contain .NOTPARALLEL.

(Because, to quote the make docs again, "If .NOTPARALLEL is mentioned as a target, then this invocation of make will be run serially, even if the ‘-j’ option is given. Any recursively invoked make command will still run recipes in parallel (unless its makefile also contains this target).")

However: when I wanted to see the actual error from using .NOTPARALLEL *only* in "tests/uefi-test-tools/Makefile", and not in the build-generated makefiles, I failed to get any error. All the output images were built just fine.

Phil: when you wrote that "The following patch didn't help" -- referring to .NOTPARALLEL, added only to "tests/uefi-test-tools/Makefile"-- , did you clean your tree first (with "make clean" or "git clean -ffdx")?

Quoted 5 lines

Because now I'm thinking that the *individual* makefiles generated by edk2's "build" base tool might actually compatible with "-j", and your testing of .NOTPARALLEL failed only because your build tree (for example the Conf/ subdir -- list it with "-A") was in a messy state from your previous -j4 attempt (where you didn't use .NOTPARALLEL at all).

FWIW, I've asked

- both on edk2-devel, about adding .NOTPARALLEL to the generated inner makefiles:

"parallelism in the module-level, generated GNUmakefile's"
https://lists.01.org/pipermail/edk2-devel/2019-January/035463.html

- and on help-make, about safely filtering the job-related options from MAKEFLAGS, between the outer and inner "make" processes:

"filtering job options from MAKEFLAGS, manually"
http://lists.gnu.org/archive/html/help-make/2019-01/msg00003.html

Thanks
Laszlo

#20 lore raw

Re: [Qemu-devel] [PATCH 4/5] tests/uefi-test-tools: add build scripts

Quoted 5 lines · Laszlo Ersek

FWIW, I've asked

- both on edk2-devel, about adding .NOTPARALLEL to the generated inner makefiles:

Quoted 8 lines

"parallelism in the module-level, generated GNUmakefile's"
https://lists.01.org/pipermail/edk2-devel/2019-January/035463.html

- and on help-make, about safely filtering the job-related options from MAKEFLAGS, between the outer and inner "make" processes:

"filtering job options from MAKEFLAGS, manually"
http://lists.gnu.org/archive/html/help-make/2019-01/msg00003.html

Results:

- The "build" utility in BaseTools generates such GNUMakefiles that dependencies between targets are fully described. This is an explicit goal and if we notice any missing deps, that's a bug to be reported and fixed. As a consequence, we can expect the recipes in those GNUMakefiles to be fully parallelizable. Therefore we need not add .NOTPARALLEL to them (in the edk2 project), or filter MAKEFLAGS, or append "-j1" to MAKEFLAGS.

- The .NOTPARALLEL target ignores any pre-requisites it gets; .NOTPARALLEL applies to the entire current invocation of make. Therefore we just need to stick a bare .NOTPARALLEL into "tests/uefi-test-tools/Makefile", to keep "build" itself isolated.

I'm going to post v2 with this one addition (i.e., .NOTPARALLEL).

Thanks,
Laszlo

#21 lore raw

PATCH 5/5 tests/data: introduce "uefi-boot-images" with the "bios-tables-test" ISOs

In reply to Laszlo Ersek

Add UEFI-bootable qcow2-compressed ISO images built from:

tests/uefi-test-tools/UefiTestToolsPkg/BiosTablesTest

Cc "Michael S. Tsirkin" <[email protected]>
Cc Ard Biesheuvel <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Philippe Mathieu-Daudé <[email protected]>
Cc Shannon Zhao <[email protected]>

---

Notes:
Again, if needed, I'd be happy to be designated as Maintainer for the files being added in this patch; please let me know the right spot in "MAINTAINERS".

 tests/data/uefi-boot-images/bios-tables-test.aarch64.iso.qcow2 | Bin 0 -> 11776 bytes
 tests/data/uefi-boot-images/bios-tables-test.arm.iso.qcow2     | Bin 0 -> 11776 bytes
 tests/data/uefi-boot-images/bios-tables-test.i386.iso.qcow2    | Bin 0 -> 12800 bytes
 tests/data/uefi-boot-images/bios-tables-test.x86_64.iso.qcow2  | Bin 0 -> 13312 bytes
 4 files changed, 0 insertions(+), 0 deletions(-)
tests/data/uefi-boot-images/bios-tables-test.aarch64.iso.qcow2 new file binary +0 -0
new file mode 100644
index 0000000000000000000000000000000000000000..fa2cf9291215c2fce59e526e1147be950cdce818
GIT binary patch
tests/data/uefi-boot-images/bios-tables-test.arm.iso.qcow2 new file binary +0 -0
new file mode 100644
index 0000000000000000000000000000000000000000..c83358f2367630e6a6b499ac26c1628dab2b0aff
GIT binary patch
tests/data/uefi-boot-images/bios-tables-test.i386.iso.qcow2 new file binary +0 -0
new file mode 100644
index 0000000000000000000000000000000000000000..2d897a221010607a5e48de4f2d8cc84024c5eae3
GIT binary patch
tests/data/uefi-boot-images/bios-tables-test.x86_64.iso.qcow2 new file binary +0 -0
new file mode 100644
index 0000000000000000000000000000000000000000..e38ec57e7f391814ab924fdc27e6917d341f652d
GIT binary patch
Signature · 1 line
2.19.1.3.g30247aa5d201