Fix tarball builds of UEFI/EDK2 firmware
cover Fix tarball builds of UEFI/EDK2 firmware
Bruce noticed that we cannot build make efi target from the v4.1.0 tarball. This is due to a failure on the part of the make-release script to pull in submodules nested under other submodules, as well as Makefile.edk2's assumptions about being in a git tree.
Suggestions for more robust solutions are definitely welcome, but for now this series takes what seems to be the most direct approach.
roms/Makefile.edk2 | 7 ++++++- scripts/make-release | 8 ++++++++ 2 files changed, 14 insertions(+), 1 deletion(-)
PATCH 1/2 make-release: pull in edk2 submodules so we can build it from tarballs
The make efi target added by 536d2173 is built from the roms/edk2 submodule, which in turn relies on additional submodules nested under roms/edk2.
The make-release script currently only pulls in top-level submodules, so these nested submodules are missing in the resulting tarball.
We could try to address this situation more generally by recursively pulling in all submodules, but this doesn't necessarily ensure the end-result will build properly (this case also required other changes).
Additionally, due to the nature of submodules, we may not always have control over how these sorts of things are dealt with, so for now we continue to handle it on a case-by-case in the make-release script.
--- scripts/make-release | 8 ++++++++ 1 file changed, 8 insertions(+)
scripts/make-release +8 -0
@@ -20,6 +20,14 @@ git checkout "v${version}" git submodule update --init (cd roms/seabios && git describe --tags --long --dirty > .version) (cd roms/skiboot && ./make_version.sh > .version) +# Fetch edk2 submodule's submodules, since it won't have access to them via +# the tarball later. +# +# A more uniform way to handle this sort of situation would be nice, but we +# don't necessarily have much control over how a submodule handles its +# submodule dependencies, so we continue to handle these on a case-by-case +# basis for now. +(cd roms/edk2 && git submodule update --init) popd tar --exclude=.git -cjf ${destination}.tar.bz2 ${destination} rm -rf ${destination}
Signature · 1 line
2.17.1
PATCH 2/2 roms/Makefile.edk2: don't pull in submodules when building from tarball
In reply to Michael RothCurrently the make efi target pulls submodules nested under the roms/edk2 submodule as dependencies. However, when we attempt to build from a tarball this fails since we are no longer in a git tree.
A preceding patch will pre-populate these submodules in the tarball, so assume this build dependency is only needed when building from a git tree.
--- roms/Makefile.edk2 | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
roms/Makefile.edk2 +6 -1
@@ -46,8 +46,13 @@ all: $(foreach flashdev,$(flashdevs),../pc-bios/edk2-$(flashdev).fd.bz2) \ # files. .INTERMEDIATE: $(foreach flashdev,$(flashdevs),../pc-bios/edk2-$(flashdev).fd) +# Fetch edk2 submodule's submodules. If it is not in a git tree, assume +# we're building from a tarball and that they've already been fetched by +# make-release/tarball scripts. submodules: - cd edk2 && git submodule update --init --force + if test -d edk2/.git; then \ + cd edk2 && git submodule update --init --force; \ + fi # See notes on the ".NOTPARALLEL" target and the "+" indicator in # "tests/uefi-test-tools/Makefile".
Signature · 1 line
2.17.1
Re: [Qemu-devel] [PATCH 2/2] roms/Makefile.edk2: don't pull in submodules when building from tarball
Quoted 27 lines · Michael Roth
Currently the
make efitarget pulls submodules nested under the roms/edk2 submodule as dependencies. However, when we attempt to build from a tarball this fails since we are no longer in a git tree.A preceding patch will pre-populate these submodules in the tarball, so assume this build dependency is only needed when building from a git tree.
Reported-by Bruce Rogers <[email protected]>Cc Laszlo Ersek <[email protected]>Cc Bruce Rogers <[email protected]>Cc [email protected] # v4.1.0Signed-off-by Michael Roth <[email protected]>--- roms/Makefile.edk2 | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)roms/Makefile.edk2 +1
-0@@ -46,8 +46,13 @@ all: $(foreach flashdev,$(flashdevs),../pc-bios/edk2-$(flashdev).fd.bz2) \ # files. .INTERMEDIATE: $(foreach flashdev,$(flashdevs),../pc-bios/edk2-$(flashdev).fd) +# Fetch edk2 submodule's submodules. If it is not in a git tree, assume
roms/Makefile.edk2 all: $(foreach flashdev,$(flashdevs),../pc-bios/edk2-$(flashdev).fd.bz2) \ · line 58
Quoted 10 lines
roms/Makefile.edk2 +5
-1+# we're building from a tarball and that they've already been fetched by +# make-release/tarball scripts. submodules: - cd edk2 && git submodule update --init --force + if test -d edk2/.git; then \ + cd edk2 && git submodule update --init --force; \ + fi # See notes on the ".NOTPARALLEL" target and the "+" indicator in # "tests/uefi-test-tools/Makefile".
Thank you,
Laszlo
Re: [Qemu-devel] [PATCH 2/2] roms/Makefile.edk2: don't pull in submodules when building from tarball
In reply to Michael RothQuoted 22 lines · Michael Roth
Currently the
make efitarget pulls submodules nested under the roms/edk2 submodule as dependencies. However, when we attempt to build from a tarball this fails since we are no longer in a git tree.A preceding patch will pre-populate these submodules in the tarball, so assume this build dependency is only needed when building from a git tree.
Reported-by Bruce Rogers <[email protected]>Cc Laszlo Ersek <[email protected]>Cc Bruce Rogers <[email protected]>Cc [email protected] # v4.1.0Signed-off-by Michael Roth <[email protected]>--- roms/Makefile.edk2 | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)roms/Makefile.edk2 +0
-0
roms/Makefile.edk2 all: $(foreach flashdev,$(flashdevs),../pc-bios/edk2-$(flashdev).fd.bz2) \ · line 51
Quoted 6 lines
roms/Makefile.edk2 +3
-0@@ -46,8 +46,13 @@ all: $(foreach flashdev,$(flashdevs),../pc-bios/edk2-$(flashdev).fd.bz2) \ # files. .INTERMEDIATE: $(foreach flashdev,$(flashdevs),../pc-bios/edk2-$(flashdev).fd) +# Fetch edk2 submodule's submodules. If it is not in a git tree, assume +# we're building from a tarball and that they've already been fetched by +# make-release/tarball scripts.
Annoying, without using the make-release tool in a fresh clone, I get qemu/roms/edk2/CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf(-1):
error 000E: File/directory not found in workspace
I vaguely remember there was a thread about not using 'git submodule update --init --recursive' in the root directory but can't find it, any idea?
Quoted 9 lines
roms/Makefile.edk2 +3
-1submodules: - cd edk2 && git submodule update --init --force + if test -d edk2/.git; then \ + cd edk2 && git submodule update --init --force; \ + fi # See notes on the ".NOTPARALLEL" target and the "+" indicator in # "tests/uefi-test-tools/Makefile".
Re: [Qemu-devel] [PATCH 2/2] roms/Makefile.edk2: don't pull in submodules when building from tarball
Quoted 34 lines · Philippe Mathieu-Daudé
Quoted 22 lines · Michael Roth
Currently the
make efitarget pulls submodules nested under the roms/edk2 submodule as dependencies. However, when we attempt to build from a tarball this fails since we are no longer in a git tree.A preceding patch will pre-populate these submodules in the tarball, so assume this build dependency is only needed when building from a git tree.
Reported-by Bruce Rogers <[email protected]>Cc Laszlo Ersek <[email protected]>Cc Bruce Rogers <[email protected]>Cc [email protected] # v4.1.0Signed-off-by Michael Roth <[email protected]>--- roms/Makefile.edk2 | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)roms/Makefile.edk2 +0
-0roms/Makefile.edk2 all: $(foreach flashdev,$(flashdevs),../pc-bios/edk2-$(flashdev).fd.bz2) \ · line 51
Quoted 6 lines
roms/Makefile.edk2 +3
-0@@ -46,8 +46,13 @@ all: $(foreach flashdev,$(flashdevs),../pc-bios/edk2-$(flashdev).fd.bz2) \ # files. .INTERMEDIATE: $(foreach flashdev,$(flashdevs),../pc-bios/edk2-$(flashdev).fd) +# Fetch edk2 submodule's submodules. If it is not in a git tree, assume +# we're building from a tarball and that they've already been fetched by +# make-release/tarball scripts.Annoying, without using the make-release tool in a fresh clone, I get qemu/roms/edk2/CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf(-1):
error 000E: File/directory not found in workspace
Quoted 3 lines
I vaguely remember there was a thread about not using 'git submodule update --init --recursive' in the root directory but can't find it, any idea?
I think that was a point made in the first patch of this series -- the "--recursive" flag is unbounded, and it might pull in several unneeded submodules.
For example, openssl itself appears to have three submodules (boringssl, krb5, pyca-cryptography), and none of those is needed for building openssl the way edk2 consumes it.
I seem to remember that patch#1 in this series stated: we'd be handling the submodules on a case-by-case basis.
Thanks
Laszlo
Quoted 10 lines
Quoted 10 lines
roms/Makefile.edk2 +3
-1submodules: - cd edk2 && git submodule update --init --force + if test -d edk2/.git; then \ + cd edk2 && git submodule update --init --force; \ + fi # See notes on the ".NOTPARALLEL" target and the "+" indicator in # "tests/uefi-test-tools/Makefile".
Quoted 4 lines · Michael Roth
Bruce noticed that we cannot build
make efitarget from the v4.1.0 tarball. This is due to a failure on the part of the make-release script to pull in submodules nested under other submodules, as well as Makefile.edk2's assumptions about being in a git tree.
Hmm I'd expect distributions interested in building EDK2, build it properly tuned, outside of QEMU. I consider what is here as default config useful for testing, not for production use. I might be wrong.
Meanwhile, it is true 'make efi' should not fail.
Quoted 2 lines
Suggestions for more robust solutions are definitely welcome, but for now this series takes what seems to be the most direct approach.
This is fine:
I'll queue these patches in my edk2-next branch since I'm preparing a pull request.
Thanks,
Phil.