Skip to contentSkip to conversation

tools/firmware: update OVMF Makefile, when necessary

PATCH New Ian Jackson 3 participants 10 messages Apr 29, 2019 – May 30, 2019

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 tools/firmware: update OVMF Makefile, when necessary

On advice from Wei, I am about to push this squashed backport to the stable trees. We think this will help fix osstest on those branches following the upgrade to Debian stretch.

Ian.

From e983e8ae84efd5e43045a3d20a820f13cb4a75bf Mon Sep 17 00:00:00 2001
From: Wei Liu <[email protected]>
Date: Wed, 28 Nov 2018 17:43:33 +0000
Subject: [PATCH] tools/firmware: update OVMF Makefile, when necessary

[ This is two commits from master aka staging-4.12: ]

OVMF has become dependent on OpenSSL, which is included as a submodule. Initialise submodules before building.

(cherry picked from commit b16281870e06f5f526029a4e69634a16dc38e8e4)

tools: only call git when necessary in OVMF Makefile

Users may choose to export a snapshot of OVMF and build it with xen.git supplied ovmf-makefile. In that case we don't need to call git submodule.

Fixes b16281870e.

(cherry picked from commit 68292c94a60eab24514ab4a8e4772af24dead807)
---
 tools/firmware/ovmf-makefile | 1 +
 1 file changed, 1 insertion(+)
tools/firmware/ovmf-makefile +1 -0
index 2838744461..55f9992145 100644
@@ -16,6 +16,7 @@ all: build
 
 .PHONY: build
 build:
+	if test -e .git ; then $(GIT) submodule update --init --recursive ; fi
 	OvmfPkg/build.sh -a X64 -b $(TARGET) -n 4
 	cp Build/OvmfX64/$(TARGET)_GCC*/FV/OVMF.fd ovmf.bin
 
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#1 lore raw

PATCH 1/1 tools/firmware: update OVMF Makefile, when necessary

Ian Jackson writes ("[PATCH STABLE] tools/firmware: update OVMF Makefile, when necessary"):

Quoted 3 lines

On advice from Wei, I am about to push this squashed backport to the stable trees. We think this will help fix osstest on those branches following the upgrade to Debian stretch.

Now done, including for staging-4.6. 4.6 is out of security support, but osstest wants to be able to build 4.6 so that it can test migration from 4.6 to 4.7, and 4.7 *is* still (just about) in security support.

I expect the 4.6 push gate to fail and to have to force push it.

It is also possible that we will experience other build fallout.

Ian.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#2 lore raw

PATCH 1/1 tools/firmware: update OVMF Makefile, when necessary

Ian Jackson writes ("[PATCH STABLE] tools/firmware: update OVMF Makefile, when necessary"):

Quoted 5 lines

Ian Jackson writes ("[PATCH STABLE] tools/firmware: update OVMF Makefile, when necessary"):

Quoted 3 lines

On advice from Wei, I am about to push this squashed backport to the stable trees. We think this will help fix osstest on those branches following the upgrade to Debian stretch.

Quoted 8 lines

Now done, including for staging-4.6. 4.6 is out of security support, but osstest wants to be able to build 4.6 so that it can test migration from 4.6 to 4.7, and 4.7 *is* still (just about) in security support.

I expect the 4.6 push gate to fail and to have to force push it.

It is also possible that we will experience other build fallout.

In fact, even with this patch, 4.6 does not build because it is missing a backport of
ebdba150bff1d914805d60efa576337bbef0c305
xenalyze: fix misleading indentation.

So I have cherry picked
ebdba150bff1d914805d60efa576337bbef0c305
which is is the 4.7 backport of that commit, onto staging-4.6.

Ian.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#3 lore raw

PATCH 1/1 tools/firmware: update OVMF Makefile, when necessaryf

In reply to Ian Jackson

Ian Jackson writes ("[PATCH STABLE] tools/firmware: update OVMF Makefile, when necessary"):

Quoted 4 lines

Now done, including for staging-4.6. 4.6 is out of security support, but osstest wants to be able to build 4.6 so that it can test migration from 4.6 to 4.7, and 4.7 *is* still (just about) in security support.

I have cherry picked another 2 build fixes from 4.7 onto staging-4.6:

7c8db58d3739c805f4c0f773b65157f306b00c2a
Fix misleading indentation warnings

e675332d5d049bbf5ce4cf1924a6414b8035963d
xenalyze: remove cr3_compare_total

Ian.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#4 lore raw

Stable trees (4.6 and 4.7), building on stretch, osstest, redux

In reply to Ian Jackson

Ian Jackson writes ("[PATCH STABLE] tools/firmware: update OVMF Makefile, when necessary"):

Quoted 8 lines

Now done, including for staging-4.6. 4.6 is out of security support, but osstest wants to be able to build 4.6 so that it can test migration from 4.6 to 4.7, and 4.7 *is* still (just about) in security support.

I expect the 4.6 push gate to fail and to have to force push it.

It is also possible that we will experience other build fallout.

I am still struggling with this. 4.7 and 4.6 still do not build. Right now there are two problems that need thought:

1. That old ipxe is just too badly broken. I spent a long while trying to backport compiler fixes but it is totally ridiculous. IMO our only sensible option is to update at least osstest's buildsx to a much newer ipxe.

This could be done by cherry picking
38ab99b26bf4298a33105ec66f3f6a3f7e05a326
ipxe: update to newer commit
(which is from xen 4.8 ish) onto our 4.7 and 4.6 branches.

If this is felt too intrusive, then I could somehow make it conditional and have osstest use it. This is not entirely trivial because we have an ad hoc patch application thing.

I'm sort of tempted to have osstest arbitrarily run
git cherry-pick 38ab99b26bf4298a33105ec66f3f6a3f7e05a326
at some appropriate point...

2. hvmloader fails to build, I think because we need 7825ae12df1f6d48c4d009cbbdf5a55aff27291b
errno: introduce EISDIR/EROFS/ENOTEMPTY to the ABI
03720ea541382a3ca80eaaec2aa11932b03aacaf
errno: declare aliases using XEN_ERRNO()
67790205df26e7c3dfeef8b8e64194ebc279220d
public/errno: Reduce complexity of inclusion
305e957ffee94fc06c4ba53ef5562f1b8c1c6b02
hvmloader: use xen/errno.h rather than the host systems errno.h

Is backporting that lot OK ?

There are also some simple backports we need:
c2a17869d5dcd845d646bf4db122cad73596a2be
libfsimage: replace deprecated readdir_r() with readdir()
b9daff9d811285f1e40669bc621c2241793f7a95
libxl: replace deprecated readdir_r() with readdir()
668e4edf92fcf7cb929eed221059a3eeb02722c3
stubdom: make GMP aware that it's being cross-compiled
2f9eb73c2e2d7fdda8e2586c20f7dbd856002eba
stubdom: fix stubdom-vtpm build

With all of the above, 4.6 builds again. I guess 4.7 will too.

Fixing that will also probably enable the 4.8 push gate to pass. It is currently blocked because it wants to test 4.7->4.8 migration and can't build 4.7.

Ian.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#5 lore raw

Re: Stable trees (4.6 and 4.7), building on stretch, osstest, redux

Am Tue, 28 May 2019 20:59:24 +0100
schrieb Ian Jackson <[email protected]>:

Quoted 1 line

I am still struggling with this. 4.7 and 4.6 still do not build.

I maintain the various staging-X.Y branches for various releases, just for my own entertainment. The remaining build error in SLE_15 (and Tumbleweed) is some asm error in ipxe. It was silently fixed by rearranging the code, so there was no single commit to cherry-pick. At some point I lost interest and did not finish the work to make it build again.

https://build.opensuse.org/package/show/home:olh:xen-4.7/xen

The individual changes:

https://github.com/olafhering/xen/compare/olh-base-staging-4.7...olh-fixes-staging-4.7
https://github.com/olafhering/qemu-xen/compare/olh-base-xen-staging-4.7...olh-fixes-xen-staging-4.7
https://github.com/olafhering/seabios/compare/olh-base-xen-staging-4.7...olh-fixes-xen-staging-4.7
https://github.com/olafhering/edk2/compare/olh-base-xen-staging-4.7...olh-fixes-xen-staging-4.7
https://github.com/olafhering/ipxe/compare/olh-base-xen-staging-4.7...olh-fixes-xen-staging-4.7

Since I last touched it a few months ago, Tumbleweed moved on and will require additional changes for gcc9. Not sure how stale Stretch is.

To see changes for other staging-X.Y branches, replace '4.7' as needed.

Olaf

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#6 lore raw

Re: Stable trees (4.6 and 4.7), building on stretch, osstest, redux

In reply to Ian Jackson
Quoted 21 lines
Quoted 1 line
Quoted 1 line · On 28.05.19 at 21:59, <[email protected]> wrote:

Ian Jackson writes ("[PATCH STABLE] tools/firmware: update OVMF Makefile, when necessary"):

Quoted 8 lines

Now done, including for staging-4.6. 4.6 is out of security support, but osstest wants to be able to build 4.6 so that it can test migration from 4.6 to 4.7, and 4.7 *is* still (just about) in security support.

I expect the 4.6 push gate to fail and to have to force push it.

It is also possible that we will experience other build fallout.

I am still struggling with this. 4.7 and 4.6 still do not build. Right now there are two problems that need thought:

1. That old ipxe is just too badly broken. I spent a long while trying to backport compiler fixes but it is totally ridiculous. IMO our only sensible option is to update at least osstest's buildsx to a much newer ipxe.

Quoted 8 lines

This could be done by cherry picking
38ab99b26bf4298a33105ec66f3f6a3f7e05a326
ipxe: update to newer commit
(which is from xen 4.8 ish) onto our 4.7 and 4.6 branches.

If this is felt too intrusive, then I could somehow make it conditional and have osstest use it. This is not entirely trivial because we have an ad hoc patch application thing.

Since the new ipxe should have been proven stable enough in the newer trees, I think cherry-picking said commit should be fine.

Quoted 11 lines

2. hvmloader fails to build, I think because we need 7825ae12df1f6d48c4d009cbbdf5a55aff27291b
errno: introduce EISDIR/EROFS/ENOTEMPTY to the ABI
03720ea541382a3ca80eaaec2aa11932b03aacaf
errno: declare aliases using XEN_ERRNO()
67790205df26e7c3dfeef8b8e64194ebc279220d
public/errno: Reduce complexity of inclusion
305e957ffee94fc06c4ba53ef5562f1b8c1c6b02
hvmloader: use xen/errno.h rather than the host systems errno.h

Is backporting that lot OK ?

I think so, yes, albeit I'm puzzled how they would end up being needed.

Quoted 11 lines

There are also some simple backports we need:
c2a17869d5dcd845d646bf4db122cad73596a2be
libfsimage: replace deprecated readdir_r() with readdir()
b9daff9d811285f1e40669bc621c2241793f7a95
libxl: replace deprecated readdir_r() with readdir()
668e4edf92fcf7cb929eed221059a3eeb02722c3
stubdom: make GMP aware that it's being cross-compiled
2f9eb73c2e2d7fdda8e2586c20f7dbd856002eba
stubdom: fix stubdom-vtpm build

Quoted 5 lines

With all of the above, 4.6 builds again. I guess 4.7 will too.

Fixing that will also probably enable the 4.8 push gate to pass. It is currently blocked because it wants to test 4.7->4.8 migration and can't build 4.7.

And similarly in turn for 4.9's need to have a building 4.8 baseline, afaict.

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#7 lore raw

Re: Stable trees (4.6 and 4.7), building on stretch, osstest, redux

Jan Beulich writes ("Re: Stable trees (4.6 and 4.7), building on stretch, osstest, redux"):

Quoted 11 lines · On 28.05.19 at 21:59, <[email protected]> wrote:
Quoted 10 lines

1. That old ipxe is just too badly broken. I spent a long while trying to backport compiler fixes but it is totally ridiculous. IMO our only sensible option is to update at least osstest's buildsx to a much newer ipxe.

This could be done by cherry picking
38ab99b26bf4298a33105ec66f3f6a3f7e05a326
ipxe: update to newer commit
(which is from xen 4.8 ish) onto our 4.7 and 4.6 branches.

Quoted 7 lines
Quoted 3 lines

If this is felt too intrusive, then I could somehow make it conditional and have osstest use it. This is not entirely trivial because we have an ad hoc patch application thing.

Since the new ipxe should have been proven stable enough in the newer trees, I think cherry-picking said commit should be fine.

That wasn't what I was expecting you to say :-). I will go with your opinion since it is certainly less work.

Quoted 10 lines
Quoted 10 lines

2. hvmloader fails to build, I think because we need 7825ae12df1f6d48c4d009cbbdf5a55aff27291b
errno: introduce EISDIR/EROFS/ENOTEMPTY to the ABI
03720ea541382a3ca80eaaec2aa11932b03aacaf
errno: declare aliases using XEN_ERRNO()
67790205df26e7c3dfeef8b8e64194ebc279220d
public/errno: Reduce complexity of inclusion
305e957ffee94fc06c4ba53ef5562f1b8c1c6b02
hvmloader: use xen/errno.h rather than the host systems errno.h

Quoted 4 lines
Quoted 1 line

Is backporting that lot OK ?

I think so, yes, albeit I'm puzzled how they would end up being needed.

We need the last of these because in stretch Linux the host system's errno header files were reorganised in a way that was incompatible with the wrong-headed use of host headers and (only some) host header directories.

The others are neded because of textual or semantic conflicts.

Quoted 11 lines
Quoted 11 lines

There are also some simple backports we need:
c2a17869d5dcd845d646bf4db122cad73596a2be
libfsimage: replace deprecated readdir_r() with readdir()
b9daff9d811285f1e40669bc621c2241793f7a95
libxl: replace deprecated readdir_r() with readdir()
668e4edf92fcf7cb929eed221059a3eeb02722c3
stubdom: make GMP aware that it's being cross-compiled
2f9eb73c2e2d7fdda8e2586c20f7dbd856002eba
stubdom: fix stubdom-vtpm build

Quoted 8 lines
Quoted 5 lines

With all of the above, 4.6 builds again. I guess 4.7 will too.

Fixing that will also probably enable the 4.8 push gate to pass. It is currently blocked because it wants to test 4.7->4.8 migration and can't build 4.7.

And similarly in turn for 4.9's need to have a building 4.8 baseline, afaict.

Yes. We have fixed the 4.8 build but it hasn't passed its push gate so 4.9 is using the "tested" branch...

OK, I will go ahead with all of the above for 4.6 and 4.7. Thanks.

Thanks also to Olaf. Your experience with the ipxe problems was similar to mine. I think just upgrading is the best approach.

Ian.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#8 lore raw

Re: Stable trees (4.6 and 4.7), building on stretch, osstest, redux

In reply to Ian Jackson

Ian Jackson writes ("Stable trees (4.6 and 4.7), building on stretch, osstest, redux"):

Quoted 1 line

I am still struggling with this. 4.7 and 4.6 still do not build.

I have now pushed all of these to 4.6 and 4.7 and it builds for me. I will kill the current, doomed, 4.6 and 4.7 flights.

Quoted 10 lines

1. That old ipxe is just too badly broken. I spent a long while trying to backport compiler fixes but it is totally ridiculous. IMO our only sensible option is to update at least osstest's buildsx to a much newer ipxe.

This could be done by cherry picking
38ab99b26bf4298a33105ec66f3f6a3f7e05a326
ipxe: update to newer commit
(which is from xen 4.8 ish) onto our 4.7 and 4.6 branches.

Quoted 7 lines

If this is felt too intrusive, then I could somehow make it conditional and have osstest use it. This is not entirely trivial because we have an ad hoc patch application thing.

I'm sort of tempted to have osstest arbitrarily run
git cherry-pick 38ab99b26bf4298a33105ec66f3f6a3f7e05a326
at some appropriate point...

This applied cleanly to both.

Quoted 11 lines

2. hvmloader fails to build, I think because we need 7825ae12df1f6d48c4d009cbbdf5a55aff27291b
errno: introduce EISDIR/EROFS/ENOTEMPTY to the ABI
03720ea541382a3ca80eaaec2aa11932b03aacaf
errno: declare aliases using XEN_ERRNO()
67790205df26e7c3dfeef8b8e64194ebc279220d
public/errno: Reduce complexity of inclusion
305e957ffee94fc06c4ba53ef5562f1b8c1c6b02
hvmloader: use xen/errno.h rather than the host systems errno.h

Is backporting that lot OK ?

These were not needed for 4.7.

Quoted 6 lines

There are also some simple backports we need:
c2a17869d5dcd845d646bf4db122cad73596a2be
libfsimage: replace deprecated readdir_r() with readdir()
b9daff9d811285f1e40669bc621c2241793f7a95
libxl: replace deprecated readdir_r() with readdir()

These were not needed for 4.7

Quoted 4 lines

668e4edf92fcf7cb929eed221059a3eeb02722c3
stubdom: make GMP aware that it's being cross-compiled
2f9eb73c2e2d7fdda8e2586c20f7dbd856002eba
stubdom: fix stubdom-vtpm build

These were needed for 4.6 and 4.7. The last quoted commit there is wrong (the quoted commit id is a local one of mine). It should read:

7791790c7ab97c85306dce749c6c8eb56d1dc0da
stubdom: fix stubdom-vtpm build

Ian.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#9 lore raw

Re: Stable trees (4.6 and 4.7), building on stretch, osstest, redux

Ian Jackson writes ("Re: Stable trees (4.6 and 4.7), building on stretch, osstest, redux"):

Quoted 2 lines

I have now pushed all of these to 4.6 and 4.7 and it builds for me. I will kill the current, doomed, 4.6 and 4.7 flights.

I have now also backported
5f28de0b0e474e01931b719fa27ca30b8aa446e0
libxl: compilation warning fix for arm & aarch64

Ian.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel