Fix qemu package build
A message that is not in the archive (sent off-list or not yet received): [email protected]
cover Fix qemu package build
Patch ping; no response received to patches submitted 5 days ago.
First, there is a problem that is currently reported by buildroot's
autobuilder: the --disable-git-update option is now deprecated and is interpreted as --with-git-submodules=validate, which can only work in a git clone. The correct replacement is --with-git-submodules=ignore.
With that fixed, package/qemu build succeeds. However, on hosts using make 3.82 (e.g CentOS 7), despite reporting build success, it fails to actually build or install any target binaries. The problem is that Qemu's makefile incorrectly interprets the --no-print-directory passed by buildroot as the -n option to make and injects -n (--just-print) into its ninja build.
Alexey Neyman (2):
qemu: do not use deprecated option
qemu: filter out long make options
package/qemu/qemu.mk | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
Signature · 6 lines
2.27.0 _______________________________________________ buildroot mailing list [email protected] http://lists.busybox.net/mailman/listinfo/buildroot
PATCH 1/2 package/qemu: do not use deprecated option
Qemu build fails with:
--disable-git-update deprecated, use --with-git-submodules=validate
ERROR: cannot validate git submodules without .git
Use --with-git-submodules=ignore instead.
--- package/qemu/qemu.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
package/qemu/qemu.mk +1 -1
@@ -198,7 +198,7 @@ define QEMU_CONFIGURE_CMDS --disable-vhost-crypto \ --disable-libxml2 \ --disable-capstone \ - --disable-git-update \ + --with-git-submodules=ignore \ --disable-opengl \ --disable-vhost-user-blk-server \ --disable-virtiofsd \
Signature · 6 lines
2.27.0 _______________________________________________ buildroot mailing list [email protected] http://lists.busybox.net/mailman/listinfo/buildroot
Re: [Buildroot] [PATCH 1/2] package/qemu: do not use deprecated option
Alexey, All,
On 2021-07-21 12:46 -0700, Alexey Neyman spake thusly:
Quoted 8 lines
Qemu build fails with:
--disable-git-update deprecated, use --with-git-submodules=validate
ERROR: cannot validate git submodules without .gitUse --with-git-submodules=ignore instead.
Signed-off-by Alexey Neyman <[email protected]>
An identical patch by Joseph was already pending in the backlog, so I applied his patch.
Thanks!
Regards,
Yann E. MORIN.
Quoted 24 lines
--- package/qemu/qemu.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)package/qemu/qemu.mk +1
-1@@ -198,7 +198,7 @@ define QEMU_CONFIGURE_CMDS --disable-vhost-crypto \ --disable-libxml2 \ --disable-capstone \ - --disable-git-update \ + --with-git-submodules=ignore \ --disable-opengl \ --disable-vhost-user-blk-server \ --disable-virtiofsd \Signature · 6 lines
2.27.0 _______________________________________________ buildroot mailing list [email protected] http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 10 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' _______________________________________________ buildroot mailing list [email protected] http://lists.busybox.net/mailman/listinfo/buildroot
PATCH 2/2 package/qemu: filter out long make options
In reply to Alexey NeymanSince qemu started using ninja, they have the following fragment in the Makefile:
MAKE.n = $(findstring n,$(firstword $(MAKEFLAGS))) NINJAFLAGS = ... $(if $(MAKE.n), -n) ...
Buildroot's generated makefile in the O= directory invokes make in the base buildroot with --no-print-directory. However, make's placement of the --no-print-directory in MAKEFLAGS varies between the versions of the host make; make 4.3 places that at the end while make 3.82 places it at the beginning. As a result, if building on a system with an older host make, qemu's makefile invokes ninja -n which does not generate any outputs.
To reproduce, on a CentOS 7 machine or docker image:
mkdir /tmp/br-build && cd /tmp/br-build
make -C ~/buildroot pc_x86_64_bios_defconfig O=pwd make menuconfig # Switch to glibc, enable "QEMU" and "QEMU tools" make all # Build succeeds
find target -name qemu-img # No binary has been built
--- package/qemu/qemu.mk | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
package/qemu/qemu.mk +3 -2
@@ -206,14 +206,15 @@ define QEMU_CONFIGURE_CMDS $(QEMU_OPTS) endef +# QEMU Makefile incorrectly interprets long options to make define QEMU_BUILD_CMDS unset TARGET_DIR; \ - $(TARGET_MAKE_ENV) $(MAKE) -C $(@D) + MAKEFLAGS="$(filter-out --%,$(MAKEFLAGS))" $(TARGET_MAKE_ENV) $(MAKE) -C $(@D) endef define QEMU_INSTALL_TARGET_CMDS unset TARGET_DIR; \ - $(TARGET_MAKE_ENV) $(MAKE) -C $(@D) $(QEMU_MAKE_ENV) DESTDIR=$(TARGET_DIR) install + MAKEFLAGS="$(filter-out --%,$(MAKEFLAGS))" $(TARGET_MAKE_ENV) $(MAKE) -C $(@D) $(QEMU_MAKE_ENV) DESTDIR=$(TARGET_DIR) install endef $(eval $(generic-package))
Signature · 6 lines
2.27.0 _______________________________________________ buildroot mailing list [email protected] http://lists.busybox.net/mailman/listinfo/buildroot
Re: [Buildroot] [PATCH 2/2] package/qemu: filter out long make options
Hello,
Quoted 14 lines · On Wed, 21 Jul 2021 12:46:05 -0700 Alexey Neyman <[email protected]> wrote:
Since qemu started using ninja, they have the following fragment in the
Makefile:MAKE.n = $(findstring n,$(firstword $(MAKEFLAGS))) NINJAFLAGS = ... $(if $(MAKE.n), -n) ...
Buildroot's generated makefile in the O= directory invokes make in the base buildroot with --no-print-directory. However, make's placement of the --no-print-directory in MAKEFLAGS varies between the versions of the host make; make 4.3 places that at the end while make 3.82 places it at the beginning. As a result, if building on a system with an older host make, qemu's makefile invokes
ninja -nwhich does not generate any outputs.
Quoted 6 lines
To reproduce, on a CentOS 7 machine or docker image:
mkdir/tmp/br-build&& cd/tmp/br-build
make -C ~/buildrootpc_x86_64_bios_defconfigO=pwdmake menuconfig # Switch to glibc, enable "QEMU" and "QEMU tools" make all # Build succeeds
find target -name qemu-img # No binary has been built
I'm not sure to grasp all the implications of this, but isn't this a bug in Qemu's build machinery, that should be fixed in Qemu's Makefile instead ?
Thomas
Signature · 7 lines
Thomas Petazzoni, co-owner and CEO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com _______________________________________________ buildroot mailing list [email protected] http://lists.busybox.net/mailman/listinfo/buildroot
Re: [Buildroot] [PATCH 2/2] package/qemu: filter out long make options
Hi,
Quoted 25 lines · Thomas Petazzoni
Hello,
Quoted 14 lines · On Wed, 21 Jul 2021 12:46:05 -0700 Alexey Neyman <[email protected]> wrote:
Since qemu started using ninja, they have the following fragment in the
Makefile:MAKE.n = $(findstring n,$(firstword $(MAKEFLAGS))) NINJAFLAGS = ... $(if $(MAKE.n), -n) ...
Buildroot's generated makefile in the O= directory invokes make in the base buildroot with --no-print-directory. However, make's placement of the --no-print-directory in MAKEFLAGS varies between the versions of the host make; make 4.3 places that at the end while make 3.82 places it at the beginning. As a result, if building on a system with an older host make, qemu's makefile invokes
ninja -nwhich does not generate any outputs.Quoted 6 lines
To reproduce, on a CentOS 7 machine or docker image:
mkdir/tmp/br-build&& cd/tmp/br-build
make -C ~/buildrootpc_x86_64_bios_defconfigO=pwdmake menuconfig # Switch to glibc, enable "QEMU" and "QEMU tools" make all # Build succeeds
find target -name qemu-img # No binary has been built
Quoted 3 lines
I'm not sure to grasp all the implications of this, but isn't this a bug in Qemu's build machinery, that should be fixed in Qemu's
Makefileinstead ?
I was on the fence about it myself. Okay, I'll try to send a fix for qemu/Makefile to Qemu developers and see if they agree to pick it up; they could argue they don't support being invoked with --no-print-directory.
Regards,
Alexey.
_______________________________________________
buildroot mailing list
[email protected]
http://lists.busybox.net/mailman/listinfo/buildroot
Re: [Buildroot] [PATCH 2/2] package/qemu: filter out long make options
Quoted 7 lines · On Wed, 21 Jul 2021 15:18:36 -0700 Alexey Neyman <[email protected]> wrote:
Quoted 3 lines
I'm not sure to grasp all the implications of this, but isn't this a bug in Qemu's build machinery, that should be fixed in Qemu's
Makefileinstead ?I was on the fence about it myself. Okay, I'll try to send a fix for qemu/Makefile to Qemu developers and see if they agree to pick it up; they could argue they don't support being invoked with --no-print-directory.
It would be odd for them to consider that as a normal behavior, --no-print-directory is really a standard make option.
Let us know what they say, of course.
Thanks a lot!
Thomas
Signature · 7 lines
Thomas Petazzoni, co-owner and CEO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com _______________________________________________ buildroot mailing list [email protected] http://lists.busybox.net/mailman/listinfo/buildroot
PATCH 1/1 package/qemu: filter out long make options
Since qemu started using ninja, they have the following fragment in the Makefile:
MAKE.n = $(findstring n,$(firstword $(MAKEFLAGS))) NINJAFLAGS = ... $(if $(MAKE.n), -n) ...
Buildroot's generated makefile in the O= directory invokes make in the base buildroot with --no-print-directory. However, make's placement of the --no-print-directory in MAKEFLAGS varies between the versions of the host make; make 4.3 places that at the end while make 3.82 places it at the beginning. As a result, if building on a system with an older host make, qemu's makefile invokes ninja -n which does not generate any outputs.
To reproduce, on a CentOS 7 machine or docker image:
mkdir /tmp/br-build && cd /tmp/br-build
make -C ~/buildroot pc_x86_64_bios_defconfig O=pwd make menuconfig # Switch to glibc, enable "QEMU" and "QEMU tools" make all # Build succeeds
find target -name qemu-img # No binary has been built
Pick up the fix commited in Qemu upstream.
--- .../0004-Makefile-ignore-long-options.patch | 41 +++++++++++++++++++ 1 file changed, 41 insertions(+) create mode 100644 package/qemu/0004-Makefile-ignore-long-options.patch
package/qemu/0004-Makefile-ignore-long-options.patch new file +41 -0
@@ -0,0 +1,41 @@ +From 14833e24dea49303ebc2464813601054b6cdfcac Mon Sep 17 00:00:00 2001 +From: Alexey Neyman <[email protected]> +Date: Wed, 21 Jul 2021 19:08:46 -0700 +Subject: [PATCH] Makefile: ignore long options + +When searching for options like -n in MAKEFLAGS, current code may result +in a false positive match when make is invoked with long options like +--no-print-directory. This has been observed with certain versions of +host make (e.g. 3.82) while building the Qemu package in buildroot. + +Filter out such long options before searching for one-character options. + +Signed-off-by: Alexey Neyman <[email protected]> +Message-Id: <[email protected]> +Signed-off-by: Paolo Bonzini <[email protected]> +--- + Makefile | 8 +++++--- + 1 file changed, 5 insertions(+), 3 deletions(-) + +diff --git a/Makefile b/Makefile +index 6c36330eef..401c623a65 100644 +--- a/Makefile ++++ b/Makefile +@@ -129,9 +129,11 @@ endif + # 4. Rules to bridge to other makefiles + + ifneq ($(NINJA),) +-MAKE.n = $(findstring n,$(firstword $(MAKEFLAGS))) +-MAKE.k = $(findstring k,$(firstword $(MAKEFLAGS))) +-MAKE.q = $(findstring q,$(firstword $(MAKEFLAGS))) ++# Filter out long options to avoid flags like --no-print-directory which ++# may result in false positive match for MAKE.n ++MAKE.n = $(findstring n,$(firstword $(filter-out --%,$(MAKEFLAGS)))) ++MAKE.k = $(findstring k,$(firstword $(filter-out --%,$(MAKEFLAGS)))) ++MAKE.q = $(findstring q,$(firstword $(filter-out --%,$(MAKEFLAGS)))) + MAKE.nq = $(if $(word 2, $(MAKE.n) $(MAKE.q)),nq) + NINJAFLAGS = $(if $V,-v) $(if $(MAKE.n), -n) $(if $(MAKE.k), -k0) \ + $(filter-out -j, $(lastword -j1 $(filter -l% -j%, $(MAKEFLAGS)))) \ +-- +2.27.0 +
Signature · 6 lines
2.27.0 _______________________________________________ buildroot mailing list [email protected] http://lists.busybox.net/mailman/listinfo/buildroot
Yann E. MORIN
AppliedRe: [Buildroot] [PATCH] package/qemu: filter out long make options
Alexey, All,
On 2021-08-07 12:15 -0700, Alexey Neyman spake thusly:
Quoted 21 lines
Since qemu started using ninja, they have the following fragment in the
Makefile:MAKE.n = $(findstring n,$(firstword $(MAKEFLAGS))) NINJAFLAGS = ... $(if $(MAKE.n), -n) ...
Buildroot's generated makefile in the O= directory invokes make in the base buildroot with --no-print-directory. However, make's placement of the --no-print-directory in MAKEFLAGS varies between the versions of the host make; make 4.3 places that at the end while make 3.82 places it at the beginning. As a result, if building on a system with an older host make, qemu's makefile invokes
ninja -nwhich does not generate any outputs.To reproduce, on a CentOS 7 machine or docker image:
mkdir/tmp/br-build&& cd/tmp/br-build
make -C ~/buildrootpc_x86_64_bios_defconfigO=pwdmake menuconfig # Switch to glibc, enable "QEMU" and "QEMU tools" make all # Build succeeds
find target -name qemu-img # No binary has been built
Quoted 3 lines
Pick up the fix commited in Qemu upstream.
Signed-off-by Alexey Neyman <[email protected]>
Applied to master, thanks.
Regards,
Yann E. MORIN.
Quoted 59 lines
--- .../0004-Makefile-ignore-long-options.patch | 41 +++++++++++++++++++ 1 file changed, 41 insertions(+) create mode 100644 package/qemu/0004-Makefile-ignore-long-options.patchpackage/qemu/0004-Makefile-ignore-long-options.patch new file +41
-0@@ -0,0 +1,41 @@ +From 14833e24dea49303ebc2464813601054b6cdfcac Mon Sep 17 00:00:00 2001 +From: Alexey Neyman <[email protected]> +Date: Wed, 21 Jul 2021 19:08:46 -0700 +Subject: [PATCH] Makefile: ignore long options + +When searching for options like -n in MAKEFLAGS, current code may result +in a false positive match when make is invoked with long options like +--no-print-directory. This has been observed with certain versions of +host make (e.g. 3.82) while building the Qemu package in buildroot. + +Filter out such long options before searching for one-character options. + +Signed-off-by: Alexey Neyman <[email protected]> +Message-Id: <[email protected]> +Signed-off-by: Paolo Bonzini <[email protected]> +--- + Makefile | 8 +++++--- + 1 file changed, 5 insertions(+), 3 deletions(-) + +diff --git a/Makefile b/Makefile +index 6c36330eef..401c623a65 100644 +--- a/Makefile ++++ b/Makefile +@@ -129,9 +129,11 @@ endif + # 4. Rules to bridge to other makefiles + + ifneq ($(NINJA),) +-MAKE.n = $(findstring n,$(firstword $(MAKEFLAGS))) +-MAKE.k = $(findstring k,$(firstword $(MAKEFLAGS))) +-MAKE.q = $(findstring q,$(firstword $(MAKEFLAGS))) ++# Filter out long options to avoid flags like --no-print-directory which ++# may result in false positive match for MAKE.n ++MAKE.n = $(findstring n,$(firstword $(filter-out --%,$(MAKEFLAGS)))) ++MAKE.k = $(findstring k,$(firstword $(filter-out --%,$(MAKEFLAGS)))) ++MAKE.q = $(findstring q,$(firstword $(filter-out --%,$(MAKEFLAGS)))) + MAKE.nq = $(if $(word 2, $(MAKE.n) $(MAKE.q)),nq) + NINJAFLAGS = $(if $V,-v) $(if $(MAKE.n), -n) $(if $(MAKE.k), -k0) \ + $(filter-out -j, $(lastword -j1 $(filter -l% -j%, $(MAKEFLAGS)))) \ +-- +2.27.0 +Signature · 6 lines
2.27.0 _______________________________________________ buildroot mailing list [email protected] http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 10 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' _______________________________________________ buildroot mailing list [email protected] http://lists.busybox.net/mailman/listinfo/buildroot