Skip to contentSkip to conversation

qemu: allow selection of host qemu-user from menuconfig

PATCH New Thomas Petazzoni 3 participants 7 messages Dec 10, 2014 – May 21, 2015

lore
Share & tools
lore
#0 lore raw

cover qemu: allow selection of host qemu-user from menuconfig

Hello,

This patch series is a respin of those old patches posted by Frank Hunleth.

The first patch is almost unchanged (just refreshed on top of the current master) and creates a menuconfig entry in 'Host utilities' to enable Qemu.

The second patch is inspired by
http://patchwork.ozlabs.org/patch/319108/ (from Frank as well), but has a completely different implementation, which I find nicer (based on make code, plus the check is done much earlier, plus we don't need to depend on the toolchain package). Ideas on how to make the 'make' checks nicer are welcome.

This set of patches superseds:

http://patchwork.ozlabs.org/patch/319108/ (from Frank)
http://patchwork.ozlabs.org/patch/299014/ (from Frank)
http://patchwork.ozlabs.org/patch/419140/ (from Thierry Bultel)

Comments welcome.

Thomas

Frank Hunleth (1):
qemu: add to host utilities menu

Thomas Petazzoni (1):
qemu: add host/target Linux version check

 package/Config.in.host      |  1 +
 package/qemu/Config.in.host |  8 ++++++++
 package/qemu/qemu.mk        | 28 ++++++++++++++++++++++++++++
 3 files changed, 37 insertions(+)
 create mode 100644 package/qemu/Config.in.host
Signature · 1 line
2.1.0
#1 lore raw

PATCH 1/2 qemu: add to host utilities menu

From: Frank Hunleth <[email protected]>

This allows qemu-user to be selected by the user. One use case for this is to call qemu-user from post build scripts to run regression tests against the build.

---
 package/Config.in.host      | 1 +
 package/qemu/Config.in.host | 8 ++++++++
 2 files changed, 9 insertions(+)
 create mode 100644 package/qemu/Config.in.host
package/Config.in.host +1 -0
index 0b7bc36..9b127c8 100644
@@ -18,6 +18,7 @@ menu "Host utilities"
 	source "package/parted/Config.in.host"
 	source "package/patchelf/Config.in.host"
 	source "package/pwgen/Config.in.host"
+	source "package/qemu/Config.in.host"
 	source "package/sam-ba/Config.in.host"
 	source "package/squashfs/Config.in.host"
 	source "package/sunxi-tools/Config.in.host"
package/qemu/Config.in.host new file +8 -0
new file mode 100644
index 0000000..a1f0531
@@ -0,0 +1,8 @@
+config BR2_PACKAGE_HOST_QEMU
+	bool "host qemu"
+	help
+	  QEMU is a generic and open source machine emulator and virtualizer.
+
+	  This option builds a user emulator for your selected architecture.
+
+	  http://www.qemu.org
Signature · 1 line
2.1.0
#2 lore raw

PATCH 2/2 qemu: add host/target Linux version check

In reply to Thomas Petazzoni

Raise an error if the host is using an older kernel than the target. Since qemu-user passes emulated system calls to the host kernel, this prevents usage of qemu-user in situations where those system calls will fail.

This is based on an original patch from Frank Hunleth <[email protected]>, but completely rewritten in a different way:

* Instead of using shell based testing, we use pure make tests, which allows to detect the problem not when host-qemu starts to build, but at the very beginning of the entire Buildroot build.

* Instead of looking at $(STAGING_DIR)/usr/include/linux/version.h (which requires having a dependency on the 'toolchain' package, which is a bit unusual for a host package), we use the BR2_TOOLCHAIN_HEADERS_AT_LEAST Config.in option which tells us the version of the kernel headers used in the toolchain.

---
 package/qemu/qemu.mk | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)
package/qemu/qemu.mk +28 -0
index 9cff5ed..6c5ddf3 100644
@@ -59,6 +59,34 @@ HOST_QEMU_ARCH = ppc
 endif
 HOST_QEMU_TARGETS = $(HOST_QEMU_ARCH)-linux-user
 
+ifeq ($(BR2_PACKAGE_HOST_QEMU),y)
+HOST_QEMU_HOST_SYSTEM_TYPE = $(shell uname -s)
+ifneq ($(HOST_QEMU_HOST_SYSTEM_TYPE),Linux)
+$(error "qemu-user can only be used on Linux hosts")
+endif
+
+HOST_QEMU_HOST_SYSTEM_VERSION_MAJOR = $(shell uname -r | cut -f1 -d'.')
+HOST_QEMU_HOST_SYSTEM_VERSION_MINOR = $(shell uname -r | cut -f2 -d'.')
+HOST_QEMU_TARGET_SYSTEM_VERSION_MAJOR = $(shell echo $(BR2_TOOLCHAIN_HEADERS_AT_LEAST) | cut -f1 -d'.')
+HOST_QEMU_TARGET_SYSTEM_VERSION_MINOR = $(shell echo $(BR2_TOOLCHAIN_HEADERS_AT_LEAST) | cut -f2 -d'.')
+HOST_QEMU_COMPARE_VERSION_MAJOR = $(shell test $(HOST_QEMU_HOST_SYSTEM_VERSION_MAJOR) -ge $(HOST_QEMU_TARGET_SYSTEM_VERSION_MAJOR) && echo OK)
+HOST_QEMU_COMPARE_VERSION_MINOR = $(shell test $(HOST_QEMU_HOST_SYSTEM_VERSION_MINOR) -ge $(HOST_QEMU_TARGET_SYSTEM_VERSION_MINOR) && echo OK)
+
+#
+# The principle of qemu-user is that it emulates the instructions of
+# the target architecture when running the binary, and then when this
+# binary does a system call, it converts this system call into a
+# system call on the host machine. This mechanism makes an assumption:
+# that the target binary will not do system calls that do not exist on
+# the host. This basically requires that the target binary should be
+# built with kernel headers that are older or the same as the kernel
+# version running on the host machine.
+#
+ifneq ($(HOST_QEMU_COMPARE_VERSION_MAJOR)$(HOST_QEMU_COMPARE_VERSION_MINOR),OKOK)
+$(error "Refusing to build qemu-user: target Linux version newer than host's.")
+endif
+endif
+
 define HOST_QEMU_CONFIGURE_CMDS
 	cd $(@D); $(HOST_CONFIGURE_OPTS) ./configure    \
 		--target-list="$(HOST_QEMU_TARGETS)"    \
Signature · 1 line
2.1.0
#3 lore raw

PATCH 2/2 qemu: add host/target Linux version check

Hi Thomas,

Quoted 14 lines · Thomas Petazzoni

Raise an error if the host is using an older kernel than the target. Since qemu-user passes emulated system calls to the host kernel, this prevents usage of qemu-user in situations where those system calls will fail.

This is based on an original patch from Frank Hunleth <[email protected]>, but completely rewritten in a different way:

* Instead of using shell based testing, we use pure make tests, which allows to detect the problem not when host-qemu starts to build, but at the very beginning of the entire Buildroot build.

Quoted 7 lines

* Instead of looking at $(STAGING_DIR)/usr/include/linux/version.h (which requires having a dependency on the 'toolchain' package, which is a bit unusual for a host package), we use the BR2_TOOLCHAIN_HEADERS_AT_LEAST Config.in option which tells us the version of the kernel headers used in the toolchain.

Thanks for revisiting and improving the old host-qemu patch. I'm using your new changes to qemu.mk right now and everything seems to work as expected. So,

Thanks,
Frank

Quoted 46 lines
---
 package/qemu/qemu.mk | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)
package/qemu/qemu.mk +28 -1
index 9cff5ed..6c5ddf3 100644
@@ -59,6 +59,34 @@ HOST_QEMU_ARCH = ppc
 endif
 HOST_QEMU_TARGETS = $(HOST_QEMU_ARCH)-linux-user
 
+ifeq ($(BR2_PACKAGE_HOST_QEMU),y)
+HOST_QEMU_HOST_SYSTEM_TYPE = $(shell uname -s)
+ifneq ($(HOST_QEMU_HOST_SYSTEM_TYPE),Linux)
+$(error "qemu-user can only be used on Linux hosts")
+endif
+
+HOST_QEMU_HOST_SYSTEM_VERSION_MAJOR = $(shell uname -r | cut -f1 -d'.')
+HOST_QEMU_HOST_SYSTEM_VERSION_MINOR = $(shell uname -r | cut -f2 -d'.')
+HOST_QEMU_TARGET_SYSTEM_VERSION_MAJOR = $(shell echo $(BR2_TOOLCHAIN_HEADERS_AT_LEAST) | cut -f1 -d'.')
+HOST_QEMU_TARGET_SYSTEM_VERSION_MINOR = $(shell echo $(BR2_TOOLCHAIN_HEADERS_AT_LEAST) | cut -f2 -d'.')
+HOST_QEMU_COMPARE_VERSION_MAJOR = $(shell test $(HOST_QEMU_HOST_SYSTEM_VERSION_MAJOR) -ge $(HOST_QEMU_TARGET_SYSTEM_VERSION_MAJOR) && echo OK)
+HOST_QEMU_COMPARE_VERSION_MINOR = $(shell test $(HOST_QEMU_HOST_SYSTEM_VERSION_MINOR) -ge $(HOST_QEMU_TARGET_SYSTEM_VERSION_MINOR) && echo OK)
+
+#
+# The principle of qemu-user is that it emulates the instructions of
+# the target architecture when running the binary, and then when this
+# binary does a system call, it converts this system call into a
+# system call on the host machine. This mechanism makes an assumption:
+# that the target binary will not do system calls that do not exist on
+# the host. This basically requires that the target binary should be
+# built with kernel headers that are older or the same as the kernel
+# version running on the host machine.
+#
+ifneq ($(HOST_QEMU_COMPARE_VERSION_MAJOR)$(HOST_QEMU_COMPARE_VERSION_MINOR),OKOK)
+$(error "Refusing to build qemu-user: target Linux version newer than host's.")
+endif
+endif
+
 define HOST_QEMU_CONFIGURE_CMDS
        cd $(@D); $(HOST_CONFIGURE_OPTS) ./configure    \
                --target-list="$(HOST_QEMU_TARGETS)"    \
--

2.1.0

Signature · 4 lines
Frank Hunleth
Troodon Software LLC
Embedded Software Development
http://troodon-software.com/
#4 lore raw

PATCH 2/2 qemu: add host/target Linux version check

In reply to Thomas Petazzoni

Thomas Petazzoni
<[email protected]> wrote
in
news:1418249519-15794-3-git-send-email-thomas.petazzoni at free-electrons.com:

Quoted 4 lines

+ifneq ($(HOST_QEMU_COMPARE_VERSION_MAJOR)$(HOST_QEMU_COMPARE_VERSION_MINOR),OKO
K) +$(error "Refusing to build qemu-user: target Linux version newer than host's.") +endif

Hi,

while testing an allyesconfig with "make source-check" this error was raised. I had to comment this line in order to run the source-check. Is it possible to run this check only during a build phase?

Regards, Bernd

#5 lore raw

PATCH 2/2 qemu: add host/target Linux version check

Dear Bernd Kuhls,

Quoted 8 lines · Bernd Kuhls
Quoted 4 lines

+ifneq ($(HOST_QEMU_COMPARE_VERSION_MAJOR)$(HOST_QEMU_COMPARE_VERSION_MINOR),OKO
K) +$(error "Refusing to build qemu-user: target Linux version newer than host's.") +endif

while testing an allyesconfig with "make source-check" this error was raised. I had to comment this line in order to run the source-check. Is it possible to run this check only during a build phase?

Yes, but that's not the proper solution. The proper solution is to enclose the test with:

ifeq ($(BR_BUILDING),y)
...
endif

See how it's used in linux.mk or uboot.mk.

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#6 lore raw

cover qemu: allow selection of host qemu-user from menuconfig

In reply to Thomas Petazzoni

Hello,

Quoted 5 lines · Thomas Petazzoni

Frank Hunleth (1):
qemu: add to host utilities menu

Thomas Petazzoni (1):
qemu: add host/target Linux version check

I've applied both patches.

Thanks,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com