meson: Clean up some dependencies regarding qemu-system
PATCH 1/1 meson: Clean up some dependencies regarding qemu-system
Reduce amount of compiled code and installed binaries if just the qemu utilities such as qemu-img, qemu-io and qemu-nbd have to be built. This helps to make those tools easier available on hosts where qemu-system or qemu-user isn't supported.
Reason for this patch is that qemu fail to build on debian for some seconday non-release architectures (e.g. hppa, sh4, ia64), as can be seen here:
https://buildd.debian.org/status/package.php?p=qemu&suite=sid
This patch helps to clean up the build so that these tools can be provided on those platforms in future.
docs/meson.build +1 -1
@@ -49,7 +49,7 @@ if build_docs 'qemu-trace-stap.1': (stap.found() ? 'man1' : ''), 'virtfs-proxy-helper.1': (have_virtfs_proxy_helper ? 'man1' : ''), 'virtiofsd.1': (have_virtiofsd ? 'man1' : ''), - 'qemu.1': 'man1', + 'qemu.1': (have_system ? 'man1' : ''), 'qemu-block-drivers.7': 'man7', 'qemu-cpu-models.7': 'man7' }
meson.build +7 -5
@@ -3021,7 +3021,6 @@ subdir('util') subdir('qom') subdir('authz') subdir('crypto') -subdir('ui') subdir('hw') subdir('gdbstub')
@@ -3060,9 +3059,10 @@ if have_system or have_user arguments: ['@INPUT@', '@EXTRA_ARGS@', '-o', '@OUTPUT@']) subdir('libdecnumber') subdir('target') + subdir('ui') + subdir('audio') endif -subdir('audio') subdir('io') subdir('chardev') subdir('fsdev')
@@ -3581,7 +3581,7 @@ subdir('qga') # Don't build qemu-keymap if xkbcommon is not explicitly enabled # when we don't build tools or system -if xkbcommon.found() +if xkbcommon.found() and have_system # used for the update-keymaps target, so include rules even if !have_tools qemu_keymap = executable('qemu-keymap', files('qemu-keymap.c', 'ui/input-keymap.c') + genh, dependencies: [qemuutil, xkbcommon], install: have_tools)
@@ -3596,7 +3596,9 @@ if have_tools dependencies: [blockdev, qemuutil, gnutls, selinux], install: true) - subdir('storage-daemon') + if have_system + subdir('storage-daemon') + endif subdir('contrib/rdmacm-mux') subdir('contrib/elf2dmp')
@@ -3611,7 +3613,7 @@ if have_tools subdir('contrib/vhost-user-scsi') endif - if targetos == 'linux' + if targetos == 'linux' and have_system executable('qemu-bridge-helper', files('qemu-bridge-helper.c'), dependencies: [qemuutil, libcap_ng], install: true,
qapi/meson.build +1 -1
@@ -51,12 +51,12 @@ qapi_all_modules = [ 'transaction', 'virtio', 'yank', + 'qdev', ] if have_system qapi_all_modules += [ 'acpi', 'audio', - 'qdev', 'pci', 'rdma', 'rocker',
tests/meson.build +3 -1
@@ -1,5 +1,7 @@ subdir('bench') -subdir('qemu-iotests') +if have_system + subdir('qemu-iotests') +endif test_qapi_outputs = [ 'qapi-builtin-types.c',
tools/meson.build +1 -1
@@ -5,7 +5,7 @@ have_virtiofsd = get_option('virtiofsd') \ error_message: 'virtiofsd requires libcap-ng-devel and seccomp-devel') \ .require(have_vhost_user, error_message: 'virtiofsd needs vhost-user-support') \ - .disable_auto_if(not have_tools and not have_system) \ + .disable_auto_if(not have_system) \ .allowed() if have_virtiofsd
Quoted 35 lines · Helge Deller
meson.build +5
-3@@ -3581,7 +3581,7 @@ subdir('qga') # Don't build qemu-keymap if xkbcommon is not explicitly enabled # when we don't build tools or system -if xkbcommon.found() +if xkbcommon.found() and have_system # used for the update-keymaps target, so include rules even if !have_tools qemu_keymap = executable('qemu-keymap', files('qemu-keymap.c', 'ui/input-keymap.c') + genh, dependencies: [qemuutil, xkbcommon], install: have_tools)@@ -3596,7 +3596,9 @@ if have_tools dependencies: [blockdev, qemuutil, gnutls, selinux], install: true) - subdir('storage-daemon') + if have_system + subdir('storage-daemon') + endif subdir('contrib/rdmacm-mux') subdir('contrib/elf2dmp')@@ -3611,7 +3613,7 @@ if have_tools subdir('contrib/vhost-user-scsi') endif - if targetos == 'linux' + if targetos == 'linux' and have_system executable('qemu-bridge-helper', files('qemu-bridge-helper.c'), dependencies: [qemuutil, libcap_ng], install: true,tools/meson.build +0
-0
tools/meson.build have_virtiofsd = get_option('virtiofsd') \ · line 11
Quoted 8 lines
tools/meson.build +1
-1@@ -5,7 +5,7 @@ have_virtiofsd = get_option('virtiofsd') \ error_message: 'virtiofsd requires libcap-ng-devel and seccomp-devel') \ .require(have_vhost_user, error_message: 'virtiofsd needs vhost-user-support') \ - .disable_auto_if(not have_tools and not have_system) \ + .disable_auto_if(not have_system) \ .allowed() if have_virtiofsd
These are wrong. qemu-bridge-helper, virtiofsd, qemu-storage-daemon and qemu-keymap *are* tools; if they fail to build due to any dependencies, or due to other compilation issues, you need to add tests to meson.build and check for the cause of the issues.
The rest is okay.
Paolo
Hi Paolo,
Quoted 42 lines · Paolo Bonzini
Quoted 41 lines · Helge Deller
meson.build +4
-2@@ -3581,7 +3581,7 @@ subdir('qga') # Don't build qemu-keymap if xkbcommon is not explicitly enabled # when we don't build tools or system -if xkbcommon.found() +if xkbcommon.found() and have_system # used for the update-keymaps target, so include rules even if !have_tools qemu_keymap = executable('qemu-keymap', files('qemu-keymap.c', 'ui/input-keymap.c') + genh, dependencies: [qemuutil, xkbcommon], install: have_tools)@@ -3596,7 +3596,9 @@ if have_tools dependencies: [blockdev, qemuutil, gnutls, selinux], install: true) - subdir('storage-daemon') + if have_system + subdir('storage-daemon') + endif subdir('contrib/rdmacm-mux') subdir('contrib/elf2dmp')@@ -3611,7 +3613,7 @@ if have_tools subdir('contrib/vhost-user-scsi')endif
- if targetos == 'linux'
+ if targetos == 'linux' andhave_systemexecutable('qemu-bridge-helper', files('qemu-bridge-helper.c'),
dependencies: [qemuutil,libcap_ng],
install: true,tools/meson.build +1
-1@@ -5,7 +5,7 @@ have_virtiofsd = get_option('virtiofsd') \ error_message: 'virtiofsd requires libcap-ng-devel and seccomp-devel') \ .require(have_vhost_user, error_message: 'virtiofsd needs vhost-user-support') \ - .disable_auto_if(not have_tools and not have_system) \ + .disable_auto_if(not have_system) \ .allowed()
Quoted 7 lines
Quoted 2 lines
if
have_virtiofsdThese are wrong. qemu-bridge-helper, virtiofsd, qemu-storage-daemon and qemu-keymap *are* tools; if they fail to build due to any dependencies, or due to other compilation issues, you need to add tests to meson.build and check for the cause of the issues.
No doubt, those are *tools*.
But aren't those only required when you run system- and/or user-emulation?
Looking at the packaging of qemu in debian:
qemu-system-common debian package consists of thse *tools*:
qemu-bridge-helper, vhost-user-gpu, virtfs-proxy-helper, virtiofsd
qemu-utils debian package consists of the *utilities*:
qemu-img, qemu-io, qemu-nbd
IMHO this categorization makes sense.
My patch was targetting at making "qemu-utils" possible to build and that's why I think the tools you mention should depend on "have_system".
Quoted 1 line
The rest is okay.
Ok, thanks!
Helge
Quoted 14 lines · Helge Deller
Hi Paolo,
Quoted 4 lines · Paolo Bonzini
These are wrong. qemu-bridge-helper, virtiofsd, qemu-storage-daemon and qemu-keymap *are* tools; if they fail to build due to any dependencies, or due to other compilation issues, you need to add tests to meson.build and check for the cause of the issues.
No doubt, those are *tools*.
But aren't those only required when you run system- and/or user-emulation?Looking at the packaging of qemu in debian:
Quoted 7 lines
qemu-system-common debian package consists of thse *tools*:
qemu-bridge-helper, vhost-user-gpu, virtfs-proxy-helper, virtiofsdqemu-utils debian package consists of the *utilities*:
qemu-img, qemu-io, qemu-nbdIMHO this categorization makes sense.
Possibly, but it's not the categorization we use upstream, which splits our binaries into three groups:
* system-emulation binaries (qemu-system-arm, etc)
* usermode-emulation binaries (qemu-arm, etc)
* tools (everything else)
(I think the guest-agent may be a fourth group.)
thanks
-- PMM
Quoted 22 lines · Helge Deller
Reduce amount of compiled code and installed binaries if just the qemu utilities such as qemu-img, qemu-io and qemu-nbd have to be built. This helps to make those tools easier available on hosts where qemu-system or qemu-user isn't supported.
Reason for this patch is that qemu fail to build on debian for some seconday non-release architectures (e.g. hppa, sh4, ia64), as can be seen here:
https://buildd.debian.org/status/package.php?p=qemu&suite=sid
This patch helps to clean up the build so that these tools can be provided on those platforms in future.Signed-off-by Helge Deller <[email protected]>index
fbdb442fdf..4a499db441100644qapi/meson.build +0
-0@@ -51,12 +51,12 @@ qapi_all_modules = [ 'transaction', 'virtio', 'yank',
qapi/meson.build qapi_all_modules = [ · line 62
Quoted 10 lines
qapi/meson.build +1
-1+ 'qdev', ] if have_system qapi_all_modules += [ 'acpi', 'audio', - 'qdev', 'pci', 'rdma', 'rocker',
Why do you want qdev out of sysemu?
Quoted 18 lines · Peter Maydell
Quoted 17 lines · Helge Deller
Hi Paolo,
Quoted 4 lines · Paolo Bonzini
These are wrong. qemu-bridge-helper, virtiofsd, qemu-storage-daemon and qemu-keymap *are* tools; if they fail to build due to any dependencies, or due to other compilation issues, you need to add tests to meson.build and check for the cause of the issues.
No doubt, those are *tools*.
But aren't those only required when you run system- and/or user-emulation?Looking at the packaging of qemu in debian:
qemu-system-common debian package consists of thse *tools*:
qemu-bridge-helper, vhost-user-gpu, virtfs-proxy-helper, virtiofsd
Quoted 8 lines
Quoted 4 lines
qemu-utils debian package consists of the *utilities*:
qemu-img, qemu-io, qemu-nbdIMHO this categorization makes sense.
Possibly, but it's not the categorization we use upstream, which splits our binaries into three groups:
* system-emulation binaries (qemu-system-arm, etc)
Subcategory:
* helpers (required to use system-emulation binaries), apparently provided by 'qemu-system-common' on Debian -- except virtiofsd which is not a helper --
Quoted 2 lines
* usermode-emulation binaries (qemu-arm, etc)
* tools (everything else)
apparently provided by 'qemu-utils' on Debian (without virtiofsd).
@Debian: Maybe virtiofsd deserves its own qemu-virtiofsd package?
Quoted 2 lines
(I think the guest-agent may be a fourth group.)
(or part of tools?)
Quoted 29 lines · Philippe Mathieu-Daudé
Quoted 18 lines · Peter Maydell
Quoted 17 lines · Helge Deller
Hi Paolo,
Quoted 4 lines · Paolo Bonzini
These are wrong. qemu-bridge-helper, virtiofsd, qemu-storage-daemon and qemu-keymap *are* tools; if they fail to build due to any dependencies, or due to other compilation issues, you need to add tests to meson.build and check for the cause of the issues.
No doubt, those are *tools*.
But aren't those only required when you run system- and/or user-emulation?Looking at the packaging of qemu in debian:
qemu-system-common debian package consists of thse *tools*:
qemu-bridge-helper, vhost-user-gpu, virtfs-proxy-helper, virtiofsdQuoted 8 lines
Quoted 4 lines
qemu-utils debian package consists of the *utilities*:
qemu-img, qemu-io, qemu-nbdIMHO this categorization makes sense.
Possibly, but it's not the categorization we use upstream, which splits our binaries into three groups:
* system-emulation binaries (qemu-system-arm, etc)
Quoted 5 lines
Subcategory:
* helpers (required to use system-emulation binaries), apparently provided by 'qemu-system-common' on Debian -- except virtiofsd which is not a helper --
No, I don't think this is a subcategory. We provide
--enable-system
--enable-user
--enable-tools
but there is no
--enable-helper
and qemu-bridge-helper, etc are built via --enable-tools.
Quoted 6 lines
Quoted 2 lines
* usermode-emulation binaries (qemu-arm, etc)
* tools (everything else)apparently provided by 'qemu-utils' on Debian (without virtiofsd).
Quoted 6 lines
@Debian: Maybe virtiofsd deserves its own qemu-virtiofsd package?
Quoted 2 lines
(I think the guest-agent may be a fourth group.)
(or part of tools?)
No, it looks like we build it via --enable-guest-agent, and --enable-tools is not checked.
thanks
-- PMM
Quoted 27 lines · Philippe Mathieu-Daudé
Quoted 26 lines · Helge Deller
Reduce amount of compiled code and installed binaries if just the qemu utilities such as qemu-img, qemu-io and qemu-nbd have to be built. This helps to make those tools easier available on hosts where qemu-system or qemu-user isn't supported.
Reason for this patch is that qemu fail to build on debian for some seconday non-release architectures (e.g. hppa, sh4, ia64), as can be seen here:
https://buildd.debian.org/status/package.php?p=qemu&suite=sid
This patch helps to clean up the build so that these tools can be provided on those platforms in future.Signed-off-by Helge Deller <[email protected]>index
fbdb442fdf..4a499db441100644qapi/meson.build +1
-0@@ -51,12 +51,12 @@ qapi_all_modules = [ 'transaction', 'virtio', 'yank', + 'qdev',]
ifhave_system
qapi_all_modules+= [
Quoted 8 lines
Quoted 6 lines
'acpi',
'audio',
- 'qdev',
'pci',
'rdma',
'rocker',Why do you want qdev out of sysemu?
I just moved it some lines below sysemu.
Helge
Quoted 53 lines · Helge Deller
Hi Paolo,
Quoted 42 lines · Paolo Bonzini
Quoted 41 lines · Helge Deller
meson.build +4
-2@@ -3581,7 +3581,7 @@ subdir('qga') # Don't build qemu-keymap if xkbcommon is not explicitly enabled # when we don't build tools or system -if xkbcommon.found() +if xkbcommon.found() and have_system # used for the update-keymaps target, so include rules even if !have_tools qemu_keymap = executable('qemu-keymap', files('qemu-keymap.c', 'ui/input-keymap.c') + genh, dependencies: [qemuutil, xkbcommon], install: have_tools)@@ -3596,7 +3596,9 @@ if have_tools dependencies: [blockdev, qemuutil, gnutls, selinux], install: true) - subdir('storage-daemon') + if have_system + subdir('storage-daemon') + endif subdir('contrib/rdmacm-mux') subdir('contrib/elf2dmp')@@ -3611,7 +3613,7 @@ if have_tools subdir('contrib/vhost-user-scsi')endif
- if targetos == 'linux'
+ if targetos == 'linux' andhave_systemexecutable('qemu-bridge-helper', files('qemu-bridge-helper.c'),
dependencies: [qemuutil,libcap_ng],
install: true,tools/meson.build +1
-1@@ -5,7 +5,7 @@ have_virtiofsd = get_option('virtiofsd') \ error_message: 'virtiofsd requires libcap-ng-devel and seccomp-devel') \ .require(have_vhost_user, error_message: 'virtiofsd needs vhost-user-support') \ - .disable_auto_if(not have_tools and not have_system) \ + .disable_auto_if(not have_system) \ .allowed()Quoted 7 lines
Quoted 2 lines
if
have_virtiofsdThese are wrong. qemu-bridge-helper, virtiofsd, qemu-storage-daemon and qemu-keymap *are* tools; if they fail to build due to any dependencies, or due to other compilation issues, you need to add tests to meson.build and check for the cause of the issues.
Quoted 7 lines
No doubt, those are *tools*.
But aren't those only required when you run system- and/or user-emulation?Looking at the packaging of qemu in debian:
qemu-system-common debian package consists of thse *tools*:
qemu-bridge-helper, vhost-user-gpu, virtfs-proxy-helper, virtiofsd
Well technically vhost-user-gpu and virtiofsd are usable (or should be) without QEMU as they are vhost-user backends and could be used with other VMMs that support vhost-user (kvm-tool, crosvm, xen-vhost-frontend).
Not withstanding that virtiofsd is now a legacy binary as the production backend for vhost-user virtio-fs is the rust rewrite:
https://gitlab.com/virtio-fs/virtiofsd
Quoted 13 lines
qemu-utils debian package consists of the *utilities*:
qemu-img, qemu-io, qemu-nbdIMHO this categorization makes sense.
My patch was targetting at making "qemu-utils" possible to build and that's why I think the tools you mention should depend on "have_system".Quoted 1 line
The rest is okay.
Ok, thanks!
Helge
Signature · 2 lines
Alex Bennée Virtualisation Tech Lead @ Linaro
Quoted 44 lines · Peter Maydell
Quoted 29 lines · Philippe Mathieu-Daudé
Quoted 18 lines · Peter Maydell
Quoted 17 lines · Helge Deller
Hi Paolo,
Quoted 4 lines · Paolo Bonzini
These are wrong. qemu-bridge-helper, virtiofsd, qemu-storage-daemon and qemu-keymap *are* tools; if they fail to build due to any dependencies, or due to other compilation issues, you need to add tests to meson.build and check for the cause of the issues.
No doubt, those are *tools*.
But aren't those only required when you run system- and/or user-emulation?Looking at the packaging of qemu in debian:
qemu-system-common debian package consists of thse *tools*:
qemu-bridge-helper, vhost-user-gpu, virtfs-proxy-helper, virtiofsdQuoted 8 lines
Quoted 4 lines
qemu-utils debian package consists of the *utilities*:
qemu-img, qemu-io, qemu-nbdIMHO this categorization makes sense.
Possibly, but it's not the categorization we use upstream, which splits our binaries into three groups:
* system-emulation binaries (qemu-system-arm, etc)Quoted 5 lines
Subcategory:
* helpers (required to use system-emulation binaries), apparently provided by 'qemu-system-common' on Debian -- except virtiofsd which is not a helper --
No, I don't think this is a subcategory. We provide
--enable-system
--enable-user
--enable-tools
but there is no
--enable-helper
and qemu-bridge-helper, etc are built via --enable-tools.
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines
(I think the guest-agent may be a fourth group.)
...No, it looks like we build it via --enable-guest-agent, and --enable-tools is not checked.
Given that info, would it then make sense to keep as is:
--enable-system
--enable-user
--enable-tools -> qemu-bridge-helper, vhost-user-gpu, virtfs-proxy-helper, virtiofsd --enable-guest-agent -> guest agents (option is there already, but not used)
and additonally add:
--enable-utils -> qemu-img, qemu-io, qemu-nbd
This would leave the helpers in --enable-tools.
Helge
20.12.2022 23:56, Helge Deller wrote:
..
Quoted 7 lines
Given that info, would it then make sense to keep as is:
--enable-system
--enable-user
--enable-tools -> qemu-bridge-helper, vhost-user-gpu, virtfs-proxy-helper, virtiofsd --enable-guest-agent -> guest agents (option is there already, but not used)
and additonally add:
--enable-utils -> qemu-img, qemu-io, qemu-nbd
tools and utils is confusing, it smells like it's the same thing.
I understand that in theory, eg virtiofsd can be used without qemu, with some other software, but in practice there's no such software, and I don't think it will emerge in a (near) future. All the "tools" listed above are system-mode helpers really, this is why I package them in qemu-SYSTEM-common in Debian. They're not very useful for now without some of qemu-system-xx. To my view, these should be part of --enable-system, at least for now. Maybe with a separate --enable-system-helpers to turn them off if needed but I don't think it's necessary.
The new "utils" category above actually *is* useful by its own, especially qemu-nbd, for example to access .qcow2 images from the host OS. This is what eg Redhat is shipping in qemu-io package. And these are good to have without --enable-system.
/mjt
Quoted 21 lines · Michael Tokarev
20.12.2022 23:56, Helge Deller wrote:
..Quoted 7 lines
Given that info, would it then make sense to keep as is:
--enable-system
--enable-user
--enable-tools -> qemu-bridge-helper, vhost-user-gpu, virtfs-proxy-helper, virtiofsd --enable-guest-agent -> guest agents (option is there already, but not used) and additonally add:
--enable-utils -> qemu-img, qemu-io, qemu-nbdtools and utils is confusing, it smells like it's the same thing.
I understand that in theory, eg virtiofsd can be used without qemu, with some other software, but in practice there's no such software, and I don't think it will emerge in a (near) future. All the "tools" listed above are system-mode helpers really, this is why I package them in qemu-SYSTEM-common in Debian. They're not very useful for now without some of qemu-system-xx. To my view, these should be part of --enable-system, at least for now. Maybe with a separate --enable-system-helpers to turn them off if needed but I don't think it's necessary.
Quoted 4 lines
The new "utils" category above actually *is* useful by its own, especially qemu-nbd, for example to access .qcow2 images from the host OS. This is what eg Redhat is shipping in qemu-io package. And these are good to have without --enable-system.
Michael, please note, that the patch I initially sent in this thread is *exactly* doing what you summarize here.
Helge