Fix QEMU compilation on Debian 10
cover Fix QEMU compilation on Debian 10
QEMU does not compile on an old Debian 10 system for the following reasons:
- Several sources include recent kernel headers that are not provided by this system, and not listed in linux-headers/
- The libvhost-user.c source file ends up including a system kernel header, instead of the up-to-date standard-headers/ version that contains the right macro definition.
David 'Digit' Turner (2):
Fix libvhost-user.c compilation.
Add missing Linux kernel headers.
linux-headers/linux/const.h | 36 +++++++ linux-headers/linux/memfd.h | 35 +++++++ linux-headers/linux/nvme_ioctl.h | 114 ++++++++++++++++++++++ linux-headers/linux/vfio.h | 15 +-- scripts/update-linux-headers.sh | 4 +- subprojects/libvhost-user/libvhost-user.c | 6 ++ 6 files changed, 202 insertions(+), 8 deletions(-) create mode 100644 linux-headers/linux/const.h create mode 100644 linux-headers/linux/memfd.h create mode 100644 linux-headers/linux/nvme_ioctl.h
Signature · 1 line
2.40.0.348.gf938b09366-goog
PATCH 1/2 Fix libvhost-user.c compilation.
The source file uses VIRTIO_F_VERSION_1 which is not defined by <linux/virtio_config.h> on Debian 10.
The system-provided <linux/virtio_config.h> which does not include the macro definition is included through <linux/vhost.h>, so fix the issue by including the standard-headers version before that.
--- subprojects/libvhost-user/libvhost-user.c | 6 ++++++ 1 file changed, 6 insertions(+)
subprojects/libvhost-user/libvhost-user.c +6 -0
@@ -32,6 +32,12 @@#include <sys/mman.h> #include <endian.h> +/* Necessary to provide VIRTIO_F_VERSION_1 on system +* with older linux headers. Must appear before +* <linux/vhost.h> below. +*/ +#include "standard-headers/linux/virtio_config.h" +#if defined(__linux__)#include <sys/syscall.h> #include <fcntl.h>
Signature · 1 line
2.40.0.348.gf938b09366-goog
Re: [PATCH 1/2] Fix libvhost-user.c compilation.
Quoted 9 lines · David 'Digit' Turner
The source file uses
VIRTIO_F_VERSION_1which is not defined by <linux/virtio_config.h> on Debian 10.The system-provided <
linux/virtio_config.h> which does not include the macro definition is included through <linux/vhost.h>, so fix the issue by including the standard-headers version before that.Signed-off-by David 'Digit' Turner <[email protected]>
I don't get it. ./linux-headers/linux/vhost.h does not seem to use <linux/virtio_config.h> for me.
In fact nothing does:
$ git grep linux/virtio_config.h include/hw/virtio/virtio.h:#include "standard-headers/linux/virtio_config.h" include/standard-headers/linux/vhost_types.h:#include "standard-headers/linux/virtio_config.h" include/standard-headers/linux/virtio_9p.h:#include "standard-headers/linux/virtio_config.h" include/standard-headers/linux/virtio_balloon.h:#include "standard-headers/linux/virtio_config.h" include/standard-headers/linux/virtio_blk.h:#include "standard-headers/linux/virtio_config.h" include/standard-headers/linux/virtio_console.h:#include "standard-headers/linux/virtio_config.h" include/standard-headers/linux/virtio_crypto.h:#include "standard-headers/linux/virtio_config.h" include/standard-headers/linux/virtio_fs.h:#include "standard-headers/linux/virtio_config.h" include/standard-headers/linux/virtio_mem.h:#include "standard-headers/linux/virtio_config.h" include/standard-headers/linux/virtio_net.h:#include "standard-headers/linux/virtio_config.h" include/standard-headers/linux/virtio_pmem.h:#include "standard-headers/linux/virtio_config.h" include/standard-headers/linux/virtio_rng.h:#include "standard-headers/linux/virtio_config.h" include/standard-headers/linux/virtio_vsock.h:#include "standard-headers/linux/virtio_config.h" linux-headers/linux/virtio_config.h:#include "standard-headers/linux/virtio_config.h" scripts/update-linux-headers.sh:cat <<EOF >$output/linux-headers/linux/virtio_config.h scripts/update-linux-headers.sh:#include "standard-headers/linux/virtio_config.h" subprojects/libvduse/libvduse.c:#include "linux-headers/linux/virtio_config.h" tests/qtest/fuzz/virtio_net_fuzz.c:#include "standard-headers/linux/virtio_config.h" tests/qtest/libqos/virtio-gpio.c:#include "standard-headers/linux/virtio_config.h" tests/qtest/libqos/virtio-pci-modern.c:#include "standard-headers/linux/virtio_config.h" tests/qtest/libqos/virtio.c:#include "standard-headers/linux/virtio_config.h"
Quoted 23 lines
--- subprojects/libvhost-user/libvhost-user.c | 6 ++++++ 1 file changed, 6 insertions(+)subprojects/libvhost-user/libvhost-user.c +6
-0@@ -32,6 +32,12 @@#include <sys/mman.h> #include <endian.h> +/* Necessary to provide VIRTIO_F_VERSION_1 on system +* with older linux headers. Must appear before +* <linux/vhost.h> below. +*/ +#include "standard-headers/linux/virtio_config.h" +#if defined(__linux__)#include <sys/syscall.h> #include <fcntl.h> Signature · 1 line
2.40.0.348.gf938b09366-goog
Re: [PATCH 1/2] Fix libvhost-user.c compilation.
Quoted 9 lines · Michael S. Tsirkin
Quoted 8 lines · David 'Digit' Turner
The source file uses
VIRTIO_F_VERSION_1which is not defined by <linux/virtio_config.h> on Debian 10.The system-provided <
linux/virtio_config.h> which does not include the macro definition is included through <linux/vhost.h>, so fix the issue by including the standard-headers version before that.
Quoted 4 lines
Quoted 1 line
Signed-off-by David 'Digit' Turner <[email protected]>I don't get it.
./linux-headers/linux/vhost.hdoes not seem to use <linux/virtio_config.h> for me.
The issue is that subprojects/libvhost-user/libvhost-user.c includes linux/vhost.h. Probably should be changed to linux-headers/linux/vhost.h, but David's patch makes sense because libvhost-user.c does use a symbol from virtio_config.h.
Paolo
Quoted 51 lines
$ git grep linux/virtio_config.h include/hw/virtio/virtio.h:#include "standard-headers/linux/virtio_config.h" include/standard-headers/linux/vhost_types.h:#include "standard-headers/linux/virtio_config.h" include/standard-headers/linux/virtio_9p.h:#include "standard-headers/linux/virtio_config.h" include/standard-headers/linux/virtio_balloon.h:#include "standard-headers/linux/virtio_config.h" include/standard-headers/linux/virtio_blk.h:#include "standard-headers/linux/virtio_config.h" include/standard-headers/linux/virtio_console.h:#include "standard-headers/linux/virtio_config.h" include/standard-headers/linux/virtio_crypto.h:#include "standard-headers/linux/virtio_config.h" include/standard-headers/linux/virtio_fs.h:#include "standard-headers/linux/virtio_config.h" include/standard-headers/linux/virtio_mem.h:#include "standard-headers/linux/virtio_config.h" include/standard-headers/linux/virtio_net.h:#include "standard-headers/linux/virtio_config.h" include/standard-headers/linux/virtio_pmem.h:#include "standard-headers/linux/virtio_config.h" include/standard-headers/linux/virtio_rng.h:#include "standard-headers/linux/virtio_config.h" include/standard-headers/linux/virtio_vsock.h:#include "standard-headers/linux/virtio_config.h" linux-headers/linux/virtio_config.h:#include "standard-headers/linux/virtio_config.h" scripts/update-linux-headers.sh:cat <<EOF >$output/linux-headers/linux/virtio_config.h scripts/update-linux-headers.sh:#include "standard-headers/linux/virtio_config.h" subprojects/libvduse/libvduse.c:#include "linux-headers/linux/virtio_config.h" tests/qtest/fuzz/virtio_net_fuzz.c:#include "standard-headers/linux/virtio_config.h" tests/qtest/libqos/virtio-gpio.c:#include "standard-headers/linux/virtio_config.h" tests/qtest/libqos/virtio-pci-modern.c:#include "standard-headers/linux/virtio_config.h" tests/qtest/libqos/virtio.c:#include "standard-headers/linux/virtio_config.h"Quoted 23 lines
--- subprojects/libvhost-user/libvhost-user.c | 6 ++++++ 1 file changed, 6 insertions(+)subprojects/libvhost-user/libvhost-user.c +6
-0@@ -32,6 +32,12 @@#include <sys/mman.h> #include <endian.h> +/* Necessary to provide VIRTIO_F_VERSION_1 on system +* with older linux headers. Must appear before +* <linux/vhost.h> below. +*/ +#include "standard-headers/linux/virtio_config.h" +#if defined(__linux__)#include <sys/syscall.h> #include <fcntl.h> Signature · 1 line
2.40.0.348.gf938b09366-goog
PATCH 2/2 Add missing Linux kernel headers.
In reply to David 'Digit' TurnerAdd <linux/memfd.h>, used by hw/display/virtio-gpu-udmabuf.c Add <linux/nvme_ioctl.h>, used by qga/commands-posix.c Add <linux/const.h> used by kvm-all.c, which requires the _BITUL() macro definition to be available.
Without these, QEMU will not compile on Debian 10 systems.
The script has then been run against the official 6.2.8 kernel source tree (current stable release), which explains why comments in <linux/vfio.h> have been updated too.
--- linux-headers/linux/const.h | 36 ++++++++++ linux-headers/linux/memfd.h | 35 ++++++++++ linux-headers/linux/nvme_ioctl.h | 114 +++++++++++++++++++++++++++++++ linux-headers/linux/vfio.h | 15 ++-- scripts/update-linux-headers.sh | 4 +- 5 files changed, 196 insertions(+), 8 deletions(-) create mode 100644 linux-headers/linux/const.h create mode 100644 linux-headers/linux/memfd.h create mode 100644 linux-headers/linux/nvme_ioctl.h
linux-headers/linux/const.h new file +36 -0
@@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* const.h: Macros for dealing with constants. */ + +#ifndef _LINUX_CONST_H +#define _LINUX_CONST_H + +/* Some constant macros are used in both assembler and +* C code. Therefore we cannot annotate them always with +* 'UL' and other type specifiers unilaterally. We +* use the following macros to deal with this. +* +* Similarly, _AT() will cast an expression with a type in C, but +* leave it unchanged in asm. +*/ + +#ifdef __ASSEMBLY__ +#define _AC (X ,Y ) X +#define _AT (T ,X ) X +#else +#define __AC (X ,Y ) (X##Y) +#define _AC (X ,Y ) __AC(X,Y) +#define _AT (T ,X ) ((T)(X)) +#endif + +#define _UL (x) (_AC(x, UL)) +#define _ULL (x) (_AC(x, ULL)) + +#define _BITUL (x) (_UL(1) << (x)) +#define _BITULL (x) (_ULL(1) << (x)) + +#define __ALIGN_KERNEL (x, a) __ALIGN_KERNEL_MASK(x, (typeof(x))(a) - 1) +#define __ALIGN_KERNEL_MASK (x, mask) (((x) + (mask)) & ~(mask)) + +#define __KERNEL_DIV_ROUND_UP (n, d) (((n) + (d) - 1) / (d)) + +#endif /* _LINUX_CONST_H */
linux-headers/linux/memfd.h new file +35 -0
@@ -0,0 +1,35 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +#ifndef _LINUX_MEMFD_H +#define _LINUX_MEMFD_H + +#include <asm-generic/hugetlb_encode.h> + +/* flags for memfd_create(2) (unsigned int) */ +#define MFD_CLOEXEC 0x0001U +#define MFD_ALLOW_SEALING 0x0002U +#define MFD_HUGETLB 0x0004U + +/* +* Huge page size encoding when MFD_HUGETLB is specified, and a huge page +* size other than the default is desired. See hugetlb_encode.h. +* All known huge page size encodings are provided here. It is the +* responsibility of the application to know which sizes are supported on +* the running system. See mmap(2) man page for details. +*/ +#define MFD_HUGE_SHIFT HUGETLB_FLAG_ENCODE_SHIFT +#define MFD_HUGE_MASK HUGETLB_FLAG_ENCODE_MASK + +#define MFD_HUGE_64KB HUGETLB_FLAG_ENCODE_64KB +#define MFD_HUGE_512KB HUGETLB_FLAG_ENCODE_512KB +#define MFD_HUGE_1MB HUGETLB_FLAG_ENCODE_1MB +#define MFD_HUGE_2MB HUGETLB_FLAG_ENCODE_2MB +#define MFD_HUGE_8MB HUGETLB_FLAG_ENCODE_8MB +#define MFD_HUGE_16MB HUGETLB_FLAG_ENCODE_16MB +#define MFD_HUGE_32MB HUGETLB_FLAG_ENCODE_32MB +#define MFD_HUGE_256MB HUGETLB_FLAG_ENCODE_256MB +#define MFD_HUGE_512MB HUGETLB_FLAG_ENCODE_512MB +#define MFD_HUGE_1GB HUGETLB_FLAG_ENCODE_1GB +#define MFD_HUGE_2GB HUGETLB_FLAG_ENCODE_2GB +#define MFD_HUGE_16GB HUGETLB_FLAG_ENCODE_16GB + +#endif /* _LINUX_MEMFD_H */
linux-headers/linux/nvme_ioctl.h new file +114 -0
114 lines not on this page · raw message
linux-headers/linux/vfio.h +9 -6
@@ -49,7 +49,11 @@/* Supports VFIO_DMA_UNMAP_FLAG_ALL */ #define VFIO_UNMAP_ALL 9 -/* Supports the vaddr flag for DMA map and unmap */ +/* +* Supports the vaddr flag for DMA map and unmap. Not supported for mediated +* devices, so this capability is subject to change as groups are added or +* removed. +*/ #define VFIO_UPDATE_VADDR 10/*
@@ -1343,8 +1347,7 @@ struct vfio_iommu_type1_info_dma_avail {* Map process virtual addresses to IO virtual addresses using the * provided struct vfio_dma_map. Caller sets argsz. READ &/ WRITE required. * -* If flags & VFIO_DMA_MAP_FLAG_VADDR, update the base vaddr for iova, and -* unblock translation of host virtual addresses in the iova range. The vaddr +* If flags & VFIO_DMA_MAP_FLAG_VADDR, update the base vaddr for iova. The vaddr * must have previously been invalidated with VFIO_DMA_UNMAP_FLAG_VADDR. To * maintain memory consistency within the user application, the updated vaddr * must address the same memory object as originally mapped. Failure to do so
@@ -1395,9 +1398,9 @@ struct vfio_bitmap {* must be 0. This cannot be combined with the get-dirty-bitmap flag. * * If flags & VFIO_DMA_UNMAP_FLAG_VADDR, do not unmap, but invalidate host -* virtual addresses in the iova range. Tasks that attempt to translate an -* iova's vaddr will block. DMA to already-mapped pages continues. This -* cannot be combined with the get-dirty-bitmap flag. +* virtual addresses in the iova range. DMA to already-mapped pages continues. +* Groups may not be added to the container while any addresses are invalid. +* This cannot be combined with the get-dirty-bitmap flag. */ struct vfio_iommu_type1_dma_unmap {__u32 argsz;
scripts/update-linux-headers.sh +2 -2
@@ -160,8 +160,8 @@ done rm -rf "$output/linux-headers/linux" mkdir -p "$output/linux-headers/linux" -for header in kvm.h vfio.h vfio_ccw.h vfio_zdev.h vhost.h \ - psci.h psp-sev.h userfaultfd.h mman.h vduse.h; do +for header in const.h kvm.h vfio.h vfio_ccw.h vfio_zdev.h vhost.h \ + psci.h psp-sev.h userfaultfd.h memfd.h mman.h nvme_ioctl.h vduse.h; do cp "$tmpdir/include/linux/$header" "$output/linux-headers/linux" done
Signature · 1 line
2.40.0.348.gf938b09366-goog
Re: [PATCH 2/2] Add missing Linux kernel headers.
Quoted 6 lines · David 'Digit' Turner
Add <
linux/memfd.h>, used byhw/display/virtio-gpu-udmabuf.cAdd <linux/nvme_ioctl.h>, used byqga/commands-posix.cAdd <linux/const.h> used bykvm-all.c, which requires the_BITUL()macro definition to be available.Without these, QEMU will not compile on Debian 10 systems.
Hm, I wonder why we hadn't seen errors in the CI then.
Quoted 5 lines
The script has then been run against the official 6.2.8 kernel source tree (current stable release), which explains why comments in <
linux/vfio.h> have been updated too.
I think we usually run the script against a release or release candidate, not stable.
Quoted 12 lines
Signed-off-by David 'Digit' Turner <[email protected]>--- linux-headers/linux/const.h | 36 ++++++++++ linux-headers/linux/memfd.h | 35 ++++++++++ linux-headers/linux/nvme_ioctl.h | 114 +++++++++++++++++++++++++++++++ linux-headers/linux/vfio.h | 15 ++-- scripts/update-linux-headers.sh | 4 +- 5 files changed, 196 insertions(+), 8 deletions(-) create mode 100644 linux-headers/linux/const.h create mode 100644 linux-headers/linux/memfd.h create mode 100644 linux-headers/linux/nvme_ioctl.h
Re: [PATCH 2/2] Add missing Linux kernel headers.
Quoted 11 lines · Cornelia Huck
Quoted 6 lines · David 'Digit' Turner
Add <
linux/memfd.h>, used byhw/display/virtio-gpu-udmabuf.cAdd <linux/nvme_ioctl.h>, used byqga/commands-posix.cAdd <linux/const.h> used bykvm-all.c, which requires the_BITUL()macro definition to be available.Without these, QEMU will not compile on Debian 10 systems.
Hm, I wonder why we hadn't seen errors in the CI then.
No idea either, but fwiw, I am building on a docker.io/debian:10-slim container.
Quoted 10 lines
Quoted 5 lines
The script has then been run against the official 6.2.8 kernel source tree (current stable release), which explains why comments in <
linux/vfio.h> have been updated too.I think we usually run the script against a release or release candidate, not stable.
I meant that this was run against the headers of the 6.2.8 official
release, which was listed as "stable" on https://kernel.org/ (that page now lists the 6.2.9 release btw)
I'd be happy to re-run it against a different set if you can tell me which one (and where to get it, just in case).
Quoted 14 lines
Quoted 12 lines
Signed-off-by David 'Digit' Turner <[email protected]>--- linux-headers/linux/const.h | 36 ++++++++++ linux-headers/linux/memfd.h | 35 ++++++++++ linux-headers/linux/nvme_ioctl.h | 114 +++++++++++++++++++++++++++++++ linux-headers/linux/vfio.h | 15 ++-- scripts/update-linux-headers.sh | 4 +- 5 files changed, 196 insertions(+), 8 deletions(-) create mode 100644 linux-headers/linux/const.h create mode 100644 linux-headers/linux/memfd.h create mode 100644 linux-headers/linux/nvme_ioctl.h
Re: [PATCH 2/2] Add missing Linux kernel headers.
Quoted 8 lines · David Turner
Quoted 6 lines · Cornelia Huck
Quoted 4 lines · David 'Digit' Turner
The script has then been run against the official 6.2.8 kernel source tree (current stable release), which explains why comments in <
linux/vfio.h> have been updated too.
Quoted 8 lines
Quoted 4 lines
I think we usually run the script against a release or release candidate, not stable.
I meant that this was run against the headers of the 6.2.8 official
release, which was listed as "stable" on https://kernel.org/ (that page now lists the 6.2.9 release btw)
I'd be happy to re-run it against a different set if you can tell me which one (and where to get it, just in case).
I think most people actually run it against a checkout of Linus' git tree, preferrably either the latest -rc version (or the latest release during the kernel merge window) -- people usually run the script because they want to use some new interfaces that were recently introduced to the kernel. (This also ensures linear history, although I don't think that's too much of a problem.)
Just make sure you note the exact version (or git hash), then it's fine. Might also be worth splitting this into adding the new files to the script and actually updating the headers.
Re: [PATCH 2/2] Add missing Linux kernel headers.
Quoted 20 lines · Cornelia Huck
Quoted 8 lines · David Turner
Quoted 6 lines · Cornelia Huck
Quoted 4 lines · David 'Digit' Turner
The script has then been run against the official 6.2.8 kernel source tree (current stable release), which explains why comments in <
linux/vfio.h> have been updated too.Quoted 8 lines
Quoted 4 lines
I think we usually run the script against a release or release candidate, not stable.
I meant that this was run against the headers of the 6.2.8 official
release, which was listed as "stable" on https://kernel.org/ (that page now lists the 6.2.9 release btw)
I'd be happy to re-run it against a different set if you can tell me which one (and where to get it, just in case).
Quoted 6 lines
I think most people actually run it against a checkout of Linus' git tree, preferrably either the latest -rc version (or the latest release during the kernel merge window) -- people usually run the script because they want to use some new interfaces that were recently introduced to the kernel. (This also ensures linear history, although I don't think that's too much of a problem.)
Yeah, I think the requirement is just "it has to be against some commit that is on the mainline of the upstream kernel", it doesn't inherently have to be an rc or a full point release. The assumption we're making here is that ABI is stable once a change hits Linus' git tree, and not stable before that.
The other requirement is "don't go backwards", ie don't sync to a commit that pre-dates whatever the last commit we synced to is.
The last sync we did was to ceaa837f96ad ("Linux 6.2-rc8").
thanks
-- PMM
Re: [PATCH 2/2] Add missing Linux kernel headers.
Quoted 37 lines · Peter Maydell
Quoted 4 lines · Cornelia Huck
Quoted 1 line · David Turner
On Wed, Apr 5, 2023 at 3:06 PM Cornelia Huck <[email protected]>
Quoted 7 lines · wrote:
Quoted 7 lines
Quoted 7 lines · David 'Digit' Turner
Quoted 4 lines
The script has then been run against the official 6.2.8 kernel source tree (current stable release), which explains why comments in <
linux/vfio.h> have been updated too.Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
I think we usually run the script against a release or release candidate, not stable.
I meant that this was run against the headers of the 6.2.8 official
release, which was listed as "stable" on https://kernel.org/ (that
page now
Quoted 2 lines
Quoted 2 lines
lists the 6.2.9 release btw)
I'd be happy to re-run it against a different set if you can tell mewhich
Quoted 8 lines
Quoted 1 line
one (and where to get it, just in case).
I think most people actually run it against a checkout of Linus' git tree, preferrably either the latest -rc version (or the latest release during the kernel merge window) -- people usually run the script because they want to use some new interfaces that were recently introduced to the kernel. (This also ensures linear history, although I don't think that's too much of a problem.)
Yeah, I think the requirement is just "it has to be against some commit that is on the mainline of the upstream kernel", it doesn't inherently have to be an rc or a full point release. The assumption we're making here is that ABI is stable once a change hits Linus' git tree, and not stable before that.
Quoted 6 lines
The other requirement is "don't go backwards", ie don't sync to a commit that pre-dates whatever the last commit we synced to is.
The last sync we did was to
ceaa837f96ad("Linux 6.2-rc8").Thank you, that makes perfect sense, I have sent another series of patches
(with the headers updade as a separate patch for consistency)
thanks
Quoted 2 lines
-- PMM
Paolo Bonzini
AppliedQueued, thanks.
Paolo