Bring in all the Linux headers we depend on in QEMU

RFC In review Anthony Liguori 9 participants 25 messages May 3, 2009 – May 4, 2009

Subsystems (2)

KVM Kbuild

lore

Share & tools

lore

Anthony Liguori

author

2009-05-03 21:37 UTC #0 lore raw

RFC 1/1 Bring in all the Linux headers we depend on in QEMU

Sorry this explanation is long winded, but this is a messy situation.

In Linux, there isn't a very consistent policy about userspace kernel header inclusion. On a typical Linux system, you're likely to find kernel headers in three places.

glibc headers (/usr/include/{linux,asm})

These headers are installed by glibc. They very often are based on much older kernel versions that the kernel you have in your distribution. For software that depends on these headers, very often this means that your software detects features being missing that are present on your kernel. Furthermore, glibc only installs the headers it needs so very often certain headers have dependencies that aren't met. A classic example is linux/compiler.h and the broken usbdevice_fs.h header that depends on it. There are still distributions today that QEMU doesn't compile on because of this.

Today, most of QEMU's code depends on these headers.

/lib/modules/$(uname -r)/build

These are the kernel headers that are installed as part of your kernel. In general, this is a pretty good place to find the headers that are associated with the kernel version you're actually running on. However, these headers are part of the kernel build tree and are not always guaranteed to be includable from userspace.

Developers, in particular, like to point things at their random kernel trees. In general though, relying on a full kernel source tree being available isn't a good idea. Kernel headers change dramatically across versions too so it's very likely that we would need to have a lot of #ifdefs dependent on kernel versions, or some of the uglier work arounds we have in usb-linux.c.

I think the best way to avoid #ifdefs and dependencies on broken/incomplete glibc headers is to include all of the Linux headers we need within QEMU. The attached patch does just this.

I think there's room for discussion about whether we really want to do this. We could potentially depend on some more common glibc headers (like asm/types.h) while bringing in less reliable headers (if_tun.h/virtio*). Including them all seems like the most robust solution to me though.

Comments?

Regards,

Anthony Liguori

Quoted 1 line

From b42aa57e94fc6b05897271b0816fa34d70670c9a Mon Sep 17 00:00:00 2001

From: Anthony Liguori <[email protected]>
Date: Sat, 2 May 2009 15:45:24 -0500
Subject: [PATCH] Import Linux headers into QEMU

These are the headers from Linux 2.6.29 that have been modified to work under QEMU. This includes the necessary scripts to generate the headers from the original Linux source tree.

I've not included the headers in this patch as they are quite big and would make review more difficult. I've included headers for all host architectures QEMU currently supports but I've only tested x86.

Signed-off-by Anthony Liguori <[email protected]>

---
 Makefile.target |    3 --
 configure       |   37 ++++++++++---------
 linux/Makefile  |  108 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 linux/README    |   19 ++++++++++
 linux/fixup.sed |   19 ++++++++++
 usb-linux.c     |    1 -
 6 files changed, 165 insertions(+), 22 deletions(-)
 create mode 100644 linux/Makefile
 create mode 100644 linux/README
 create mode 100644 linux/fixup.sed

Makefile.target +0 -3

index f735105..474245a 100644
@@ -124,9 +124,6 @@ CFLAGS+=-I/opt/SUNWspro/prod/include/cc
 endif
 endif
 
-kvm.o: CFLAGS+=$(KVM_CFLAGS)
-kvm-all.o: CFLAGS+=$(KVM_CFLAGS)
-
 all: $(PROGS)
 
 #########################################################

configure +19 ~~-18~~

index 82fb60a..379a2a6 100755
@@ -1081,6 +1081,23 @@ EOF
   fi
 fi
 
+# Linux kernel headers CFLAGS
+if test -z "$kerneldir" ; then
+    linux_cflags="-I$source_path/linux"
+else
+    linux_cflags="-I$kerneldir/include"
+    if test \( "$cpu" = "i386" -o "$cpu" = "x86_64" \) \
+	-a -d "$kerneldir/arch/x86/include" ; then
+	linux_cflags="$linux_cflags -I$kerneldir/arch/x86/include"
+    elif test "$cpu" = "ppc" -a -d "$kerneldir/arch/powerpc/include" ; then
+	linux_cflags="$linux_cflags -I$kerneldir/arch/powerpc/include"
+    elif test -d "$kerneldir/arch/$cpu/include" ; then
+	linux_cflags="$linux_cflags -I$kerneldir/arch/$cpu/include"
+    fi
+fi
+
+OS_CFLAGS="$OS_CFLAGS $linux_cflags"
+
 ##########################################
 # kvm probe
 if test "$kvm" = "yes" ; then

@@ -1100,27 +1117,14 @@ if test "$kvm" = "yes" ; then
 #endif
 int main(void) { return 0; }
 EOF
-  if test "$kerneldir" != "" ; then
-      kvm_cflags=-I"$kerneldir"/include
-      if test \( "$cpu" = "i386" -o "$cpu" = "x86_64" \) \
-         -a -d "$kerneldir/arch/x86/include" ; then
-            kvm_cflags="$kvm_cflags -I$kerneldir/arch/x86/include"
-	elif test "$cpu" = "ppc" -a -d "$kerneldir/arch/powerpc/include" ; then
-	    kvm_cflags="$kvm_cflags -I$kerneldir/arch/powerpc/include"
-        elif test -d "$kerneldir/arch/$cpu/include" ; then
-            kvm_cflags="$kvm_cflags -I$kerneldir/arch/$cpu/include"
-      fi
-  else
-      kvm_cflags=""
-  fi
-  if $cc $ARCH_CFLAGS -o $TMPE ${OS_CFLAGS} $kvm_cflags $TMPC \
+  if $cc $ARCH_CFLAGS -o $TMPE ${OS_CFLAGS} $TMPC \
       > /dev/null 2>/dev/null ; then
     :
   else
     kvm="no";
     if [ -x "`which awk 2>/dev/null`" ] && \
        [ -x "`which grep 2>/dev/null`" ]; then
-      kvmerr=`LANG=C $cc $ARCH_CFLAGS -o $TMPE ${OS_CFLAGS} $kvm_cflags $TMPC 2>&1 \
+      kvmerr=`LANG=C $cc $ARCH_CFLAGS -o $TMPE ${OS_CFLAGS} $TMPC 2>&1 \
 	| grep "error: " \
 	| awk -F "error: " '{if (NR>1) printf(", "); printf("%s",$2);}'`
       if test "$kvmerr" != "" ; then

@@ -1817,7 +1821,6 @@ case "$target_cpu" in
     fi
     if test "$kvm" = "yes" ; then
       echo "CONFIG_KVM=yes" >> $config_mak
-      echo "KVM_CFLAGS=$kvm_cflags" >> $config_mak
       echo "#define CONFIG_KVM 1" >> $config_h
     fi
     if test "$xen" = "yes" -a "$target_softmmu" = "yes";

@@ -1838,7 +1841,6 @@ case "$target_cpu" in
     fi
     if test "$kvm" = "yes" ; then
       echo "CONFIG_KVM=yes" >> $config_mak
-      echo "KVM_CFLAGS=$kvm_cflags" >> $config_mak
       echo "#define CONFIG_KVM 1" >> $config_h
     fi
     if test "$xen" = "yes" -a "$target_softmmu" = "yes"

@@ -1906,7 +1908,6 @@ case "$target_cpu" in
     echo "#define TARGET_PPCEMB 1" >> $config_h
     if test "$kvm" = "yes" ; then
       echo "CONFIG_KVM=yes" >> $config_mak
-      echo "KVM_CFLAGS=$kvm_cflags" >> $config_mak
       echo "#define CONFIG_KVM 1" >> $config_h
     fi
     gdb_xml_files="power-core.xml power-fpu.xml power-altivec.xml power-spe.xml"

linux/Makefile new file +108 -0

108 lines not on this page · raw message

linux/README new file +19 -0

new file mode 100644
index 0000000..e29d39b
@@ -0,0 +1,19 @@
+This directory contains a snapshot of Linux kernel headers.  The included
+Makefile was used to import these headers into QEMU.
+
+This process is intended for use by the QEMU maintainers.  It is not intended
+to work against arbitrary kernel versions nor is the intention for end-users to
+use this to update the included headers.
+
+If a contributor needs a newer kernel header for a feature implemented
+by a particular patch, please run the appropriate make sync-* command and
+send the results of that command as a separate patch.  Also make sure to update
+this file to reflect the kernel version used.
+
+While it is okay for development versions of QEMU to contain headers from
+non-release versions of the Linux kernel, any official release of QEMU must use
+headers from a released version of the Linux kernel to ensure ABI compatibility.
+
+Kernel versions used:
+
+git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git v2.6.30-rc4

linux/fixup.sed new file +19 -0

new file mode 100644
index 0000000..c0ad124
@@ -0,0 +1,19 @@
+# Header file fix ups.  This is tailored for importing the kvm linux headers
+
+# Expand asm/ includes to avoid having to do symlink trickery
+s:^#include <asm/\(.*\)>:                              \
+#if defined(__x86_64__) || defined(__i386__)            \
+#include "asm-x86/\1"                                   \
+#elif defined(__powerpc__)                              \
+#include "asm-powerpc/\1"                               \
+#elif defined(__sparc__)                                \
+#include "asm-sparc/\1"                                 \
+#elif defined(__hppa__)                                 \
+#include "asm-parisc/\1"                                \
+#elif defined(__arm__)                                  \
+#include "asm-arm/\1"                                   \
+#else                                                   \
+#error Using Linux headers on unknown architecture      \
+#endif                                                  \
+:g
+s:^#include <linux/\(.*\)>:#include "linux/\1":g

usb-linux.c +0 -1

index 70d7a1c..1392579 100644
@@ -39,7 +39,6 @@
 #include <signal.h>
 
 #include <linux/usbdevice_fs.h>
-#include <linux/version.h>
 #include "hw/usb.h"
 
 /* We redefine it to avoid version problems */

Signature · 1 line

1.6.0.6

Stefan Weil

2009-05-04 06:51 UTC #1 lore raw

Quoted 47 lines · Anthony Liguori

Sorry this explanation is long winded, but this is a messy situation.
In Linux, there isn't a very consistent policy about userspace kernel header inclusion. On a typical Linux system, you're likely to find kernel headers in three places.
glibc headers (/usr/include/{linux,asm})
These headers are installed by glibc. They very often are based on much older kernel versions that the kernel you have in your distribution. For software that depends on these headers, very often this means that your software detects features being missing that are present on your kernel. Furthermore, glibc only installs the headers it needs so very often certain headers have dependencies that aren't met. A classic example is linux/compiler.h and the broken usbdevice_fs.h header that depends on it. There are still distributions today that QEMU doesn't compile on because of this.
Today, most of QEMU's code depends on these headers.
/lib/modules/$(uname -r)/build
These are the kernel headers that are installed as part of your kernel. In general, this is a pretty good place to find the headers that are associated with the kernel version you're actually running on. However, these headers are part of the kernel build tree and are not always guaranteed to be includable from userspace.
<random kernel tree>
Developers, in particular, like to point things at their random kernel trees. In general though, relying on a full kernel source tree being available isn't a good idea. Kernel headers change dramatically across versions too so it's very likely that we would need to have a lot of #ifdefs dependent on kernel versions, or some of the uglier work arounds we have in usb-linux.c.
I think the best way to avoid #ifdefs and dependencies on broken/incomplete glibc headers is to include all of the Linux headers we need within QEMU. The attached patch does just this.
I think there's room for discussion about whether we really want to do this. We could potentially depend on some more common glibc headers (like asm/types.h) while bringing in less reliable headers (if_tun.h/virtio*). Including them all seems like the most robust solution to me though.

Quoted 5 lines

Comments?
Regards,
Anthony Liguori

For Debian systems, those headers are installed by package linux-libc-dev. There are also packages for cross compilation in emdebian (linux-libc-dev-mips-cross, linux-libc-dev-powerpc-cross, ...).

Yes, those headers did not always match the features of the current kernel, so --enable-kvm did not work. This is fixed now - there is a linux-libc-dev 2.6.29-3 which is up-to-date.

So, at the moment I see no need to fill the QEMU source tree with linux header files.

For special needs the configure option (--kerneldir=PATH) should be sufficient.

Regards

Stefan Weil

Avi Kivity

2009-05-04 07:53 UTC #10 lore raw

In reply to Anthony Liguori

Quoted 6 lines · Anthony Liguori

Sorry this explanation is long winded, but this is a messy situation.
In Linux, there isn't a very consistent policy about userspace kernel header inclusion. On a typical Linux system, you're likely to find kernel headers in three places.
glibc headers (/usr/include/{linux,asm})

Quoted 8 lines

These headers are installed by glibc. They very often are based on much older kernel versions that the kernel you have in your distribution. For software that depends on these headers, very often this means that your software detects features being missing that are present on your kernel. Furthermore, glibc only installs the headers it needs so very often certain headers have dependencies that aren't met. A classic example is linux/compiler.h and the broken usbdevice_fs.h header that depends on it. There are still distributions today that QEMU doesn't compile on because of this.
Today, most of QEMU's code depends on these headers.
/lib/modules/$(uname -r)/build
These are the kernel headers that are installed as part of your kernel. In general, this is a pretty good place to find the headers that are associated with the kernel version you're actually running on. However, these headers are part of the kernel build tree and are not always guaranteed to be includable from userspace.

I thought these were for external modules, not for userspace.

Quoted 9 lines

<random kernel tree>
Developers, in particular, like to point things at their random kernel trees. In general though, relying on a full kernel source tree being available isn't a good idea. Kernel headers change dramatically across versions too so it's very likely that we would need to have a lot of #ifdefs dependent on kernel versions, or some of the uglier work arounds we have in usb-linux.c.
I think the best way to avoid #ifdefs and dependencies on broken/incomplete glibc headers is to include all of the Linux headers we need within QEMU. The attached patch does just this.
I think there's room for discussion about whether we really want to do this. We could potentially depend on some more common glibc headers (like asm/types.h) while bringing in less reliable headers (if_tun.h/virtio*). Including them all seems like the most robust solution to me though.
Comments?

I think we need to use the output of 'make headers-install', which removes things like __user and CONFIG_*.

Signature · 1 line

Do not meddle in the internals of kernels, for they are subtle and quick to panic.

Edgar E. Iglesias

2009-05-04 08:18 UTC #2 lore raw

In reply to Stefan Weil

Quoted 58 lines · Stefan Weil

Quoted 47 lines · Anthony Liguori
Sorry this explanation is long winded, but this is a messy situation.
In Linux, there isn't a very consistent policy about userspace kernel header inclusion. On a typical Linux system, you're likely to find kernel headers in three places.
glibc headers (/usr/include/{linux,asm})
These headers are installed by glibc. They very often are based on much older kernel versions that the kernel you have in your distribution. For software that depends on these headers, very often this means that your software detects features being missing that are present on your kernel. Furthermore, glibc only installs the headers it needs so very often certain headers have dependencies that aren't met. A classic example is linux/compiler.h and the broken usbdevice_fs.h header that depends on it. There are still distributions today that QEMU doesn't compile on because of this.
Today, most of QEMU's code depends on these headers.
/lib/modules/$(uname -r)/build
These are the kernel headers that are installed as part of your kernel. In general, this is a pretty good place to find the headers that are associated with the kernel version you're actually running on. However, these headers are part of the kernel build tree and are not always guaranteed to be includable from userspace.
<random kernel tree>
Developers, in particular, like to point things at their random kernel trees. In general though, relying on a full kernel source tree being available isn't a good idea. Kernel headers change dramatically across versions too so it's very likely that we would need to have a lot of #ifdefs dependent on kernel versions, or some of the uglier work arounds we have in usb-linux.c.
I think the best way to avoid #ifdefs and dependencies on broken/incomplete glibc headers is to include all of the Linux headers we need within QEMU. The attached patch does just this.
I think there's room for discussion about whether we really want to do this. We could potentially depend on some more common glibc headers (like asm/types.h) while bringing in less reliable headers (if_tun.h/virtio*). Including them all seems like the most robust solution to me though.
Quoted 5 lines
Comments?
Regards,
Anthony Liguori
For Debian systems, those headers are installed by package linux-libc-dev. There are also packages for cross compilation in emdebian (linux-libc-dev-mips-cross, linux-libc-dev-powerpc-cross, ...).

Quoted 6 lines

Yes, those headers did not always match the features of the current kernel, so --enable-kvm did not work. This is fixed now - there is a linux-libc-dev 2.6.29-3 which is up-to-date.
So, at the moment I see no need to fill the QEMU source tree with linux header files.

I agree. The kvm issue seems unfortunate and I don't have any suggestions on how to avoid it in the future but for other issues, like restructured header files or renamed struct members etc I think there is a risk we become sloppy in keeping up to date with current practices.

I don't feel very strongly about it but my gut feeling tells me we shouldn't be doing this.

Cheers

Avi Kivity

2009-05-04 09:09 UTC #13 lore raw

In reply to Anthony Liguori

Quoted 16 lines · Anthony Liguori

Sorry this explanation is long winded, but this is a messy situation.
In Linux, there isn't a very consistent policy about userspace kernel header inclusion. On a typical Linux system, you're likely to find kernel headers in three places.
glibc headers (/usr/include/{linux,asm})
These headers are installed by glibc. They very often are based on much older kernel versions that the kernel you have in your distribution. For software that depends on these headers, very often this means that your software detects features being missing that are present on your kernel. Furthermore, glibc only installs the headers it needs so very often certain headers have dependencies that aren't met. A classic example is linux/compiler.h and the broken usbdevice_fs.h header that depends on it. There are still distributions today that QEMU doesn't compile on because of this.
Today, most of QEMU's code depends on these headers.
/lib/modules/$(uname -r)/build
These are the kernel headers that are installed as part of your kernel. In general, this is a pretty good place to find the headers that are associated with the kernel version you're actually running on. However, these headers are part of the kernel build tree and are not always guaranteed to be includable from userspace.
<random kernel tree>

Quoted 8 lines

Developers, in particular, like to point things at their random kernel trees. In general though, relying on a full kernel source tree being available isn't a good idea. Kernel headers change dramatically across versions too so it's very likely that we would need to have a lot of #ifdefs dependent on kernel versions, or some of the uglier work arounds we have in usb-linux.c.
I think the best way to avoid #ifdefs and dependencies on broken/incomplete glibc headers is to include all of the Linux headers we need within QEMU. The attached patch does just this.
I think there's room for discussion about whether we really want to do this. We could potentially depend on some more common glibc headers (like asm/types.h) while bringing in less reliable headers (if_tun.h/virtio*). Including them all seems like the most robust solution to me though.
Comments?

Thinking again about it, this is not really necessary.

In general a distro provides kernel headers matched to the running kernel. For example F10 provides kernel-headers-2.6.27.21-170.2.56.fc10.x86_64 to go along with kernel-2.6.27.21-170.2.56.fc10.x86_64. So a user running a distro kernel (the majority, given that most people don't inflict pain upon themselves unnecessarily) will have exactly the features exported by the kernel.

If a user compiles their own kernel, they will also have the complete kernel sources. We could use --kerneldir, perhaps requiring that the user do a 'make headers-install' first and point kerneldir to the result.

The only deviation for this is kvm, which also comes as an external kernel module and therefore cannot rely on the installed kernel headers. We could make the external module package (kvm-kmod) supply its own set of headers and install them somewhere, or we can carry them in qemu (much more convenient). But I don't think we need to carry such a large subset of the kernel headers (which is liable to change as kernel headers are added).

Signature · 1 line

Do not meddle in the internals of kernels, for they are subtle and quick to panic.

Paul Brook

2009-05-04 09:35 UTC #11 lore raw

In reply to Avi Kivity

Quoted 2 lines

I think we need to use the output of 'make headers-install', which removes things like __user and CONFIG_*.

Yes. Assuming we do decide to import a set of headers, they should definitely be the sanitised version created by make headers-install.

Paul

Arnd Bergmann

2009-05-04 11:29 UTC #20 lore raw

In reply to Anthony Liguori

Quoted 3 lines · Anthony Liguori

A classic example is linux/compiler.h and the broken usbdevice_fs.h header that depends on it. There are still distributions today that QEMU doesn't compile on because of this.

Can you clarify this? I can't find any version of usbdevice_fs.h that ever included linux/compiler.h (make headers_check would warn about that), and the only construct used in there that comes from compiler.h is the __user annotation, which gets stripped in 'make headers_install', and has been since 2006.

Quoted 10 lines

configure +9 -0

+# Linux kernel headers CFLAGS
+if test -z "$kerneldir" ; then
+    linux_cflags="-I$source_path/linux"
+else
+    linux_cflags="-I$kerneldir/include"
+    if test \( "$cpu" = "i386" -o "$cpu" = "x86_64" \) \
+       -a -d "$kerneldir/arch/x86/include" ; then
+       linux_cflags="$linux_cflags -I$kerneldir/arch/x86/include"
+    elif test "$cpu" = "ppc" -a -d "$kerneldir/arch/powerpc/include" ;

then + linux_cflags="$linux_cflags -I$kerneldir/arch/powerpc/include"

configure EOF · line 1097

Quoted 4 lines

configure +4 -0

+    elif test -d "$kerneldir/arch/$cpu/include" ; then
+       linux_cflags="$linux_cflags -I$kerneldir/arch/$cpu/include"
+    fi
+fi

arch/*/include is not the right place to look for user headers. I think it would be better to assume that the user only points to valid exported headers, so look for linux/version.h to check that the files have been configured and look for the absense of kvm_host.h to make sure that the user did not point to plain kernel sources.

The exported headers already handle the asm/ links correctly, so I think you never need to do anything architecture specific like your fixup.sed.

Quoted 12 lines

linux/Makefile new file +12 -0

+CORE_HDRS=linux/types.h linux/posix_types.h linux/stddef.h linux/compiler.h
+CORE_HDRS+=linux/byteorder/little_endian.h linux/byteorder/big_endian.h
+CORE_HDRS+=linux/swab.h linux/ioctl.h
+
+CORE_HDRS+=asm-generic/int-ll64.h asm-generic/int-l64.h asm-generic/ioctl.h
+
+CORE_HDRS+=asm-x86/types.h asm-x86/posix_types.h
+CORE_HDRS+=asm-x86/posix_types_32.h asm-x86/posix_types_64.h
+CORE_HDRS+=asm-x86/byteorder.h asm-x86/swab.h asm-x86/ioctl.h
+
+CORE_HDRS+=asm-powerpc/types.h asm-powerpc/posix_types.h
+CORE_HDRS+=asm-powerpc/byteorder.h asm-powerpc/swab.h asm-powerpc/ioctl.h

linux/Makefile line 25

Quoted 10 lines

linux/Makefile new file +10 -0

+
+CORE_HDRS+=asm-sparc/types.h asm-sparc/posix_types.h
+CORE_HDRS+=asm-sparc/byteorder.h asm-sparc/swab.h asm-sparc/ioctl.h
+CORE_HDRS+=asm-sparc/asi.h 
+
+CORE_HDRS+=asm-arm/types.h asm-arm/posix_types.h
+CORE_HDRS+=asm-arm/byteorder.h asm-arm/swab.h asm-arm/ioctl.h
+
+CORE_HDRS+=asm-parisc/types.h asm-parisc/posix_types.h
+CORE_HDRS+=asm-parisc/byteorder.h asm-parisc/swab.h asm-parisc/ioctl.h

I don't see the need to copy all the core headers. These should have been working for ages, and hardly ever see changes that are relevant to kvm.

The exceptions are linux/stddef.h and linux/compiler.h, which are not exported and should never be used outside of the kernel.

linux/Makefile line 34

Quoted 8 lines

linux/Makefile new file +8 -0

+# Kernel Virtual Machine interface
+KVM_HDRS=linux/kvm.h linux/kvm_para.h
+KVM_HDRS+=asm-x86/kvm.h asm-x86/kvm_para.h
+KVM_HDRS+=asm-powerpc/kvm.h asm-powerpc/kvm_para.h
+
+# VirtIO paravirtual IO framework
+VIRTIO_HDRS=linux/virtio_config.h linux/virtio_net.h linux/virtio_blk.h
+VIRTIO_HDRS+=linux/virtio_console.h linux/virtio_balloon.h

These should be copied into the qemu source tree, but not at configure time. They should just reflect the latest upstream version. Qemu already needs to handle older kernel versions at run time, and by having the very latest version in the source tree, you can make sure that qemu will run on any kernel version.

For asm/kvm.h and asm/kvm-para.h, you can have hard-coded files multiplexing between the architectures, as you would otherwise generate from your fixup.sed.

Quoted 4 lines

linux/Makefile new file +4 -0

+# tun/tap interfaces
+TUN_HDRS=linux/if_tun.h linux/if_ether.h
+
+# timers

linux/Makefile line 49

Quoted 10 lines

linux/Makefile new file +10 -0

+TIMER_HDRS=linux/rtc.h linux/hpet.h
+
+# USB pass through
+USB_HDRS=linux/usbdevice_fs.h linux/magic.h
+
+# IDE/FD
+DISK_HDRS=linux/cdrom.h linux/fd.h
+
+# Parallel port
+PPORT_HDRS=linux/ppdev.h linux/parport.h

For all of these, I would again fall back on the distro-provided headers. You might not get the latest versions, but at least you can assume that any kernel that the distro provides will also at least support the ABI from these headers.

	Arnd <><

Mark McLoughlin

2009-05-04 12:43 UTC #14 lore raw

In reply to Avi Kivity

Quoted 7 lines · Avi Kivity

In general a distro provides kernel headers matched to the running kernel. For example F10 provides kernel-headers-2.6.27.21-170.2.56.fc10.x86_64 to go along with kernel-2.6.27.21-170.2.56.fc10.x86_64. So a user running a distro kernel (the majority, given that most people don't inflict pain upon themselves unnecessarily) will have exactly the features exported by the kernel.

Right, but if you e.g. try to build a newer qemu-kvm on F10, you currently need newer kvm kernel headers - IMHO, we should use #ifdef to allow newer qemu-kvm build with older kvm headers.

Cheers,
Mark.

Arnd Bergmann

2009-05-04 13:02 UTC #15 lore raw

Quoted 3 lines · Mark McLoughlin

Right, but if you e.g. try to build a newer qemu-kvm on F10, you currently need newer kvm kernel headers - IMHO, we should use #ifdef to allow newer qemu-kvm build with older kvm headers.

I think the kvm and virtio headers should just be shipped with qemu-kvm in their latest versions, rather than relying on the ones from the kernel. Everything else should come from the distro-supplied glibc kernel headers.

	Arnd <><

Anthony Liguori

author

2009-05-04 13:13 UTC #5 lore raw

In reply to Stefan Weil

Quoted 7 lines · Stefan Weil

Anthony Liguori schrieb:
For Debian systems, those headers are installed by package linux-libc-dev. There are also packages for cross compilation in emdebian (linux-libc-dev-mips-cross, linux-libc-dev-powerpc-cross, ...).

Quoted 7 lines

Yes, those headers did not always match the features of the current kernel, so --enable-kvm did not work. This is fixed now - there is a linux-libc-dev 2.6.29-3 which is up-to-date.
So, at the moment I see no need to fill the QEMU source tree with linux header files.

We can not just rely on everyone who uses QEMU to use the latest version of Debian...

The fact is, linux-libc-dev is *not* meant for applications to use as the official kernel ABI. We shouldn't depend on it.

Regards,

Anthony Liguori

author

2009-05-04 13:14 UTC #12 lore raw

In reply to Avi Kivity

Quoted 11 lines · Avi Kivity

Quoted 9 lines
<random kernel tree>
Developers, in particular, like to point things at their random kernel trees. In general though, relying on a full kernel source tree being available isn't a good idea. Kernel headers change dramatically across versions too so it's very likely that we would need to have a lot of #ifdefs dependent on kernel versions, or some of the uglier work arounds we have in usb-linux.c.
I think the best way to avoid #ifdefs and dependencies on broken/incomplete glibc headers is to include all of the Linux headers we need within QEMU. The attached patch does just this.
I think there's room for discussion about whether we really want to do this. We could potentially depend on some more common glibc headers (like asm/types.h) while bringing in less reliable headers (if_tun.h/virtio*). Including them all seems like the most robust solution to me though.
Comments?
I think we need to use the output of 'make headers-install', which removes things like __user and CONFIG_*.

I was thinking about that as a possibility too. We still need the same basic infrastructure though.

Regards,

Anthony Liguori

author

2009-05-04 13:16 UTC #3 lore raw

In reply to Edgar E. Iglesias

Quoted 3 lines · Edgar E. Iglesias

I don't feel very strongly about it but my gut feeling tells me we shouldn't be doing this.

We have to. It's not just KVM, it's virtio, tun/tap, and as we add more things to the Linux kernel to support QEMU, it'll just grow larger.

This is how applications are supposed to use kernel headers. It's unpleasant, but that's just the way Linux is today.

Regards,

Anthony Liguori

Quoted 2 lines

Cheers

Anthony Liguori

author

2009-05-04 13:21 UTC #21 lore raw

In reply to Arnd Bergmann

Quoted 7 lines · Arnd Bergmann

Quoted 4 lines · Anthony Liguori
A classic example is linux/compiler.h and the broken usbdevice_fs.h header that depends on it. There are still distributions today that QEMU doesn't compile on because of this.

Quoted 6 lines

Can you clarify this? I can't find any version of usbdevice_fs.h that ever included linux/compiler.h (make headers_check would warn about that), and the only construct used in there that comes from compiler.h is the __user annotation, which gets stripped in 'make headers_install', and has been since 2006.

Distros that were released before 2006 certainly had this problem. The issue is that usbdevice_fs.h depends on __user.

Quoted 19 lines

linux/Makefile new file +19 -0

+CORE_HDRS=linux/types.h linux/posix_types.h linux/stddef.h linux/compiler.h
+CORE_HDRS+=linux/byteorder/little_endian.h linux/byteorder/big_endian.h
+CORE_HDRS+=linux/swab.h linux/ioctl.h
+
+CORE_HDRS+=asm-generic/int-ll64.h asm-generic/int-l64.h asm-generic/ioctl.h
+
+CORE_HDRS+=asm-x86/types.h asm-x86/posix_types.h
+CORE_HDRS+=asm-x86/posix_types_32.h asm-x86/posix_types_64.h
+CORE_HDRS+=asm-x86/byteorder.h asm-x86/swab.h asm-x86/ioctl.h
+
+CORE_HDRS+=asm-powerpc/types.h asm-powerpc/posix_types.h
+CORE_HDRS+=asm-powerpc/byteorder.h asm-powerpc/swab.h asm-powerpc/ioctl.h
+
+CORE_HDRS+=asm-sparc/types.h asm-sparc/posix_types.h
+CORE_HDRS+=asm-sparc/byteorder.h asm-sparc/swab.h asm-sparc/ioctl.h
+CORE_HDRS+=asm-sparc/asi.h 
+
+CORE_HDRS+=asm-arm/types.h asm-arm/posix_types.h
+CORE_HDRS+=asm-arm/byteorder.h asm-arm/swab.h asm-arm/ioctl.h

Quoted 9 lines

linux/Makefile line 25
Quoted 4 lines
linux/Makefile new file +3 -0
+
+CORE_HDRS+=asm-parisc/types.h asm-parisc/posix_types.h
+CORE_HDRS+=asm-parisc/byteorder.h asm-parisc/swab.h asm-parisc/ioctl.h
I don't see the need to copy all the core headers. These should have been working for ages, and hardly ever see changes that are relevant to kvm.

If we want to use virtio_*.h instead of duplicating the copies as we are now, then we need all of the core headers too or else it won't be able to compile on systems that do not have Linux libc headers (like win32).

Quoted 7 lines

linux/Makefile new file +7 -0

+# Kernel Virtual Machine interface
+KVM_HDRS=linux/kvm.h linux/kvm_para.h
+KVM_HDRS+=asm-x86/kvm.h asm-x86/kvm_para.h
+KVM_HDRS+=asm-powerpc/kvm.h asm-powerpc/kvm_para.h
+
+# VirtIO paravirtual IO framework
+VIRTIO_HDRS=linux/virtio_config.h linux/virtio_net.h linux/virtio_blk.h

Quoted 9 lines

Quoted 2 lines
+VIRTIO_HDRS+=linux/virtio_console.h linux/virtio_balloon.h
These should be copied into the qemu source tree, but not at configure time. They should just reflect the latest upstream version. Qemu already needs to handle older kernel versions at run time, and by having the very latest version in the source tree, you can make sure that qemu will run on any kernel version.

Yes, if it isn't clear, this Makefile is meant to be used by the maintainers to bring the headers into git. I didn't post the headers because it would have made the note annoyingly long.

Regards,

Anthony Liguori

author

2009-05-04 13:24 UTC #18 lore raw

In reply to Avi Kivity

Quoted 7 lines · Avi Kivity

Quoted 2 lines · Anthony Liguori
Comments?
Thinking again about it, this is not really necessary.
In general a distro provides kernel headers matched to the running kernel. For example F10 provides kernel-headers-2.6.27.21-170.2.56.fc10.x86_64 to go along with kernel-2.6.27.21-170.2.56.fc10.x86_64. So a user running a distro kernel (the majority, given that most people don't inflict pain upon themselves unnecessarily) will have exactly the features exported by the kernel.

kernel-headers is not usually installed by default. Also, I'd rather not deal with #ifdef code as we introduce new features like TUN_VNET_HDR.

Regards,

Anthony Liguori

author

2009-05-04 13:25 UTC #16 lore raw

In reply to Arnd Bergmann

Quoted 7 lines · Arnd Bergmann

Quoted 4 lines · Mark McLoughlin
Right, but if you e.g. try to build a newer qemu-kvm on F10, you currently need newer kvm kernel headers - IMHO, we should use #ifdef to allow newer qemu-kvm build with older kvm headers.

Quoted 5 lines

I think the kvm and virtio headers should just be shipped with qemu-kvm in their latest versions, rather than relying on the ones from the kernel. Everything else should come from the distro-supplied glibc kernel headers.

Just to reiterate, because I should have mentioned it in the original note, we need virtio to be buildable on non-Linux systems so those headers must not depend on having distro glibc headers.

Regards,

Anthony Liguori

Quoted 2 lines

	Arnd <><

Avi Kivity

2009-05-04 13:26 UTC #17 lore raw

In reply to Mark McLoughlin

Quoted 10 lines · Mark McLoughlin

Quoted 2 lines · Avi Kivity
In general a distro provides kernel headers matched to the running kernel. For example F10 provides kernel-headers-2.6.27.21-170.2.56.fc10.x86_64 to go along with kernel-2.6.27.21-170.2.56.fc10.x86_64. So a user running a distro kernel (the majority, given that most people don't inflict pain upon themselves unnecessarily) will have exactly the features exported by the kernel.
Right, but if you e.g. try to build a newer qemu-kvm on F10, you currently need newer kvm kernel headers - IMHO, we should use #ifdef to allow newer qemu-kvm build with older kvm headers.

qemu build against new headers should work fine on older hosts -- we discover features at runtime. But I agree it's nice to be able to build against older headers.

Signature · 1 line

Do not meddle in the internals of kernels, for they are subtle and quick to panic.

Avi Kivity

2009-05-04 13:28 UTC #6 lore raw

In reply to Anthony Liguori

Quoted 11 lines · Anthony Liguori

Quoted 10 lines · Stefan Weil
Anthony Liguori schrieb:
For Debian systems, those headers are installed by package linux-libc-dev. There are also packages for cross compilation in emdebian (linux-libc-dev-mips-cross, linux-libc-dev-powerpc-cross, ...).
Yes, those headers did not always match the features of the current kernel, so --enable-kvm did not work. This is fixed now - there is a linux-libc-dev 2.6.29-3 which is up-to-date.

Quoted 7 lines

Quoted 3 lines
So, at the moment I see no need to fill the QEMU source tree with linux header files.
We can not just rely on everyone who uses QEMU to use the latest version of Debian...
The fact is, linux-libc-dev is *not* meant for applications to use as the official kernel ABI. We shouldn't depend on it.

At least on Fedora, kernel-headers is. It is installed in /usr/include/linux and is synced (sorta) to the installed kernel.

Carrying a subset of kernel headers is a bit too much, IMO.

kvm is a special case since it is available externally.

Signature · 1 line

Do not meddle in the internals of kernels, for they are subtle and quick to panic.

Anthony Liguori

author

2009-05-04 13:30 UTC #7 lore raw

Quoted 1 line · Avi Kivity

At least on Fedora, kernel-headers is. It is installed in /usr/include/linux and is synced (sorta) to the installed kernel.

It's not the case with Ubuntu.

Quoted 1 line

Carrying a subset of kernel headers is a bit too much, IMO.

Carrying virtio, kvm, and if_tun would be sufficient IMO. I think depending on /usr/include/linux is okay for kvm and if_tun, but virtio needs to be buildable without /usr/include/linux.

Regards,

Anthony Liguori

Avi Kivity

2009-05-04 13:38 UTC #22 lore raw

In reply to Anthony Liguori

Quoted 7 lines · Anthony Liguori

Quoted 5 lines
Quoted 1 line
I don't see the need to copy all the core headers. These should have been working for ages, and hardly ever see changes that are relevant to kvm.
If we want to use virtio_*.h instead of duplicating the copies as we are now, then we need all of the core headers too or else it won't be able to compile on systems that do not have Linux libc headers (like win32).

qemu provides virtio, it doesn't consume it. We can merge the virtio headers and remove the linuxisms.

Signature · 1 line

Do not meddle in the internals of kernels, for they are subtle and quick to panic.

Avi Kivity

2009-05-04 13:41 UTC #19 lore raw

In reply to Anthony Liguori

Quoted 5 lines · Anthony Liguori

Quoted 3 lines
Thinking again about it, this is not really necessary.
In general a distro provides kernel headers matched to the running kernel. For example F10 provides kernel-headers-2.6.27.21-170.2.56.fc10.x86_64 to go along with kernel-2.6.27.21-170.2.56.fc10.x86_64. So a user running a distro kernel (the majority, given that most people don't inflict pain upon themselves unnecessarily) will have exactly the features exported by the kernel.
kernel-headers is not usually installed by default.

It is:

$ rpm -q --whatrequires kernel-headers
glibc-headers-2.9-3.x86_64

Signature · 1 line

Do not meddle in the internals of kernels, for they are subtle and quick to panic.

Edgar E. Iglesias

2009-05-04 13:44 UTC #4 lore raw

In reply to Anthony Liguori

Quoted 7 lines · Anthony Liguori

Quoted 3 lines · Edgar E. Iglesias
I don't feel very strongly about it but my gut feeling tells me we shouldn't be doing this.
We have to. It's not just KVM, it's virtio, tun/tap, and as we add more things to the Linux kernel to support QEMU, it'll just grow larger.

I'm not sure we have too. QEMU users that build from source can IMO be expected to update kernel headers if required and if our configure is explicit about it. "You didn't get feature x because you have an old kernel or old kernel headers" kind of warning.

Another alternative is to provide an option so users can specify where to find alternative kernel-headers. IIRC other I've seen this approach in several other projects.

I agree with you that the compat ifdefs are annoying though...

Quoted 2 lines

This is how applications are supposed to use kernel headers. It's unpleasant, but that's just the way Linux is today.

Do you mean that all apps using linux header files should bring those in?

Cheers

Christoph Hellwig

2009-05-04 14:01 UTC #8 lore raw

In reply to Anthony Liguori

Quoted 2 lines · Anthony Liguori

The fact is, linux-libc-dev is *not* meant for applications to use as the official kernel ABI. We shouldn't depend on it.

Umm, it is. That's exactly the reason what it is for. Note that the name of the package varies depending on the distro, but those headers in /usr/include/linux/ are the ABI for features not shimed by libc.

Christoph Hellwig

2009-05-04 14:04 UTC #23 lore raw

In reply to Avi Kivity

Quoted 2 lines · Avi Kivity

qemu provides virtio, it doesn't consume it. We can merge the virtio headers and remove the linuxisms.

Yeah. virtio is a one the (virtual) wire protocol, not a kernel ABI in the tradition sense. qemu should have it's own defintion. For kernel feature qemu uses (mostly kvm, but also the scsi generic ioctl for example) it should just use the installed kernel headers, and not build the feature if they are too old.

Arnd Bergmann

2009-05-04 14:18 UTC #24 lore raw

In reply to Anthony Liguori

Quoted 11 lines · Anthony Liguori

Quoted 7 lines · Arnd Bergmann
Can you clarify this? I can't find any version of usbdevice_fs.h that ever included linux/compiler.h (make headers_check would warn about that), and the only construct used in there that comes from compiler.h is the __user annotation, which gets stripped in 'make headers_install', and has been since 2006.
Distros that were released before 2006 certainly had this problem. The issue is that usbdevice_fs.h depends on __user.

Right. But I don't think we will see new cases like this in the future and we can work around the existing breakage by doing things like

#undef __user
#define __user
#include <linux/usbdevice_fs.h>

Quoted 8 lines

Quoted 4 lines
I don't see the need to copy all the core headers. These should have been working for ages, and hardly ever see changes that are relevant to kvm.
If we want to use virtio_*.h instead of duplicating the copies as we are now, then we need all of the core headers too or else it won't be able to compile on systems that do not have Linux libc headers (like win32).

There is a big difference between the kvm headers and the virtio headers then. For virtio, it will be easy to automatically change them not to rely on the core headers, because the only thing they use from there are really the integer types (__u32 and others). We can either define them in qemu, or replace them with C99 inttypes using sed.

The kvm headers are much harder, because they actually rely on architecture specific stuff like ioctl.h, but then again, I don't think that you want to provide kvm.h on Win32, unless you are cross-compiling for a Linux host (implying that you already have the full set of headers).

Quoted 10 lines

Quoted 6 lines
These should be copied into the qemu source tree, but not at configure time. They should just reflect the latest upstream version. Qemu already needs to handle older kernel versions at run time, and by having the very latest version in the source tree, you can make sure that qemu will run on any kernel version.
Yes, if it isn't clear, this Makefile is meant to be used by the maintainers to bring the headers into git. I didn't post the headers because it would have made the note annoyingly long.

Ok, it makes more sense now.

	Arnd <><

Lennart Sorensen

2009-05-04 14:27 UTC #9 lore raw

In reply to Anthony Liguori

Quoted 5 lines · Anthony Liguori

We can not just rely on everyone who uses QEMU to use the latest version of Debian...
The fact is, linux-libc-dev is *not* meant for applications to use as the official kernel ABI. We shouldn't depend on it.

Actually I think that is exactly what it is intended to be and exactly what you should do.

Signature · 1 line

Len Sorensen