Skip to contentSkip to conversation

remove pieces of source code

PATCH In review Glauber Costa 14 participants 25 messages May 28, 2009 – Jun 6, 2009 Kbuild

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 remove pieces of source code

Have you ever seen a girl so beautiful that you, geeky,
think: "I'll never stand a chance"?

But sometimes, you decide to make your move anyway. There's always the chance that in that very day she'll be specially in good mood, and you'll get what you want.

With the exception of the fact that qemu is not a girl, that's more or less what I'm trying to do here: Hopefully, nobody will notice what I'm trying to do, and will commmit it. Later, when realizing, it will be too late. Victory will be mine.

Or maybe people will even agree. For that, I'll try briefly to arguee my point, without disclosing to much, avoiding jeopardizing the strategy I explained above:

This patch removes a piece of code that is unmaintaned, that does not receive an update for years,
that get bug reports on the list that nobody fixes, because nobody really understands,
that places some artificial constraints on other subsystems

---
 Makefile.target         |    3 -
 configure               |   36 --
 cpu-all.h               |    9 -
 cpu-common.h            |    5 -
 cpu-exec.c              |   43 +--
 exec-all.h              |   35 --
 exec.c                  |   91 +-----
 hw/pc.c                 |   12 +-
 hw/virtio-pci.c         |    2 +-
 hw/virtio.c             |    2 +-
 kqemu.c                 |  999 -----------------------------------------------
 kqemu.h                 |  154 --------
 monitor.c               |   53 ---
 osdep.c                 |  117 ------
 qemu-doc.texi           |    2 -
 qemu-options.hx         |   19 -
 qemu-tech.texi          |    4 +-
 softmmu_template.h      |    6 -
 sysemu.h                |    4 -
 target-i386/TODO        |    1 -
 target-i386/cpu.h       |   14 -
 target-i386/helper.c    |   21 +-
 target-i386/op_helper.c |   42 --
 vl.c                    |   75 +----
 24 files changed, 12 insertions(+), 1737 deletions(-)
 delete mode 100644 kqemu.c
 delete mode 100644 kqemu.h
Makefile.target +0 -3
index 445d55f..2ded52f 100644
@@ -135,9 +135,6 @@ all: $(PROGS)
 # cpu emulator library
 LIBOBJS=exec.o translate-all.o cpu-exec.o\
         translate.o host-utils.o
-ifdef CONFIG_KQEMU
-LIBOBJS+= kqemu.o
-endif
 # TCG code generator
 LIBOBJS+= tcg/tcg.o tcg/tcg-runtime.o
 CPPFLAGS+=-I$(SRC_PATH)/tcg -I$(SRC_PATH)/tcg/$(ARCH)
configure +0 -36
index b6d18d1..7d4e226 100755
@@ -172,7 +172,6 @@ vnc_sasl="yes"
 bsd="no"
 linux="no"
 solaris="no"
-kqemu="no"
 profiler="no"
 cocoa="no"
 softmmu="yes"
@@ -215,40 +214,25 @@ case $targetos in
 CYGWIN*)
 mingw32="yes"
 OS_CFLAGS="-mno-cygwin"
-if [ "$cpu" = "i386" ] ; then
-    kqemu="yes"
-fi
 audio_possible_drivers="sdl"
 ;;
 MINGW32*)
 mingw32="yes"
-if [ "$cpu" = "i386" ] ; then
-    kqemu="yes"
-fi
 audio_possible_drivers="dsound sdl fmod"
 ;;
 GNU/kFreeBSD)
 audio_drv_list="oss"
 audio_possible_drivers="oss sdl esd pa"
-if [ "$cpu" = "i386" -o "$cpu" = "x86_64" ] ; then
-    kqemu="yes"
-fi
 ;;
 FreeBSD)
 bsd="yes"
 audio_drv_list="oss"
 audio_possible_drivers="oss sdl esd pa"
-if [ "$cpu" = "i386" -o "$cpu" = "x86_64" ] ; then
-    kqemu="yes"
-fi
 ;;
 DragonFly)
 bsd="yes"
 audio_drv_list="oss"
 audio_possible_drivers="oss sdl esd pa"
-if [ "$cpu" = "i386" -o "$cpu" = "x86_64" ] ; then
-    kqemu="yes"
-fi
 aio="no"
 ;;
 NetBSD)
@@ -309,9 +293,6 @@ SunOS)
                 exit 1
             fi
         fi
-        if test "$solarisrev" -ge 9 ; then
-            kqemu="yes"
-        fi
     fi
     if test -f /usr/include/sys/soundcard.h ; then
         audio_drv_list="oss"
@@ -330,7 +311,6 @@ linux="yes"
 linux_user="yes"
 usb="linux"
 if [ "$cpu" = "i386" -o "$cpu" = "x86_64" ] ; then
-    kqemu="yes"
     audio_possible_drivers="$audio_possible_drivers fmod"
 fi
 ;;
@@ -425,8 +405,6 @@ for opt do
   ;;
   --disable-vde) vde="no"
   ;;
-  --disable-kqemu) kqemu="no"
-  ;;
   --disable-xen) xen="no"
   ;;
   --disable-brlapi) brlapi="no"
@@ -574,9 +552,6 @@ echo "  --interp-prefix=PREFIX   where to find shared libraries, etc."
 echo "                           use %M for cpu name [$interp_prefix]"
 echo "  --target-list=LIST       set target list [$target_list]"
 echo ""
-echo "kqemu kernel acceleration support:"
-echo "  --disable-kqemu          disable kqemu support"
-echo ""
 echo "Advanced options (experts only):"
 echo "  --source-path=PATH       path of source code [$source_path]"
 echo "  --cross-prefix=PREFIX    use PREFIX for compile tools [$cross_prefix]"
@@ -1351,7 +1326,6 @@ fi
 if test -n "$sparc_cpu"; then
     echo "Target Sparc Arch $sparc_cpu"
 fi
-echo "kqemu support     $kqemu"
 echo "xen support       $xen"
 echo "brlapi support    $brlapi"
 echo "Documentation     $build_docs"
@@ -1862,11 +1836,6 @@ case "$target_cpu" in
     echo "TARGET_ARCH=i386" >> $config_mak
     echo "#define TARGET_ARCH \"i386\"" >> $config_h
     echo "#define TARGET_I386 1" >> $config_h
-    if test $kqemu = "yes" -a "$target_softmmu" = "yes"
-    then
-      echo "CONFIG_KQEMU=yes" >> $config_mak
-      echo "#define CONFIG_KQEMU 1" >> $config_h
-    fi
     if test "$kvm" = "yes" ; then
       echo "CONFIG_KVM=yes" >> $config_mak
       echo "KVM_CFLAGS=$kvm_cflags" >> $config_mak
@@ -1884,11 +1853,6 @@ case "$target_cpu" in
     echo "#define TARGET_ARCH \"x86_64\"" >> $config_h
     echo "#define TARGET_I386 1" >> $config_h
     echo "#define TARGET_X86_64 1" >> $config_h
-    if test $kqemu = "yes" -a "$target_softmmu" = "yes" -a $cpu = "x86_64"
-    then
-      echo "CONFIG_KQEMU=yes" >> $config_mak
-      echo "#define CONFIG_KQEMU 1" >> $config_h
-    fi
     if test "$kvm" = "yes" ; then
       echo "CONFIG_KVM=yes" >> $config_mak
       echo "KVM_CFLAGS=$kvm_cflags" >> $config_mak
cpu-all.h +0 -9
index dc9b034..8eeef05 100644
@@ -873,7 +873,6 @@ int cpu_memory_rw_debug(CPUState *env, target_ulong addr,
 
 #define VGA_DIRTY_FLAG       0x01
 #define CODE_DIRTY_FLAG      0x02
-#define KQEMU_DIRTY_FLAG     0x04
 #define MIGRATION_DIRTY_FLAG 0x08
 
 /* read dirty bit (return 0 or 1) */
@@ -1056,14 +1055,6 @@ static inline int64_t profile_getclock(void)
     return cpu_get_real_ticks();
 }
 
-extern int64_t kqemu_time, kqemu_time_start;
-extern int64_t qemu_time, qemu_time_start;
-extern int64_t tlb_flush_time;
-extern int64_t kqemu_exec_count;
-extern int64_t dev_time;
-extern int64_t kqemu_ret_int_count;
-extern int64_t kqemu_ret_excp_count;
-extern int64_t kqemu_ret_intr_count;
 #endif
 
 #endif /* CPU_ALL_H */
cpu-common.h +0 -5
index 4640924..2ce8448 100644
@@ -10,12 +10,7 @@
 #include "bswap.h"
 
 /* address in the RAM (different from a physical address) */
-#ifdef CONFIG_KQEMU
-/* FIXME: This is wrong.  */
-typedef uint32_t ram_addr_t;
-#else
 typedef unsigned long ram_addr_t;
-#endif
 
 /* memory API */
 
cpu-exec.c +1 -42

63 lines not on this page · raw message

exec-all.h +0 -35
index f91e646..7f8a786 100644
@@ -352,41 +352,6 @@ static inline int can_do_io(CPUState *env)
 }
 #endif
 
-#ifdef CONFIG_KQEMU
-#define KQEMU_MODIFY_PAGE_MASK (0xff & ~(VGA_DIRTY_FLAG | CODE_DIRTY_FLAG))
-
-#define MSR_QPI_COMMBASE 0xfabe0010
-
-int kqemu_init(CPUState *env);
-int kqemu_cpu_exec(CPUState *env);
-void kqemu_flush_page(CPUState *env, target_ulong addr);
-void kqemu_flush(CPUState *env, int global);
-void kqemu_set_notdirty(CPUState *env, ram_addr_t ram_addr);
-void kqemu_modify_page(CPUState *env, ram_addr_t ram_addr);
-void kqemu_set_phys_mem(uint64_t start_addr, ram_addr_t size, 
-                        ram_addr_t phys_offset);
-void kqemu_cpu_interrupt(CPUState *env);
-void kqemu_record_dump(void);
-
-extern uint32_t kqemu_comm_base;
-
-extern ram_addr_t kqemu_phys_ram_size;
-extern uint8_t *kqemu_phys_ram_base;
-
-static inline int kqemu_is_ok(CPUState *env)
-{
-    return(env->kqemu_enabled &&
-           (env->cr[0] & CR0_PE_MASK) &&
-           !(env->hflags & HF_INHIBIT_IRQ_MASK) &&
-           (env->eflags & IF_MASK) &&
-           !(env->eflags & VM_MASK) &&
-           (env->kqemu_enabled == 2 ||
-            ((env->hflags & HF_CPL_MASK) == 3 &&
-             (env->eflags & IOPL_MASK) != IOPL_MASK)));
-}
-
-#endif
-
 typedef void (CPUDebugExcpHandler)(CPUState *env);
 
 CPUDebugExcpHandler *cpu_set_debug_excp_handler(CPUDebugExcpHandler *handler);
exec.c +2 -89

170 lines not on this page · raw message

hw/pc.c +1 -11
index 66cc780..43c0657 100644
@@ -104,17 +104,7 @@ static void ioportF0_write(void *opaque, uint32_t addr, uint32_t data)
 /* TSC handling */
 uint64_t cpu_get_tsc(CPUX86State *env)
 {
-    /* Note: when using kqemu, it is more logical to return the host TSC
-       because kqemu does not trap the RDTSC instruction for
-       performance reasons */
-#ifdef CONFIG_KQEMU
-    if (env->kqemu_enabled) {
-        return cpu_get_real_ticks();
-    } else
-#endif
-    {
-        return cpu_get_ticks();
-    }
+    return cpu_get_ticks();
 }
 
 /* SMM support */
hw/virtio-pci.c +1 -1
index c072423..06b6077 100644
@@ -58,7 +58,7 @@
 
 /* QEMU doesn't strictly need write barriers since everything runs in
  * lock-step.  We'll leave the calls to wmb() in though to make it obvious for
- * KVM or if kqemu gets SMP support.
+ * KVM if gets SMP support.
  */
 #define wmb() do { } while (0)
 
hw/virtio.c +1 -1
index 45a49fa..bd95cfc 100644
@@ -22,7 +22,7 @@
 
 /* QEMU doesn't strictly need write barriers since everything runs in
  * lock-step.  We'll leave the calls to wmb() in though to make it obvious for
- * KVM or if kqemu gets SMP support.
+ * KVM if gets SMP support.
  */
 #define wmb() do { } while (0)
 
kqemu.c deleted +0 -999

999 lines not on this page · raw message

kqemu.h deleted +0 -154

154 lines not on this page · raw message

monitor.c +0 -53

81 lines not on this page · raw message

osdep.c +0 -117

134 lines not on this page · raw message

qemu-doc.texi +0 -2
index 837c2a4..78810ae 100644
@@ -358,8 +358,6 @@ show virtual to physical memory mappings (i386 only)
 show the active virtual memory mappings (i386 only)
 @item info hpet
 show state of HPET (i386 only)
-@item info kqemu
-show KQEMU information
 @item info kvm
 show KVM information
 @item info usb
qemu-options.hx +0 -19
index 87af798..3e1b209 100644
@@ -1319,25 +1319,6 @@ STEXI
 Set the filename for the BIOS.
 ETEXI
 
-#ifdef CONFIG_KQEMU
-DEF("kernel-kqemu", 0, QEMU_OPTION_kernel_kqemu, \
-    "-kernel-kqemu   enable KQEMU full virtualization (default is user mode only)\n")
-#endif
-STEXI
-@item -kernel-kqemu
-Enable KQEMU full virtualization (default is user mode only).
-ETEXI
-
-#ifdef CONFIG_KQEMU
-DEF("no-kqemu", 0, QEMU_OPTION_no_kqemu, \
-    "-no-kqemu       disable KQEMU kernel module usage\n")
-#endif
-STEXI
-@item -no-kqemu
-Disable KQEMU kernel module usage. KQEMU options are only available if
-KQEMU support is enabled when compiling.
-ETEXI
-
 #ifdef CONFIG_KVM
 DEF("enable-kvm", 0, QEMU_OPTION_enable_kvm, \
     "-enable-kvm     enable KVM full virtualization support\n")
qemu-tech.texi +2 -2
index ed2d35b..3e62f40 100644
@@ -116,8 +116,8 @@ QEMU full system emulation features:
 QEMU uses a full software MMU for maximum portability.
 
 @item
-QEMU can optionally use an in-kernel accelerator, like kqemu and
-kvm. The accelerators execute some of the guest code natively, while
+QEMU can optionally use an in-kernel accelerator, like kvm. 
+The accelerators execute some of the guest code natively, while
 continuing to emulate the rest of the machine.
 
 @item
softmmu_template.h +0 -6
index 4b33aef..e519c51 100644
@@ -76,9 +76,6 @@ static inline DATA_TYPE glue(io_read, SUFFIX)(target_phys_addr_t physaddr,
     res |= (uint64_t)io_mem_read[index][2](io_mem_opaque[index], physaddr + 4) << 32;
 #endif
 #endif /* SHIFT > 2 */
-#ifdef CONFIG_KQEMU
-    env->last_io_time = cpu_get_time_fast();
-#endif
     return res;
 }
 
@@ -221,9 +218,6 @@ static inline void glue(io_write, SUFFIX)(target_phys_addr_t physaddr,
     io_mem_write[index][2](io_mem_opaque[index], physaddr + 4, val >> 32);
 #endif
 #endif /* SHIFT > 2 */
-#ifdef CONFIG_KQEMU
-    env->last_io_time = cpu_get_time_fast();
-#endif
 }
 
 void REGPARM glue(glue(__st, SUFFIX), MMUSUFFIX)(target_ulong addr,
sysemu.h +0 -4
index 92501ed..b9e2e8c 100644
@@ -120,10 +120,6 @@ extern int no_quit;
 extern int semihosting_enabled;
 extern int old_param;
 
-#ifdef CONFIG_KQEMU
-extern int kqemu_allowed;
-#endif
-
 #define MAX_NODES 64
 extern int nb_numa_nodes;
 extern uint64_t node_mem[MAX_NODES];
target-i386/TODO +0 -1
index b671e24..c8ada07 100644
@@ -30,4 +30,3 @@ Optimizations/Features:
 - evaluate x87 stack pointer statically
 - find a way to avoid translating several time the same TB if CR0.TS
   is set or not.
-- move kqemu support outside target-i386.
target-i386/cpu.h +0 -14
index d7b32d4..b61e073 100644
@@ -664,11 +664,6 @@ typedef struct CPUX86State {
         uint64_t mask;
     } mtrr_var[8];
 
-#ifdef CONFIG_KQEMU
-    int kqemu_enabled;
-    int last_io_time;
-#endif
-
     /* For KVM */
     uint64_t interrupt_bitmap[256 / 64];
     uint32_t mp_state;
@@ -823,15 +818,6 @@ uint64_t cpu_get_tsc(CPUX86State *env);
 #define X86_DUMP_FPU  0x0001 /* dump FPU state too */
 #define X86_DUMP_CCOP 0x0002 /* dump qemu flag cache */
 
-#ifdef CONFIG_KQEMU
-static inline int cpu_get_time_fast(void)
-{
-    int low, high;
-    asm volatile("rdtsc" : "=a" (low), "=d" (high));
-    return low;
-}
-#endif
-
 #define TARGET_PAGE_BITS 12
 
 #define cpu_init cpu_x86_init
target-i386/helper.c +3 -18
index 7fc5366..4339b01 100644
@@ -947,14 +947,10 @@ target_phys_addr_t cpu_get_phys_page_debug(CPUState *env, target_ulong addr)
 
 /* XXX: This value should match the one returned by CPUID
  * and in exec.c */
-#if defined(CONFIG_KQEMU)
-#define PHYS_ADDR_MASK 0xfffff000LL
+#if defined(TARGET_X86_64)
+#define PHYS_ADDR_MASK 0xfffffff000LL
 #else
-# if defined(TARGET_X86_64)
-# define PHYS_ADDR_MASK 0xfffffff000LL
-# else
-# define PHYS_ADDR_MASK 0xffffff000LL
-# endif
+#define PHYS_ADDR_MASK 0xffffff000LL
 #endif
 
 /* return value:
@@ -1647,21 +1643,13 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
 /* XXX: This value must match the one used in the MMU code. */ 
         if (env->cpuid_ext2_features & CPUID_EXT2_LM) {
             /* 64 bit processor */
-#if defined(CONFIG_KQEMU)
-            *eax = 0x00003020;	/* 48 bits virtual, 32 bits physical */
-#else
 /* XXX: The physical address space is limited to 42 bits in exec.c. */
             *eax = 0x00003028;	/* 48 bits virtual, 40 bits physical */
-#endif
         } else {
-#if defined(CONFIG_KQEMU)
-            *eax = 0x00000020;	/* 32 bits physical */
-#else
             if (env->cpuid_features & CPUID_PSE36)
                 *eax = 0x00000024; /* 36 bits physical */
             else
                 *eax = 0x00000020; /* 32 bits physical */
-#endif
         }
         *ebx = 0;
         *ecx = 0;
@@ -1706,9 +1694,6 @@ CPUX86State *cpu_x86_init(const char *cpu_model)
         return NULL;
     }
     cpu_reset(env);
-#ifdef CONFIG_KQEMU
-    kqemu_init(env);
-#endif
 
     qemu_init_vcpu(env);
 
target-i386/op_helper.c +0 -42

77 lines not on this page · raw message

vl.c +1 -74

156 lines not on this page · raw message

Signature · 1 line
1.5.6.6
#1 lore raw
Quoted 16 lines · Glauber Costa

Have you ever seen a girl so beautiful that you, geeky,
think: "I'll never stand a chance"?

But sometimes, you decide to make your move anyway. There's always the chance that in that very day she'll be specially in good mood, and you'll get what you want.

With the exception of the fact that qemu is not a girl, that's more or less what I'm trying to do here: Hopefully, nobody will notice what I'm trying to do, and will commmit it. Later, when realizing, it will be too late. Victory will be mine.

Or maybe people will even agree. For that, I'll try briefly to arguee my point, without disclosing to much, avoiding jeopardizing the strategy I explained above:

Quoted 7 lines

This patch removes a piece of code that is unmaintaned, that does not receive an update for years,
that get bug reports on the list that nobody fixes, because nobody really understands,
that places some artificial constraints on other subsystems

Let's actually build a proper case instead of closing our eyes and hitting enter. Here are the downsides of kqemu I know of:

o Since it's enabled by default, it forces the default build to support
< 4GB of guest memory
o It attempts to use /dev/shm for guest memory which means a special
option is needed in the default build to use more than 1/2 of host ram size
o It touches an awful lot of places in QEMU
o Some of the BIOS changes are particularly nasty and will prevent
having a unified BIOS between QEMU and Bochs
o The kernel bits will never go upstream for Linux
o No one actively supports kqemu in upstream QEMU

That said, here are the arguments for keeping kqemu

o Even though it's unmaintained, it seems to work for people
o There is no alternative for non-Linux users and folks with non-VT/SVM
hardware

Regards,

Anthony Liguori

#2 lore raw
Quoted 23 lines · Anthony Liguori
Quoted 22 lines · Glauber Costa

Have you ever seen a girl so beautiful that you, geeky,
think: "I'll never stand a chance"?

But sometimes, you decide to make your move anyway. There's always the chance that in that very day she'll be specially in good mood, and you'll get what you want.

With the exception of the fact that qemu is not a girl, that's more or less what I'm trying to do here: Hopefully, nobody will notice what I'm trying to do, and will commmit it. Later, when realizing, it will be too late. Victory will be mine.

Or maybe people will even agree. For that, I'll try briefly to arguee my point, without disclosing to much, avoiding jeopardizing the strategy I explained above:

This patch removes a piece of code that is unmaintaned, that does not receive an update for years,
that get bug reports on the list that nobody fixes, because nobody really understands,
that places some artificial constraints on other subsystems

Quoted 7 lines
Quoted 1 line

Let's actually build a proper case instead of closing our eyes and hitting enter. Here are the downsides of kqemu I know of:

o Since it's enabled by default, it forces the default build to support
< 4GB of guest memory

Making -no-kqemu the default appears as a reasonable first step then - to kill those silly "Could not open '/dev/kqemu'" warnings) and also to collect complains like: "What the heck happened to kqemu?"

Quoted 7 lines

o It attempts to use /dev/shm for guest memory which means a special
option is needed in the default build to use more than 1/2 of host ram size
o It touches an awful lot of places in QEMU
o Some of the BIOS changes are particularly nasty and will prevent
having a unified BIOS between QEMU and Bochs
o The kernel bits will never go upstream for Linux
o No one actively supports kqemu in upstream QEMU

We did some work on it a few months ago, trying to enhance its support for segmented guests. It turned out to require unreasonable effort and would still perform not significantly better than plain qemu in this context (and our customer dropped the idea to support legacy systems anyway). The results are a few low-level fixes and enhancements (that I still want to post once cleaned up) and the confirmation of what is likely already clear to people who had a look at the kernel bits: They are almost unmaintainable and can cause severe headache when trying to understand them.

Quoted 4 lines

That said, here are the arguments for keeping kqemu

o Even though it's unmaintained, it seems to work for people

At some point, I bet, at least the Linux bindings will break, and no one will be interested or able to fix that anymore. Same may happen to other platforms (doesn't Windows 7 come with a new driver model?).

Quoted 2 lines

o There is no alternative for non-Linux users and folks with non-VT/SVM
hardware

The non-HVM argument will become widely irrelevant (for desktops) very soon. The non-Linux issue will likely persist - unless someone feels so much pain to write some KVM for those platforms. But as long as there is a kqemu version that builds and works for them, I think we should keep QEMU's support. But it should no longer be a first-class citizen: off by default, factored out into more hooks, maybe even de-optimized where it blocks development or increases the maintenance effort of QEMU.

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
#3 lore raw
Quoted 31 lines · Jan Kiszka
Quoted 29 lines · Anthony Liguori
Quoted 16 lines · Glauber Costa

Have you ever seen a girl so beautiful that you, geeky,
think: "I'll never stand a chance"?

But sometimes, you decide to make your move anyway. There's always the chance that in that very day she'll be specially in good mood, and you'll get what you want.

With the exception of the fact that qemu is not a girl, that's more or less what I'm trying to do here: Hopefully, nobody will notice what I'm trying to do, and will commmit it. Later, when realizing, it will be too late. Victory will be mine.

Or maybe people will even agree. For that, I'll try briefly to arguee my point, without disclosing to much, avoiding jeopardizing the strategy I explained above:

Quoted 8 lines

This patch removes a piece of code that is unmaintaned, that does not receive an update for years,
that get bug reports on the list that nobody fixes, because nobody really understands,
that places some artificial constraints on other subsystems

Let's actually build a proper case instead of closing our eyes and hitting enter. Here are the downsides of kqemu I know of:

Quoted 8 lines
Quoted 3 lines

o Since it's enabled by default, it forces the default build to support
< 4GB of guest memory

Making -no-kqemu the default appears as a reasonable first step then - to kill those silly "Could not open '/dev/kqemu'" warnings) and also to collect complains like: "What the heck happened to kqemu?"

Yes. Note that -no-kqemu doesn't fix the above complaint but it fixes the following one. So unless there are major objections, I'd like to make -no-kqemu the default for 0.11. We can then discuss whether to make kqemu deprecated and scheduled for removal in 0.12.

Quoted 34 lines
Quoted 8 lines

o It attempts to use /dev/shm for guest memory which means a special
option is needed in the default build to use more than 1/2 of host ram size
o It touches an awful lot of places in QEMU
o Some of the BIOS changes are particularly nasty and will prevent
having a unified BIOS between QEMU and Bochs
o The kernel bits will never go upstream for Linux
o No one actively supports kqemu in upstream QEMU

We did some work on it a few months ago, trying to enhance its support for segmented guests. It turned out to require unreasonable effort and would still perform not significantly better than plain qemu in this context (and our customer dropped the idea to support legacy systems anyway). The results are a few low-level fixes and enhancements (that I still want to post once cleaned up) and the confirmation of what is likely already clear to people who had a look at the kernel bits: They are almost unmaintainable and can cause severe headache when trying to understand them.

Quoted 4 lines

That said, here are the arguments for keeping kqemu

o Even though it's unmaintained, it seems to work for people

At some point, I bet, at least the Linux bindings will break, and no one will be interested or able to fix that anymore. Same may happen to other platforms (doesn't Windows 7 come with a new driver model?).

Quoted 3 lines

o There is no alternative for non-Linux users and folks with non-VT/SVM
hardware

Quoted 8 lines

The non-HVM argument will become widely irrelevant (for desktops) very soon. The non-Linux issue will likely persist - unless someone feels so much pain to write some KVM for those platforms. But as long as there is a kqemu version that builds and works for them, I think we should keep QEMU's support. But it should no longer be a first-class citizen: off by default, factored out into more hooks, maybe even de-optimized where it blocks development or increases the maintenance effort of QEMU.

If we disable in configure, then we should remove it from the tree. The feeling is that code that's disabled by default is too likely to bitrot.

I think you've made a reasonable suggestion though. So unless there are strong feelings otherwise, I think we should do -no-kqemu by default for 0.11, see what the reaction is, then figure out whether we want to deprecate/remove.

Regards,

Anthony Liguori

Quoted 3 lines

Jan

#4 lore raw
Quoted 101 lines · Anthony Liguori
Quoted 34 lines · Jan Kiszka
Quoted 31 lines · Anthony Liguori
Quoted 16 lines · Glauber Costa

Have you ever seen a girl so beautiful that you, geeky,
think: "I'll never stand a chance"?

But sometimes, you decide to make your move anyway. There's always the chance that in that very day she'll be specially in good mood, and you'll get what you want.

With the exception of the fact that qemu is not a girl, that's more or less what I'm trying to do here: Hopefully, nobody will notice what I'm trying to do, and will commmit it. Later, when realizing, it will be too late. Victory will be mine.

Or maybe people will even agree. For that, I'll try briefly to arguee my point, without disclosing to much, avoiding jeopardizing the strategy I explained above:

Quoted 9 lines

This patch removes a piece of code that is unmaintaned, that does not receive an update for years,
that get bug reports on the list that nobody fixes, because nobody really understands,
that places some artificial constraints on other subsystems

Let's actually build a proper case instead of closing our eyes and hitting enter. Here are the downsides of kqemu I know of:

Quoted 9 lines
Quoted 4 lines

o Since it's enabled by default, it forces the default build to support
< 4GB of guest memory

Making -no-kqemu the default appears as a reasonable first step then - to kill those silly "Could not open '/dev/kqemu'" warnings) and also to collect complains like: "What the heck happened to kqemu?"

Yes. Note that -no-kqemu doesn't fix the above complaint but it fixes the following one. So unless there are major objections, I'd like to make -no-kqemu the default for 0.11. We can then discuss whether to make kqemu deprecated and scheduled for removal in 0.12.

Quoted 36 lines
Quoted 9 lines

o It attempts to use /dev/shm for guest memory which means a special
option is needed in the default build to use more than 1/2 of host ram size
o It touches an awful lot of places in QEMU
o Some of the BIOS changes are particularly nasty and will prevent
having a unified BIOS between QEMU and Bochs
o The kernel bits will never go upstream for Linux
o No one actively supports kqemu in upstream QEMU

We did some work on it a few months ago, trying to enhance its support for segmented guests. It turned out to require unreasonable effort and would still perform not significantly better than plain qemu in this context (and our customer dropped the idea to support legacy systems anyway). The results are a few low-level fixes and enhancements (that I still want to post once cleaned up) and the confirmation of what is likely already clear to people who had a look at the kernel bits: They are almost unmaintainable and can cause severe headache when trying to understand them.

Quoted 5 lines

That said, here are the arguments for keeping kqemu

o Even though it's unmaintained, it seems to work for people

At some point, I bet, at least the Linux bindings will break, and no one will be interested or able to fix that anymore. Same may happen to other platforms (doesn't Windows 7 come with a new driver model?).

Quoted 4 lines

o There is no alternative for non-Linux users and folks with non-VT/SVM
hardware

Quoted 9 lines

The non-HVM argument will become widely irrelevant (for desktops) very soon. The non-Linux issue will likely persist - unless someone feels so much pain to write some KVM for those platforms. But as long as there is a kqemu version that builds and works for them, I think we should keep QEMU's support. But it should no longer be a first-class citizen: off by default, factored out into more hooks, maybe even de-optimized where it blocks development or increases the maintenance effort of QEMU.

If we disable in configure, then we should remove it from the tree. The feeling is that code that's disabled by default is too likely to bitrot.

Quoted 14 lines

I think you've made a reasonable suggestion though. So unless there are strong feelings otherwise, I think we should do -no-kqemu by default for 0.11, see what the reaction is, then figure out whether we want to deprecate/remove.

Regards,

Anthony Liguori

Quoted 4 lines

Jan

Default setting -no-kqemu is ok for all platforms without kqemu support or with working kvm support.

For Win32, I prefer to have kqemu support enabled by default.

Regards,

Stefan Weil

#5 lore raw
In reply to Anthony Liguori
Quoted 7 lines · Anthony Liguori

If we disable in configure, then we should remove it from the tree. The feeling is that code that's disabled by default is too likely to bitrot.

I think you've made a reasonable suggestion though. So unless there are strong feelings otherwise, I think we should do -no-kqemu by default for 0.11, see what the reaction is, then figure out whether we want to deprecate/remove.

One option that has not been suggested is building binaries both with and without kqemu enabled by default. This would allow limitations that are imposed by enabling kqemu to be avoided, but wouldn't help to avoid any limitations imposed by the mere existence of kqemu support in the code. It also gives the user an extra binary that they shouldn't really have to know or care about, and prevents any "what happened to kqemu?!" feedback from being received.

However, as an intermediate step, perhaps it might be worth considering.

Either way, kqemu will clearly disappear at some point, unless someone with the required knowledge, skill, time, interest and pervasiveness intervenes.

Silly question, btw -- I've heard on several occasions that kqemu is not auditable. Is it even possible to produce a replacement with reasonable performance that *is* auditable?

(Whilst I certainly have the interest, I'm not sure whether I possess a sufficient quantity of the other four required attributes to implement something like this for KVM, if such a thing is even possible.)

Cheers,

Signature · 1 line
Stuart Brady
#6 lore raw
Quoted 4 lines · Stuart Brady

Silly question, btw -- I've heard on several occasions that kqemu is not auditable. Is it even possible to produce a replacement with reasonable performance that *is* auditable?

Is it possible to produce a large scale system-level program (esp. with smp) that is auditable? I think not.

kqemu is particularly difficult because it interacts with the hardware in complex ways.

Quoted 3 lines

(Whilst I certainly have the interest, I'm not sure whether I possess a sufficient quantity of the other four required attributes to implement something like this for KVM, if such a thing is even possible.)

It's technically possible, but I don't think it's realistic. No one wants to code for yesterday's hardware; every day there are fewer machines that need kqemu.

If you're interested in virtualization, but a cpu that supports virtualization. If you want to keep your old cpu, keep your old software as well.

Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
#7 lore raw
In reply to Jan Kiszka
Quoted 5 lines · Jan Kiszka
Quoted 2 lines · Anthony Liguori

o There is no alternative for non-Linux users and folks with non-VT/SVM
hardware

Quoted 7 lines

The non-HVM argument will become widely irrelevant (for desktops) very soon. The non-Linux issue will likely persist - unless someone feels so much pain to write some KVM for those platforms. But as long as there is a kqemu version that builds and works for them, I think we should keep QEMU's support. But it should no longer be a first-class citizen: off by default, factored out into more hooks, maybe even de-optimized where it blocks development or increases the maintenance effort of QEMU.

The non-HVM argument will always exist, and could actually get worse if lots of vendors start shipping with embedded hypervisors, such that your 'real' OS is already virtualized behind your back. That said I do agree with you - we're still better off killing/deprecating kqemu and using any spare effort to improve nested-SVM/VT support which is a more useful long term feature.

Regards,
Daniel

Signature · 4 lines
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
#8 lore raw
Quoted 13 lines · Daniel P. Berrange
Quoted 11 lines · Jan Kiszka
Quoted 2 lines · Anthony Liguori

o There is no alternative for non-Linux users and folks with non-VT/SVM
hardware

The non-HVM argument will become widely irrelevant (for desktops) very soon. The non-Linux issue will likely persist - unless someone feels so much pain to write some KVM for those platforms. But as long as there is a kqemu version that builds and works for them, I think we should keep QEMU's support. But it should no longer be a first-class citizen: off by default, factored out into more hooks, maybe even de-optimized where it blocks development or increases the maintenance effort of QEMU.

Quoted 3 lines

The non-HVM argument will always exist, and could actually get worse if lots of vendors start shipping with embedded hypervisors, such that your 'real' OS is already virtualized behind your back. That said I do

Nested virtualization like this would have been one nice-to-have via kqemu. And in fact, most of our development took place inside kvm. But the performance was worse than qemu inside kvm, at least for our use cases.

That said, there might be scenarios where kqemu inside whatever hypervisor provides better performance than emulation, but I can only recommend to evaluate this carefully. The overhead of virtualizing kqemu's monitor activities can be enormous.

Quoted 4 lines

agree with you - we're still better off killing/deprecating kqemu and using any spare effort to improve nested-SVM/VT support which is a more useful long term feature.

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
#9 lore raw
In reply to Jan Kiszka
Quoted 29 lines · Jan Kiszka
Quoted 28 lines · Anthony Liguori
Quoted 16 lines · Glauber Costa

Have you ever seen a girl so beautiful that you, geeky,
think: "I'll never stand a chance"?

But sometimes, you decide to make your move anyway. There's always the chance that in that very day she'll be specially in good mood, and you'll get what you want.

With the exception of the fact that qemu is not a girl, that's more or less what I'm trying to do here: Hopefully, nobody will notice what I'm trying to do, and will commmit it. Later, when realizing, it will be too late. Victory will be mine.

Or maybe people will even agree. For that, I'll try briefly to arguee my point, without disclosing to much, avoiding jeopardizing the strategy I explained above:

Quoted 7 lines

This patch removes a piece of code that is unmaintaned, that does not receive an update for years,
that get bug reports on the list that nobody fixes, because nobody really understands,
that places some artificial constraints on other subsystems

Let's actually build a proper case instead of closing our eyes and hitting enter. Here are the downsides of kqemu I know of:

Quoted 6 lines
Quoted 2 lines

o Since it's enabled by default, it forces the default build to support
< 4GB of guest memory

Making -no-kqemu the default appears as a reasonable first step then - to kill those silly "Could not open '/dev/kqemu'" warnings) and also to collect complains like: "What the heck happened to kqemu?"

We have at least one _stable_ release at this point that includes kqemu. So why not just remove it for the next one?

Quoted 11 lines
Quoted 3 lines

o There is no alternative for non-Linux users and folks with non-VT/SVM
hardware

The non-HVM argument will become widely irrelevant (for desktops) very soon. The non-Linux issue will likely persist - unless someone feels so much pain to write some KVM for those platforms. But as long as there is a kqemu version that builds and works for them, I think we should keep QEMU's support. But it should no longer be a first-class citizen: off by default, factored out into more hooks, maybe even de-optimized where it blocks development or increases the maintenance effort of QEMU.

Any change we make to decrease its citizenship status will likely break it very fast. Reality is noone here really understands it.

#10 lore raw
In reply to Jan Kiszka
Quoted 9 lines
Quoted 3 lines

That said, here are the arguments for keeping kqemu

o Even though it's unmaintained, it seems to work for people

At some point, I bet, at least the Linux bindings will break, and no one
will be interested or able to fix that anymore. Same may happen to other
platforms (doesn't Windows 7 come with a new driver model?).

Yes and MS even made supplications to hw vendors to write drivers for it, as they got slapped by their own monopoly practices :D Instead they should just ask them to release specs so everyone can write drivers for their own OS and restore fair competition...

Quoted 6 lines
Quoted 3 lines

o There is no alternative for non-Linux users and folks with non-
VT/SVM
hardware

The non-HVM argument will become widely irrelevant (for desktops) very

Hmm not everyone has the money to renew their hw every year or so. I still have an AthlonXP and a PentiumM based laptop here, which do work fine.

Quoted 4 lines

soon. The non-Linux issue will likely persist - unless someone feels so
much pain to write some KVM for those platforms. But as long as there is

<rant reason="Sorry you just cought me on a bad day">
Well, some FOSS devs have a tendancy those years to act like proprietary devs, disregarding other OSes as "non existant, not relevant" and so "not worth caring", which is both quite irritating and wrong, since many of those actually account for the technodiversity necessary to keep "innovation" going. I still remember all the buzz I read about Linux getting "tickless", wow, I mean like, BeOS had it 10 years ago (and Irix probably also but it wasn't really desktop oriented).

Just like ALSA, which is written by Linux, for Linux, without everyone else in mind, discrediting OSS API, which actually is defacto std on UNIX, and making it unportable to anything else.

Maybe those things like KVM could be written in a portable way... OSSv4 proves kernel code can be written in a portable way, despite them having to maintain a huge ugly kludge to account for the total lack of a stable DDM API in Linux... and again the total disregard from Linux devs dismissing the problem as "you aren't in the kernel tree, you don't exist". Of course they wouldn't include OSSv4 in the tree since it's meant to be portable anyway.

Still, Haiku proves one can go forward yet have a stable driver API. the OSSv4 BeOS port runs fine in Haiku :
http://revolf.free.fr/Alchimie-7/Alchimie7_OSS_Haiku.en.pdf
yet we have a new DDM, bluetooth support, ...

</rant>

Couldn't they just write their KVM code cleanly ?

François.

#11 lore raw
Quoted 51 lines · François Revol
Quoted 8 lines
Quoted 3 lines

That said, here are the arguments for keeping kqemu

o Even though it's unmaintained, it seems to work for people

At some point, I bet, at least the Linux bindings will break, and no one
will be interested or able to fix that anymore. Same may happen to other
platforms (doesn't Windows 7 come with a new driver model?).

Yes and MS even made supplications to hw vendors to write drivers for it, as they got slapped by their own monopoly practices :D Instead they should just ask them to release specs so everyone can write drivers for their own OS and restore fair competition...

Quoted 5 lines
Quoted 3 lines

o There is no alternative for non-Linux users and folks with non-
VT/SVM
hardware

The non-HVM argument will become widely irrelevant (for desktops) very

Hmm not everyone has the money to renew their hw every year or so. I still have an AthlonXP and a PentiumM based laptop here, which do work fine.

Quoted 4 lines

soon. The non-Linux issue will likely persist - unless someone feels so
much pain to write some KVM for those platforms. But as long as there is

<rant reason="Sorry you just cought me on a bad day">
Well, some FOSS devs have a tendancy those years to act like proprietary devs, disregarding other OSes as "non existant, not relevant" and so "not worth caring", which is both quite irritating and wrong, since many of those actually account for the technodiversity necessary to keep "innovation" going. I still remember all the buzz I read about Linux getting "tickless", wow, I mean like, BeOS had it 10 years ago (and Irix probably also but it wasn't really desktop oriented).

Just like ALSA, which is written by Linux, for Linux, without everyone else in mind, discrediting OSS API, which actually is defacto std on UNIX, and making it unportable to anything else.

Maybe those things like KVM could be written in a portable way... OSSv4 proves kernel code can be written in a portable way, despite them having to maintain a huge ugly kludge to account for the total lack of a stable DDM API in Linux... and again the total disregard from Linux devs dismissing the problem as "you aren't in the kernel tree, you don't exist". Of course they wouldn't include OSSv4 in the tree since it's meant to be portable anyway.

Quoted 8 lines

Still, Haiku proves one can go forward yet have a stable driver API. the OSSv4 BeOS port runs fine in Haiku :
http://revolf.free.fr/Alchimie-7/Alchimie7_OSS_Haiku.en.pdf
yet we have a new DDM, bluetooth support, ...

</rant>

Couldn't they just write their KVM code cleanly ?

Rant back: If you contribute to the KVM project, you would have a chance to influence its direction (always given that you provide a corresponding added value). But plain ranting doesn't change a single bit. That's how open source works.

Jan

#12 lore raw
In reply to Jan Kiszka
Quoted 5 lines · Jan Kiszka
Quoted 2 lines

o There is no alternative for non-Linux users and folks with non-VT/SVM
hardware

The non-HVM argument will become widely irrelevant (for desktops) very soon.

Only if your looking at the virtualisation market. As far as I know

- There are no non-Intel, non-AMD CPUs which support HVM
- Laptops are outselling desktops these days
- Little low-power PCs are gaining in popularity at home

Personally I have HVM on one of my laptops but not the other. I have several "media player" nano-ITX PCs, and I don't think any of them support HVM, even the Intel ones. I have one dual-AMD desktop machine which does not have HVM. I have access to two big Intel Xeon servers. Only one has HVM, and annoyingly the faster one does not have HVM, even though it's 64-bit.

That's only 2 out 7 PC types I have ready access to which have HVM.

I do understand the wish to drop KQEMU, and I understand the wish of the staff developers to not want to spend time supporting platforms they aren't using themselves, and aren't their customers. Especially when it complicates the code base.

But it will make QEMU a less useful tool for some.

I suspect if QEMU development was still a "hobby" project, the "coolness factor" of being able to do things like KQEMU would win over "target market" rule which I guess is more motivating for paid developers.

Anyway, we had this discussion before and the obvious conclusion was that the only viable way to keep KQEMU is if there are volunteers stepping up to maintain it, both the userspace and kernel portions.

Or ideally, to replace it with something KVM-compatible, as that would reduce the maintenance burden.

For replacing KQEMU on Linux, that's realistic I think. Either it's done, or there's no maintainer anyway.

But for non-Linux hosts I see a non-technical problem:

- Without API documentation you have to read the KVM source code.

- Those goes doubly so for the fiddly bits like kernel APIC emulation and virtio.

- But it may not be legally safe to read the KVM source code and reimplement it in such detail for a GPL-incompatible target kernel. (It might be seen as a form of translation).

(We've already seen someone implementing a virtio driver on the list who was worried about GPL implications).

So how can anyone implement a KVM-compatible replacement for KQEMU on other host platforms?

-- Jamie

#13 lore raw
Quoted 6 lines · Jamie Lokier
Quoted 4 lines · Jan Kiszka
Quoted 2 lines

o There is no alternative for non-Linux users and folks with non-VT/SVM
hardware

The non-HVM argument will become widely irrelevant (for desktops) very soon.

Quoted 6 lines

Only if your looking at the virtualisation market. As far as I know

- There are no non-Intel, non-AMD CPUs which support HVM
- Laptops are outselling desktops these days
- Little low-power PCs are gaining in popularity at home

VIA will soon (or already started to?) deliver working VT-x support with their CPUs. As HVM also became a marketing tag, probably only very resource-restricted systems will soon be sold without it, and on those kqemu is no fun anyway (at best only for very very resource-restricted guests).

Quoted 15 lines

Personally I have HVM on one of my laptops but not the other. I have several "media player" nano-ITX PCs, and I don't think any of them support HVM, even the Intel ones. I have one dual-AMD desktop machine which does not have HVM. I have access to two big Intel Xeon servers. Only one has HVM, and annoyingly the faster one does not have HVM, even though it's 64-bit.

That's only 2 out 7 PC types I have ready access to which have HVM.

I do understand the wish to drop KQEMU, and I understand the wish of the staff developers to not want to spend time supporting platforms they aren't using themselves, and aren't their customers. Especially when it complicates the code base.

Quoted 5 lines

But it will make QEMU a less useful tool for some.

I suspect if QEMU development was still a "hobby" project, the "coolness factor" of being able to do things like KQEMU would win over "target market" rule which I guess is more motivating for paid developers.

That wouldn't change any of the technical problems it has. If we had no broadly available HVM support today, /then/ you could bet that a kqemu rewrite would be on the way, either driven by hobbyists or commercial use.

Quoted 13 lines

Anyway, we had this discussion before and the obvious conclusion was that the only viable way to keep KQEMU is if there are volunteers stepping up to maintain it, both the userspace and kernel portions.

Or ideally, to replace it with something KVM-compatible, as that would reduce the maintenance burden.

For replacing KQEMU on Linux, that's realistic I think. Either it's done, or there's no maintainer anyway.

But for non-Linux hosts I see a non-technical problem:

Quoted 8 lines

- Without API documentation you have to read the KVM source code.

- Those goes doubly so for the fiddly bits like kernel APIC emulation and virtio.

- But it may not be legally safe to read the KVM source code and reimplement it in such detail for a GPL-incompatible target kernel. (It might be seen as a form of translation).

For that case (and Avi still haven written the API docs): Take (at least) two people, one reading the source and writing that docs and the others reading the docs and re-implementing a backend under whatever license they want.

Quoted 6 lines

(We've already seen someone implementing a virtio driver on the list who was worried about GPL implications).

So how can anyone implement a KVM-compatible replacement for KQEMU on other host platforms?

It's not that easy. The major design challenge is that kqemu is running in cooperative mode with TCG, using the latter to handle cases the in-kernel monitor cannot (or doesn't want to due to performance/latency impact). Kvm does not need this as its guest-host switches are less frequent and less heavy. We played with an approach to drive kqemu into the same direction (patch and trap those un-virtualizable instructions) and reduced the exists. The next step would have been to adopt kvm's I/O handling philosophy (or even code), but problems of our approach on heavily segmented guests stopped the effort.

There might have been another issue of the patch&trap approach:
self-checking guests (you can't easily hide the binary changes from the guest). Then the only option would have been to translate the guest code like TCG already does, but optimized for host = guest = x86. But that effort would have been enormous and simply no longer worth it.

That said, I think the best one can do for kqemu remaining a qemu citizen is to move it out of the way the latter wants to take, e.g. eliminating the reason for

/* address in the RAM (different from a physical address) */ #ifdef CONFIG_KQEMU
/* FIXME: This is wrong. */
typedef uint32_t ram_addr_t;
#else
typedef unsigned long ram_addr_t;
#endif

or reducing the impact of its hooks. One could, as another example, push the hooks in cpu_in/out to helpers injected into the TCG output.

Jan

#14 lore raw
In reply to Anthony Liguori
Quoted 23 lines · Anthony Liguori
Quoted 22 lines · Glauber Costa

Have you ever seen a girl so beautiful that you, geeky,
think: "I'll never stand a chance"?

But sometimes, you decide to make your move anyway. There's always the chance that in that very day she'll be specially in good mood, and you'll get what you want.

With the exception of the fact that qemu is not a girl, that's more or less what I'm trying to do here: Hopefully, nobody will notice what I'm trying to do, and will commmit it. Later, when realizing, it will be too late. Victory will be mine.

Or maybe people will even agree. For that, I'll try briefly to arguee my point, without disclosing to much, avoiding jeopardizing the strategy I explained above:

This patch removes a piece of code that is unmaintaned, that does not receive an update for years,
that get bug reports on the list that nobody fixes, because nobody really understands,
that places some artificial constraints on other subsystems

Quoted 4 lines
Quoted 1 line

Let's actually build a proper case instead of closing our eyes and hitting enter. Here are the downsides of kqemu I know of:

Okay... you do realize I was kidding, and I never really expected this to happen at first, right? ;-)

Quoted 11 lines

o Since it's enabled by default, it forces the default build to support
< 4GB of guest memory
o It attempts to use /dev/shm for guest memory which means a special
option is needed in the default build to use more than 1/2 of host ram size
o It touches an awful lot of places in QEMU
o Some of the BIOS changes are particularly nasty and will prevent
having a unified BIOS between QEMU and Bochs
o The kernel bits will never go upstream for Linux
o No one actively supports kqemu in upstream QEMU

Quoted 3 lines

That said, here are the arguments for keeping kqemu

o Even though it's unmaintained, it seems to work for people

But traffic in the mailing list indicates that it is less and less the case. And more importantly: As it bitrots, nobody fixes it. so...

Quoted 2 lines

o There is no alternative for non-Linux users and folks with non-VT/SVM
hardware

Sure, but I don't think kqemu is such an alternative. ;-)

#15 lore raw
Quoted 3 lines · Glauber Costa
Quoted 1 line

o Even though it's unmaintained, it seems to work for people

But traffic in the mailing list indicates that it is less and less the case. And more importantly: As it bitrots, nobody fixes it. so...

Data point: Running 64bit guests doesn't work stable for me. Guest starts booting up, but qemu aborts before it reaches the login prompt. 32bit guests seem to work ok, but I don't use it that much.

cheers,
Gerd

#16 lore raw
In reply to Glauber Costa
Quoted 7 lines · Glauber Costa

But traffic in the mailing list indicates that it is less and less the case. And more importantly: As it bitrots, nobody fixes it. so...

Quoted 2 lines

o There is no alternative for non-Linux users and folks with non-VT/SVM
hardware

Sure, but I don't think kqemu is such an alternative. ;-)

It is. Without kqemu I'll have no alternative, but switch to VBox or VMWare. And none of them work with qcow images.

#17 lore raw
Quoted 11 lines · Consul
Quoted 7 lines · Glauber Costa

But traffic in the mailing list indicates that it is less and less the case. And more importantly: As it bitrots, nobody fixes it. so...

Quoted 2 lines

o There is no alternative for non-Linux users and folks with non-VT/SVM
hardware

Sure, but I don't think kqemu is such an alternative. ;-)

It is. Without kqemu I'll have no alternative, but switch to VBox or VMWare. And none of them work with qcow images.

I'm not saying that because of any kind of functionality. I'm saying it is not an alternative, because nobody really maintains it. It's bitrotting, and unless someone wants to step up, soon or later you won't be able to use it anyway.

#18 lore raw

Am 29.05.2009 um 20:49 schrieb Glauber Costa:

Quoted 8 lines · Consul
Quoted 7 lines
Quoted 6 lines · Glauber Costa

But traffic in the mailing list indicates that it is less and less the case. And more importantly: As it bitrots, nobody fixes it. so...

Quoted 2 lines

o There is no alternative for non-Linux users and folks with non-VT/SVM hardware

Quoted 6 lines
Quoted 4 lines
Quoted 1 line

Sure, but I don't think kqemu is such an alternative. ;-)

It is. Without kqemu I'll have no alternative, but switch to VBox or VMWare. And none of them work with qcow images.

I'm not saying that because of any kind of functionality. I'm saying it is not an alternative, because nobody really maintains it.

KVM is no alternative either for us because nobody really maintains documentation of what interface and functionality we'd have to implement on Solaris, Windows, Mac OS X etc. where no KVM is available today.

I'd expect some documentation here: http://www.linux-kvm.org/page/Documents
-- but no luck. No "Porting KVM For Dummies". ;(

I see no problem dumping kqemu support once KVM is available on the kqemu platforms, but now is really a bad point in time.

Andreas

#19 lore raw
Quoted 22 lines · Andreas Färber

Am 29.05.2009 um 20:49 schrieb Glauber Costa:

Quoted 10 lines · Consul
Quoted 9 lines
Quoted 8 lines · Glauber Costa

But traffic in the mailing list indicates that it is less and less the case.
And more importantly: As it bitrots, nobody fixes it. so...

Quoted 3 lines

o There is no alternative for non-Linux users and folks with non-VT/SVM
hardware

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

Sure, but I don't think kqemu is such an alternative. ;-)

It is. Without kqemu I'll have no alternative, but switch to VBox or VMWare.
And none of them work with qcow images.

I'm not saying that because of any kind of functionality. I'm saying it is
not an alternative, because nobody really maintains it.

Quoted 7 lines

KVM is no alternative either for us because nobody really maintains documentation of what interface and functionality we'd have to implement on Solaris, Windows, Mac OS X etc. where no KVM is available today.

I'd expect some documentation here:
http://www.linux-kvm.org/page/Documents -- but no luck. No "Porting KVM For Dummies". ;(

There was and likely will never be any documentation for kqemu either. But having worked on both, I can say that kvm is by an order of magnitude easier to understand from its sources than kqemu.

Sorry, but if you are really interested in such a port, having to dig through source code and ask questions to the community if something remains unclear should not be the actual problem.

I've once ported the whole Linux IrDA stack to Windows (as its own stack just sucked). There was no real design document at that point too, but well structured code and a very responsive community - which as quite happy about patches against bugs I found running it "on the evil side".

Quoted 3 lines

I see no problem dumping kqemu support once KVM is available on the kqemu platforms, but now is really a bad point in time.

Take it the other way around: This (yet only discussed!) step comes with the chance to trigger such a development.

Jan

#20 lore raw

Am 31.05.2009 um 11:15 schrieb Jan Kiszka:

Quoted 11 lines
Quoted 8 lines · Andreas Färber

KVM is no alternative either for us because nobody really maintains documentation of what interface and functionality we'd have to implement on Solaris, Windows, Mac OS X etc. where no KVM is available today.

I'd expect some documentation here:
http://www.linux-kvm.org/page/Documents -- but no luck. No "Porting KVM For Dummies". ;(

There was and likely will never be any documentation for kqemu either.

And I'm not complaining about kqemu. No one has suggested me to start new ports of kqemu.

Quoted 6 lines

But having worked on both, I can say that kvm is by an order of magnitude easier to understand from its sources than kqemu.

Sorry, but if you are really interested in such a port, having to dig through source code and ask questions to the community if something remains unclear should not be the actual problem.

It is. My using the kqemu accelerator neither makes me an expert on x86 architecture nor on any sort of kernel modules. (Just like some understanding of TCG and certain other instruction sets doesn't make me too knowledgeable about IRQ, DMA, MMU, TLB and all the other TLAs)

The porting issues start with very basic considerations: IIUC the relevant part of KVM porting-wise is the kernel side (assuming that sufficient userspace code has arrived in upstream QEMU already), which by nature of Linux and according to the FAQ is GPL'ed. Meaning that if I attempted to port KVM to a new platform, the kernel module, Kernel Extension, daemon, Service, driver or whatever form of implementation may be needed on the target OS would be required to be GPL'ed as well. Can I create a GPL'ed Kernel Extension for Mac OS X? Questionable, since it would probably end up in the same address space as proprietary/non-GPL code. Similar potential issues with CDDL'ed (Open)Solaris kernel, BSD kernels, MIT/X11'ed Haiku kernel and fully closed-source Windows. So, to be on the safe side, I can't start implementing KVM by reading the KVM source code but would need documentation of the interface that needs to be implemented (cf. virtio drivers). If KVM wants to be the one standardized interface instead of one of three implementations then it needs formalization. There are contradicting answers on whether non-GPL code may legally dynamically load GPL code, making code and inline comments not necessarily a sufficient replacement for proper documentation. Especially if it's not my personal wish, but KVM developers trying to force people to port their software in order to make their life easier.

Arguing about Open Source, you are forgetting that not everyone gets paid for their Open Source contributions to QEMU. If you work on KVM/QEMU for a living and have the time to work on large'ish porting or refactoring projects, that's one facet of Open Source only.
Many other people aren't selling but merely using QEMU, be it as a tool for (Open Source) application or operating system development or for research, because they favor it as a FOSS solution, contributing bugfixes and to ongoing platform support. These are often not the same people building QEMU on a per-commit basis, testing every apparently unrelated patch before it gets committed and breaks things for them, and are thus hit the most when some member of the now highly active KVM community decides to shuffle features around or to drop something in some upcoming release for valid reasons. Are they really "bad" community members? Should they better get lost, let you do your KVM work and go buy closed-source VMware instead? Or let partially open source VirtualBox freeze their system and ruin their Superdrive? I don't think so. Having flourishing KVM code inside QEMU is totally fine, but pointing the pistol at others, asking to port KVM to a new platform with a fixed timeframe and without documentation, even if KVM developers happen to dislike kqemu hooks in the sources and QEMU maintainers - being on Linux mainly - have turned to KVM, is not okay. The problem is not unwillingness, but rather time, information, skills, communicativeness and politeness. It makes a major difference if someone steps up and posts on qemu-devel, "hey, we'd like to replace kqemu with KVM, let's talk about how and when we can make this happen", rather than submitting a patch rendering the kqemu kernel module unusable, leaving the majority of its current users standing in the rain without replacement accelerator. That's got little to do with the spirit of Open Source.

I have a dream of something like the Open Group specification, which describes a virtualization standard in terms of required C types, preprocessor defines, functions, valid parameters, expected return values etc. It would allow multiple source-compatible but differing implementations and would in theory even allow to consolidate the current virtualization island solutions into sane, interoperable solutions. Is this too far from KVM reality?

Andreas

P.S. The "KVM on BSD" page (http://www.linux-kvm.org/page/BSD) does in fact instruct people to use kqemu!

#21 lore raw
Quoted 1 line · Andreas Färber

I have a dream of something like the Open Group specification, which describes a virtualization standard in terms of required C types, preprocessor defines, functions, valid parameters, expected return values etc. It would allow multiple source-compatible but differing implementations and would in theory even allow to consolidate the current virtualization island solutions into sane, interoperable solutions. Is this too far from KVM reality?

Aren't those things standardized only after everyone loses interest in them?

It's on my TODO to document the kvm interfaces. Unfortunately documentation jobs tend to stay low on the priority list. If someone wants to volunteer to do this I will render them any assistance I can.

Signature · 1 line
error compiling committee.c: too many arguments to function
#22 lore raw

In message: <[email protected]>
Avi Kivity <[email protected]> writes:
: Andreas Färber wrote:
: > I have a dream of something like the Open Group specification, which : > describes a virtualization standard in terms of required C types, : > preprocessor defines, functions, valid parameters, expected return : > values etc. It would allow multiple source-compatible but differing : > implementations and would in theory even allow to consolidate the : > current virtualization island solutions into sane, interoperable : > solutions. Is this too far from KVM reality?
:
: Aren't those things standardized only after everyone loses interest in them? :
: It's on my TODO to document the kvm interfaces. Unfortunately : documentation jobs tend to stay low on the priority list. If someone : wants to volunteer to do this I will render them any assistance I can.

Documenting is a good way to learn interfaces as well. Especially if you can get people who know what they really do (or should do) review your initial idea of what's going on... :)

Warner

#23 lore raw
In reply to Andreas Färber
Quoted 9 lines · Andreas Färber

Am 31.05.2009 um 11:15 schrieb Jan Kiszka:

Quoted 3 lines

Sorry, but if you are really interested in such a port, having to dig through source code and ask questions to the community if something remains unclear should not be the actual problem.

It is. My using the kqemu accelerator neither makes me an expert on x86 architecture nor on any sort of kernel modules. (Just like some understanding of TCG and certain other instruction sets doesn't make me too knowledgeable about IRQ, DMA, MMU, TLB and all the other TLAs)

A certain level of experience (or drive to build it up) is required for this. This will never be a "mechanical" porting job (for the same reason, kqemu is doomed to die once things seriously start to fall apart and there is still no one to dig deeper into it).

Quoted 13 lines

The porting issues start with very basic considerations: IIUC the relevant part of KVM porting-wise is the kernel side (assuming that sufficient userspace code has arrived in upstream QEMU already), which by nature of Linux and according to the FAQ is GPL'ed. Meaning that if I attempted to port KVM to a new platform, the kernel module, Kernel Extension, daemon, Service, driver or whatever form of implementation may be needed on the target OS would be required to be GPL'ed as well. Can I create a GPL'ed Kernel Extension for Mac OS X? Questionable, since it would probably end up in the same address space as proprietary/non-GPL code. Similar potential issues with CDDL'ed (Open)Solaris kernel, BSD kernels, MIT/X11'ed Haiku kernel and fully closed-source Windows.

Kqemu is already GPLv2, so the picture would not really change. And unless the license of the target OS does not exclude binding against GPL'ed drivers, there is no problem to be expected.

Quoted 11 lines

So, to be on the safe side, I can't start
implementing KVM by reading the KVM source code but would need documentation of the interface that needs to be implemented (cf. virtio drivers). If KVM wants to be the one standardized interface instead of one of three implementations then it needs formalization. There are contradicting answers on whether non-GPL code may legally dynamically load GPL code, making code and inline comments not necessarily a sufficient replacement for proper documentation. Especially if it's not my personal wish, but KVM developers trying to force people to port their software in order to make their life easier.

Quoted 4 lines

Arguing about Open Source, you are forgetting that not everyone gets paid for their Open Source contributions to QEMU. If you work on KVM/QEMU for a living and have the time to work on large'ish porting or refactoring projects, that's one facet of Open Source only.

Like many others, I'm doing both, ie. not all of my open source work is directly paid.

Quoted 17 lines

Many other people aren't selling but merely using QEMU, be it as a tool for (Open Source) application or operating system development or for research, because they favor it as a FOSS solution, contributing bugfixes and to ongoing platform support. These are often not the same people building QEMU on a per-commit basis, testing every apparently unrelated patch before it gets committed and breaks things for them, and are thus hit the most when some member of the now highly active KVM community decides to shuffle features around or to drop something in some upcoming release for valid reasons. Are they really "bad" community members? Should they better get lost, let you do your KVM work and go buy closed-source VMware instead? Or let partially open source VirtualBox freeze their system and ruin their Superdrive? I don't think so. Having flourishing KVM code inside QEMU is totally fine, but pointing the pistol at others, asking to port KVM to a new platform with a fixed timeframe and without documentation, even if KVM developers happen to dislike kqemu hooks in the sources and QEMU maintainers - being on Linux mainly - have turned to KVM, is not okay. The problem is

Quoted 8 lines

not unwillingness, but rather time, information, skills, communicativeness and politeness. It makes a major difference if someone steps up and posts on qemu-devel, "hey, we'd like to replace kqemu with KVM, let's talk about how and when we can make this happen", rather than submitting a patch rendering the kqemu kernel module unusable, leaving the majority of its current users standing in the rain without replacement accelerator. That's got little to do with the spirit of Open Source.

As we are doing open source, we do have this discussion here and now, we do consider options how to evolve qemu best (instead of just sticking *forever* with legacy support that is no longer making progress and is starting to cause troubles), and we do not just discontinue it like it may happen with some commercial software.

Recall: Removing kqemu did not make into the repos, it did not even make it on the agenda for 0.11. So there will like be at least another year to look for alternatives on other platforms + the ability to use stable 0.11 even longer - at least as long as kqemu works.

Quoted 8 lines

I have a dream of something like the Open Group specification, which describes a virtualization standard in terms of required C types, preprocessor defines, functions, valid parameters, expected return values etc. It would allow multiple source-compatible but differing implementations and would in theory even allow to consolidate the current virtualization island solutions into sane, interoperable solutions. Is this too far from KVM reality?

As I tried to explain: All nice ideas also require someone to work on them or to pay for this or to otherwise convincingly motivate people to do this.

Quoted 6 lines

Andreas

P.S. The "KVM on BSD" page (http://www.linux-kvm.org/page/BSD) does in fact instruct people to use kqemu!

Should probably be fixed or even dropped (it suggests that on FreeBSD kqemu == kvm).

Jan

#24 lore raw

Am 31.05.2009 um 17:10 schrieb Jan Kiszka:

Quoted 8 lines
Quoted 5 lines · Andreas Färber

Can I create a GPL'ed Kernel Extension for Mac OS X? Questionable, since it would probably end up in the same address space as proprietary/non-GPL code. Similar potential issues with CDDL'ed (Open)Solaris kernel, BSD kernels, MIT/X11'ed Haiku kernel and fully closed-source Windows.

Kqemu is already GPLv2, so the picture would not really change.

I've extended the picture here - the kqemu platforms I'm aware of are Linux, Windows, Solaris, BSDs, BeOS. Mac OS X was not among them. And kqemu for Solaris is already partially CDDL in addition to GPLv2.

Quoted 8 lines
Quoted 5 lines

Arguing about Open Source, you are forgetting that not everyone gets paid for their Open Source contributions to QEMU. If you work on KVM/QEMU for a living and have the time to work on large'ish porting or refactoring projects, that's one facet of Open Source only.

Like many others, I'm doing both, ie. not all of my open source work is directly paid.

You, plural! :)

As a matter of fact, with KVM the pace of development has become hard to follow - within days there are hundreds of mails in your inbox.
My point being that it's nice if this leads to shining new cool features for all of us, but not so nice if suddenly put under the same pressure to deliver features. For example, compare a KVM upstream patch not being reviewed for a week to Cocoa being broken for a number of weeks during the DisplayState refactoring or ppc host being broken for months during TCG conversion - different paces.

Andreas