Add HAX support

PATCH Superseded Vincent Palatin 3 participants 7 messages Nov 17, 2016 – Dec 8, 2016 Kbuild

Share & tools

Vincent Palatin

author

2016-11-17 11:14 UTC #0 lore raw

v3 cover Add HAX support

I took a stab at trying to rebase/upstream the support for Intel HAXM. (Hardware Accelerated Execution Manager).
Intel HAX is kernel-based hardware acceleration module for Windows and MacOSX.

I have based my work on the last version of the source code I found:
the emu-2.2-release branch in the external/qemu-android repository as used by the Android emulator.
In patch 2/3, I have forward-ported the core HAX code from there. It has been modified to build and run along with the current code base. It has been simplifying by removing non-UG hardware support / Darwin support / Android-specific leftovers.

CAVEAT: this patchset is still RFC because in addition of potentially containing a few outdated constructs and a questionable mess in qemu_cpu_kick_thread(), there are 2 unsolved issues:
- in rare cases, the HAX kernel module asks us to emulate a MMIO access rather than just using the 'fast MMIO' path. This path is still not implemented (as it was in the original patch). It's unclear why/when this is happening (eg you can run a full linux image without hitting it but the default iPXE option ROM is hitting it in the Intel network driver).
- the MemoryListener implementation cannot remove a memory mapping (e.g. for digging the VGA 'hole' in the lowmem). This shortcoming breaks the std vga implementation but in the current HAX module API, I don't see any (documented) way of implementing this.

In patch 3/3, I have put the plumbing into the QEMU code base, I did some clean up there and it is reasonably intrusive: i.e.

 Makefile.target           |  1 +
 configure                 | 18 ++++++++++
 cpus.c                    | 87 ++++++++++++++++++++++++++++++++++++++++++++++-
 exec.c                    | 16 +++++++++
 hw/intc/apic_common.c     |  3 +-
 include/qom/cpu.h         |  5 +++
 include/sysemu/hw_accel.h |  9 +++++
 qemu-options.hx           | 11 ++++++
 target-i386/Makefile.objs |  4 +++
 vl.c                      | 15 ++++++--
 10 files changed, 164 insertions(+), 5 deletions(-)

The patch 1/3 just extracts from KVM specific header the cpu_synchronize_ functions that HAX is also using.

I have tested the end result on a Windows 10 Pro machine (with UG support) with the Intel HAXM module 6.0.4 and a large ChromiumOS x86_64 image to exercise various code paths. It looks stable.
I also did a quick regression testing of the integration by running a Linux build with KVM enabled.
Just running 'qemu-system-x86_64 -enable-hax' is broken due to the caveats described above.

Changes from v2 to v3:
- fix saving/restoring FPU registers as suggested by Paolo.
- fix Windows build on all targets as contributed by Stefan Weil.
- clean-up IO / MMIO emulation.
- more clean-up of emulation leftovers.

Changes from v1 to v2:
- fix all style issues in the original code to get it through checkpatch.pl.
- remove Darwin support, it was barely tested and not fully functional.
- remove the support for CPU without UG mode.
- fix most review comments

Vincent Palatin (3):
kvm: move cpu synchronization code
target-i386: Add Intel HAX files
Plumb the HAXM-based hardware acceleration support

 Makefile.target             |    1 +
 configure                   |   18 +
 cpus.c                      |   88 ++-
 exec.c                      |   16 +
 gdbstub.c                   |    1 +
 hax-stub.c                  |   39 ++
 hw/i386/kvm/apic.c          |    1 +
 hw/i386/kvmvapic.c          |    1 +
 hw/intc/apic_common.c       |    3 +-
 hw/misc/vmport.c            |    2 +-
 hw/ppc/pnv_xscom.c          |    2 +-
 hw/ppc/ppce500_spin.c       |    4 +-
 hw/ppc/spapr.c              |    2 +-
 hw/ppc/spapr_hcall.c        |    2 +-
 hw/s390x/s390-pci-inst.c    |    1 +
 include/qom/cpu.h           |    5 +
 include/sysemu/hax.h        |   56 ++
 include/sysemu/hw_accel.h   |   48 ++
 include/sysemu/kvm.h        |   23 -
 monitor.c                   |    2 +-
 qemu-options.hx             |   11 +
 qom/cpu.c                   |    2 +-
 target-arm/cpu.c            |    2 +-
 target-i386/Makefile.objs   |    4 +
 target-i386/hax-all.c       | 1250 +++++++++++++++++++++++++++++++++++++++++++
 target-i386/hax-i386.h      |   85 +++
 target-i386/hax-interface.h |  357 ++++++++++++
 target-i386/hax-slot.c      |  333 ++++++++++++
 target-i386/hax-slot.h      |   58 ++
 target-i386/hax-windows.c   |  489 +++++++++++++++++
 target-i386/hax-windows.h   |   89 +++
 target-i386/helper.c        |    1 +
 target-i386/kvm.c           |    1 +
 target-ppc/mmu-hash64.c     |    2 +-
 target-ppc/translate_init.c |    2 +-
 target-s390x/gdbstub.c      |    1 +
 vl.c                        |   15 +-
 37 files changed, 2978 insertions(+), 39 deletions(-)
 create mode 100644 hax-stub.c
 create mode 100644 include/sysemu/hax.h
 create mode 100644 include/sysemu/hw_accel.h
 create mode 100644 target-i386/hax-all.c
 create mode 100644 target-i386/hax-i386.h
 create mode 100644 target-i386/hax-interface.h
 create mode 100644 target-i386/hax-slot.c
 create mode 100644 target-i386/hax-slot.h
 create mode 100644 target-i386/hax-windows.c
 create mode 100644 target-i386/hax-windows.h

Signature · 1 line

2.8.0.rc3.226.g39d4020

Vincent Palatin

author

2016-11-17 11:14 UTC #1 lore raw

PATCH v3 1/3 kvm: move cpu synchronization code

Move the generic cpu_synchronize_ functions to the common hw_accel.h header, in order to prepare for the addition of a second hardware accelerator.

Signed-off-by Stefan Weil <[email protected]>

Signed-off-by Vincent Palatin <[email protected]>

---
 cpus.c                      |  1 +
 gdbstub.c                   |  1 +
 hw/i386/kvm/apic.c          |  1 +
 hw/i386/kvmvapic.c          |  1 +
 hw/misc/vmport.c            |  2 +-
 hw/ppc/pnv_xscom.c          |  2 +-
 hw/ppc/ppce500_spin.c       |  4 ++--
 hw/ppc/spapr.c              |  2 +-
 hw/ppc/spapr_hcall.c        |  2 +-
 hw/s390x/s390-pci-inst.c    |  1 +
 include/sysemu/hw_accel.h   | 39 +++++++++++++++++++++++++++++++++++++++
 include/sysemu/kvm.h        | 23 -----------------------
 monitor.c                   |  2 +-
 qom/cpu.c                   |  2 +-
 target-arm/cpu.c            |  2 +-
 target-i386/helper.c        |  1 +
 target-i386/kvm.c           |  1 +
 target-ppc/mmu-hash64.c     |  2 +-
 target-ppc/translate_init.c |  2 +-
 target-s390x/gdbstub.c      |  1 +
 20 files changed, 58 insertions(+), 34 deletions(-)
 create mode 100644 include/sysemu/hw_accel.h

cpus.c +1 -0

index 5213351..fc78502 100644
@@ -33,6 +33,7 @@
 #include "sysemu/block-backend.h"
 #include "exec/gdbstub.h"
 #include "sysemu/dma.h"
+#include "sysemu/hw_accel.h"
 #include "sysemu/kvm.h"
 #include "qmp-commands.h"
 #include "exec/exec-all.h"

gdbstub.c +1 -0

index de62d26..de9b62b 100644
@@ -32,6 +32,7 @@
 #define MAX_PACKET_LENGTH 4096
 
 #include "qemu/sockets.h"
+#include "sysemu/hw_accel.h"
 #include "sysemu/kvm.h"
 #include "exec/semihost.h"
 #include "exec/exec-all.h"

hw/i386/kvm/apic.c +1 -0

index 01cbaa8..328f80c 100644
@@ -14,6 +14,7 @@
 #include "cpu.h"
 #include "hw/i386/apic_internal.h"
 #include "hw/pci/msi.h"
+#include "sysemu/hw_accel.h"
 #include "sysemu/kvm.h"
 #include "target-i386/kvm_i386.h"

hw/i386/kvmvapic.c +1 -0

index b30d1b9..2f767b6 100644
@@ -14,6 +14,7 @@
 #include "exec/exec-all.h"
 #include "sysemu/sysemu.h"
 #include "sysemu/cpus.h"
+#include "sysemu/hw_accel.h"
 #include "sysemu/kvm.h"
 #include "hw/i386/apic_internal.h"
 #include "hw/sysbus.h"

hw/misc/vmport.c +1 -1

index c763811..be40930 100644
@@ -25,7 +25,7 @@
 #include "hw/hw.h"
 #include "hw/isa/isa.h"
 #include "hw/i386/pc.h"
-#include "sysemu/kvm.h"
+#include "sysemu/hw_accel.h"
 #include "hw/qdev.h"
 
 //#define VMPORT_DEBUG

hw/ppc/pnv_xscom.c +1 -1

index 5aaa264..abcb85d 100644
@@ -20,7 +20,7 @@
 #include "qapi/error.h"
 #include "hw/hw.h"
 #include "qemu/log.h"
-#include "sysemu/kvm.h"
+#include "sysemu/hw_accel.h"
 #include "target-ppc/cpu.h"
 #include "hw/sysbus.h"

hw/ppc/ppce500_spin.c +2 -2

index cf958a9..eb219ab 100644
@@ -29,9 +29,9 @@
 
 #include "qemu/osdep.h"
 #include "hw/hw.h"
-#include "sysemu/sysemu.h"
 #include "hw/sysbus.h"
-#include "sysemu/kvm.h"
+#include "sysemu/hw_accel.h"
+#include "sysemu/sysemu.h"
 #include "e500.h"
 
 #define MAX_CPUS 32

hw/ppc/spapr.c +1 -1

index 0cbab24..174f4d3 100644
@@ -36,7 +36,7 @@
 #include "sysemu/device_tree.h"
 #include "sysemu/block-backend.h"
 #include "sysemu/cpus.h"
-#include "sysemu/kvm.h"
+#include "sysemu/hw_accel.h"
 #include "kvm_ppc.h"
 #include "migration/migration.h"
 #include "mmu-hash64.h"

hw/ppc/spapr_hcall.c +1 -1

index 9a9bedf..b2a8e48 100644
@@ -1,5 +1,6 @@
 #include "qemu/osdep.h"
 #include "qapi/error.h"
+#include "sysemu/hw_accel.h"
 #include "sysemu/sysemu.h"
 #include "qemu/log.h"
 #include "cpu.h"

@@ -9,7 +10,6 @@
 #include "mmu-hash64.h"
 #include "cpu-models.h"
 #include "trace.h"
-#include "sysemu/kvm.h"
 #include "kvm_ppc.h"
 #include "hw/ppc/spapr_ovec.h"

hw/s390x/s390-pci-inst.c +1 -0

index 0864d9b..4d0775c 100644
@@ -18,6 +18,7 @@
 #include "s390-pci-bus.h"
 #include "exec/memory-internal.h"
 #include "qemu/error-report.h"
+#include "sysemu/hw_accel.h"
 
 /* #define DEBUG_S390PCI_INST */
 #ifdef DEBUG_S390PCI_INST

include/sysemu/hw_accel.h new file +39 -0

new file mode 100644
index 0000000..03812cf
@@ -0,0 +1,39 @@
+/*
+ * QEMU Hardware accelertors support
+ *
+ * Copyright 2016 Google, Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#ifndef QEMU_HW_ACCEL_H
+#define QEMU_HW_ACCEL_H
+
+#include "qom/cpu.h"
+#include "sysemu/hax.h"
+#include "sysemu/kvm.h"
+
+static inline void cpu_synchronize_state(CPUState *cpu)
+{
+    if (kvm_enabled()) {
+        kvm_cpu_synchronize_state(cpu);
+    }
+}
+
+static inline void cpu_synchronize_post_reset(CPUState *cpu)
+{
+    if (kvm_enabled()) {
+        kvm_cpu_synchronize_post_reset(cpu);
+    }
+}
+
+static inline void cpu_synchronize_post_init(CPUState *cpu)
+{
+    if (kvm_enabled()) {
+        kvm_cpu_synchronize_post_init(cpu);
+    }
+}
+
+#endif /* QEMU_HW_ACCEL_H */

include/sysemu/kvm.h +0 ~~-23~~

index df67cc0..3045ee7 100644
@@ -461,29 +461,6 @@ void kvm_cpu_synchronize_state(CPUState *cpu);
 void kvm_cpu_synchronize_post_reset(CPUState *cpu);
 void kvm_cpu_synchronize_post_init(CPUState *cpu);
 
-/* generic hooks - to be moved/refactored once there are more users */
-
-static inline void cpu_synchronize_state(CPUState *cpu)
-{
-    if (kvm_enabled()) {
-        kvm_cpu_synchronize_state(cpu);
-    }
-}
-
-static inline void cpu_synchronize_post_reset(CPUState *cpu)
-{
-    if (kvm_enabled()) {
-        kvm_cpu_synchronize_post_reset(cpu);
-    }
-}
-
-static inline void cpu_synchronize_post_init(CPUState *cpu)
-{
-    if (kvm_enabled()) {
-        kvm_cpu_synchronize_post_init(cpu);
-    }
-}
-
 /**
  * kvm_irqchip_add_msi_route - Add MSI route for specific vector
  * @s:      KVM state

monitor.c +1 -1

index 0841d43..d38956f 100644
@@ -50,7 +50,7 @@
 #include "sysemu/balloon.h"
 #include "qemu/timer.h"
 #include "migration/migration.h"
-#include "sysemu/kvm.h"
+#include "sysemu/hw_accel.h"
 #include "qemu/acl.h"
 #include "sysemu/tpm.h"
 #include "qapi/qmp/qerror.h"

qom/cpu.c +1 -1

index 03d9190..2c5274f 100644
@@ -22,7 +22,7 @@
 #include "qapi/error.h"
 #include "qemu-common.h"
 #include "qom/cpu.h"
-#include "sysemu/kvm.h"
+#include "sysemu/hw_accel.h"
 #include "qemu/notify.h"
 #include "qemu/log.h"
 #include "exec/log.h"

target-arm/cpu.c +1 -1

index 99f0dbe..744cc11 100644
@@ -31,7 +31,7 @@
 #endif
 #include "hw/arm/arm.h"
 #include "sysemu/sysemu.h"
-#include "sysemu/kvm.h"
+#include "sysemu/hw_accel.h"
 #include "kvm_arm.h"
 
 static void arm_cpu_set_pc(CPUState *cs, vaddr value)

target-i386/helper.c +1 -0

index 4ecc091..7530280 100644
@@ -24,6 +24,7 @@
 #include "kvm_i386.h"
 #ifndef CONFIG_USER_ONLY
 #include "sysemu/sysemu.h"
+#include "sysemu/hw_accel.h"
 #include "monitor/monitor.h"
 #include "hw/i386/apic_internal.h"
 #endif

target-i386/kvm.c +1 -0

index 1c0864e..d9ad69e 100644
@@ -23,6 +23,7 @@
 #include "qemu-common.h"
 #include "cpu.h"
 #include "sysemu/sysemu.h"
+#include "sysemu/hw_accel.h"
 #include "sysemu/kvm_int.h"
 #include "kvm_i386.h"
 #include "hyperv.h"

target-ppc/mmu-hash64.c +1 -1

index fdb7a78..0efc8c6 100644
@@ -23,7 +23,7 @@
 #include "exec/exec-all.h"
 #include "exec/helper-proto.h"
 #include "qemu/error-report.h"
-#include "sysemu/kvm.h"
+#include "sysemu/hw_accel.h"
 #include "kvm_ppc.h"
 #include "mmu-hash64.h"
 #include "exec/log.h"

target-ppc/translate_init.c +1 -1

index 208fa1e..1a049af 100644
@@ -21,10 +21,10 @@
 #include "qemu/osdep.h"
 #include "disas/bfd.h"
 #include "exec/gdbstub.h"
-#include "sysemu/kvm.h"
 #include "kvm_ppc.h"
 #include "sysemu/arch_init.h"
 #include "sysemu/cpus.h"
+#include "sysemu/hw_accel.h"
 #include "cpu-models.h"
 #include "mmu-hash32.h"
 #include "mmu-hash64.h"

target-s390x/gdbstub.c +1 -0

index 3d223de..3c652fb 100644
@@ -23,6 +23,7 @@
 #include "exec/exec-all.h"
 #include "exec/gdbstub.h"
 #include "qemu/bitops.h"
+#include "sysemu/hw_accel.h"
 
 int s390_cpu_gdb_read_register(CPUState *cs, uint8_t *mem_buf, int n)
 {

Signature · 1 line

2.8.0.rc3.226.g39d4020

Vincent Palatin

author

2016-11-17 11:14 UTC #2 lore raw

PATCH v3 2/3 target-i386: Add Intel HAX files

In reply to Vincent Palatin

That's a forward port of the core HAX interface code from the emu-2.2-release branch in the external/qemu-android repository as used by the Android emulator.

The original commit was "target-i386: Add Intel HAX to android emulator" saying:
"""
Backport of 2b3098ff27bab079caab9b46b58546b5036f5c0c
from studio-1.4-dev into emu-master-dev

Intel HAX (harware acceleration) will enhance android emulator performance in Windows and Mac OS X in the systems powered by Intel processors with "Intel Hardware Accelerated Execution Manager" package installed when user runs android emulator with Intel target.

Signed-off-by David Chou <[email protected]>

"""

It has been modified to build and run along with the current code base. The formatting has been fixed to go through scripts/checkpatch.pl, and the DPRINTF macros have been updated to get the instanciations checked by the compiler.

The FPU registers saving/restoring has been updated to match the current QEMU registers layout.

The implementation has been simplified by doing the following modifications:
- removing the code for supporting the hardware without Unrestricted Guest (UG) mode (including all the code to fallback on TCG emulation).
- not including the Darwin support (which is not yet debugged/tested).
- simplifying the initialization by removing the leftovers from the Android specific code, then trimming down the remaining logic.
- removing the unused MemoryListener callbacks.

Signed-off-by Vincent Palatin <[email protected]>

---
 hax-stub.c                  |   39 ++
 include/sysemu/hax.h        |   56 ++
 target-i386/hax-all.c       | 1250 +++++++++++++++++++++++++++++++++++++++++++
 target-i386/hax-i386.h      |   85 +++
 target-i386/hax-interface.h |  357 ++++++++++++
 target-i386/hax-slot.c      |  333 ++++++++++++
 target-i386/hax-slot.h      |   58 ++
 target-i386/hax-windows.c   |  489 +++++++++++++++++
 target-i386/hax-windows.h   |   89 +++
 9 files changed, 2756 insertions(+)
 create mode 100644 hax-stub.c
 create mode 100644 include/sysemu/hax.h
 create mode 100644 target-i386/hax-all.c
 create mode 100644 target-i386/hax-i386.h
 create mode 100644 target-i386/hax-interface.h
 create mode 100644 target-i386/hax-slot.c
 create mode 100644 target-i386/hax-slot.h
 create mode 100644 target-i386/hax-windows.c
 create mode 100644 target-i386/hax-windows.h

hax-stub.c new file +39 -0

new file mode 100644
index 0000000..a532dba
@@ -0,0 +1,39 @@
+/*
+ * QEMU HAXM support
+ *
+ * Copyright (c) 2015, Intel Corporation
+ *
+ * Copyright 2016 Google, Inc.
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qemu-common.h"
+#include "cpu.h"
+#include "sysemu/hax.h"
+
+int hax_sync_vcpus(void)
+{
+    return 0;
+}
+
+int hax_populate_ram(uint64_t va, uint32_t size)
+{
+    return -ENOSYS;
+}
+
+int hax_init_vcpu(CPUState *cpu)
+{
+    return -ENOSYS;
+}
+
+int hax_smp_cpu_exec(CPUState *cpu)
+{
+    return -ENOSYS;
+}

include/sysemu/hax.h new file +56 -0

56 lines not on this page · raw message

target-i386/hax-all.c new file +1250 -0

1,250 lines not on this page · raw message

target-i386/hax-i386.h new file +85 -0

85 lines not on this page · raw message

target-i386/hax-interface.h new file +357 -0

357 lines not on this page · raw message

target-i386/hax-slot.c new file +333 -0

333 lines not on this page · raw message

target-i386/hax-slot.h new file +58 -0

58 lines not on this page · raw message

target-i386/hax-windows.c new file +489 -0

489 lines not on this page · raw message

target-i386/hax-windows.h new file +89 -0

89 lines not on this page · raw message

Signature · 1 line

2.8.0.rc3.226.g39d4020

Vincent Palatin

author

2016-11-17 11:14 UTC #3 lore raw

PATCH v3 3/3 Plumb the HAXM-based hardware acceleration support

In reply to Vincent Palatin

Use the Intel HAX is kernel-based hardware acceleration module for Windows (similar to KVM on Linux).

Based on the "target-i386: Add Intel HAX to android emulator" patch from David Chou <[email protected]>

Signed-off-by Vincent Palatin <[email protected]>

---
 Makefile.target           |  1 +
 configure                 | 18 ++++++++++
 cpus.c                    | 87 ++++++++++++++++++++++++++++++++++++++++++++++-
 exec.c                    | 16 +++++++++
 hw/intc/apic_common.c     |  3 +-
 include/qom/cpu.h         |  5 +++
 include/sysemu/hw_accel.h |  9 +++++
 qemu-options.hx           | 11 ++++++
 target-i386/Makefile.objs |  4 +++
 vl.c                      | 15 ++++++--
 10 files changed, 164 insertions(+), 5 deletions(-)

Makefile.target +1 -0

index 7a5080e..dab81e7 100644
@@ -96,6 +96,7 @@ obj-y += target-$(TARGET_BASE_ARCH)/
 obj-y += disas.o
 obj-y += tcg-runtime.o
 obj-$(call notempty,$(TARGET_XML_FILES)) += gdbstub-xml.o
+obj-$(call lnot,$(CONFIG_HAX)) += hax-stub.o
 obj-$(call lnot,$(CONFIG_KVM)) += kvm-stub.o
 
 obj-$(CONFIG_LIBDECNUMBER) += libdecnumber/decContext.o

configure +18 -0

index 7d2a34e..236a7b7 100755
@@ -230,6 +230,7 @@ vhost_net="no"
 vhost_scsi="no"
 vhost_vsock="no"
 kvm="no"
+hax="no"
 colo="yes"
 rdma=""
 gprof="no"

@@ -563,6 +564,7 @@ CYGWIN*)
 ;;
 MINGW32*)
   mingw32="yes"
+  hax="yes"
   audio_possible_drivers="dsound sdl"
   if check_include dsound.h; then
     audio_drv_list="dsound"

@@ -610,6 +612,7 @@ OpenBSD)
 Darwin)
   bsd="yes"
   darwin="yes"
+  hax="yes"
   LDFLAGS_SHARED="-bundle -undefined dynamic_lookup"
   if [ "$cpu" = "x86_64" ] ; then
     QEMU_CFLAGS="-arch x86_64 $QEMU_CFLAGS"

@@ -919,6 +922,10 @@ for opt do
   ;;
   --enable-kvm) kvm="yes"
   ;;
+  --disable-hax) hax="no"
+  ;;
+  --enable-hax) hax="yes"
+  ;;
   --disable-colo) colo="no"
   ;;
   --enable-colo) colo="yes"

@@ -1371,6 +1378,7 @@ disabled with --disable-FEATURE, default is enabled if available:
   fdt             fdt device tree
   bluez           bluez stack connectivity
   kvm             KVM acceleration support
+  hax             HAX acceleration support
   colo            COarse-grain LOck-stepping VM for Non-stop Service
   rdma            RDMA-based migration support
   vde             support for vde network

@@ -5043,6 +5051,7 @@ echo "ATTR/XATTR support $attr"
 echo "Install blobs     $blobs"
 echo "KVM support       $kvm"
 echo "COLO support      $colo"
+echo "HAX support       $hax"
 echo "RDMA support      $rdma"
 echo "TCG interpreter   $tcg_interpreter"
 echo "fdt support       $fdt"

@@ -6027,6 +6036,15 @@ case "$target_name" in
       fi
     fi
 esac
+if test "$hax" = "yes" ; then
+  if test "$target_softmmu" = "yes" ; then
+    case "$target_name" in
+    i386|x86_64)
+      echo "CONFIG_HAX=y" >> $config_target_mak
+    ;;
+    esac
+  fi
+fi
 if test "$target_bigendian" = "yes" ; then
   echo "TARGET_WORDS_BIGENDIAN=y" >> $config_target_mak
 fi

cpus.c +86 -1

123 lines not on this page · raw message

exec.c +16 -0

index 3d867f1..fa24cd1 100644
@@ -31,6 +31,7 @@
 #include "hw/xen/xen.h"
 #endif
 #include "sysemu/kvm.h"
+#include "sysemu/hax.h"
 #include "sysemu/sysemu.h"
 #include "qemu/timer.h"
 #include "qemu/config-file.h"

@@ -1624,6 +1625,21 @@ static void ram_block_add(RAMBlock *new_block, Error **errp)
                 qemu_mutex_unlock_ramlist();
                 return;
             }
+            /*
+             * In Hax, the qemu allocate the virtual address, and HAX kernel
+             * populate the memory with physical memory. Currently we have no
+             * paging, so user should make sure enough free memory in advance
+             */
+            if (hax_enabled()) {
+                int ret;
+                ret = hax_populate_ram((uint64_t)(uintptr_t)new_block->host,
+                                       new_block->max_length);
+                if (ret < 0) {
+                    error_setg(errp, "Hax failed to populate ram");
+                    return;
+                }
+            }
+
             memory_try_enable_merging(new_block->host, new_block->max_length);
         }
     }

hw/intc/apic_common.c +2 -1

index d78c885..3945dfd 100644
@@ -26,6 +26,7 @@
 #include "hw/i386/apic.h"
 #include "hw/i386/apic_internal.h"
 #include "trace.h"
+#include "sysemu/hax.h"
 #include "sysemu/kvm.h"
 #include "hw/qdev.h"
 #include "hw/sysbus.h"

@@ -316,7 +317,7 @@ static void apic_common_realize(DeviceState *dev, Error **errp)
 
     /* Note: We need at least 1M to map the VAPIC option ROM */
     if (!vapic && s->vapic_control & VAPIC_ENABLE_MASK &&
-        ram_size >= 1024 * 1024) {
+        !hax_enabled() && ram_size >= 1024 * 1024) {
         vapic = sysbus_create_simple("kvmvapic", -1, NULL);
     }
     s->vapic = vapic;

include/qom/cpu.h +5 -0

index 3f79a8e..ca4d0fb 100644
@@ -227,6 +227,8 @@ struct CPUWatchpoint {
 struct KVMState;
 struct kvm_run;
 
+struct hax_vcpu_state;
+
 #define TB_JMP_CACHE_BITS 12
 #define TB_JMP_CACHE_SIZE (1 << TB_JMP_CACHE_BITS)

@@ -392,6 +394,9 @@ struct CPUState {
        (absolute value) offset as small as possible.  This reduces code
        size, especially for hosts without large memory offsets.  */
     uint32_t tcg_exit_req;
+
+    bool hax_vcpu_dirty;
+    struct hax_vcpu_state *hax_vcpu;
 };
 
 QTAILQ_HEAD(CPUTailQ, CPUState);

include/sysemu/hw_accel.h +9 -0

index 03812cf..c9b3105 100644
@@ -20,6 +20,9 @@ static inline void cpu_synchronize_state(CPUState *cpu)
     if (kvm_enabled()) {
         kvm_cpu_synchronize_state(cpu);
     }
+    if (hax_enabled()) {
+        hax_cpu_synchronize_state(cpu);
+    }
 }
 
 static inline void cpu_synchronize_post_reset(CPUState *cpu)

@@ -27,6 +30,9 @@ static inline void cpu_synchronize_post_reset(CPUState *cpu)
     if (kvm_enabled()) {
         kvm_cpu_synchronize_post_reset(cpu);
     }
+    if (hax_enabled()) {
+        hax_cpu_synchronize_post_reset(cpu);
+    }
 }
 
 static inline void cpu_synchronize_post_init(CPUState *cpu)

@@ -34,6 +40,9 @@ static inline void cpu_synchronize_post_init(CPUState *cpu)
     if (kvm_enabled()) {
         kvm_cpu_synchronize_post_init(cpu);
     }
+    if (hax_enabled()) {
+        hax_cpu_synchronize_post_init(cpu);
+    }
 }
 
 #endif /* QEMU_HW_ACCEL_H */

qemu-options.hx +11 -0

index 4536e18..bd28219 100644
@@ -3259,6 +3259,17 @@ Enable KVM full virtualization support. This option is only available
 if KVM support is enabled when compiling.
 ETEXI
 
+DEF("enable-hax", 0, QEMU_OPTION_enable_hax, \
+    "-enable-hax     enable HAX virtualization support\n", QEMU_ARCH_I386)
+STEXI
+@item -enable-hax
+@findex -enable-hax
+Enable HAX (Hardware-based Acceleration eXecution) support. This option
+is only available if HAX support is enabled when compiling. HAX is only
+applicable to MAC and Windows platform, and thus does not conflict with
+KVM.
+ETEXI
+
 DEF("xen-domid", HAS_ARG, QEMU_OPTION_xen_domid,
     "-xen-domid id   specify xen guest domain id\n", QEMU_ARCH_ALL)
 DEF("xen-create", 0, QEMU_OPTION_xen_create,

target-i386/Makefile.objs +4 -0

index b223d79..f14cda9 100644
@@ -5,3 +5,7 @@ obj-y += gdbstub.o
 obj-$(CONFIG_SOFTMMU) += machine.o arch_memory_mapping.o arch_dump.o monitor.o
 obj-$(CONFIG_KVM) += kvm.o hyperv.o
 obj-$(call lnot,$(CONFIG_KVM)) += kvm-stub.o
+# HAX support
+ifdef CONFIG_WIN32
+obj-$(CONFIG_HAX) += hax-all.o hax-slot.o hax-windows.o
+endif

vl.c +12 -3

index 319f641..b11fa50 100644
@@ -92,6 +92,7 @@ int main(int argc, char **argv)
 #include "sysemu/cpus.h"
 #include "migration/colo.h"
 #include "sysemu/kvm.h"
+#include "sysemu/hax.h"
 #include "qapi/qmp/qjson.h"
 #include "qemu/option.h"
 #include "qemu/config-file.h"

@@ -1959,7 +1960,7 @@ static void main_loop(void)
     int64_t ti;
 #endif
     do {
-        nonblocking = !kvm_enabled() && !xen_enabled() && last_io > 0;
+        nonblocking = tcg_enabled() && last_io > 0;
 #ifdef CONFIG_PROFILER
         ti = profile_getclock();
 #endif

@@ -3724,6 +3725,10 @@ int main(int argc, char **argv, char **envp)
                 olist = qemu_find_opts("machine");
                 qemu_opts_parse_noisily(olist, "accel=kvm", false);
                 break;
+            case QEMU_OPTION_enable_hax:
+                olist = qemu_find_opts("machine");
+                qemu_opts_parse_noisily(olist, "accel=hax", false);
+                break;
             case QEMU_OPTION_M:
             case QEMU_OPTION_machine:
                 olist = qemu_find_opts("machine");

@@ -4418,8 +4423,8 @@ int main(int argc, char **argv, char **envp)
 
     cpu_ticks_init();
     if (icount_opts) {
-        if (kvm_enabled() || xen_enabled()) {
-            error_report("-icount is not allowed with kvm or xen");
+        if (!tcg_enabled()) {
+            error_report("-icount is not allowed with hardware virtualization");
             exit(1);
         }
         configure_icount(icount_opts, &error_abort);

@@ -4555,6 +4560,10 @@ int main(int argc, char **argv, char **envp)
 
     numa_post_machine_init();
 
+    if (hax_enabled()) {
+        hax_sync_vcpus();
+    }
+
     if (qemu_opts_foreach(qemu_find_opts("fw_cfg"),
                           parse_fw_cfg, fw_cfg_find(), NULL) != 0) {
         exit(1);

Signature · 1 line

2.8.0.rc3.226.g39d4020

Yu Ning

2016-12-08 09:34 UTC #4 lore raw

In reply to Vincent Palatin

As a HAXM developer at Intel, I just want to come out and show our support for Vincent's upstreaming effort. We'd love to see HAXM support code land in upstream QEMU, and will do what is necessary to make that happen.

We've been working on the issues that Vincent raised and have made some progress - see below.

Regards,
Yu

Quoted 12 lines · Vincent Palatin

I took a stab at trying to rebase/upstream the support for Intel HAXM. (Hardware Accelerated Execution Manager).
Intel HAX is kernel-based hardware acceleration module for Windows and MacOSX.
I have based my work on the last version of the source code I found:
the emu-2.2-release branch in the external/qemu-android repository as used by the Android emulator.
In patch 2/3, I have forward-ported the core HAX code from there. It has been modified to build and run along with the current code base. It has been simplifying by removing non-UG hardware support / Darwin support / Android-specific leftovers.

Quoted 8 lines

CAVEAT: this patchset is still RFC because in addition of potentially containing a few outdated constructs and a questionable mess in qemu_cpu_kick_thread(), there are 2 unsolved issues:
- in rare cases, the HAX kernel module asks us to emulate a MMIO access rather than just using the 'fast MMIO' path. This path is still not implemented (as it was in the original patch). It's unclear why/when this is happening (eg you can run a full linux image without hitting it but the default iPXE option ROM is hitting it in the Intel network driver).

This is now resolved, thanks to Vincent's help with debugging. The next HAXM driver release will include the fix.

Quoted 4 lines

- the MemoryListener implementation cannot remove a memory mapping (e.g. for digging the VGA 'hole' in the lowmem). This shortcoming breaks the std vga implementation but in the current HAX module API, I don't see any (documented) way of implementing this.

The current HAXM driver does not support removing a memory mapping. We just got that implemented this week, but testing has revealed another MMIO bug, which we are trying to fix.

Quoted 92 lines

In patch 3/3, I have put the plumbing into the QEMU code base, I did some clean up there and it is reasonably intrusive: i.e.
  Makefile.target           |  1 +
  configure                 | 18 ++++++++++
  cpus.c                    | 87 ++++++++++++++++++++++++++++++++++++++++++++++-
  exec.c                    | 16 +++++++++
  hw/intc/apic_common.c     |  3 +-
  include/qom/cpu.h         |  5 +++
  include/sysemu/hw_accel.h |  9 +++++
  qemu-options.hx           | 11 ++++++
  target-i386/Makefile.objs |  4 +++
  vl.c                      | 15 ++++++--
  10 files changed, 164 insertions(+), 5 deletions(-)
The patch 1/3 just extracts from KVM specific header the cpu_synchronize_ functions that HAX is also using.
I have tested the end result on a Windows 10 Pro machine (with UG support) with the Intel HAXM module 6.0.4 and a large ChromiumOS x86_64 image to exercise various code paths. It looks stable.
I also did a quick regression testing of the integration by running a Linux build with KVM enabled.
Just running 'qemu-system-x86_64 -enable-hax' is broken due to the caveats described above.
Changes from v2 to v3:
- fix saving/restoring FPU registers as suggested by Paolo.
- fix Windows build on all targets as contributed by Stefan Weil.
- clean-up IO / MMIO emulation.
- more clean-up of emulation leftovers.
Changes from v1 to v2:
- fix all style issues in the original code to get it through checkpatch.pl.
- remove Darwin support, it was barely tested and not fully functional.
- remove the support for CPU without UG mode.
- fix most review comments
Vincent Palatin (3):
kvm: move cpu synchronization code
target-i386: Add Intel HAX files
Plumb the HAXM-based hardware acceleration support
  Makefile.target             |    1 +
  configure                   |   18 +
  cpus.c                      |   88 ++-
  exec.c                      |   16 +
  gdbstub.c                   |    1 +
  hax-stub.c                  |   39 ++
  hw/i386/kvm/apic.c          |    1 +
  hw/i386/kvmvapic.c          |    1 +
  hw/intc/apic_common.c       |    3 +-
  hw/misc/vmport.c            |    2 +-
  hw/ppc/pnv_xscom.c          |    2 +-
  hw/ppc/ppce500_spin.c       |    4 +-
  hw/ppc/spapr.c              |    2 +-
  hw/ppc/spapr_hcall.c        |    2 +-
  hw/s390x/s390-pci-inst.c    |    1 +
  include/qom/cpu.h           |    5 +
  include/sysemu/hax.h        |   56 ++
  include/sysemu/hw_accel.h   |   48 ++
  include/sysemu/kvm.h        |   23 -
  monitor.c                   |    2 +-
  qemu-options.hx             |   11 +
  qom/cpu.c                   |    2 +-
  target-arm/cpu.c            |    2 +-
  target-i386/Makefile.objs   |    4 +
  target-i386/hax-all.c       | 1250 +++++++++++++++++++++++++++++++++++++++++++
  target-i386/hax-i386.h      |   85 +++
  target-i386/hax-interface.h |  357 ++++++++++++
  target-i386/hax-slot.c      |  333 ++++++++++++
  target-i386/hax-slot.h      |   58 ++
  target-i386/hax-windows.c   |  489 +++++++++++++++++
  target-i386/hax-windows.h   |   89 +++
  target-i386/helper.c        |    1 +
  target-i386/kvm.c           |    1 +
  target-ppc/mmu-hash64.c     |    2 +-
  target-ppc/translate_init.c |    2 +-
  target-s390x/gdbstub.c      |    1 +
  vl.c                        |   15 +-
  37 files changed, 2978 insertions(+), 39 deletions(-)
create mode 100644 hax-stub.c
create mode 100644 include/sysemu/hax.h
create mode 100644 include/sysemu/hw_accel.h
create mode 100644 target-i386/hax-all.c
create mode 100644 target-i386/hax-i386.h
create mode 100644 target-i386/hax-interface.h
create mode 100644 target-i386/hax-slot.c
create mode 100644 target-i386/hax-slot.h
create mode 100644 target-i386/hax-windows.c
create mode 100644 target-i386/hax-windows.h

Stefan Weil

2016-12-08 15:32 UTC #5 lore raw

Am 08.12.2016 um 10:34 schrieb Yu Ning:

Quoted 10 lines

As a HAXM developer at Intel, I just want to come out and show our support for Vincent's upstreaming effort. We'd love to see HAXM support code land in upstream QEMU, and will do what is necessary to make that happen.
We've been working on the issues that Vincent raised and have made some progress - see below.
Regards,
Yu

Thanks a lot - I think that HAXM integration will make fast progress then.

AFAIK, there remain at least two other features which need to be addressed:

* support for real mode (e. g. BIOS code)

* support for macOS

Stefan

Vincent Palatin

author

2016-12-08 15:48 UTC #6 lore raw

Quoted 10 lines · Stefan Weil

Am 08.12.2016 um 10:34 schrieb Yu Ning:
Quoted 8 lines
As a HAXM developer at Intel, I just want to come out and show our support for Vincent's upstreaming effort. We'd love to see HAXM support code land in upstream QEMU, and will do what is necessary to make that happen.
We've been working on the issues that Vincent raised and have made some progress - see below.

Quoted 8 lines

Quoted 2 lines
Regards,
Yu
Thanks a lot - I think that HAXM integration will make fast progress then.
AFAIK, there remain at least two other features which need to be addressed:
* support for real mode (e. g. BIOS code)

It does support it actually, it was failing early due to a bug in the kernel module.
Thanks to the diligent work by Yu Ning on Intel side, this part is solved (as soon as they release the new kernel module version).

Quoted 1 line

* support for macOS

I have a patch for it, it is mainly lacking bake/test time.

Signature · 1 line

Vincent