hax: Support for Linux hosts

PATCH In review Alexandro Sanchez Bach 5 participants 12 messages Nov 14, 2018 – Feb 2, 2019

lore

Share & tools

lore

Alexandro Sanchez Bach

author

2018-11-14 13:05 UTC #0 lore raw

PATCH 1/1 hax: Support for Linux hosts

Intel HAXM supports now 32-bit and 64-bit Linux hosts. This patch includes the corresponding userland changes.

Since the Darwin userland backend is POSIX-compliant, the hax-darwin.{c,h} files have been renamed to hax-posix.{c,h}. This prefix is consistent with the naming used in the rest of QEMU.

Signed-off-by Alexandro Sanchez Bach <[email protected]>

---
 target/i386/Makefile.objs                 |  5 ++++-
 target/i386/hax-i386.h                    | 10 +++++++++-
 target/i386/{hax-darwin.c => hax-posix.c} |  0
 target/i386/{hax-darwin.h => hax-posix.h} |  0
 4 files changed, 13 insertions(+), 2 deletions(-)
 rename target/i386/{hax-darwin.c => hax-posix.c} (100%)
 rename target/i386/{hax-darwin.h => hax-posix.h} (100%)

target/i386/Makefile.objs +4 -1

index 04678f5503..be36d31d23 100644
@@ -12,8 +12,11 @@ obj-$(call lnot,$(CONFIG_SEV)) += sev-stub.o
 ifdef CONFIG_WIN32
 obj-$(CONFIG_HAX) += hax-all.o hax-mem.o hax-windows.o
 endif
+ifdef CONFIG_LINUX
+obj-$(CONFIG_HAX) += hax-all.o hax-mem.o hax-posix.o
+endif
 ifdef CONFIG_DARWIN
-obj-$(CONFIG_HAX) += hax-all.o hax-mem.o hax-darwin.o
+obj-$(CONFIG_HAX) += hax-all.o hax-mem.o hax-posix.o
 obj-$(CONFIG_HVF) += hvf/
 endif
 obj-$(CONFIG_WHPX) += whpx-all.o

target/i386/hax-i386.h +9 -1

index 6abc156f88..f87a64c9df 100644
@@ -20,6 +20,10 @@
 typedef int hax_fd;
 #endif
 
+#ifdef CONFIG_LINUX
+typedef int hax_fd;
+#endif
+
 #ifdef CONFIG_WIN32
 typedef HANDLE hax_fd;
 #endif

@@ -83,7 +87,11 @@ void hax_memory_init(void);
 
 
 #ifdef CONFIG_DARWIN
-#include "target/i386/hax-darwin.h"
+#include "target/i386/hax-posix.h"
+#endif
+
+#ifdef CONFIG_LINUX
+#include "target/i386/hax-posix.h"
 #endif
 
 #ifdef CONFIG_WIN32

target/i386/hax-posix.c renamed from target/i386/hax-darwin.c +0 -0

similarity index 100%
rename from target/i386/hax-darwin.c
rename to target/i386/hax-posix.c

target/i386/hax-posix.h renamed from target/i386/hax-darwin.h +0 -0

similarity index 100%
rename from target/i386/hax-darwin.h
rename to target/i386/hax-posix.h

Signature · 1 line

2.19.1

Alex Bennée

2018-11-14 15:04 UTC #1 lore raw

Quoted 22 lines · Alexandro Sanchez Bach

Intel HAXM supports now 32-bit and 64-bit Linux hosts. This patch includes the corresponding userland changes.
Since the Darwin userland backend is POSIX-compliant, the hax-darwin.{c,h} files have been renamed to hax-posix.{c,h}. This prefix is consistent with the naming used in the rest of QEMU.
Signed-off-by Alexandro Sanchez Bach <[email protected]>
---
 target/i386/Makefile.objs                 |  5 ++++-
 target/i386/hax-i386.h                    | 10 +++++++++-
 target/i386/{hax-darwin.c => hax-posix.c} |  0
 target/i386/{hax-darwin.h => hax-posix.h} |  0
 4 files changed, 13 insertions(+), 2 deletions(-)
 rename target/i386/{hax-darwin.c => hax-posix.c} (100%)
 rename target/i386/{hax-darwin.h => hax-posix.h} (100%)
target/i386/Makefile.objs +0 -0
index 04678f5503..be36d31d23 100644

target/i386/Makefile.objs obj-$(call lnot,$(CONFIG_SEV)) += sev-stub.o · line 19

Quoted 9 lines

target/i386/Makefile.objs +4 -1

@@ -12,8 +12,11 @@ obj-$(call lnot,$(CONFIG_SEV)) += sev-stub.o
 ifdef CONFIG_WIN32
 obj-$(CONFIG_HAX) += hax-all.o hax-mem.o hax-windows.o
 endif
+ifdef CONFIG_LINUX
+obj-$(CONFIG_HAX) += hax-all.o hax-mem.o hax-posix.o
+endif
 ifdef CONFIG_DARWIN
-obj-$(CONFIG_HAX) += hax-all.o hax-mem.o hax-darwin.o
+obj-$(CONFIG_HAX) += hax-all.o hax-mem.o hax-posix.o

This is starting to get repetitive, perhaps:

  obj-$(CONFIG_HAX) += hax-all.o hax-mem.o
  ifneq ($(CONFIG_LINUX)$(CONFIG_DARWIN),)
  obj-$(CONFIG_HAX) += hax-posix.o
  endif
  ifdef CONFIG_WIN32
  obj-$(CONFIG_HAX) += hax-windows.o
  endif

Would be better?

Quoted 8 lines

target/i386/Makefile.objs +0 -0
 obj-$(CONFIG_HVF) += hvf/
 endif
 obj-$(CONFIG_WHPX) += whpx-all.o
target/i386/hax-i386.h +0 -0
index 6abc156f88..f87a64c9df 100644

target/i386/hax-i386.h line 25

Quoted 6 lines

target/i386/hax-i386.h +3 -0

@@ -20,6 +20,10 @@
 typedef int hax_fd;
 #endif
 
+#ifdef CONFIG_LINUX
+typedef int hax_fd;
+#endif

Again this could be merged with the other POSIX like above.

Quoted 5 lines

target/i386/hax-i386.h +1 -0

+
 #ifdef CONFIG_WIN32
 typedef HANDLE hax_fd;
 #endif

target/i386/hax-i386.h void hax_memory_init(void); · line 95

Quoted 10 lines

target/i386/hax-i386.h +5 -1

@@ -83,7 +87,11 @@ void hax_memory_init(void);
 
 
 #ifdef CONFIG_DARWIN
-#include "target/i386/hax-darwin.h"
+#include "target/i386/hax-posix.h"
+#endif
+
+#ifdef CONFIG_LINUX
+#include "target/i386/hax-posix.h"
 #endif

And here.

In fact looking at my config-host.mak we have a CONFIG_POSIX so you could use that.

target/i386/hax-posix.h

Quoted 10 lines

target/i386/hax-i386.h +0 -0
 #ifdef CONFIG_WIN32
target/i386/hax-posix.c renamed from target/i386/hax-darwin.c +0 -0
similarity index 100%
rename from target/i386/hax-darwin.c
rename to target/i386/hax-posix.c
target/i386/hax-posix.h renamed from target/i386/hax-darwin.h +0 -0
similarity index 100%
rename from target/i386/hax-darwin.h
rename to target/i386/hax-posix.h

--
Alex Bennée

Alexandro Sanchez Bach

author

2018-11-15 01:35 UTC #2 lore raw

PATCH 1/1 hax: Support for Linux hosts

Intel HAXM supports now 32-bit and 64-bit Linux hosts. This patch includes the corresponding userland changes.

Since the Darwin userland backend is POSIX-compliant, the hax-darwin.{c,h} files have been renamed to hax-posix.{c,h}. This prefix is consistent with the naming used in the rest of QEMU.

Signed-off-by Alexandro Sanchez Bach <[email protected]>

---
 target/i386/Makefile.objs                 | 4 +++-
 target/i386/hax-i386.h                    | 6 +++---
 target/i386/{hax-darwin.c => hax-posix.c} | 0
 target/i386/{hax-darwin.h => hax-posix.h} | 0
 4 files changed, 6 insertions(+), 4 deletions(-)
 rename target/i386/{hax-darwin.c => hax-posix.c} (100%)
 rename target/i386/{hax-darwin.h => hax-posix.h} (100%)

target/i386/Makefile.objs +3 -1

index 04678f5503..070e701856 100644
@@ -12,8 +12,10 @@ obj-$(call lnot,$(CONFIG_SEV)) += sev-stub.o
 ifdef CONFIG_WIN32
 obj-$(CONFIG_HAX) += hax-all.o hax-mem.o hax-windows.o
 endif
+ifdef CONFIG_POSIX
+obj-$(CONFIG_HAX) += hax-all.o hax-mem.o hax-posix.o
+endif
 ifdef CONFIG_DARWIN
-obj-$(CONFIG_HAX) += hax-all.o hax-mem.o hax-darwin.o
 obj-$(CONFIG_HVF) += hvf/
 endif
 obj-$(CONFIG_WHPX) += whpx-all.o

target/i386/hax-i386.h +3 -3

index 6abc156f88..f13fa4638f 100644
@@ -16,7 +16,7 @@
 #include "cpu.h"
 #include "sysemu/hax.h"
 
-#ifdef CONFIG_DARWIN
+#ifdef CONFIG_POSIX
 typedef int hax_fd;
 #endif

@@ -82,8 +82,8 @@ hax_fd hax_mod_open(void);
 void hax_memory_init(void);
 
 
-#ifdef CONFIG_DARWIN
-#include "target/i386/hax-darwin.h"
+#ifdef CONFIG_POSIX
+#include "target/i386/hax-posix.h"
 #endif
 
 #ifdef CONFIG_WIN32

target/i386/hax-posix.c renamed from target/i386/hax-darwin.c +0 -0

similarity index 100%
rename from target/i386/hax-darwin.c
rename to target/i386/hax-posix.c

target/i386/hax-posix.h renamed from target/i386/hax-darwin.h +0 -0

similarity index 100%
rename from target/i386/hax-darwin.h
rename to target/i386/hax-posix.h

Signature · 1 line

2.19.1

Paolo Bonzini

2018-11-16 12:53 UTC #3 lore raw

In reply to Alexandro Sanchez Bach

Quoted 6 lines · Alexandro Sanchez Bach

Intel HAXM supports now 32-bit and 64-bit Linux hosts. This patch includes the corresponding userland changes.
Since the Darwin userland backend is POSIX-compliant, the hax-darwin.{c,h} files have been renamed to hax-posix.{c,h}. This prefix is consistent with the naming used in the rest of QEMU.

What's the advantage of HAXM when Linux hosts can just run KVM? I guess avoiding bitrot?

Paolo

Alexandro Sanchez

author

2018-11-16 15:00 UTC #4 lore raw

Hey Paolo,

Quoted 1 line

What's the advantage of HAXM when Linux hosts can just run KVM?

For end-users, it's true that there's several disadvantages and barely any
advantages: There's some guests run only on HAXM, since it can handle MMIO accesses using BMI instructions ...and that's about it. :-) The HAXM API also offers protecting/unprotecting arbitrary chunks of guest memory, which has some advantages over KVM_GET_DIRTY_LOG. It's used in [1], but has not been upstreamed yet.

For developers, there's few advantages over KVM that make work slightly easier/simpler:
- No need to recompile+install a kernel, but just build+load a kernel module, which takes few seconds, e.g. [2]. I'm aware of your kvm-kmod (really helpful!), but Linux updates require patching it, which in turn makes it incompatible with older Linux releases.
- Automated building, testing (and soon nightly builds as well).
- Extensive emulator tests [3] (generic virtualization tests also on the way).
- Inspired by Bareflank, support for 3rd party "extensions". [4]

Of course, I'm aware that KVM is much more mature/compatible/performant that HAXM and many other alternatives. My point is that it might be a bit more comfortable to work with HAXM (and we are trying to improve this further every day).

Another point concerns what I feel is the raison d'etre for HAXM:

There seems to be an unnecessary fragmentation of hypervisor APIs: KVM (Linux), HVF (Mac), WHPX (Windows), which resembles the fragmentation of graphics APIs. Afaik, no cross-platform type-2 hypervisor exists aside from HAXM and Bareflank. This is fine for QEMU, since all major accelerators are implemented, but other smaller projects might suffer from this. Of course, we could wrap all APIs under a single userland library, but aside from adding another layer of abstraction/translation, the underlying problem
persists: Each new feature needs to be re-implemented inside each hypervisor (and HVF/WHPX are closed-source!).

Type-2 hypervisors don't *need* to be tightly coupled to a specific kernel:
If you look at HAXM, a thin platform-specific layer exists, the rest is shared code. I believe that's much more convenient.

Cheers,
Alexandro

[1]
https://android-review.googlesource.com/c/platform/external/qemu/+/643674/
[2] https://travis-ci.com/intel/haxm/builds/91615353
[3] https://github.com/intel/haxm/blob/master/tests/test_emulator.cpp
[4] https://github.com/intel/haxm/pull/121

Quoted 16 lines · Paolo Bonzini

Quoted 1 line · Alexandro Sanchez Bach
Intel HAXM supports now 32-bit and 64-bit Linux hosts. This patch
includes
Quoted 3 lines
the corresponding userland changes.
Since the Darwin userland backend is POSIX-compliant, the
hax-darwin.{c,h}
Quoted 1 line
files have been renamed to hax-posix.{c,h}. This prefix is consistent
with
Quoted 1 line
the naming used in the rest of QEMU.
What's the advantage of HAXM when Linux hosts can just run KVM? I guess avoiding bitrot?
Paolo

Kamil Rytarowski

2018-11-22 07:23 UTC #5 lore raw

In reply to Paolo Bonzini

Quoted 4 lines · Paolo Bonzini

Quoted 3 lines · Alexandro Sanchez Bach
Intel HAXM supports now 32-bit and 64-bit Linux hosts. This patch includes the corresponding userland changes.

Quoted 9 lines

Quoted 3 lines
Since the Darwin userland backend is POSIX-compliant, the hax-darwin.{c,h} files have been renamed to hax-posix.{c,h}. This prefix is consistent with the naming used in the rest of QEMU.
What's the advantage of HAXM when Linux hosts can just run KVM? I guess avoiding bitrot?
Paolo

This patch is also useful for NetBSD, even if it's not a Linux host. There is a driver in progress again (thanks to the newly added Linux port, it's now much easier to get done).

I recommend to merge this patch.

Kamil Rytarowski

2018-11-24 23:50 UTC #6 lore raw

Quoted 12 lines · Kamil Rytarowski

Quoted 11 lines · Paolo Bonzini
Quoted 6 lines · Alexandro Sanchez Bach
Intel HAXM supports now 32-bit and 64-bit Linux hosts. This patch includes the corresponding userland changes.
Since the Darwin userland backend is POSIX-compliant, the hax-darwin.{c,h} files have been renamed to hax-posix.{c,h}. This prefix is consistent with the naming used in the rest of QEMU.
What's the advantage of HAXM when Linux hosts can just run KVM? I guess avoiding bitrot?

Quoted 9 lines

Quoted 2 lines
Paolo
This patch is also useful for NetBSD, even if it's not a Linux host. There is a driver in progress again (thanks to the newly added Linux port, it's now much easier to get done).
I recommend to merge this patch.

For the record, I've a functional version of HAXM for NetBSD as host. Once you will merge this patch, I will submit another one to configure to enable haxm for NetBSD.

I need to keep the patch by Alexandro in a local copy of qemu.

Paolo Bonzini

2018-11-25 17:15 UTC #7 lore raw

Quoted 20 lines · Kamil Rytarowski

Quoted 19 lines · Kamil Rytarowski
Quoted 4 lines · Paolo Bonzini
Quoted 3 lines · Alexandro Sanchez Bach
Intel HAXM supports now 32-bit and 64-bit Linux hosts. This patch includes the corresponding userland changes.
Quoted 9 lines
Quoted 3 lines
Since the Darwin userland backend is POSIX-compliant, the hax-darwin.{c,h} files have been renamed to hax-posix.{c,h}. This prefix is consistent with the naming used in the rest of QEMU.
What's the advantage of HAXM when Linux hosts can just run KVM? I guess avoiding bitrot?
Paolo
This patch is also useful for NetBSD, even if it's not a Linux host. There is a driver in progress again (thanks to the newly added Linux port, it's now much easier to get done).

Quoted 8 lines

Quoted 2 lines
I recommend to merge this patch.
For the record, I've a functional version of HAXM for NetBSD as host. Once you will merge this patch, I will submit another one to configure to enable haxm for NetBSD.
I need to keep the patch by Alexandro in a local copy of qemu.

Sure, it will be accepted for the release after 3.1.

Paolo

Kamil Rytarowski

2019-01-08 17:11 UTC #8 lore raw

Quoted 28 lines · Paolo Bonzini

Quoted 27 lines · Kamil Rytarowski
Quoted 12 lines · Kamil Rytarowski
Quoted 11 lines · Paolo Bonzini
Quoted 6 lines · Alexandro Sanchez Bach
Intel HAXM supports now 32-bit and 64-bit Linux hosts. This patch includes the corresponding userland changes.
Since the Darwin userland backend is POSIX-compliant, the hax-darwin.{c,h} files have been renamed to hax-posix.{c,h}. This prefix is consistent with the naming used in the rest of QEMU.
What's the advantage of HAXM when Linux hosts can just run KVM? I guess avoiding bitrot?
Quoted 9 lines
Quoted 2 lines
Paolo
This patch is also useful for NetBSD, even if it's not a Linux host. There is a driver in progress again (thanks to the newly added Linux port, it's now much easier to get done).
I recommend to merge this patch.
For the record, I've a functional version of HAXM for NetBSD as host. Once you will merge this patch, I will submit another one to configure to enable haxm for NetBSD.

Quoted 7 lines

Quoted 1 line
I need to keep the patch by Alexandro in a local copy of qemu.
Sure, it will be accepted for the release after 3.1.
Paolo

Ping.

Paolo Bonzini

2019-01-08 18:27 UTC #9 lore raw

Quoted 31 lines · Kamil Rytarowski

Quoted 30 lines · Paolo Bonzini
Quoted 28 lines · Kamil Rytarowski
Quoted 12 lines · Kamil Rytarowski
Quoted 11 lines · Paolo Bonzini
Quoted 6 lines · Alexandro Sanchez Bach
Intel HAXM supports now 32-bit and 64-bit Linux hosts. This patch includes the corresponding userland changes.
Since the Darwin userland backend is POSIX-compliant, the hax-darwin.{c,h} files have been renamed to hax-posix.{c,h}. This prefix is consistent with the naming used in the rest of QEMU.
What's the advantage of HAXM when Linux hosts can just run KVM? I guess avoiding bitrot?
Quoted 9 lines
Quoted 2 lines
Paolo
This patch is also useful for NetBSD, even if it's not a Linux host. There is a driver in progress again (thanks to the newly added Linux port, it's now much easier to get done).
I recommend to merge this patch.
For the record, I've a functional version of HAXM for NetBSD as host. Once you will merge this patch, I will submit another one to configure to enable haxm for NetBSD.
I need to keep the patch by Alexandro in a local copy of qemu.

Quoted 7 lines

Quoted 5 lines
Sure, it will be accepted for the release after 3.1.
Paolo
Ping.

It's already queued, but the pull request was pretty big and it had some issues (quite an understatement).

Paolo

Philippe Mathieu-Daudé

2019-01-09 11:00 UTC #10 lore raw

Quoted 34 lines · Paolo Bonzini

Quoted 33 lines · Kamil Rytarowski
Quoted 32 lines · Paolo Bonzini
Quoted 20 lines · Kamil Rytarowski
Quoted 19 lines · Kamil Rytarowski
Quoted 4 lines · Paolo Bonzini
Quoted 3 lines · Alexandro Sanchez Bach
Intel HAXM supports now 32-bit and 64-bit Linux hosts. This patch includes the corresponding userland changes.
Quoted 9 lines
Quoted 3 lines
Since the Darwin userland backend is POSIX-compliant, the hax-darwin.{c,h} files have been renamed to hax-posix.{c,h}. This prefix is consistent with the naming used in the rest of QEMU.
What's the advantage of HAXM when Linux hosts can just run KVM? I guess avoiding bitrot?
Paolo
This patch is also useful for NetBSD, even if it's not a Linux host. There is a driver in progress again (thanks to the newly added Linux port, it's now much easier to get done).
Quoted 8 lines
Quoted 2 lines
I recommend to merge this patch.
For the record, I've a functional version of HAXM for NetBSD as host. Once you will merge this patch, I will submit another one to configure to enable haxm for NetBSD.
I need to keep the patch by Alexandro in a local copy of qemu.
Sure, it will be accepted for the release after 3.1.

Quoted 8 lines

Quoted 5 lines
Quoted 3 lines
Paolo
Ping.
It's already queued, but the pull request was pretty big and it had some issues (quite an understatement).

Hopefully you queued Alexandro's v2 (which isn't named v2) with Alex comments fixed, thanks!

Phil.

Kamil Rytarowski

2019-02-02 15:15 UTC #11 lore raw

In reply to Paolo Bonzini

Quoted 28 lines · Paolo Bonzini

Quoted 27 lines · Kamil Rytarowski
Quoted 12 lines · Kamil Rytarowski
Quoted 11 lines · Paolo Bonzini
Quoted 6 lines · Alexandro Sanchez Bach
Intel HAXM supports now 32-bit and 64-bit Linux hosts. This patch includes the corresponding userland changes.
Since the Darwin userland backend is POSIX-compliant, the hax-darwin.{c,h} files have been renamed to hax-posix.{c,h}. This prefix is consistent with the naming used in the rest of QEMU.
What's the advantage of HAXM when Linux hosts can just run KVM? I guess avoiding bitrot?
Quoted 9 lines
Quoted 2 lines
Paolo
This patch is also useful for NetBSD, even if it's not a Linux host. There is a driver in progress again (thanks to the newly added Linux port, it's now much easier to get done).
I recommend to merge this patch.
For the record, I've a functional version of HAXM for NetBSD as host. Once you will merge this patch, I will submit another one to configure to enable haxm for NetBSD.

Quoted 7 lines

Quoted 1 line
I need to keep the patch by Alexandro in a local copy of qemu.
Sure, it will be accepted for the release after 3.1.
Paolo

I've pushed the haxm patch for NetBSD through qemu-trivial.

https://lists.gnu.org/archive/html/qemu-trivial/2019-01/msg00161.html

A proof that it is usable:

http://blog.netbsd.org/tnf/entry/the_hardware_assisted_virtualization_challenge