Skip to contentSkip to conversation

WHPX support for Arm

RFC Superseded Mohamed Mediouni 4 participants 26 messages Jul 31, 2025 – Aug 1, 2025 Arm/Amlogic Meson SoC support

v1 of 22 Superseded Latest is v22 →

lore
Share & tools
lore
#0 lore raw

cover WHPX support for Arm

This one took way longer for me to publish than I should have.

There are a number of lingering bugs in this one including u-boot not working.

Interrupt controller save/restore is entirely missing in this RFC, and some other state bits are likely still missing too.

ITS not blocked by default yet, remember to use its=off when testing this series. You might also want the GICv3 + GICv2m support patch as part of the HVF vGIC patch series, which is not duplicated here.

PS: on both this and HVF, interrupt controller initialisation needs to be done early so I ended up with hardcoded addresses. Wonder if the right way to go might be to defer virt and vCPU initialisation until late in the process post-gic_realize...

Other than that, this boots both EDK2 and Linux in SMP, when using devicetree or ACPI.

Mohamed Mediouni (9):
whpx: Move around files before introducing AArch64 support
whpx: reshuffle common code
whpx: common: use whpx_cpu_instance_init on x86 only
whpx: interrupt controller support
hw/virt: make Qemu aware that WHPX has a vGICv3
hw: intc: arm_gicv3_common: add whpx
whpx: add arm64 support
whpx: copy over memory tracking logic from hvf target/arm: cpu: mark WHPX as supporting PSCI 1.1

 accel/meson.build                             |   1 +
 accel/whpx/meson.build                        |   7 +
 {target/i386 => accel}/whpx/whpx-accel-ops.c  |   8 +-
 accel/whpx/whpx-common.c                      | 670 ++++++++++++++++
 hw/arm/virt.c                                 |   3 +
 hw/intc/arm_gicv3_common.c                    |   3 +
 hw/intc/arm_gicv3_whpx.c                      | 285 +++++++
 hw/intc/meson.build                           |   1 +
 .../whpx => include/system}/whpx-accel-ops.h  |   4 +-
 include/system/whpx-all.h                     |  12 +
 include/system/whpx-common.h                  |  22 +
 .../whpx => include/system}/whpx-internal.h   |  11 +-
 meson.build                                   |   5 +-
 target/arm/cpu.c                              |   3 +-
 target/arm/meson.build                        |   1 +
 target/arm/whpx/meson.build                   |   3 +
 target/arm/whpx/whpx-all.c                    | 744 ++++++++++++++++++
 target/i386/whpx/meson.build                  |   1 -
 target/i386/whpx/whpx-all.c                   | 524 +-----------
 target/i386/whpx/whpx-apic.c                  |   2 +-
 20 files changed, 1780 insertions(+), 530 deletions(-)
 create mode 100644 accel/whpx/meson.build
 rename {target/i386 => accel}/whpx/whpx-accel-ops.c (96%)
 create mode 100644 accel/whpx/whpx-common.c
 create mode 100644 hw/intc/arm_gicv3_whpx.c
 rename {target/i386/whpx => include/system}/whpx-accel-ops.h (92%)
 create mode 100644 include/system/whpx-all.h
 create mode 100644 include/system/whpx-common.h
 rename {target/i386/whpx => include/system}/whpx-internal.h (97%)
 create mode 100644 target/arm/whpx/meson.build
 create mode 100644 target/arm/whpx/whpx-all.c
Signature · 1 line
2.39.5 (Apple Git-154)
#1 lore raw

RFC 1/9 whpx: Move around files before introducing AArch64 support

Switch to a design where we can share whpx code between x86 and AArch64 when it makes sense to do so.

---
 accel/meson.build                                     | 1 +
 accel/whpx/meson.build                                | 6 ++++++
 {target/i386 => accel}/whpx/whpx-accel-ops.c          | 8 ++++++--
 {target/i386/whpx => include/system}/whpx-accel-ops.h | 0
 {target/i386/whpx => include/system}/whpx-internal.h  | 7 ++++++-
 target/i386/whpx/meson.build                          | 1 -
 target/i386/whpx/whpx-all.c                           | 4 ++--
 7 files changed, 21 insertions(+), 6 deletions(-)
 create mode 100644 accel/whpx/meson.build
 rename {target/i386 => accel}/whpx/whpx-accel-ops.c (96%)
 rename {target/i386/whpx => include/system}/whpx-accel-ops.h (100%)
 rename {target/i386/whpx => include/system}/whpx-internal.h (98%)
accel/meson.build +1 -0
index 25b0f100b5..de927a3b37 100644
@@ -6,6 +6,7 @@ user_ss.add(files('accel-user.c'))
 subdir('tcg')
 if have_system
   subdir('hvf')
+  subdir('whpx')
   subdir('qtest')
   subdir('kvm')
   subdir('xen')
accel/whpx/meson.build new file +6 -0
new file mode 100644
index 0000000000..659efb13c9
@@ -0,0 +1,6 @@
+whpx_ss = ss.source_set()
+whpx_ss.add(files(
+  'whpx-accel-ops.c',
+))
+
+specific_ss.add_all(when: 'CONFIG_WHPX', if_true: whpx_ss)
\ No newline at end of file
accel/whpx/whpx-accel-ops.c renamed from target/i386/whpx/whpx-accel-ops.c +6 -2
similarity index 96%
rename from target/i386/whpx/whpx-accel-ops.c
rename to accel/whpx/whpx-accel-ops.c
index da58805b1a..364d99b660 100644
@@ -16,8 +16,8 @@
 #include "qemu/guest-random.h"
 
 #include "system/whpx.h"
-#include "whpx-internal.h"
-#include "whpx-accel-ops.h"
+#include "system/whpx-internal.h"
+#include "system/whpx-accel-ops.h"
 
 static void *whpx_cpu_thread_fn(void *arg)
 {
@@ -80,7 +80,11 @@ static void whpx_kick_vcpu_thread(CPUState *cpu)
 
 static bool whpx_vcpu_thread_is_idle(CPUState *cpu)
 {
+#ifndef __aarch64__
     return !whpx_apic_in_platform();
+#else
+    return 0;
+#endif
 }
 
 static void whpx_accel_ops_class_init(ObjectClass *oc, const void *data)
include/system/whpx-accel-ops.h renamed from target/i386/whpx/whpx-accel-ops.h +0 -0
similarity index 100%
rename from target/i386/whpx/whpx-accel-ops.h
rename to include/system/whpx-accel-ops.h
include/system/whpx-internal.h renamed from target/i386/whpx/whpx-internal.h +6 -1
similarity index 98%
rename from target/i386/whpx/whpx-internal.h
rename to include/system/whpx-internal.h
index 6633e9c4ca..af1e46a491 100644
@@ -3,8 +3,9 @@
 
 #include <windows.h>
 #include <winhvplatform.h>
+#ifdef __x86_64__
 #include <winhvemulation.h>
-
+#endif
 typedef enum WhpxBreakpointState {
     WHPX_BP_CLEARED = 0,
     WHPX_BP_SET_PENDING,
@@ -97,12 +98,16 @@ void whpx_apic_get(DeviceState *s);
 
 /* Define function typedef */
 LIST_WINHVPLATFORM_FUNCTIONS(WHP_DEFINE_TYPE)
+#ifdef __x86_64__
 LIST_WINHVEMULATION_FUNCTIONS(WHP_DEFINE_TYPE)
+#endif
 LIST_WINHVPLATFORM_FUNCTIONS_SUPPLEMENTAL(WHP_DEFINE_TYPE)
 
 struct WHPDispatch {
     LIST_WINHVPLATFORM_FUNCTIONS(WHP_DECLARE_MEMBER)
+#ifdef __x86_64__
     LIST_WINHVEMULATION_FUNCTIONS(WHP_DECLARE_MEMBER)
+#endif
     LIST_WINHVPLATFORM_FUNCTIONS_SUPPLEMENTAL(WHP_DECLARE_MEMBER)
 };
 
target/i386/whpx/meson.build +0 -1
index 9c54aaad39..c3aaaff9fd 100644
@@ -1,5 +1,4 @@
 i386_system_ss.add(when: 'CONFIG_WHPX', if_true: files(
   'whpx-all.c',
   'whpx-apic.c',
-  'whpx-accel-ops.c',
 ))
target/i386/whpx/whpx-all.c +2 -2
index b72dcff3c8..5a431fc3c7 100644
@@ -31,8 +31,8 @@
 #include "accel/accel-cpu-target.h"
 #include <winerror.h>
 
-#include "whpx-internal.h"
-#include "whpx-accel-ops.h"
+#include "system/whpx-internal.h"
+#include "system/whpx-accel-ops.h"
 
 #include <winhvplatform.h>
 #include <winhvemulation.h>
Signature · 1 line
2.39.5 (Apple Git-154)
Reviewed-by on v1 1/9
#2 lore raw

Re: [RFC 1/9] whpx: Move around files before introducing AArch64 support

Hi Mohamed,

Quoted 12 lines · Mohamed Mediouni

Switch to a design where we can share whpx code between x86 and AArch64 when it makes sense to do so.

---
  accel/meson.build                                     | 1 +
  accel/whpx/meson.build                                | 6 ++++++
  {target/i386 => accel}/whpx/whpx-accel-ops.c          | 8 ++++++--
  {target/i386/whpx => include/system}/whpx-accel-ops.h | 0
  {target/i386/whpx => include/system}/whpx-internal.h  | 7 ++++++-
  target/i386/whpx/meson.build                          | 1 -
  target/i386/whpx/whpx-all.c                           | 4 ++--
  7 files changed, 21 insertions(+), 6 deletions(-)
Quoted 4 lines

create mode 100644 accel/whpx/meson.build
rename {target/i386 => accel}/whpx/whpx-accel-ops.c (96%) rename {target/i386/whpx => include/system}/whpx-accel-ops.h (100%) rename {target/i386/whpx => include/system}/whpx-internal.h (98%)

Missing updates in MAINTAINERS, otherwise:

#3 lore raw

Re: [RFC 1/9] whpx: Move around files before introducing AArch64 support

In reply to Mohamed Mediouni

accel/whpx/meson.build line 6

Quoted 10 lines · Mohamed Mediouni
accel/whpx/meson.build new file +6 -0
@@ -0,0 +1,6 @@
+whpx_ss = ss.source_set()
+whpx_ss.add(files(
+  'whpx-accel-ops.c',
+))
+
+specific_ss.add_all(when: 'CONFIG_WHPX', if_true: whpx_ss)
\ No newline at end of file

Add the newline.

Quoted 19 lines
accel/whpx/whpx-accel-ops.c renamed from target/i386/whpx/whpx-accel-ops.c +2 -2
similarity index 96%
rename from target/i386/whpx/whpx-accel-ops.c
rename to accel/whpx/whpx-accel-ops.c
index da58805b1a..364d99b660 100644
@@ -16,8 +16,8 @@
  #include "qemu/guest-random.h"
 
  #include "system/whpx.h"
-#include "whpx-internal.h"
-#include "whpx-accel-ops.h"
+#include "system/whpx-internal.h"
+#include "system/whpx-accel-ops.h"
 
  static void *whpx_cpu_thread_fn(void *arg)
  {

accel/whpx/whpx-accel-ops.c static void whpx_kick_vcpu_thread(CPUState *cpu) · line 88

Quoted 9 lines
accel/whpx/whpx-accel-ops.c renamed from target/i386/whpx/whpx-accel-ops.c +4 -0
@@ -80,7 +80,11 @@ static void whpx_kick_vcpu_thread(CPUState *cpu)
 
  static bool whpx_vcpu_thread_is_idle(CPUState *cpu)
  {
+#ifndef __aarch64__
      return !whpx_apic_in_platform();
+#else
+    return 0;
+#endif
  }

Better as code movement only.
Save new code for a subsequent patch.

Quoted 19 lines
include/system/whpx-internal.h renamed from target/i386/whpx/whpx-internal.h +3 -1
@@ -3,8 +3,9 @@
 
  #include <windows.h>
  #include <winhvplatform.h>
+#ifdef __x86_64__
  #include <winhvemulation.h>
-
+#endif
  typedef enum WhpxBreakpointState {
      WHPX_BP_CLEARED = 0,
      WHPX_BP_SET_PENDING,
@@ -97,12 +98,16 @@ void whpx_apic_get(DeviceState *s);
 
  /* Define function typedef */
  LIST_WINHVPLATFORM_FUNCTIONS(WHP_DEFINE_TYPE)
+#ifdef __x86_64__
  LIST_WINHVEMULATION_FUNCTIONS(WHP_DEFINE_TYPE)

include/system/whpx-internal.h void whpx_apic_get(DeviceState *s); · line 112

Quoted 10 lines
include/system/whpx-internal.h renamed from target/i386/whpx/whpx-internal.h +3 -0
+#endif
  LIST_WINHVPLATFORM_FUNCTIONS_SUPPLEMENTAL(WHP_DEFINE_TYPE)
 
  struct WHPDispatch {
      LIST_WINHVPLATFORM_FUNCTIONS(WHP_DECLARE_MEMBER)
+#ifdef __x86_64__
      LIST_WINHVEMULATION_FUNCTIONS(WHP_DECLARE_MEMBER)
+#endif
      LIST_WINHVPLATFORM_FUNCTIONS_SUPPLEMENTAL(WHP_DECLARE_MEMBER)
  };

Likewise.

r~

#4 lore raw

RFC 2/9 whpx: reshuffle common code

In reply to Mohamed Mediouni

Some code can be shared between x86_64 and arm64 WHPX. Do so as much as reasonable.

---
 accel/whpx/meson.build          |   1 +
 accel/whpx/whpx-common.c        | 578 ++++++++++++++++++++++++++++++++
 include/system/whpx-accel-ops.h |   4 +-
 include/system/whpx-all.h       |  12 +
 include/system/whpx-common.h    |  22 ++
 include/system/whpx-internal.h  |   4 +-
 target/i386/whpx/whpx-all.c     | 520 +---------------------------
 target/i386/whpx/whpx-apic.c    |   2 +-
 8 files changed, 622 insertions(+), 521 deletions(-)
 create mode 100644 accel/whpx/whpx-common.c
 create mode 100644 include/system/whpx-all.h
 create mode 100644 include/system/whpx-common.h
accel/whpx/meson.build +1 -0
index 659efb13c9..344b2b45d0 100644
@@ -1,6 +1,7 @@
 whpx_ss = ss.source_set()
 whpx_ss.add(files(
   'whpx-accel-ops.c',
+  'whpx-common.c'
 ))
 
 specific_ss.add_all(when: 'CONFIG_WHPX', if_true: whpx_ss)
\ No newline at end of file
accel/whpx/whpx-common.c new file +578 -0

578 lines not on this page · raw message

include/system/whpx-accel-ops.h +2 -2
index 54cfc25a14..ed9d4c49f4 100644
@@ -7,8 +7,8 @@
  * See the COPYING file in the top-level directory.
  */
 
-#ifndef TARGET_I386_WHPX_ACCEL_OPS_H
-#define TARGET_I386_WHPX_ACCEL_OPS_H
+#ifndef SYSTEM_WHPX_ACCEL_OPS_H
+#define SYSTEM_WHPX_ACCEL_OPS_H
 
 #include "system/cpus.h"
 
include/system/whpx-all.h new file +12 -0
new file mode 100644
index 0000000000..382b028c83
@@ -0,0 +1,12 @@
+#ifndef SYSTEM_WHPX_ALL_H
+#define SYSTEM_WHPX_ALL_H
+
+/* Called by whpx-common */
+int whpx_vcpu_run(CPUState *cpu);
+void whpx_get_registers(CPUState *cpu);
+void whpx_set_registers(CPUState *cpu, int level);
+int whpx_accel_init(AccelState *as, MachineState *ms);
+#ifdef __x86_64__
+void whpx_cpu_instance_init(CPUState *cs);
+#endif
+#endif
include/system/whpx-common.h new file +22 -0
new file mode 100644
index 0000000000..57c73448a7
@@ -0,0 +1,22 @@
+#ifndef SYSTEM_WHPX_COMMON_H
+#define SYSTEM_WHPX_COMMON_H
+
+struct AccelCPUState {
+#ifdef __x86_64__
+    WHV_EMULATOR_HANDLE emulator;
+    bool window_registered;
+    bool interruptable;
+    bool ready_for_pic_interrupt;
+    uint64_t tpr;
+    uint64_t apic_base;
+    bool interruption_pending;
+#endif
+    /* Must be the last field as it may have a tail */
+    WHV_RUN_VP_EXIT_CONTEXT exit_ctx;
+};
+
+int whpx_first_vcpu_starting(CPUState *cpu);
+int whpx_last_vcpu_stopping(CPUState *cpu);
+void whpx_memory_init(void);
+struct whpx_breakpoint *whpx_lookup_breakpoint_by_addr(uint64_t address);
+#endif
\ No newline at end of file
include/system/whpx-internal.h +2 -2
index af1e46a491..4b613c4bb5 100644
@@ -1,5 +1,5 @@
-#ifndef TARGET_I386_WHPX_INTERNAL_H
-#define TARGET_I386_WHPX_INTERNAL_H
+#ifndef SYSTEM_WHPX_INTERNAL_H
+#define SYSTEM_WHPX_INTERNAL_H
 
 #include <windows.h>
 #include <winhvplatform.h>
target/i386/whpx/whpx-all.c +4 -516

563 lines not on this page · raw message

target/i386/whpx/whpx-apic.c +1 -1
index e1ef6d4e6d..badb404b63 100644
@@ -18,7 +18,7 @@
 #include "hw/pci/msi.h"
 #include "system/hw_accel.h"
 #include "system/whpx.h"
-#include "whpx-internal.h"
+#include "system/whpx-internal.h"
 
 struct whpx_lapic_state {
     struct {
Signature · 1 line
2.39.5 (Apple Git-154)
#5 lore raw

RFC 3/9 whpx: common: use whpx_cpu_instance_init on x86 only

In reply to Mohamed Mediouni

We aren't using it on arm64.

---
 accel/whpx/whpx-common.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
accel/whpx/whpx-common.c +2 -1
index 0b23deb7c4..43d0200afd 100644
@@ -486,9 +486,10 @@ static void whpx_set_kernel_irqchip(Object *obj, Visitor *v,
 
 static void whpx_cpu_accel_class_init(ObjectClass *oc, const void *data)
 {
+#ifdef __x86_64__
     AccelCPUClass *acc = ACCEL_CPU_CLASS(oc);
-
     acc->cpu_instance_init = whpx_cpu_instance_init;
+#endif
 }
 
 static const TypeInfo whpx_cpu_accel_type = {
Signature · 1 line
2.39.5 (Apple Git-154)
#6 lore raw

Re: [RFC 3/9] whpx: common: use whpx_cpu_instance_init on x86 only

Hi Mohamed,

Quoted 12 lines · Mohamed Mediouni

We aren't using it on arm64.

---
  accel/whpx/whpx-common.c | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)
accel/whpx/whpx-common.c +0 -0
index 0b23deb7c4..43d0200afd 100644

accel/whpx/whpx-common.c static void whpx_set_kernel_irqchip(Object *obj, Visitor *v, · line 493

Quoted 9 lines
accel/whpx/whpx-common.c +2 -1
@@ -486,9 +486,10 @@ static void whpx_set_kernel_irqchip(Object *obj, Visitor *v,
 
  static void whpx_cpu_accel_class_init(ObjectClass *oc, const void *data)
  {
+#ifdef __x86_64__
      AccelCPUClass *acc = ACCEL_CPU_CLASS(oc);
-
      acc->cpu_instance_init = whpx_cpu_instance_init;
+#endif
  }

I'd rather keep whpx-*common* without any #ifdef'ry: add hooks when architecture-specific code need to be handled, and add empty arch stub when nothing to be done there (preferably with a comment in the stub).

#7 lore raw

RFC 4/9 whpx: interrupt controller support

In reply to Mohamed Mediouni
---
 hw/intc/arm_gicv3_whpx.c | 285 +++++++++++++++++++++++++++++++++++++++
 hw/intc/meson.build      |   1 +
 2 files changed, 286 insertions(+)
 create mode 100644 hw/intc/arm_gicv3_whpx.c
hw/intc/arm_gicv3_whpx.c new file +285 -0
new file mode 100644
index 0000000000..97ab164518
@@ -0,0 +1,285 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * ARM Generic Interrupt Controller using HVF platform support
+ *
+ * Copyright (c) 2025 Mohamed Mediouni
+ * Based on vGICv3 KVM code by Pavel Fedin
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "hw/intc/arm_gicv3_common.h"
+#include "qemu/error-report.h"
+#include "qemu/module.h"
+#include "system/runstate.h"
+#include "system/whpx.h"
+#include "system/whpx-internal.h"
+#include "gicv3_internal.h"
+#include "vgic_common.h"
+#include "qom/object.h"
+#include "target/arm/cpregs.h"
+
+#include "hw/arm/bsa.h"
+#include <winhvplatform.h>
+#include <winhvplatformdefs.h>
+#include <winnt.h>
+
+struct WHPXARMGICv3Class {
+    ARMGICv3CommonClass parent_class;
+    DeviceRealize parent_realize;
+    ResettablePhases parent_phases;
+};
+
+#define TYPE_WHPX_GICV3 "whpx-arm-gicv3"
+typedef struct WHPXARMGICv3Class WHPXARMGICv3Class;
+
+/* This is reusing the GICv3State typedef from ARM_GICV3_ITS_COMMON */
+DECLARE_OBJ_CHECKERS(GICv3State, WHPXARMGICv3Class,
+                     WHPX_GICV3, TYPE_WHPX_GICV3);
+
+static void whpx_gicv3_check(GICv3State *s)
+{
+}
+
+static void whpx_gicv3_put_cpu(CPUState *cpu_state, run_on_cpu_data arg)
+{
+}
+
+static void whpx_gicv3_put(GICv3State *s)
+{
+    int ncpu;
+
+    whpx_gicv3_check(s);
+
+    for (ncpu = 0; ncpu < s->num_cpu; ncpu++) {
+        run_on_cpu_data data;
+        data.host_ptr = &s->cpu[ncpu];
+        run_on_cpu(s->cpu[ncpu].cpu, whpx_gicv3_put_cpu, data);
+    }
+}
+
+static void whpx_gicv3_get_cpu(CPUState *cpu_state, run_on_cpu_data arg)
+{
+}
+
+static void whpx_gicv3_get(GICv3State *s)
+{
+    int ncpu;
+
+    for (ncpu = 0; ncpu < s->num_cpu; ncpu++) {
+        run_on_cpu_data data;
+        data.host_ptr = &s->cpu[ncpu];
+        run_on_cpu(s->cpu[ncpu].cpu, whpx_gicv3_get_cpu, data);
+    }
+}
+
+static void whpx_gicv3_set_irq(void *opaque, int irq, int level)
+{
+    struct whpx_state *whpx = &whpx_global;
+
+    GICv3State *s = (GICv3State *)opaque;
+    if (irq > s->num_irq) {
+        return;
+    }
+    WHV_INTERRUPT_TYPE interrupt_type = WHvArm64InterruptTypeFixed;
+    WHV_INTERRUPT_CONTROL interrupt_control = 
+    {interrupt_type = WHvArm64InterruptTypeFixed,
+    .RequestedVector = GIC_INTERNAL + irq, .InterruptControl.Asserted=1};
+
+    whp_dispatch.WHvRequestInterrupt(whpx->partition, &interrupt_control, sizeof(interrupt_control));
+
+    if (!level) {
+        interrupt_control.InterruptControl.Asserted = 0;
+        whp_dispatch.WHvRequestInterrupt(whpx->partition, &interrupt_control, sizeof(interrupt_control));
+    }
+
+}
+
+static void whpx_gicv3_icc_reset(CPUARMState *env, const ARMCPRegInfo *ri)
+{
+    GICv3State *s;
+    GICv3CPUState *c;
+
+    c = (GICv3CPUState *)env->gicv3state;
+    s = c->gic;
+
+    c->icc_pmr_el1 = 0;
+    /*
+     * Architecturally the reset value of the ICC_BPR registers
+     * is UNKNOWN. We set them all to 0 here; when the kernel
+     * uses these values to program the ICH_VMCR_EL2 fields that
+     * determine the guest-visible ICC_BPR register values, the
+     * hardware's "writing a value less than the minimum sets
+     * the field to the minimum value" behaviour will result in
+     * them effectively resetting to the correct minimum value
+     * for the host GIC.
+     */
+    c->icc_bpr[GICV3_G0] = 0;
+    c->icc_bpr[GICV3_G1] = 0;
+    c->icc_bpr[GICV3_G1NS] = 0;
+
+    c->icc_sre_el1 = 0x7;
+    memset(c->icc_apr, 0, sizeof(c->icc_apr));
+    memset(c->icc_igrpen, 0, sizeof(c->icc_igrpen));
+
+    if (s->migration_blocker) {
+        return;
+    }
+
+    c->icc_ctlr_el1[GICV3_S] = c->icc_ctlr_el1[GICV3_NS];
+}
+
+static void whpx_gicv3_reset_hold(Object *obj, ResetType type)
+{
+    GICv3State *s = ARM_GICV3_COMMON(obj);
+    WHPXARMGICv3Class *kgc = WHPX_GICV3_GET_CLASS(s);
+
+    if (kgc->parent_phases.hold) {
+        kgc->parent_phases.hold(obj, type);
+    }
+
+    whpx_gicv3_put(s);
+}
+
+
+/*
+ * CPU interface registers of GIC needs to be reset on CPU reset.
+ * For the calling arm_gicv3_icc_reset() on CPU reset, we register
+ * below ARMCPRegInfo. As we reset the whole cpu interface under single
+ * register reset, we define only one register of CPU interface instead
+ * of defining all the registers.
+ */
+static const ARMCPRegInfo gicv3_cpuif_reginfo[] = {
+    { .name = "ICC_CTLR_EL1", .state = ARM_CP_STATE_BOTH,
+      .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 12, .opc2 = 4,
+      /*
+       * If ARM_CP_NOP is used, resetfn is not called,
+       * So ARM_CP_NO_RAW is appropriate type.
+       */
+      .type = ARM_CP_NO_RAW,
+      .access = PL1_RW,
+      .readfn = arm_cp_read_zero,
+      .writefn = arm_cp_write_ignore,
+      /*
+       * We hang the whole cpu interface reset routine off here
+       * rather than parcelling it out into one little function
+       * per register
+       */
+      .resetfn = whpx_gicv3_icc_reset,
+    },
+};
+
+static void whpx_set_reg(CPUState *cpu, WHV_REGISTER_NAME reg, WHV_REGISTER_VALUE val)
+{
+    struct whpx_state *whpx = &whpx_global;
+    HRESULT hr;
+    hr = whp_dispatch.WHvSetVirtualProcessorRegisters(whpx->partition, cpu->cpu_index,
+         &reg, 1, &val);
+
+    if (FAILED(hr)) {
+        error_report("WHPX: Failed to set register %08x, hr=%08lx", reg, hr);
+    }
+}
+
+static void whpx_gicv3_realize(DeviceState *dev, Error **errp)
+{
+    GICv3State *s = WHPX_GICV3(dev);
+    WHPXARMGICv3Class *kgc = WHPX_GICV3_GET_CLASS(s);
+    Error *local_err = NULL;
+    int i;
+
+    kgc->parent_realize(dev, &local_err);
+    if (local_err) {
+        error_propagate(errp, local_err);
+        return;
+    }
+
+    if (s->revision != 3) {
+        error_setg(errp, "unsupported GIC revision %d for platform GIC",
+                   s->revision);
+    }
+
+    if (s->security_extn) {
+        error_setg(errp, "the platform vGICv3 does not implement the "
+                   "security extensions");
+        return;
+    }
+
+    if (s->nmi_support) {
+        error_setg(errp, "NMI is not supported with the platform GIC");
+        return;
+    }
+
+    if (s->nb_redist_regions > 1) {
+        error_setg(errp, "Multiple VGICv3 redistributor regions are not "
+                   "supported by WHPX");
+        error_append_hint(errp, "A maximum of %d VCPUs can be used",
+                          s->redist_region_count[0]);
+        return;
+    }
+
+    gicv3_init_irqs_and_mmio(s, whpx_gicv3_set_irq, NULL);
+/*
+    WHV_ARM64_IC_PARAMETERS param = {
+        .EmulationMode = WHvArm64IcEmulationModeGicV3,
+        .GicV3Parameters = {
+            .GicdBaseAddress = s->iomem_dist.addr,
+            .GicPpiPerformanceMonitorsInterrupt = VIRTUAL_PMU_IRQ,
+            .GicPpiOverflowInterruptFromCntv = ARCH_TIMER_VIRT_IRQ
+        }
+    };
+    hr = whp_dispatch.WHvSetPartitionProperty(
+        whpx->partition,
+        WHvPartitionPropertyCodeArm64IcParameters,
+        &param,
+        sizeof(WHV_ARM64_IC_PARAMETERS));
+    if (FAILED(hr)) {
+        error_report("WHPX: failed to set up interrupt controller");
+    }
+*/
+    for (i = 0; i < s->num_cpu; i++) {
+        CPUState *cpu_state = qemu_get_cpu(i);
+        ARMCPU *cpu = ARM_CPU(cpu_state);
+        WHV_REGISTER_VALUE val = {.Reg64 = 0x080A0000 + (0x20000 * i)};
+        whpx_set_reg(cpu_state, WHvArm64RegisterGicrBaseGpa, val);
+        define_arm_cp_regs(cpu, gicv3_cpuif_reginfo);
+        printf("addr: 0x%lld\n", s->redist_regions[0].iomem.addr);
+    }
+
+    if (s->maint_irq) {
+        error_setg(errp, "Nested virtualisation not currently supported by WHPX.");
+        return;
+    }
+}
+
+static void whpx_gicv3_class_init(ObjectClass *klass, const void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(klass);
+    ResettableClass *rc = RESETTABLE_CLASS(klass);
+    ARMGICv3CommonClass *agcc = ARM_GICV3_COMMON_CLASS(klass);
+    WHPXARMGICv3Class *kgc = WHPX_GICV3_CLASS(klass);
+
+    agcc->pre_save = whpx_gicv3_get;
+    agcc->post_load = whpx_gicv3_put;
+
+    device_class_set_parent_realize(dc, whpx_gicv3_realize,
+                                    &kgc->parent_realize);
+    resettable_class_set_parent_phases(rc, NULL, whpx_gicv3_reset_hold, NULL,
+                                       &kgc->parent_phases);
+}
+
+static const TypeInfo whpx_arm_gicv3_info = {
+    .name = TYPE_WHPX_GICV3,
+    .parent = TYPE_ARM_GICV3_COMMON,
+    .instance_size = sizeof(GICv3State),
+    .class_init = whpx_gicv3_class_init,
+    .class_size = sizeof(WHPXARMGICv3Class),
+};
+
+static void whpx_gicv3_register_types(void)
+{
+    type_register_static(&whpx_arm_gicv3_info);
+}
+
+type_init(whpx_gicv3_register_types)
hw/intc/meson.build +1 -0
index 3137521a4a..9342ff0e2c 100644
@@ -41,6 +41,7 @@ specific_ss.add(when: 'CONFIG_APIC', if_true: files('apic.c', 'apic_common.c'))
 specific_ss.add(when: 'CONFIG_ARM_GIC', if_true: files('arm_gicv3_cpuif_common.c'))
 specific_ss.add(when: 'CONFIG_ARM_GICV3', if_true: files('arm_gicv3_cpuif.c'))
 specific_ss.add(when: 'CONFIG_ARM_GIC_KVM', if_true: files('arm_gic_kvm.c'))
+specific_ss.add(when: 'CONFIG_WHPX', if_true: files('arm_gicv3_whpx.c'))
 specific_ss.add(when: ['CONFIG_ARM_GIC_KVM', 'TARGET_AARCH64'], if_true: files('arm_gicv3_kvm.c', 'arm_gicv3_its_kvm.c'))
 specific_ss.add(when: 'CONFIG_ARM_V7M', if_true: files('armv7m_nvic.c'))
 specific_ss.add(when: 'CONFIG_GRLIB', if_true: files('grlib_irqmp.c'))
Signature · 1 line
2.39.5 (Apple Git-154)
#8 lore raw

RFC 5/9 hw/virt: make Qemu aware that WHPX has a vGICv3

In reply to Mohamed Mediouni

WHPX is a vGICv3-only target without vGICv2 or user-mode irqchip support.

---
 hw/arm/virt.c | 3 +++
 1 file changed, 3 insertions(+)
hw/arm/virt.c +3 -0
index ef6be3660f..4996c2075e 100644
@@ -49,6 +49,7 @@
 #include "system/tcg.h"
 #include "system/kvm.h"
 #include "system/hvf.h"
+#include "system/whpx.h"
 #include "system/qtest.h"
 #include "hw/loader.h"
 #include "qapi/error.h"
@@ -2058,6 +2059,8 @@ static void finalize_gic_version(VirtMachineState *vms)
         /* KVM w/o kernel irqchip can only deal with GICv2 */
         gics_supported |= VIRT_GIC_VERSION_2_MASK;
         accel_name = "KVM with kernel-irqchip=off";
+    } else if (whpx_enabled()) {
+        gics_supported |= VIRT_GIC_VERSION_3_MASK;
     } else if (tcg_enabled() || hvf_enabled() || qtest_enabled())  {
         gics_supported |= VIRT_GIC_VERSION_2_MASK;
         if (module_object_class_by_name("arm-gicv3")) {
Signature · 1 line
2.39.5 (Apple Git-154)
#9 lore raw

Re: [RFC 5/9] hw/virt: make Qemu aware that WHPX has a vGICv3

Quoted 21 lines · Mohamed Mediouni

WHPX is a vGICv3-only target without vGICv2 or user-mode irqchip support.

---
  hw/arm/virt.c | 3 +++
  1 file changed, 3 insertions(+)
hw/arm/virt.c +1 -0
index ef6be3660f..4996c2075e 100644
@@ -49,6 +49,7 @@
  #include "system/tcg.h"
  #include "system/kvm.h"
  #include "system/hvf.h"
+#include "system/whpx.h"
  #include "system/qtest.h"
  #include "hw/loader.h"
  #include "qapi/error.h"

hw/arm/virt.c static void finalize_gic_version(VirtMachineState *vms) · line 2066

Quoted 8 lines
hw/arm/virt.c +2 -0
@@ -2058,6 +2059,8 @@ static void finalize_gic_version(VirtMachineState *vms)
          /* KVM w/o kernel irqchip can only deal with GICv2 */
          gics_supported |= VIRT_GIC_VERSION_2_MASK;
          accel_name = "KVM with kernel-irqchip=off";
+    } else if (whpx_enabled()) {
+        gics_supported |= VIRT_GIC_VERSION_3_MASK;
      } else if (tcg_enabled() || hvf_enabled() || qtest_enabled())  {
          gics_supported |= VIRT_GIC_VERSION_2_MASK;
          if (module_object_class_by_name("arm-gicv3")) {

Should this patch be the last of the series to avoid breaking git bisectability?

#10 lore raw

Re: [RFC 5/9] hw/virt: make Qemu aware that WHPX has a vGICv3

Quoted 27 lines · Philippe Mathieu-Daudé
Quoted 22 lines · Mohamed Mediouni

WHPX is a vGICv3-only target without vGICv2 or user-mode irqchip support.

---
 hw/arm/virt.c | 3 +++
 1 file changed, 3 insertions(+)
hw/arm/virt.c +1 -0
index ef6be3660f..4996c2075e 100644
@@ -49,6 +49,7 @@
 #include "system/tcg.h"
 #include "system/kvm.h"
 #include "system/hvf.h"
+#include "system/whpx.h"
 #include "system/qtest.h"
 #include "hw/loader.h"
 #include "qapi/error.h"
@@ -2058,6 +2059,8 @@ static void finalize_gic_version(VirtMachineState *vms)
         /* KVM w/o kernel irqchip can only deal with GICv2 */
         gics_supported |= VIRT_GIC_VERSION_2_MASK;
         accel_name = "KVM with kernel-irqchip=off";
Quoted 8 lines

hw/arm/virt.c static void finalize_gic_version(VirtMachineState *vms) · line 2066

Quoted 5 lines
hw/arm/virt.c +2 -0
+    } else if (whpx_enabled()) {
+        gics_supported |= VIRT_GIC_VERSION_3_MASK;
     } else if (tcg_enabled() || hvf_enabled() || qtest_enabled())  {
         gics_supported |= VIRT_GIC_VERSION_2_MASK;
         if (module_object_class_by_name("arm-gicv3")) {

Should this patch be the last of the series to avoid breaking git bisectability?

Not needed for buildability I think, as system/whpx.h was already present before because of x86 whpx

#11 lore raw

RFC 6/9 hw: intc: arm_gicv3_common: add whpx

In reply to Mohamed Mediouni

Redirect to the platform-specific vGICv3.

---
 hw/intc/arm_gicv3_common.c | 3 +++
 1 file changed, 3 insertions(+)
hw/intc/arm_gicv3_common.c +3 -0
index e438d8c042..a83b075517 100644
@@ -32,6 +32,7 @@
 #include "gicv3_internal.h"
 #include "hw/arm/linux-boot-if.h"
 #include "system/kvm.h"
+#include "system/whpx.h"
 
 
 static void gicv3_gicd_no_migration_shift_bug_post_load(GICv3State *cs)
@@ -662,6 +663,8 @@ const char *gicv3_class_name(void)
 {
     if (kvm_irqchip_in_kernel()) {
         return "kvm-arm-gicv3";
+    } else if (whpx_enabled()) {
+        return "whpx-arm-gicv3";
     } else {
         if (kvm_enabled()) {
             error_report("Userspace GICv3 is not supported with KVM");
Signature · 1 line
2.39.5 (Apple Git-154)
#12 lore raw

Re: [RFC 6/9] hw: intc: arm_gicv3_common: add whpx

Quoted 20 lines · Mohamed Mediouni

Redirect to the platform-specific vGICv3.

---
  hw/intc/arm_gicv3_common.c | 3 +++
  1 file changed, 3 insertions(+)
hw/intc/arm_gicv3_common.c +1 -0
index e438d8c042..a83b075517 100644
@@ -32,6 +32,7 @@
  #include "gicv3_internal.h"
  #include "hw/arm/linux-boot-if.h"
  #include "system/kvm.h"
+#include "system/whpx.h"
 
 
  static void gicv3_gicd_no_migration_shift_bug_post_load(GICv3State *cs)

hw/intc/arm_gicv3_common.c const char *gicv3_class_name(void) · line 667

Quoted 5 lines
hw/intc/arm_gicv3_common.c +2 -0
@@ -662,6 +663,8 @@ const char *gicv3_class_name(void)
  {
      if (kvm_irqchip_in_kernel()) {
          return "kvm-arm-gicv3";
+    } else if (whpx_enabled()) {
+        return "whpx-arm-gicv3";

While you follow a pre-existing pattern, here I'd prefer to use TYPE_WHPX_GICV3 (after defining it in hw/intc/arm_gicv3_common.h).

Quoted 3 lines
hw/intc/arm_gicv3_common.c +0 -0
      } else {
          if (kvm_enabled()) {
              error_report("Userspace GICv3 is not supported with KVM");
#13 lore raw

RFC 7/9 whpx: add arm64 support

In reply to Mohamed Mediouni
---
 accel/whpx/whpx-common.c    |   1 +
 meson.build                 |   5 +-
 target/arm/meson.build      |   1 +
 target/arm/whpx/meson.build |   3 +
 target/arm/whpx/whpx-all.c  | 744 ++++++++++++++++++++++++++++++++++++
 5 files changed, 752 insertions(+), 2 deletions(-)
 create mode 100644 target/arm/whpx/meson.build
 create mode 100644 target/arm/whpx/whpx-all.c
accel/whpx/whpx-common.c +1 -0
index 43d0200afd..58fd6287ed 100644
@@ -15,6 +15,7 @@
 #include "gdbstub/helpers.h"
 #include "qemu/accel.h"
 #include "accel/accel-ops.h"
+#include "system/memory.h"
 #include "system/whpx.h"
 #include "system/cpus.h"
 #include "system/runstate.h"
meson.build +3 -2
index e53cd5b413..f5823e3c86 100644
@@ -327,7 +327,8 @@ accelerator_targets += { 'CONFIG_XEN': xen_targets }
 
 if cpu == 'aarch64'
   accelerator_targets += {
-    'CONFIG_HVF': ['aarch64-softmmu']
+    'CONFIG_HVF': ['aarch64-softmmu'],
+    'CONFIG_WHPX': ['aarch64-softmmu']
   }
 elif cpu == 'x86_64'
   accelerator_targets += {
@@ -885,7 +886,7 @@ if get_option('kvm').allowed() and host_os == 'linux'
   accelerators += 'CONFIG_KVM'
 endif
 if get_option('whpx').allowed() and host_os == 'windows'
-  if get_option('whpx').enabled() and host_machine.cpu() != 'x86_64'
+  if get_option('whpx').enabled() and host_machine.cpu() in ['x86_64', 'aarch64']
     error('WHPX requires 64-bit host')
   elif cc.has_header('winhvplatform.h', required: get_option('whpx')) and \
        cc.has_header('winhvemulation.h', required: get_option('whpx'))
target/arm/meson.build +1 -0
index 07d9271aa4..e28bd3f8e2 100644
@@ -46,6 +46,7 @@ arm_common_system_ss.add(files(
 ))
 
 subdir('hvf')
+subdir('whpx')
 
 if 'CONFIG_TCG' in config_all_accel
    subdir('tcg')
target/arm/whpx/meson.build new file +3 -0
new file mode 100644
index 0000000000..1de2ef0283
@@ -0,0 +1,3 @@
+arm_system_ss.add(when: 'CONFIG_WHPX', if_true: files(
+  'whpx-all.c',
+))
target/arm/whpx/whpx-all.c new file +744 -0

745 lines not on this page · raw message

Signature · 1 line
2.39.5 (Apple Git-154)
#14 lore raw

Re: [RFC 7/9] whpx: add arm64 support

meson.build if get_option('kvm').allowed() and host_os == 'linux' · line 890

Quoted 7 lines · Mohamed Mediouni
meson.build +1 -1
@@ -885,7 +886,7 @@ if get_option('kvm').allowed() and host_os == 'linux'
    accelerators += 'CONFIG_KVM'
  endif
  if get_option('whpx').allowed() and host_os == 'windows'
-  if get_option('whpx').enabled() and host_machine.cpu() != 'x86_64'
+  if get_option('whpx').enabled() and host_machine.cpu() in ['x86_64', 'aarch64']
      error('WHPX requires 64-bit host')

This is wrong, since the sense of "in" is incorrect.

But I think this really should be

  if cpu == 'i386'
    if get_option('whpx').enabled()
      error('WHPX requires 64-bit host')
    endif
    # Leave CONFIG_WHPX disabled
  else
    if cc.has_header('winhvplatform.h', required: get_option('whpx')) and \
       cc.has_header('winhvemulation.h', required: get_option('whpx'))
      accelerators += 'CONFIG_WHPX'
    endif
  endif

because that's the only way that the error message makes sense.

General reject of --enable-whpx on unsupported host will be handled later with

Quoted 3 lines
(quoted diff) +0 -0
 if 'CONFIG_WHPX' not in accelerators and get_option('whpx').enabled()
  error('WHPX not available on this platform')

endif

r~

#15 lore raw

RFC 8/9 whpx: copy over memory tracking logic from hvf

In reply to Mohamed Mediouni

This allows edk2 to work, although u-boot is still not functional.

---
 accel/whpx/whpx-common.c | 198 ++++++++++++++++++++++++++++-----------
 1 file changed, 144 insertions(+), 54 deletions(-)
accel/whpx/whpx-common.c +144 -54
index 58fd6287ed..19415767e4 100644
@@ -265,89 +265,171 @@ void whpx_vcpu_kick(CPUState *cpu)
  * Memory support.
  */
 
-static void whpx_update_mapping(hwaddr start_pa, ram_addr_t size,
-                                void *host_va, int add, int rom,
-                                const char *name)
+ /* hvf_slot flags */
+#define WHPX_SLOT_LOG (1 << 0)
+typedef struct whpx_slot {
+    uint64_t start;
+    uint64_t size;
+    uint8_t *mem;
+    int slot_id;
+    uint32_t flags;
+    MemoryRegion *region;
+} whpx_slot;
+
+typedef struct WHPXState {
+    whpx_slot slots[32];
+    int num_slots;
+} WHPXState;
+
+ WHPXState *whpx_state;
+
+ struct mac_slot {
+    int present;
+    uint64_t size;
+    uint64_t gpa_start;
+    uint64_t gva;
+};
+
+struct mac_slot mac_slots[32];
+
+static int do_hvf_set_memory(whpx_slot *slot, WHV_MAP_GPA_RANGE_FLAGS flags)
 {
     struct whpx_state *whpx = &whpx_global;
+    struct mac_slot *macslot;
     HRESULT hr;
 
-    /*
-    if (add) {
-        printf("WHPX: ADD PA:%p Size:%p, Host:%p, %s, '%s'\n",
-               (void*)start_pa, (void*)size, host_va,
-               (rom ? "ROM" : "RAM"), name);
-    } else {
-        printf("WHPX: DEL PA:%p Size:%p, Host:%p,      '%s'\n",
-               (void*)start_pa, (void*)size, host_va, name);
+    macslot = &mac_slots[slot->slot_id];
+
+    if (macslot->present) {
+        if (macslot->size != slot->size) {
+            macslot->present = 0;
+            hr = whp_dispatch.WHvUnmapGpaRange(whpx->partition, macslot->gpa_start, macslot->size);
+            if (FAILED(hr)) {
+                abort();
+            }
+        }
     }
-    */
-
-    if (add) {
-        hr = whp_dispatch.WHvMapGpaRange(whpx->partition,
-                                         host_va,
-                                         start_pa,
-                                         size,
-                                         (WHvMapGpaRangeFlagRead |
-                                          WHvMapGpaRangeFlagExecute |
-                                          (rom ? 0 : WHvMapGpaRangeFlagWrite)));
-    } else {
-        hr = whp_dispatch.WHvUnmapGpaRange(whpx->partition,
-                                           start_pa,
-                                           size);
+
+    if (!slot->size) {
+        return 0;
     }
 
-    if (FAILED(hr)) {
-        error_report("WHPX: Failed to %s GPA range '%s' PA:%p, Size:%p bytes,"
-                     " Host:%p, hr=%08lx",
-                     (add ? "MAP" : "UNMAP"), name,
-                     (void *)(uintptr_t)start_pa, (void *)size, host_va, hr);
+    macslot->present = 1;
+    macslot->gpa_start = slot->start;
+    macslot->size = slot->size;
+    hr = whp_dispatch.WHvMapGpaRange(whpx->partition, slot->mem, slot->start, slot->size, flags);
+    return 0;
+}
+
+static whpx_slot *whpx_find_overlap_slot(uint64_t start, uint64_t size)
+{
+    whpx_slot *slot;
+    int x;
+    for (x = 0; x < whpx_state->num_slots; ++x) {
+        slot = &whpx_state->slots[x];
+        if (slot->size && start < (slot->start + slot->size) &&
+            (start + size) > slot->start) {
+            return slot;
+        }
     }
+    return NULL;
 }
 
-static void whpx_process_section(MemoryRegionSection *section, int add)
+static void whpx_set_phys_mem(MemoryRegionSection *section, bool add)
 {
-    MemoryRegion *mr = section->mr;
-    hwaddr start_pa = section->offset_within_address_space;
-    ram_addr_t size = int128_get64(section->size);
-    unsigned int delta;
-    uint64_t host_va;
+    whpx_slot *mem;
+    MemoryRegion *area = section->mr;
+    bool writable = !area->readonly && !area->rom_device;
+    WHV_MAP_GPA_RANGE_FLAGS flags;
+    uint64_t page_size = qemu_real_host_page_size();
+
+    if (!memory_region_is_ram(area)) {
+        if (writable) {
+            return;
+        } else if (!memory_region_is_romd(area)) {
+            /*
+             * If the memory device is not in romd_mode, then we actually want
+             * to remove the hvf memory slot so all accesses will trap.
+             */
+             add = false;
+        }
+    }
 
-    if (!memory_region_is_ram(mr)) {
-        return;
+    if (!QEMU_IS_ALIGNED(int128_get64(section->size), page_size) ||
+        !QEMU_IS_ALIGNED(section->offset_within_address_space, page_size)) {
+        /* Not page aligned, so we can not map as RAM */
+        add = false;
     }
 
-    delta = qemu_real_host_page_size() - (start_pa & ~qemu_real_host_page_mask());
-    delta &= ~qemu_real_host_page_mask();
-    if (delta > size) {
-        return;
+    mem = whpx_find_overlap_slot(
+            section->offset_within_address_space,
+            int128_get64(section->size));
+
+    if (mem && add) {
+        if (mem->size == int128_get64(section->size) &&
+            mem->start == section->offset_within_address_space &&
+            mem->mem == (memory_region_get_ram_ptr(area) +
+            section->offset_within_region)) {
+            return; /* Same region was attempted to register, go away. */
+        }
+    }
+
+    /* Region needs to be reset. set the size to 0 and remap it. */
+    if (mem) {
+        mem->size = 0;
+        if (do_hvf_set_memory(mem, 0)) {
+            error_report("Failed to reset overlapping slot");
+            abort();
+        }
     }
-    start_pa += delta;
-    size -= delta;
-    size &= qemu_real_host_page_mask();
-    if (!size || (start_pa & ~qemu_real_host_page_mask())) {
+
+    if (!add) {
         return;
     }
 
-    host_va = (uintptr_t)memory_region_get_ram_ptr(mr)
-            + section->offset_within_region + delta;
+    if (area->readonly ||
+        (!memory_region_is_ram(area) && memory_region_is_romd(area))) {
+        flags = WHvMapGpaRangeFlagRead | WHvMapGpaRangeFlagExecute;
+    } else {
+        flags = WHvMapGpaRangeFlagRead | WHvMapGpaRangeFlagWrite | WHvMapGpaRangeFlagExecute;
+    }
+
+    /* Now make a new slot. */
+    int x;
+
+    for (x = 0; x < whpx_state->num_slots; ++x) {
+        mem = &whpx_state->slots[x];
+        if (!mem->size) {
+            break;
+        }
+    }
+
+    if (x == whpx_state->num_slots) {
+        error_report("No free slots");
+        abort();
+    }
 
-    whpx_update_mapping(start_pa, size, (void *)(uintptr_t)host_va, add,
-                        memory_region_is_rom(mr), mr->name);
+    mem->size = int128_get64(section->size);
+    mem->mem = memory_region_get_ram_ptr(area) + section->offset_within_region;
+    mem->start = section->offset_within_address_space;
+    mem->region = area;
+
+    if (do_hvf_set_memory(mem, flags)) {
+        error_report("Error registering new memory slot");
+        abort();
+    }
 }
 
 static void whpx_region_add(MemoryListener *listener,
                            MemoryRegionSection *section)
 {
-    memory_region_ref(section->mr);
-    whpx_process_section(section, 1);
+    whpx_set_phys_mem(section, true);
 }
 
 static void whpx_region_del(MemoryListener *listener,
                            MemoryRegionSection *section)
 {
-    whpx_process_section(section, 0);
-    memory_region_unref(section->mr);
+    whpx_set_phys_mem(section, false);
 }
 
 static void whpx_transaction_begin(MemoryListener *listener)
@@ -531,6 +613,14 @@ static void whpx_accel_instance_init(Object *obj)
     memset(whpx, 0, sizeof(struct whpx_state));
     /* Turn on kernel-irqchip, by default */
     whpx->kernel_irqchip_allowed = true;
+
+    int x;
+    whpx_state = malloc(sizeof(WHPXState));
+    whpx_state->num_slots = ARRAY_SIZE(whpx_state->slots);
+    for (x = 0; x < whpx_state->num_slots; ++x) {
+        whpx_state->slots[x].size = 0;
+        whpx_state->slots[x].slot_id = x;
+    }
 }
 
 static const TypeInfo whpx_accel_type = {
Signature · 1 line
2.39.5 (Apple Git-154)
#16 lore raw

RFC 9/9 target/arm: cpu: mark WHPX as supporting PSCI 1.1

In reply to Mohamed Mediouni
---
 target/arm/cpu.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
target/arm/cpu.c +2 -1
index e2b2337399..3b69c9786a 100644
@@ -23,6 +23,7 @@
 #include "qemu/timer.h"
 #include "qemu/log.h"
 #include "exec/page-vary.h"
+#include "system/whpx.h"
 #include "target/arm/idau.h"
 #include "qemu/module.h"
 #include "qapi/error.h"
@@ -1496,7 +1497,7 @@ static void arm_cpu_initfn(Object *obj)
     cpu->psci_version = QEMU_PSCI_VERSION_0_1; /* By default assume PSCI v0.1 */
     cpu->kvm_target = QEMU_KVM_ARM_TARGET_NONE;
 
-    if (tcg_enabled() || hvf_enabled()) {
+    if (tcg_enabled() || hvf_enabled() || whpx_enabled()) {
         /* TCG and HVF implement PSCI 1.1 */
         cpu->psci_version = QEMU_PSCI_VERSION_1_1;
     }
Signature · 1 line
2.39.5 (Apple Git-154)
#17 lore raw
In reply to Mohamed Mediouni

Hi Mohamed,

Quoted 11 lines · Mohamed Mediouni

This one took way longer for me to publish than I should have.

There are a number of lingering bugs in this one including u-boot not working.

Interrupt controller save/restore is entirely missing in this RFC, and some other state bits are likely still missing too.

ITS not blocked by default yet, remember to use its=off when testing this series. You might also want the GICv3 + GICv2m support patch as part of the HVF vGIC patch series, which is not duplicated here.

Quoted 6 lines

PS: on both this and HVF, interrupt controller initialisation needs to be done early so I ended up with hardcoded addresses. Wonder if the right way to go might be to defer virt and vCPU initialisation until late in the process post-gic_realize...

Other than that, this boots both EDK2 and Linux in SMP, when using devicetree or ACPI.

thanks for posting this, that's an exciting series!

I applied it on top of your other series ([email protected]) and solved the conflicts. However, it would really help if you could push that exact branch somewhere, so people can easily pull it and try. I'm fine if you want to duplicate gic patches in this series as well.

I tried to direct boot a kernel (6.15 defconfig) and ran into this error [1]:

$ ./build/qemu-system-aarch64.exe -M virt,its=off -cpu cortex-a76 -m 2G -nographic -accel whpx -kernel out/Image.gz out/host.ext4

Could you please share your exact command line?
Does it work with direct kernel boot also?

Kind Regards,
Pierrick

[1] Error when booting:

[    1.381525] Internal error: Oops: 0000000096000002 [#1]  SMP
[    1.458060] Modules linked in:
[    1.461172] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted 6.15.0-00001-g7797e43a2520 #1 PREEMPT
[    1.470502] Hardware name: linux,dummy-virt (DT)
[    1.475102] pstate: 204000c5 (nzCv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[    1.482023] pc : pci_generic_config_read+0x38/0xb8
[    1.486970] lr : pci_generic_config_read+0x24/0xb8
[    1.491734] sp : ffff80008000b940
[    1.495079] x29: ffff80008000b940 x28: 0000000000000000 x27: ffffbb8fe13f00ec
[    1.502473] x26: ffffbb8fe14f9060 x25: ffffbb8fe14f9078 x24: ffffbb8fe1c99990
[    1.509564] x23: 0000000000000000 x22: ffff80008000b9f4 x21: ffff574c83b7c000
[    1.516636] x20: ffff80008000b964 x19: 0000000000000004 x18: 0000000000000006
[    1.523722] x17: 6666666666666666 x16: 6678302d30303030 x15: 0720072007200720
[    1.531583] x14: 0720072007200720 x13: 0720072007200720 x12: ffffbb8fe1736838
[    1.539094] x11: 0000000000000058 x10: 0000000000000018 x9 : ffffbb8fe1736838
[    1.546212] x8 : 00000000000000c5 x7 : ffffbb8fe187ff40 x6 : 00000000000000ff
[    1.553370] x5 : 0000000000000000 x4 : 0000000000000000 x3 : ffff800090000000
[    1.560676] x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffff800090000000
[    1.567890] Call trace:
[    1.570509]  pci_generic_config_read+0x38/0xb8 (P)
[    1.575362]  pci_bus_read_config_dword+0x80/0xe4
[    1.580231]  pci_bus_generic_read_dev_vendor_id+0x30/0x164
[    1.586164]  pci_scan_single_device+0x118/0x18c
[    1.590912]  pci_scan_slot+0x58/0x214
[    1.594782]  pci_scan_child_bus_extend+0x40/0x234
[    1.599606]  pci_scan_root_bus_bridge+0x64/0xd8
[    1.604194]  pci_host_probe+0x30/0xec
[    1.607972]  pci_host_common_probe+0x128/0x1c0
[    1.612430]  platform_probe+0x68/0xdc
[    1.616220]  really_probe+0xbc/0x2c0
[    1.619814]  __driver_probe_device+0x78/0x120
[    1.624435]  driver_probe_device+0x3c/0x154
[    1.628620]  __driver_attach+0x90/0x1a0
[    1.632509]  bus_for_each_dev+0x7c/0xdc
[    1.636449]  driver_attach+0x24/0x30
[    1.640067]  bus_add_driver+0xe4/0x208
[    1.643774]  driver_register+0x68/0x130
[    1.647667]  __platform_driver_register+0x24/0x30
[    1.652704]  gen_pci_driver_init+0x1c/0x28
[    1.657181]  do_one_initcall+0x60/0x1d4
[    1.661172]  kernel_init_freeable+0x210/0x274
[    1.665749]  kernel_init+0x20/0x140
[    1.669341]  ret_from_fork+0x10/0x20
[    1.673034] Code: 7100067f 540002c0 71000a7f 54000180 (b9400000)
#18 lore raw
Quoted 19 lines · Pierrick Bouvier

Hi Mohamed,

Quoted 11 lines · Mohamed Mediouni

This one took way longer for me to publish than I should have.
There are a number of lingering bugs in this one including u-boot not working.
Interrupt controller save/restore is entirely missing in this RFC, and some other state bits are likely still missing too.
ITS not blocked by default yet, remember to use its=off when testing this series.
You might also want the GICv3 + GICv2m support patch as part of the HVF vGIC patch series, which is not duplicated here.
PS: on both this and HVF, interrupt controller initialisation needs to be done early so I ended up with hardcoded addresses. Wonder if the right way to go might be to defer virt and vCPU initialisation until late in the process post-gic_realize...
Other than that, this boots both EDK2 and Linux in SMP, when using devicetree or ACPI.

Quoted 5 lines

thanks for posting this, that's an exciting series!

I applied it on top of your other series ([email protected]) and solved the conflicts. However, it would really help if you could push that exact branch somewhere, so people can easily pull it and try. I'm fine if you want to duplicate gic patches in this series as well.

Hello,

My branches are at https://github.com/mediouni-m/qemu

whpx-v1 corresponding to this RFC, but latest rev of the whpx branch has some fixes

Have some additional notes and binaries here too: https://github.com/mediouni-m/qemu/releases/tag/whpx-v1.1

Thank you,
-Mohamed

Quoted 2 lines

I tried to direct boot a kernel (6.15 defconfig) and ran into this error [1]:

$ ./build/qemu-system-aarch64.exe -M virt,its=off -cpu cortex-a76 -m 2G -nographic -accel whpx -kernel out/Image.gz out/host.ext4

Syntax that I use is -M virt,accel=whpx,its=off -m 2048-cpu cortex-a72 -bios share/edk2-aarch64-code.fd.

And on some kernel versions, you’ll also need irqchip.gicv3_nolpi=1.

Quoted 6 lines

Could you please share your exact command line?
Does it work with direct kernel boot also?

Kind Regards,
Pierrick

Quoted 8 lines

[1] Error when booting:

[    1.381525] Internal error: Oops: 0000000096000002 [#1]  SMP
[    1.458060] Modules linked in:
[    1.461172] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted 6.15.0-00001-g7797e43a2520 #1 PREEMPT
[    1.470502] Hardware name: linux,dummy-virt (DT)
[    1.475102] pstate: 204000c5 (nzCv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[    1.482023] pc : pci_generic_config_read+0x38/0xb8
[    1.486970] lr : pci_generic_config_read+0x24/0xb8

I don’t think I saw this particular one before… which Windows version and hardware are you testing this on?

(Message was sent from the wrong email alias on mobile, resending) Thank you,
-Mohamed

#19 lore raw
Quoted 19 lines · Mohamed Mediouni
Quoted 19 lines · Pierrick Bouvier

Hi Mohamed,

Quoted 11 lines · Mohamed Mediouni

This one took way longer for me to publish than I should have.
There are a number of lingering bugs in this one including u-boot not working.
Interrupt controller save/restore is entirely missing in this RFC, and some other state bits are likely still missing too.
ITS not blocked by default yet, remember to use its=off when testing this series.
You might also want the GICv3 + GICv2m support patch as part of the HVF vGIC patch series, which is not duplicated here.
PS: on both this and HVF, interrupt controller initialisation needs to be done early so I ended up with hardcoded addresses. Wonder if the right way to go might be to defer virt and vCPU initialisation until late in the process post-gic_realize...
Other than that, this boots both EDK2 and Linux in SMP, when using devicetree or ACPI.

Quoted 9 lines
Quoted 5 lines

thanks for posting this, that's an exciting series!

I applied it on top of your other series ([email protected]) and solved the conflicts. However, it would really help if you could push that exact branch somewhere, so people can easily pull it and try. I'm fine if you want to duplicate gic patches in this series as well.

Hello,

My branches are at https://github.com/mediouni-m/qemu

Thanks, it's worth adding it in cover letter for next versions.

Quoted 28 lines

whpx-v1 corresponding to this RFC, but latest rev of the whpx branch has some fixes

Have some additional notes and binaries here too: https://github.com/mediouni-m/qemu/releases/tag/whpx-v1.1

Thank you,
-Mohamed

Quoted 2 lines

I tried to direct boot a kernel (6.15 defconfig) and ran into this error [1]:

$ ./build/qemu-system-aarch64.exe -M virt,its=off -cpu cortex-a76 -m 2G -nographic -accel whpx -kernel out/Image.gz out/host.ext4

Syntax that I use is -M virt,accel=whpx,its=off -m 2048-cpu cortex-a72 -bios share/edk2-aarch64-code.fd.

And on some kernel versions, you’ll also need irqchip.gicv3_nolpi=1.

Quoted 6 lines

Could you please share your exact command line?
Does it work with direct kernel boot also?

Kind Regards,
Pierrick

Quoted 8 lines

[1] Error when booting:

[    1.381525] Internal error: Oops: 0000000096000002 [#1]  SMP
[    1.458060] Modules linked in:
[    1.461172] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted 6.15.0-00001-g7797e43a2520 #1 PREEMPT
[    1.470502] Hardware name: linux,dummy-virt (DT)
[    1.475102] pstate: 204000c5 (nzCv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[    1.482023] pc : pci_generic_config_read+0x38/0xb8
[    1.486970] lr : pci_generic_config_read+0x24/0xb8
Quoted 2 lines

I don’t think I saw this particular one before… which Windows version and hardware are you testing this on?

I see the same error as before.

I tried also binaries from https://github.com/mediouni-m/qemu/releases/tag/whpx-v1.1, when directly booting kernel, I still see the same pci issue with both binaries and my compiled whpx-v1.3. When booting edk2 provided, I ran into this other error instead with both binaries [1].

I'm running latest Windows 11 (stable channel, fully updated), on a microsoft volterra (devkit). It might be an issue specific to this platform.

In case you're interested, we can arrange an access to the machine, but I understand if it's not your priority now.

[1]
Windows Hypervisor Platform accelerator is operational
UEFI firmware (version edk2-stable202408-prebuilt.qemu.org built at 16:28:50 on Sep 12 2024)
ArmTrngLib could not be correctly initialized.
Error: Image at 000BFDB6000 start failed: 00000001
Error: Image at 000BFD6D000 start failed: Not Found
MapGcdMmioSpace: failed to add GCD memory space for region [0x4010000000+0x10000000)
ASSERT_EFI_ERROR (Status = Unsupported)
ASSERT [PciHostBridgeDxe] /home/kraxel/projects/qemu/roms/edk2/OvmfPkg/Fdt/FdtPciHostBridgeLib/FdtPciHostBridgeLib.c(326): !(((INTN)(RETURN_STATUS)(Status)) < 0)

Quoted 4 lines

(Message was sent from the wrong email alias on mobile, resending) Thank you,
-Mohamed

Thanks,
Pierrick

#20 lore raw
Quoted 58 lines · Pierrick Bouvier
Quoted 17 lines · Mohamed Mediouni
Quoted 17 lines · Pierrick Bouvier

Hi Mohamed,

Quoted 11 lines · Mohamed Mediouni

This one took way longer for me to publish than I should have.
There are a number of lingering bugs in this one including u-boot not working.
Interrupt controller save/restore is entirely missing in this RFC, and some other state bits are likely still missing too.
ITS not blocked by default yet, remember to use its=off when testing this series.
You might also want the GICv3 + GICv2m support patch as part of the HVF vGIC patch series, which is not duplicated here.
PS: on both this and HVF, interrupt controller initialisation needs to be done early so I ended up with hardcoded addresses. Wonder if the right way to go might be to defer virt and vCPU initialisation until late in the process post-gic_realize...
Other than that, this boots both EDK2 and Linux in SMP, when using devicetree or ACPI.

Quoted 8 lines
Quoted 5 lines

thanks for posting this, that's an exciting series!

I applied it on top of your other series ([email protected]) and solved the conflicts. However, it would really help if you could push that exact branch somewhere, so people can easily pull it and try. I'm fine if you want to duplicate gic patches in this series as well.

Hello,
My branches are at https://github.com/mediouni-m/qemu

Thanks, it's worth adding it in cover letter for next versions.

Quoted 22 lines

whpx-v1 corresponding to this RFC, but latest rev of the whpx branch has some fixes
Have some additional notes and binaries here too: https://github.com/mediouni-m/qemu/releases/tag/whpx-v1.1
Thank you,
-Mohamed

Quoted 2 lines

I tried to direct boot a kernel (6.15 defconfig) and ran into this error [1]:

$ ./build/qemu-system-aarch64.exe -M virt,its=off -cpu cortex-a76 -m 2G -nographic -accel whpx -kernel out/Image.gz out/host.ext4

Syntax that I use is -M virt,accel=whpx,its=off -m 2048-cpu cortex-a72 -bios share/edk2-aarch64-code.fd. And on some kernel versions, you’ll also need irqchip.gicv3_nolpi=1.

Quoted 6 lines

Could you please share your exact command line?
Does it work with direct kernel boot also?

Kind Regards,
Pierrick

Quoted 8 lines

[1] Error when booting:

[    1.381525] Internal error: Oops: 0000000096000002 [#1]  SMP
[    1.458060] Modules linked in:
[    1.461172] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted 6.15.0-00001-g7797e43a2520 #1 PREEMPT
[    1.470502] Hardware name: linux,dummy-virt (DT)
[    1.475102] pstate: 204000c5 (nzCv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[    1.482023] pc : pci_generic_config_read+0x38/0xb8
[    1.486970] lr : pci_generic_config_read+0x24/0xb8
Quoted 2 lines

I don’t think I saw this particular one before… which Windows version and hardware are you testing this on?

Quoted 7 lines

I see the same error as before.

I tried also binaries from https://github.com/mediouni-m/qemu/releases/tag/whpx-v1.1, when directly booting kernel, I still see the same pci issue with both binaries and my compiled whpx-v1.3. When booting edk2 provided, I ran into this other error instead with both binaries [1].

I'm running latest Windows 11 (stable channel, fully updated), on a microsoft volterra (devkit). It might be an issue specific to this platform.

I didn’t test anything on the stable branch for now but only on Canary so far.
Just cursorily tested (EDK2 only) an X Elite device on prod (26100.4652) and this issue doesn’t appear.

I have 8cx Gen 3 and 8cx Gen 1 (SQ1) devices around, will test on those older SoCs later and see.

Random idea for testing: what if you put -M highmem=off, does that change anything?

Thanks,
-Mohamed

Quoted 18 lines

In case you're interested, we can arrange an access to the machine, but I understand if it's not your priority now.

[1]
Windows Hypervisor Platform accelerator is operational
UEFI firmware (version edk2-stable202408-prebuilt.qemu.org built at 16:28:50 on Sep 12 2024)
ArmTrngLib could not be correctly initialized.
Error: Image at 000BFDB6000 start failed: 00000001
Error: Image at 000BFD6D000 start failed: Not Found
MapGcdMmioSpace: failed to add GCD memory space for region [0x4010000000+0x10000000)
ASSERT_EFI_ERROR (Status = Unsupported)
ASSERT [PciHostBridgeDxe] /home/kraxel/projects/qemu/roms/edk2/OvmfPkg/Fdt/FdtPciHostBridgeLib/FdtPciHostBridgeLib.c(326): !(((INTN)(RETURN_STATUS)(Status)) < 0)

Quoted 3 lines

(Message was sent from the wrong email alias on mobile, resending) Thank you,
-Mohamed

Thanks,
Pierrick

#21 lore raw
Quoted 63 lines · Mohamed Mediouni
Quoted 63 lines · Pierrick Bouvier
Quoted 17 lines · Mohamed Mediouni
Quoted 17 lines · Pierrick Bouvier

Hi Mohamed,

Quoted 11 lines · Mohamed Mediouni

This one took way longer for me to publish than I should have.
There are a number of lingering bugs in this one including u-boot not working.
Interrupt controller save/restore is entirely missing in this RFC, and some other state bits are likely still missing too.
ITS not blocked by default yet, remember to use its=off when testing this series.
You might also want the GICv3 + GICv2m support patch as part of the HVF vGIC patch series, which is not duplicated here.
PS: on both this and HVF, interrupt controller initialisation needs to be done early so I ended up with hardcoded addresses. Wonder if the right way to go might be to defer virt and vCPU initialisation until late in the process post-gic_realize...
Other than that, this boots both EDK2 and Linux in SMP, when using devicetree or ACPI.

Quoted 8 lines
Quoted 5 lines

thanks for posting this, that's an exciting series!

I applied it on top of your other series ([email protected]) and solved the conflicts. However, it would really help if you could push that exact branch somewhere, so people can easily pull it and try. I'm fine if you want to duplicate gic patches in this series as well.

Hello,
My branches are at https://github.com/mediouni-m/qemu

Thanks, it's worth adding it in cover letter for next versions.

Quoted 22 lines

whpx-v1 corresponding to this RFC, but latest rev of the whpx branch has some fixes
Have some additional notes and binaries here too: https://github.com/mediouni-m/qemu/releases/tag/whpx-v1.1
Thank you,
-Mohamed

Quoted 2 lines

I tried to direct boot a kernel (6.15 defconfig) and ran into this error [1]:

$ ./build/qemu-system-aarch64.exe -M virt,its=off -cpu cortex-a76 -m 2G -nographic -accel whpx -kernel out/Image.gz out/host.ext4

Syntax that I use is -M virt,accel=whpx,its=off -m 2048-cpu cortex-a72 -bios share/edk2-aarch64-code.fd. And on some kernel versions, you’ll also need irqchip.gicv3_nolpi=1.

Quoted 6 lines

Could you please share your exact command line?
Does it work with direct kernel boot also?

Kind Regards,
Pierrick

Quoted 8 lines

[1] Error when booting:

[    1.381525] Internal error: Oops: 0000000096000002 [#1]  SMP
[    1.458060] Modules linked in:
[    1.461172] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted 6.15.0-00001-g7797e43a2520 #1 PREEMPT
[    1.470502] Hardware name: linux,dummy-virt (DT)
[    1.475102] pstate: 204000c5 (nzCv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[    1.482023] pc : pci_generic_config_read+0x38/0xb8
[    1.486970] lr : pci_generic_config_read+0x24/0xb8
Quoted 2 lines

I don’t think I saw this particular one before… which Windows version and hardware are you testing this on?

I see the same error as before.

I tried also binaries from https://github.com/mediouni-m/qemu/releases/tag/whpx-v1.1, when directly booting kernel, I still see the same pci issue with both binaries and my compiled whpx-v1.3. When booting edk2 provided, I ran into this other error instead with both binaries [1].

Quoted 9 lines
Quoted 2 lines

I'm running latest Windows 11 (stable channel, fully updated), on a microsoft volterra (devkit). It might be an issue specific to this platform.

I didn’t test anything on the stable branch for now but only on Canary so far.
Just cursorily tested (EDK2 only) an X Elite device on prod (26100.4652) and this issue doesn’t appear.

I have 8cx Gen 3 and 8cx Gen 1 (SQ1) devices around, will test on those older SoCs later and see.

Random idea for testing: what if you put -M highmem=off, does that change anything?

Good guess, it solves the problem with edk2, and direct boots linux kernel successfully now.

Quoted 22 lines

Thanks,
-Mohamed

Quoted 18 lines

In case you're interested, we can arrange an access to the machine, but I understand if it's not your priority now.

[1]
Windows Hypervisor Platform accelerator is operational
UEFI firmware (version edk2-stable202408-prebuilt.qemu.org built at 16:28:50 on Sep 12 2024)
ArmTrngLib could not be correctly initialized.
Error: Image at 000BFDB6000 start failed: 00000001
Error: Image at 000BFD6D000 start failed: Not Found
MapGcdMmioSpace: failed to add GCD memory space for region [0x4010000000+0x10000000)
ASSERT_EFI_ERROR (Status = Unsupported)
ASSERT [PciHostBridgeDxe] /home/kraxel/projects/qemu/roms/edk2/OvmfPkg/Fdt/FdtPciHostBridgeLib/FdtPciHostBridgeLib.c(326): !(((INTN)(RETURN_STATUS)(Status)) < 0)

Quoted 3 lines

(Message was sent from the wrong email alias on mobile, resending) Thank you,
-Mohamed

Thanks,
Pierrick

#22 lore raw
Quoted 68 lines · Pierrick Bouvier
Quoted 63 lines · Mohamed Mediouni
Quoted 63 lines · Pierrick Bouvier
Quoted 17 lines · Mohamed Mediouni
Quoted 17 lines · Pierrick Bouvier

Hi Mohamed,

Quoted 11 lines · Mohamed Mediouni

This one took way longer for me to publish than I should have.
There are a number of lingering bugs in this one including u-boot not working.
Interrupt controller save/restore is entirely missing in this RFC, and some other state bits are likely still missing too.
ITS not blocked by default yet, remember to use its=off when testing this series.
You might also want the GICv3 + GICv2m support patch as part of the HVF vGIC patch series, which is not duplicated here.
PS: on both this and HVF, interrupt controller initialisation needs to be done early so I ended up with hardcoded addresses. Wonder if the right way to go might be to defer virt and vCPU initialisation until late in the process post-gic_realize...
Other than that, this boots both EDK2 and Linux in SMP, when using devicetree or ACPI.

Quoted 8 lines
Quoted 5 lines

thanks for posting this, that's an exciting series!

I applied it on top of your other series ([email protected]) and solved the conflicts. However, it would really help if you could push that exact branch somewhere, so people can easily pull it and try. I'm fine if you want to duplicate gic patches in this series as well.

Hello,
My branches are at https://github.com/mediouni-m/qemu

Thanks, it's worth adding it in cover letter for next versions.

Quoted 22 lines

whpx-v1 corresponding to this RFC, but latest rev of the whpx branch has some fixes
Have some additional notes and binaries here too: https://github.com/mediouni-m/qemu/releases/tag/whpx-v1.1
Thank you,
-Mohamed

Quoted 2 lines

I tried to direct boot a kernel (6.15 defconfig) and ran into this error [1]:

$ ./build/qemu-system-aarch64.exe -M virt,its=off -cpu cortex-a76 -m 2G -nographic -accel whpx -kernel out/Image.gz out/host.ext4

Syntax that I use is -M virt,accel=whpx,its=off -m 2048-cpu cortex-a72 -bios share/edk2-aarch64-code.fd. And on some kernel versions, you’ll also need irqchip.gicv3_nolpi=1.

Quoted 6 lines

Could you please share your exact command line?
Does it work with direct kernel boot also?

Kind Regards,
Pierrick

Quoted 8 lines

[1] Error when booting:

[    1.381525] Internal error: Oops: 0000000096000002 [#1]  SMP
[    1.458060] Modules linked in:
[    1.461172] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted 6.15.0-00001-g7797e43a2520 #1 PREEMPT
[    1.470502] Hardware name: linux,dummy-virt (DT)
[    1.475102] pstate: 204000c5 (nzCv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[    1.482023] pc : pci_generic_config_read+0x38/0xb8
[    1.486970] lr : pci_generic_config_read+0x24/0xb8
Quoted 2 lines

I don’t think I saw this particular one before… which Windows version and hardware are you testing this on?

I see the same error as before.

I tried also binaries from https://github.com/mediouni-m/qemu/releases/tag/whpx-v1.1, when directly booting kernel, I still see the same pci issue with both binaries and my compiled whpx-v1.3. When booting edk2 provided, I ran into this other error instead with both binaries [1].

I'm running latest Windows 11 (stable channel, fully updated), on a microsoft volterra (devkit). It might be an issue specific to this platform.

Quoted 8 lines
Quoted 5 lines

I didn’t test anything on the stable branch for now but only on Canary so far.
Just cursorily tested (EDK2 only) an X Elite device on prod (26100.4652) and this issue doesn’t appear. I have 8cx Gen 3 and 8cx Gen 1 (SQ1) devices around, will test on those older SoCs later and see. Random idea for testing: what if you put -M highmem=off, does that change anything?

Good guess, it solves the problem with edk2, and direct boots linux kernel successfully now.

If you don’t mind, could you please test the latest commit I just put on the Git repo (whpx branch)? It should address this properly.

Thank you,
-Mohamed

#23 lore raw
Quoted 11 lines · Mohamed Mediouni
Quoted 8 lines
Quoted 5 lines

I didn’t test anything on the stable branch for now but only on Canary so far.
Just cursorily tested (EDK2 only) an X Elite device on prod (26100.4652) and this issue doesn’t appear. I have 8cx Gen 3 and 8cx Gen 1 (SQ1) devices around, will test on those older SoCs later and see. Random idea for testing: what if you put -M highmem=off, does that change anything?

Good guess, it solves the problem with edk2, and direct boots linux kernel successfully now.

If you don’t mind, could you please test the latest commit I just put on the Git repo (whpx branch)? It should address this properly.

Sure.
Without highmem=off, I get a new error:
Physical address width: 36
C:\msys64\home\tcwg\qemu\build\qemu-system-aarch64.exe: VCPU supports less PA bits (36) than requested by the memory map (40)

#24 lore raw
Quoted 11 lines · Pierrick Bouvier
Quoted 6 lines · Mohamed Mediouni
Quoted 6 lines
Quoted 5 lines

I didn’t test anything on the stable branch for now but only on Canary so far.
Just cursorily tested (EDK2 only) an X Elite device on prod (26100.4652) and this issue doesn’t appear. I have 8cx Gen 3 and 8cx Gen 1 (SQ1) devices around, will test on those older SoCs later and see. Random idea for testing: what if you put -M highmem=off, does that change anything?

Quoted 9 lines
Quoted 3 lines
Quoted 2 lines

Good guess, it solves the problem with edk2, and direct boots linux kernel successfully now.

If you don’t mind, could you please test the latest commit I just put on the Git repo (whpx branch)? It should address this properly.

Sure.
Without highmem=off, I get a new error:
Physical address width: 36
C:\msys64\home\tcwg\qemu\build\qemu-system-aarch64.exe: VCPU supports less PA bits (36) than requested by the memory map (40)

Oh this confirms what I thought, just pushed a new commit which should be a complete fix. :)

And accessorily is also a good thing for Snapdragon X too, as 40 VA bits were used there despite the hardware having 39.

Thank you,
-Mohamed

#25 lore raw
Quoted 13 lines · Mohamed Mediouni
Quoted 13 lines · Pierrick Bouvier
Quoted 8 lines · Mohamed Mediouni
Quoted 8 lines
Quoted 5 lines

I didn’t test anything on the stable branch for now but only on Canary so far.
Just cursorily tested (EDK2 only) an X Elite device on prod (26100.4652) and this issue doesn’t appear. I have 8cx Gen 3 and 8cx Gen 1 (SQ1) devices around, will test on those older SoCs later and see. Random idea for testing: what if you put -M highmem=off, does that change anything?

Good guess, it solves the problem with edk2, and direct boots linux kernel successfully now.

Quoted 9 lines
Quoted 7 lines
Quoted 1 line

If you don’t mind, could you please test the latest commit I just put on the Git repo (whpx branch)? It should address this properly.

Sure.
Without highmem=off, I get a new error:
Physical address width: 36
C:\msys64\home\tcwg\qemu\build\qemu-system-aarch64.exe: VCPU supports less PA bits (36) than requested by the memory map (40)

Oh this confirms what I thought, just pushed a new commit which should be a complete fix. :)

It works fine now, on top of c6e3d4d.
Excellent work!

Quoted 2 lines

And accessorily is also a good thing for Snapdragon X too, as 40 VA bits were used there despite the hardware having 39.

Yes.

Quoted 2 lines

Thank you,
-Mohamed

This seems like a good time for cleaning up, address the few comments that have been made, and send a v2. Feel free to add the gic related patches in it, so we have a "complete" series to review.

Thanks,
Pierrick