Skip to contentSkip to conversation

WHPX support for Arm

PATCH Superseded Mohamed Mediouni 5 participants 36 messages Jan 21 – Jan 27 Arm/Amlogic Meson SoC support

v17 of 22 Superseded Latest is v22 →

lore
Share & tools
lore
#0 lore raw

v17 cover WHPX support for Arm

Link to branch: https://github.com/mediouni-m/qemu whpx (tag for this submission: whpx-v18)

Missing features:
- VM save-restore: interrupt controller state notably
- SVE register sync: I didn't have the time to test this on pre-release hardware with SVE2 support yet. So SVE2 is currently masked for VMs when running this.

Known bugs:
- U-Boot still doesn't work (hangs when trying to parse firmware) but EDK2 does.

Note:

"target/arm/kvm: add constants for new PSCI versions" taken from the mailing list.

"accel/system: Introduce hwaccel_enabled() helper" taken from the mailing list, added here as part of this series to make it compilable as a whole.

And another note:

Seems that unlike HVF there isn't direct correspondence between WHv registers and the actual register layout, so didn't do changes there to a sysreg.inc.

Updates for v17:
- Reworking the MSI-X configuration patches
- Accordingly, its= retains the same behavior as before, so add a test case for GICv3 + GICv2m.
- Squashing "whpx: arm64: check for physical address width after WHPX availability" with "whpx: arm64: clamp down IPA size"
- Not worthwhile to have a separate GICv3 + GICv2m patch, squash it into the "rework MSI-X configuration" patch
- Address review comment on "whpx: arm64: gicv3: add migration blocker"

Updates since v16:
- Address some last review comments
- Rebase with the latest origin/staging

Updates since v15:
- Some last tiny changes addressed.

Updates since v14:
- Working around https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87950

It turns out that we can't avoid default: in case with an enum on GCC without warnings, although Clang allows that behavior.

Updates since v13:
- Addressing review comments from Akihiko Odaki
- Squash GICv2 ACPI workaround patch and have it documented as a separate function
- Rebase with current origin/staging

Updates since v12:
- Address review comments and make sure that checkpatch shows no errors, make test also ran
- Tested the Windows x86_64 build too in addition to Windows arm64 and macOS arm64
- Hopefully the last revision, should be ready to merge.

Updates since v11:
- Address review comments
- Rebase up to latest staging
- Switch to assuming Qemu 11.0 as the newest machine model

Updates since v10:
- Bring forward to latest Qemu
- Fix a typo in the GICv3+GICv2m PR

Updates since v9:
- Adding partition reset on the reboot side of things...

Updates since v8:
- v9 and v8 were not submitted properly because of my MTA not behaving, sorry for that.
- v10 introduces a new argument, -M msi=, to handle MSI-X configuration more granularly.
- That surfaced what I think is a bug (?), with vms->its=1 on GICv2 configurations... or I did understand everything wrong.
- Oopsie due to email provider ratelimiting.

Updates since v7:
- Oops, fixing bug in "hw/arm: virt: cleanly fail on attempt to use the platform vGIC together with ITS". Other commits are unchanged.

Updates since v6:
- Rebasing
- Fixing a bug in the GICv3+GICv2m case for ACPI table generation
- getting rid of the slots infrastructure for memory management
- Place the docs commit right after the "cleanly fail on attempt to run GICv3+GICv2m on an unsupported config" one as that's what switches ITS to a tristate.
- Fixing a build issue when getting rid of the arch-specific arm64 hvf-stub.

Updates since v5:
- Rebasing
- Address review comments
- Rework ITS enablement to a tristate
- On x86: move away from deprecated APIs to get/set APIC state

Updates since v4:
- Taking into account review comments
- Add migration blocker in the vGICv3 code due to missing interrupt controller save/restore
- Debug register sync

Updates since v3:
- Disabling SVE on WHPX
- Taking into account review comments incl:

- fixing x86 support
- reduce the amount of __x86_64__ checks in common code to the minimum (winhvemulation) which can be reduced even further down the road.
- generalize get_physical_address_range into something common between hvf and whpx

Updates since v2:
- Fixed up a rebase screwup for whpx-internal.h
- Fixed ID_AA64ISAR1_EL1 and ID_AA64ISAR2_EL1 feature probe for -cpu host
- Switched to ID_AA64PFR1_EL1/ID_AA64DFR0_EL1 instead of their non-AA64 variant

Updates since v1:
- Shutdowns and reboots
- MPIDR_EL1 register sync
- Fixing GICD_TYPER_LPIS value
- IPA size clamping
- -cpu host now implemented

Mohamed Mediouni (19):
hw: arm: virt: rework MSI-X configuration
qtest: hw/arm: virt: add new test case for GICv3 + GICv2m
docs: arm: update virt machine model description
whpx: Move around files before introducing AArch64 support
whpx: reshuffle common code
whpx: ifdef out winhvemulation on non-x86_64
whpx: common: add WHPX_INTERCEPT_DEBUG_TRAPS define
hw, target, accel: whpx: change apic_in_platform to kernel_irqchip
whpx: interrupt controller support
whpx: add arm64 support
whpx: change memory management logic
target/arm: cpu: mark WHPX as supporting PSCI 1.3
whpx: arm64: clamp down IPA size
hw/arm, accel/hvf, whpx: unify get_physical_address_range between WHPX
and HVF
whpx: arm64: implement -cpu host
target/arm: whpx: instantiate GIC early
whpx: arm64: gicv3: add migration blocker
whpx: enable arm64 builds
whpx: arm64: add partition-wide reset on the reboot path

Philippe Mathieu-Daudé (1):
accel/system: Introduce hwaccel_enabled() helper

Sebastian Ott (1):
target/arm/kvm: add constants for new PSCI versions

 MAINTAINERS                                   |   6 +
 accel/hvf/hvf-all.c                           |   7 +-
 accel/meson.build                             |   1 +
 accel/stubs/whpx-stub.c                       |   1 +
 accel/whpx/meson.build                        |   7 +
 {target/i386 => accel}/whpx/whpx-accel-ops.c  |   6 +-
 accel/whpx/whpx-common.c                      | 536 ++++++++++
 docs/system/arm/virt.rst                      |  13 +-
 hw/arm/virt-acpi-build.c                      |  27 +-
 hw/arm/virt.c                                 | 160 ++-
 hw/i386/x86-cpu.c                             |   4 +-
 hw/intc/arm_gicv3_common.c                    |   3 +
 hw/intc/arm_gicv3_whpx.c                      | 247 +++++
 hw/intc/meson.build                           |   1 +
 include/hw/arm/virt.h                         |   8 +-
 include/hw/core/boards.h                      |   3 +-
 include/hw/intc/arm_gicv3_common.h            |   3 +
 include/system/hvf_int.h                      |   4 +
 include/system/hw_accel.h                     |  13 +
 .../whpx => include/system}/whpx-accel-ops.h  |   4 +-
 include/system/whpx-all.h                     |  20 +
 include/system/whpx-common.h                  |  26 +
 .../whpx => include/system}/whpx-internal.h   |  16 +-
 include/system/whpx.h                         |   5 +-
 meson.build                                   |  20 +-
 target/arm/cpu.c                              |   3 +
 target/arm/cpu64.c                            |  17 +-
 target/arm/hvf-stub.c                         |  20 -
 target/arm/hvf/hvf.c                          |   6 +-
 target/arm/hvf_arm.h                          |   3 -
 target/arm/kvm-consts.h                       |   2 +
 target/arm/meson.build                        |   2 +-
 target/arm/whpx/meson.build                   |   5 +
 target/arm/whpx/whpx-all.c                    | 985 ++++++++++++++++++
 target/arm/whpx/whpx-stub.c                   |  15 +
 target/arm/whpx_arm.h                         |  17 +
 target/i386/cpu-apic.c                        |   2 +-
 target/i386/hvf/hvf.c                         |  11 +
 target/i386/whpx/meson.build                  |   1 -
 target/i386/whpx/whpx-all.c                   | 569 +---------
 target/i386/whpx/whpx-apic.c                  |   2 +-
 tests/data/acpi/aarch64/virt/APIC.msi_gicv2m  | Bin 0 -> 188 bytes
 tests/data/acpi/aarch64/virt/IORT.msi_gicv2m  | Bin 0 -> 172 bytes
 tests/qtest/bios-tables-test.c                |  21 +
 44 files changed, 2186 insertions(+), 636 deletions(-)
 create mode 100644 accel/whpx/meson.build
 rename {target/i386 => accel}/whpx/whpx-accel-ops.c (96%)
 create mode 100644 accel/whpx/whpx-common.c
 create mode 100644 hw/intc/arm_gicv3_whpx.c
 rename {target/i386/whpx => include/system}/whpx-accel-ops.h (92%)
 create mode 100644 include/system/whpx-all.h
 create mode 100644 include/system/whpx-common.h
 rename {target/i386/whpx => include/system}/whpx-internal.h (95%)
 delete mode 100644 target/arm/hvf-stub.c
 create mode 100644 target/arm/whpx/meson.build
 create mode 100644 target/arm/whpx/whpx-all.c
 create mode 100644 target/arm/whpx/whpx-stub.c
 create mode 100644 target/arm/whpx_arm.h
 create mode 100644 tests/data/acpi/aarch64/virt/APIC.msi_gicv2m
 create mode 100644 tests/data/acpi/aarch64/virt/IORT.msi_gicv2m
Signature · 1 line
2.50.1 (Apple Git-155)
#1 lore raw

PATCH v17 1/21 target/arm/kvm: add constants for new PSCI versions

From: Sebastian Ott <[email protected]>

Add constants for PSCI version 1_2 and 1_3.

---
 target/arm/kvm-consts.h | 2 ++
 1 file changed, 2 insertions(+)
target/arm/kvm-consts.h +2 -0
index 54ae5da7ce..9fba3e886d 100644
@@ -97,6 +97,8 @@ MISMATCH_CHECK(QEMU_PSCI_1_0_FN_PSCI_FEATURES, PSCI_1_0_FN_PSCI_FEATURES);
 #define QEMU_PSCI_VERSION_0_2                     0x00002
 #define QEMU_PSCI_VERSION_1_0                     0x10000
 #define QEMU_PSCI_VERSION_1_1                     0x10001
+#define QEMU_PSCI_VERSION_1_2                     0x10002
+#define QEMU_PSCI_VERSION_1_3                     0x10003
 
 MISMATCH_CHECK(QEMU_PSCI_0_2_RET_TOS_MIGRATION_NOT_REQUIRED, PSCI_0_2_TOS_MP);
 /* We don't bother to check every possible version value */
Signature · 1 line
2.50.1 (Apple Git-155)
Reviewed-by on v17 1/21
#2 lore raw

Re: [PATCH v17 01/21] target/arm/kvm: add constants for new PSCI versions

Quoted 11 lines · Mohamed Mediouni

From: Sebastian Ott <[email protected]>

Add constants for PSCI version 1_2 and 1_3.

---
  target/arm/kvm-consts.h | 2 ++
  1 file changed, 2 insertions(+)
#3 lore raw

PATCH v17 2/21 accel/system: Introduce hwaccel_enabled() helper

In reply to Mohamed Mediouni

From: Philippe Mathieu-Daudé <[email protected]>

hwaccel_enabled() return whether any hardware accelerator is enabled.

---
 include/system/hw_accel.h | 13 +++++++++++++
 1 file changed, 13 insertions(+)
include/system/hw_accel.h +13 -0
index 55497edc29..628a50e066 100644
@@ -40,4 +40,17 @@ void cpu_synchronize_pre_loadvm(CPUState *cpu);
 void cpu_synchronize_post_reset(CPUState *cpu);
 void cpu_synchronize_post_init(CPUState *cpu);
 
+/**
+ * hwaccel_enabled:
+ *
+ * Returns: %true if a hardware accelerator is enabled, %false otherwise.
+ */
+static inline bool hwaccel_enabled(void)
+{
+    return hvf_enabled()
+        || kvm_enabled()
+        || nvmm_enabled()
+        || whpx_enabled();
+}
+
 #endif /* QEMU_HW_ACCEL_H */
Signature · 1 line
2.50.1 (Apple Git-155)
#4 lore raw

PATCH v17 3/21 hw: arm: virt: rework MSI-X configuration

In reply to Mohamed Mediouni

Introduce a -M msi= argument to be able to control MSI-X support independently from ITS, as part of supporting GICv3 + GICv2m platforms.

Remove vms->its as it's no longer needed after that change.

---
 hw/arm/virt-acpi-build.c |  27 ++++++--
 hw/arm/virt.c            | 137 +++++++++++++++++++++++++++++++++++++--
 include/hw/arm/virt.h    |   8 ++-
 3 files changed, 157 insertions(+), 15 deletions(-)
hw/arm/virt-acpi-build.c +20 -7
index 03b4342574..187dd4e272 100644
@@ -440,6 +440,17 @@ static void create_rc_its_idmaps(GArray *its_idmaps, GArray *smmuv3_devs)
     }
 }
 
+/*
+ * In the prior Qemu ACPI table handling, GICv2 configurations
+ * had vms->its=1... That's broken.
+ *
+ * Match that assumption to match the existing ACPI tables that
+ * have been shipping for quite a while.
+ */
+static int is_gicv2_acpi_workaround_needed(VirtMachineState *vms) {
+    return vms->gic_version == 2;
+}
+
 /*
  * Input Output Remapping Table (IORT)
  * Conforms to "IO Remapping Table System Software on ARM Platforms",
@@ -473,7 +484,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
         nb_nodes = num_smmus + 1; /* RC and SMMUv3 */
         rc_mapping_count = rc_smmu_idmaps_len;
 
-        if (vms->its) {
+        if (virt_is_its_enabled(vms) || is_gicv2_acpi_workaround_needed(vms)) {
             /*
              * Knowing the ID ranges from the RC to the SMMU, it's possible to
              * determine the ID ranges from RC that go directly to ITS.
@@ -484,7 +495,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
             rc_mapping_count += rc_its_idmaps->len;
         }
     } else {
-        if (vms->its) {
+        if (virt_is_its_enabled(vms) || is_gicv2_acpi_workaround_needed(vms)) {
             nb_nodes = 2; /* RC and ITS */
             rc_mapping_count = 1; /* Direct map to ITS */
         } else {
@@ -499,7 +510,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
     build_append_int_noprefix(table_data, IORT_NODE_OFFSET, 4);
     build_append_int_noprefix(table_data, 0, 4); /* Reserved */
 
-    if (vms->its) {
+    if (virt_is_its_enabled(vms) || is_gicv2_acpi_workaround_needed(vms)) {
         /* Table 12 ITS Group Format */
         build_append_int_noprefix(table_data, 0 /* ITS Group */, 1); /* Type */
         node_size =  20 /* fixed header size */ + 4 /* 1 GIC ITS Identifier */;
@@ -518,7 +529,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
         int smmu_mapping_count, offset_to_id_array;
         int irq = sdev->irq;
 
-        if (vms->its) {
+        if (virt_is_its_enabled(vms) || is_gicv2_acpi_workaround_needed(vms)) {
             smmu_mapping_count = 1; /* ITS Group node */
             offset_to_id_array = SMMU_V3_ENTRY_SIZE; /* Just after the header */
         } else {
@@ -611,7 +622,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
             }
         }
 
-        if (vms->its) {
+        if (virt_is_its_enabled(vms) || is_gicv2_acpi_workaround_needed(vms)) {
             /*
              * Map bypassed (don't go through the SMMU) RIDs (input) to
              * ITS Group node directly: RC -> ITS.
@@ -946,7 +957,7 @@ build_madt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
                                           memmap[VIRT_HIGH_GIC_REDIST2].size);
         }
 
-        if (vms->its) {
+        if (virt_is_its_enabled(vms)) {
             /*
              * ACPI spec, Revision 6.0 Errata A
              * (original 6.0 definition has invalid Length)
@@ -960,7 +971,9 @@ build_madt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
             build_append_int_noprefix(table_data, memmap[VIRT_GIC_ITS].base, 8);
             build_append_int_noprefix(table_data, 0, 4);    /* Reserved */
         }
-    } else {
+    }
+
+    if (virt_is_gicv2m_enabled(vms)) {
         const uint16_t spi_base = vms->irqmap[VIRT_GIC_V2M] + ARM_SPI_BASE;
 
         /* 5.2.12.16 GIC MSI Frame Structure */
hw/arm/virt.c +130 -7

196 lines not on this page · raw message

include/hw/arm/virt.h +7 -1
index 5907d41dbb..6ca849e3fd 100644
@@ -101,6 +101,10 @@ typedef enum VirtIOMMUType {
 
 typedef enum VirtMSIControllerType {
     VIRT_MSI_CTRL_NONE,
+    /* This value is overriden at runtime.*/
+    VIRT_MSI_CTRL_AUTO,
+    /* Legacy option: its=off provides a GICv2m when using GICv2 */
+    VIRT_MSI_LEGACY_OPT_ITS_OFF,
     VIRT_MSI_CTRL_GICV2M,
     VIRT_MSI_CTRL_ITS,
 } VirtMSIControllerType;
@@ -146,7 +150,6 @@ struct VirtMachineState {
     bool highmem_ecam;
     bool highmem_mmio;
     bool highmem_redists;
-    bool its;
     bool tcg_its;
     bool virt;
     bool ras;
@@ -216,4 +219,7 @@ static inline int virt_gicv3_redist_region_count(VirtMachineState *vms)
             vms->highmem_redists) ? 2 : 1;
 }
 
+bool virt_is_its_enabled(VirtMachineState *vms);
+bool virt_is_gicv2m_enabled(VirtMachineState *vms);
+
 #endif /* QEMU_ARM_VIRT_H */
Signature · 1 line
2.50.1 (Apple Git-155)
#5 lore raw

Re: [PATCH v17 03/21] hw: arm: virt: rework MSI-X configuration

Quoted 21 lines · Mohamed Mediouni

Introduce a -M msi= argument to be able to control MSI-X support independently from ITS, as part of supporting GICv3 + GICv2m platforms.

Remove vms->its as it's no longer needed after that change.

---
 hw/arm/virt-acpi-build.c |  27 ++++++--
 hw/arm/virt.c            | 137 +++++++++++++++++++++++++++++++++++++--
 include/hw/arm/virt.h    |   8 ++-
 3 files changed, 157 insertions(+), 15 deletions(-)
hw/arm/virt-acpi-build.c +0 -0
index 03b4342574..187dd4e272 100644
@@ -440,6 +440,17 @@ static void create_rc_its_idmaps(GArray *its_idmaps, GArray *smmuv3_devs)
     }
 }
 

hw/arm/virt-acpi-build.c static void create_rc_its_idmaps(GArray *its_idmaps, GArray *smmuv3_devs) · line 452

Quoted 10 lines
hw/arm/virt-acpi-build.c +10 -0
+/*
+ * In the prior Qemu ACPI table handling, GICv2 configurations
+ * had vms->its=1... That's broken.
+ *
+ * Match that assumption to match the existing ACPI tables that
+ * have been shipping for quite a while.
+ */
+static int is_gicv2_acpi_workaround_needed(VirtMachineState *vms) {
+    return vms->gic_version == 2;
+}

We don't need to keep identical bug-for-bug ACPI tables like that. If we were incorrectly reporting an ITS in a GICv2-only ACPI table, that was a bug and we can fix it. (This might need adjusting of the golden reference ACPI data in some of the bios-tables-tests if we were testing that, so it ought to go in its own patch.)

Quoted 24 lines
hw/arm/virt-acpi-build.c +24 -0
+bool virt_is_its_enabled(VirtMachineState *vms)
+{
+    switch (vms->msi_controller) {
+    case VIRT_MSI_CTRL_NONE:
+        return false;
+    case VIRT_MSI_CTRL_ITS:
+        return true;
+    case VIRT_MSI_CTRL_GICV2M:
+        return false;
+    case VIRT_MSI_CTRL_AUTO:
+        /*
+         * Earlier Qemu considered its=on as the default when using the GICv2.
+         * It is safe to do this because this is called is prior to
+         * finalize_msi_controller.
+         */
+        return true;
+    default:
+        g_assert_not_reached();
+    }
+}
+
+bool virt_is_gicv2m_enabled(VirtMachineState *vms)
+{
+    switch (vms->msi_controller) {

hw/arm/virt-acpi-build.c static void create_rc_its_idmaps(GArray *its_idmaps, GArray *smmuv3_devs)

Quoted 10 lines
hw/arm/virt-acpi-build.c +10 -0
+    case VIRT_MSI_CTRL_NONE:
+        return false;
+    case VIRT_MSI_CTRL_ITS:
+    case VIRT_MSI_CTRL_GICV2M:
+    case VIRT_MSI_CTRL_AUTO:
+        return !virt_is_its_enabled(vms);
+    default:
+        g_assert_not_reached();
+    }
+}

Why do we need these? The idea of finalize_msi_controller() is that it fixes vms->msi_controller so that you can just directly say vms->msi_controller == VIRT_MSI_CTRL_ITS or vms->msi_controller == VIRT_MSI_CTRL_GICV2M rather than having to have more complicated logic.

thanks
-- PMM

#6 lore raw

Re: [PATCH v17 03/21] hw: arm: virt: rework MSI-X configuration

Quoted 26 lines · Peter Maydell
Quoted 21 lines · Mohamed Mediouni

Introduce a -M msi= argument to be able to control MSI-X support independently from ITS, as part of supporting GICv3 + GICv2m platforms.

Remove vms->its as it's no longer needed after that change.

---

hw/arm/virt-acpi-build.c |  27 ++++++--
hw/arm/virt.c            | 137 +++++++++++++++++++++++++++++++++++++--
include/hw/arm/virt.h    |   8 ++-
3 files changed, 157 insertions(+), 15 deletions(-)
hw/arm/virt-acpi-build.c +0 -0
index 03b4342574..187dd4e272 100644
@@ -440,6 +440,17 @@ static void create_rc_its_idmaps(GArray *its_idmaps, GArray *smmuv3_devs)
    }
 }
 
Quoted 17 lines

hw/arm/virt-acpi-build.c static void create_rc_its_idmaps(GArray *its_idmaps, GArray *smmuv3_devs) · line 452

Quoted 10 lines
hw/arm/virt-acpi-build.c +10 -0
+/*
+ * In the prior Qemu ACPI table handling, GICv2 configurations
+ * had vms->its=1... That's broken.
+ *
+ * Match that assumption to match the existing ACPI tables that
+ * have been shipping for quite a while.
+ */
+static int is_gicv2_acpi_workaround_needed(VirtMachineState *vms) {
+    return vms->gic_version == 2;
+}

We don't need to keep identical bug-for-bug ACPI tables like that. If we were incorrectly reporting an ITS in a GICv2-only ACPI table, that was a bug and we can fix it. (This might need adjusting of the golden reference ACPI data in some of the bios-tables-tests if we were testing that, so it ought to go in its own patch.)

Hello,

I’m a bit concerned about breaking hibernation in this case…

My intent was keeping this behavior for now and then add a machine model version dependent toggle in a follow-up patch.

Quoted 32 lines
Quoted 32 lines
hw/arm/virt-acpi-build.c +32 -0
+bool virt_is_its_enabled(VirtMachineState *vms)
+{
+    switch (vms->msi_controller) {
+    case VIRT_MSI_CTRL_NONE:
+        return false;
+    case VIRT_MSI_CTRL_ITS:
+        return true;
+    case VIRT_MSI_CTRL_GICV2M:
+        return false;
+    case VIRT_MSI_CTRL_AUTO:
+        /*
+         * Earlier Qemu considered its=on as the default when using the GICv2.
+         * It is safe to do this because this is called is prior to
+         * finalize_msi_controller.
+         */
+        return true;
+    default:
+        g_assert_not_reached();
+    }
+}
+
+bool virt_is_gicv2m_enabled(VirtMachineState *vms)
+{
+    switch (vms->msi_controller) {
+    case VIRT_MSI_CTRL_NONE:
+        return false;
+    case VIRT_MSI_CTRL_ITS:
+    case VIRT_MSI_CTRL_GICV2M:
+    case VIRT_MSI_CTRL_AUTO:
+        return !virt_is_its_enabled(vms);
+    default:
+        g_assert_not_reached();
Quoted 9 lines

hw/arm/virt-acpi-build.c

Quoted 2 lines
hw/arm/virt-acpi-build.c +2 -0
+    }
+}

Why do we need these? The idea of finalize_msi_controller() is that it fixes vms->msi_controller so that you can just directly say vms->msi_controller == VIRT_MSI_CTRL_ITS or vms->msi_controller == VIRT_MSI_CTRL_GICV2M rather than having to have more complicated logic.

It’s because of object_class_property_add_str(oc, "msi", virt_get_msi, virt_set_msi) (and the same applies for the its property), is it safe to assume that virt_get_msi/virt_get_its will never be called from that prior to finalize?

Quoted 3 lines

thanks
-- PMM

#7 lore raw

Re: [PATCH v17 03/21] hw: arm: virt: rework MSI-X configuration

Quoted 20 lines · Mohamed Mediouni
Quoted 20 lines · Peter Maydell

hw/arm/virt-acpi-build.c static void create_rc_its_idmaps(GArray *its_idmaps, GArray *smmuv3_devs) · line 452

Quoted 10 lines · Mohamed Mediouni
hw/arm/virt-acpi-build.c +10 -0
+/*
+ * In the prior Qemu ACPI table handling, GICv2 configurations
+ * had vms->its=1... That's broken.
+ *
+ * Match that assumption to match the existing ACPI tables that
+ * have been shipping for quite a while.
+ */
+static int is_gicv2_acpi_workaround_needed(VirtMachineState *vms) {
+    return vms->gic_version == 2;
+}

We don't need to keep identical bug-for-bug ACPI tables like that. If we were incorrectly reporting an ITS in a GICv2-only ACPI table, that was a bug and we can fix it. (This might need adjusting of the golden reference ACPI data in some of the bios-tables-tests if we were testing that, so it ought to go in its own patch.)

Quoted 5 lines

Hello,

I’m a bit concerned about breaking hibernation in this case…

My intent was keeping this behavior for now and then add a machine model version dependent toggle in a follow-up patch.

I'm not an ACPI table expert but my understanding is that it's OK to change the ACPI table contents without having to make those changes machine-version dependent. See e.g. commit d6afe18b7242, which changed the ACPI tables for the its=off case and did not make those changes machine-version specific.

Quoted 9 lines
Quoted 8 lines
Quoted 1 line

+bool virt_is_its_enabled(VirtMachineState *vms)

Why do we need these? The idea of finalize_msi_controller() is that it fixes vms->msi_controller so that you can just directly say vms->msi_controller == VIRT_MSI_CTRL_ITS or vms->msi_controller == VIRT_MSI_CTRL_GICV2M rather than having to have more complicated logic.

It’s because of object_class_property_add_str(oc, "msi", virt_get_msi, virt_set_msi) (and the same applies for the its property), is it safe to assume that virt_get_msi/virt_get_its will never be called from that prior to finalize?

The get and set property functions need to handle the msi_controller field not having been finalized, but they are only getting and setting it.

thanks
-- PMM

#8 lore raw

Re: [PATCH v17 03/21] hw: arm: virt: rework MSI-X configuration

Quoted 29 lines · Peter Maydell
Quoted 24 lines · Mohamed Mediouni
Quoted 20 lines · Peter Maydell

hw/arm/virt-acpi-build.c static void create_rc_its_idmaps(GArray *its_idmaps, GArray *smmuv3_devs) · line 452

Quoted 10 lines · Mohamed Mediouni
hw/arm/virt-acpi-build.c +10 -0
+/*
+ * In the prior Qemu ACPI table handling, GICv2 configurations
+ * had vms->its=1... That's broken.
+ *
+ * Match that assumption to match the existing ACPI tables that
+ * have been shipping for quite a while.
+ */
+static int is_gicv2_acpi_workaround_needed(VirtMachineState *vms) {
+    return vms->gic_version == 2;
+}

We don't need to keep identical bug-for-bug ACPI tables like that. If we were incorrectly reporting an ITS in a GICv2-only ACPI table, that was a bug and we can fix it. (This might need adjusting of the golden reference ACPI data in some of the bios-tables-tests if we were testing that, so it ought to go in its own patch.)

Hello,

I’m a bit concerned about breaking hibernation in this case…

Quoted 8 lines
Quoted 1 line

My intent was keeping this behavior for now and then add a machine model version dependent toggle in a follow-up patch.

I'm not an ACPI table expert but my understanding is that it's OK to change the ACPI table contents without having to make those changes machine-version dependent. See e.g. commit d6afe18b7242, which changed the ACPI tables for the its=off case and did not make those changes machine-version specific.

Hi,

That commit didn’t affect the default/regular config so it wouldn't have been too problematic in practice.

Have had countless issues with how brittle hibernation is* - and more or less subtle ACPI table changes tend to break it.

* and Qemu doesn’t provide an FACS (nevermind one with an HW signature) to force a clean reboot
- which Windows ignores anyway

Quoted 17 lines
Quoted 9 lines
Quoted 8 lines
Quoted 1 line

+bool virt_is_its_enabled(VirtMachineState *vms)

Why do we need these? The idea of finalize_msi_controller() is that it fixes vms->msi_controller so that you can just directly say vms->msi_controller == VIRT_MSI_CTRL_ITS or vms->msi_controller == VIRT_MSI_CTRL_GICV2M rather than having to have more complicated logic.

It’s because of object_class_property_add_str(oc, "msi", virt_get_msi, virt_set_msi) (and the same applies for the its property), is it safe to assume that virt_get_msi/virt_get_its will never be called from that prior to finalize?

The get and set property functions need to handle the msi_controller field not having been finalized, but they are only getting and setting it.

thanks
-- PMM

#9 lore raw

Re: [PATCH v17 03/21] hw: arm: virt: rework MSI-X configuration

Quoted 38 lines · Mohamed Mediouni
Quoted 30 lines · Peter Maydell
Quoted 24 lines · Mohamed Mediouni
Quoted 20 lines · Peter Maydell

hw/arm/virt-acpi-build.c static void create_rc_its_idmaps(GArray *its_idmaps, GArray *smmuv3_devs) · line 452

Quoted 10 lines · Mohamed Mediouni
hw/arm/virt-acpi-build.c +10 -0
+/*
+ * In the prior Qemu ACPI table handling, GICv2 configurations
+ * had vms->its=1... That's broken.
+ *
+ * Match that assumption to match the existing ACPI tables that
+ * have been shipping for quite a while.
+ */
+static int is_gicv2_acpi_workaround_needed(VirtMachineState *vms) {
+    return vms->gic_version == 2;
+}

We don't need to keep identical bug-for-bug ACPI tables like that. If we were incorrectly reporting an ITS in a GICv2-only ACPI table, that was a bug and we can fix it. (This might need adjusting of the golden reference ACPI data in some of the bios-tables-tests if we were testing that, so it ought to go in its own patch.)

Hello,

I’m a bit concerned about breaking hibernation in this case…

Quoted 7 lines
Quoted 1 line

My intent was keeping this behavior for now and then add a machine model version dependent toggle in a follow-up patch.

I'm not an ACPI table expert but my understanding is that it's OK to change the ACPI table contents without having to make those changes machine-version dependent. See e.g. commit d6afe18b7242, which changed the ACPI tables for the its=off case and did not make those changes machine-version specific.

Quoted 4 lines

That commit didn’t affect the default/regular config so it wouldn't have been too problematic in practice.

Have had countless issues with how brittle hibernation is* - and more or less subtle ACPI table changes tend to break it.

I don't want to add back-compat handling for this to the virt board unless one of our ACPI table experts says that yes we do need to keep the ACPI table contents identical for older versioned machine types.

thanks
-- PMM

#10 lore raw

Re: [PATCH v17 03/21] hw: arm: virt: rework MSI-X configuration

Quoted 44 lines · Peter Maydell
Quoted 38 lines · Mohamed Mediouni
Quoted 30 lines · Peter Maydell
Quoted 24 lines · Mohamed Mediouni
Quoted 20 lines · Peter Maydell

hw/arm/virt-acpi-build.c static void create_rc_its_idmaps(GArray *its_idmaps, GArray *smmuv3_devs) · line 452

Quoted 10 lines · Mohamed Mediouni
hw/arm/virt-acpi-build.c +10 -0
+/*
+ * In the prior Qemu ACPI table handling, GICv2 configurations
+ * had vms->its=1... That's broken.
+ *
+ * Match that assumption to match the existing ACPI tables that
+ * have been shipping for quite a while.
+ */
+static int is_gicv2_acpi_workaround_needed(VirtMachineState *vms) {
+    return vms->gic_version == 2;
+}

We don't need to keep identical bug-for-bug ACPI tables like that. If we were incorrectly reporting an ITS in a GICv2-only ACPI table, that was a bug and we can fix it. (This might need adjusting of the golden reference ACPI data in some of the bios-tables-tests if we were testing that, so it ought to go in its own patch.)

Hello,

I’m a bit concerned about breaking hibernation in this case…

Quoted 7 lines
Quoted 1 line

My intent was keeping this behavior for now and then add a machine model version dependent toggle in a follow-up patch.

I'm not an ACPI table expert but my understanding is that it's OK to change the ACPI table contents without having to make those changes machine-version dependent. See e.g. commit d6afe18b7242, which changed the ACPI tables for the its=off case and did not make those changes machine-version specific.

Quoted 4 lines

That commit didn’t affect the default/regular config so it wouldn't have been too problematic in practice.

Have had countless issues with how brittle hibernation is* - and more or less subtle ACPI table changes tend to break it.

Quoted 7 lines

I don't want to add back-compat handling for this to the virt board unless one of our ACPI table experts says that yes we do need to keep the ACPI table contents identical for older versioned machine types.

thanks
-- PMM

Generally what we have handles VM migration, not hybernation. The issue with hybernation is guests might cache some data to speed up init. If you want to go overboard you would have to never change anything at all in the firmware. Whether they do it in any specific instance is hard to predict 100% but our approach generally is to try with a couple of popular guests, and not to worry too much ahead of time otherwise. For several reasons: one is most changes are harmless, another one there's no special race here, if there's an issue you can trigger it predictably, yet another one is hybernation is a rarely used feature.

For the specific change, I'd guess just test and if fine - fix it without a compat.

Signature · 1 line
MST
#11 lore raw

Re: [PATCH v17 03/21] hw: arm: virt: rework MSI-X configuration

Quoted 68 lines · Michael S. Tsirkin
Quoted 44 lines · Peter Maydell
Quoted 38 lines · Mohamed Mediouni
Quoted 30 lines · Peter Maydell
Quoted 24 lines · Mohamed Mediouni
Quoted 20 lines · Peter Maydell

Review

Quoted 10 lines · Mohamed Mediouni
(quoted diff) +10 -0
+/*
+ * In the prior Qemu ACPI table handling, GICv2 configurations
+ * had vms->its=1... That's broken.
+ *
+ * Match that assumption to match the existing ACPI tables that
+ * have been shipping for quite a while.
+ */
+static int is_gicv2_acpi_workaround_needed(VirtMachineState *vms) {
+    return vms->gic_version == 2;
+}

We don't need to keep identical bug-for-bug ACPI tables like that. If we were incorrectly reporting an ITS in a GICv2-only ACPI table, that was a bug and we can fix it. (This might need adjusting of the golden reference ACPI data in some of the bios-tables-tests if we were testing that, so it ought to go in its own patch.)

Hello,

I’m a bit concerned about breaking hibernation in this case…

Quoted 7 lines
Quoted 1 line

My intent was keeping this behavior for now and then add a machine model version dependent toggle in a follow-up patch.

I'm not an ACPI table expert but my understanding is that it's OK to change the ACPI table contents without having to make those changes machine-version dependent. See e.g. commit d6afe18b7242, which changed the ACPI tables for the its=off case and did not make those changes machine-version specific.

Quoted 4 lines

That commit didn’t affect the default/regular config so it wouldn't have been too problematic in practice.

Have had countless issues with how brittle hibernation is* - and more or less subtle ACPI table changes tend to break it.

Quoted 7 lines

I don't want to add back-compat handling for this to the virt board unless one of our ACPI table experts says that yes we do need to keep the ACPI table contents identical for older versioned machine types.

thanks
-- PMM

Generally what we have handles VM migration, not hybernation. The issue with hybernation is guests might cache some data to speed up init. If you want to go overboard you would have to never change anything at all in the firmware. Whether they do it in any specific instance is hard to predict 100% but our approach generally is to try with a couple of popular guests, and not to worry too much ahead of time otherwise. For several reasons: one is most changes are harmless, another one there's no special race here, if there's an issue you can trigger it predictably, yet another one is hybernation is a rarely used feature.

Quoted 2 lines

For the specific change, I'd guess just test and if fine - fix it without a compat.

Hi,

It turns out that even the Ubuntu stock kernel of all things has hibernation disabled on arm64. Had to install the linux-aws variant to test it and that testing didn’t go too well without even making any Qemu changes between suspend and resume.

So considering it as unimportant…

(And just sent a new rev with putting the new behaviour, had to tackle a bug on the way)

Thank you,
-Mohamed

Quoted 4 lines
Signature · 1 line
MST
#12 lore raw

PATCH v17 4/21 qtest: hw/arm: virt: add new test case for GICv3 + GICv2m

In reply to Mohamed Mediouni
---
 tests/data/acpi/aarch64/virt/APIC.msi_gicv2m | Bin 0 -> 188 bytes
 tests/data/acpi/aarch64/virt/IORT.msi_gicv2m | Bin 0 -> 172 bytes
 tests/qtest/bios-tables-test.c               |  21 +++++++++++++++++++
 3 files changed, 21 insertions(+)
 create mode 100644 tests/data/acpi/aarch64/virt/APIC.msi_gicv2m
 create mode 100644 tests/data/acpi/aarch64/virt/IORT.msi_gicv2m
tests/data/acpi/aarch64/virt/APIC.msi_gicv2m new file binary +0 -0
new file mode 100644
index 0000000000000000000000000000000000000000..16a01a17c0af605daf64f3cd2de3572be9e60cab
GIT binary patch
tests/data/acpi/aarch64/virt/IORT.msi_gicv2m new file binary +0 -0
new file mode 100644
index 0000000000000000000000000000000000000000..c10da4e61dd00e7eb062558a2735d49ca0b20620
GIT binary patch
tests/qtest/bios-tables-test.c +21 -0
index 6b892ef23e..fe6af5412f 100644
@@ -2251,6 +2251,25 @@ static void test_acpi_aarch64_virt_tcg_its_off(void)
     free_test_data(&data);
 }
 
+static void test_acpi_aarch64_virt_tcg_msi_gicv2m(void)
+{
+    test_data data = {
+        .machine = "virt",
+        .arch = "aarch64",
+        .variant = ".msi_gicv2m",
+        .tcg_only = true,
+        .uefi_fl1 = "pc-bios/edk2-aarch64-code.fd",
+        .uefi_fl2 = "pc-bios/edk2-arm-vars.fd",
+        .cd = "tests/data/uefi-boot-images/bios-tables-test.aarch64.iso.qcow2",
+        .ram_start = 0x40000000ULL,
+        .scan_len = 128ULL * 1024 * 1024,
+    };
+
+    test_acpi_one("-cpu cortex-a57 "
+                  "-M gic-version=3,iommu=smmuv3,msi=gicv2m", &data);
+    free_test_data(&data);
+}
+
 static void test_acpi_q35_viot(void)
 {
     test_data data = {
@@ -2833,6 +2852,8 @@ int main(int argc, char *argv[])
                            test_acpi_aarch64_virt_tcg_topology);
             qtest_add_func("acpi/virt/its_off",
                            test_acpi_aarch64_virt_tcg_its_off);
+            qtest_add_func("acpi/virt/msi_gicv2m",
+                           test_acpi_aarch64_virt_tcg_msi_gicv2m);
             qtest_add_func("acpi/virt/numamem",
                            test_acpi_aarch64_virt_tcg_numamem);
             qtest_add_func("acpi/virt/memhp", test_acpi_aarch64_virt_tcg_memhp);
Signature · 1 line
2.50.1 (Apple Git-155)
#13 lore raw

PATCH v17 5/21 docs: arm: update virt machine model description

In reply to Mohamed Mediouni

Update the documentation to match current QEMU.

Remove the mention of pre-2.7 machine models as those aren't provided anymore.

---
 docs/system/arm/virt.rst | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)
docs/system/arm/virt.rst +8 -5
index e5570773ba..3799d2237e 100644
@@ -41,9 +41,10 @@ The virt board supports:
 - User-creatable SMMUv3 devices (see below for example)
 - hotpluggable DIMMs
 - hotpluggable NVDIMMs
-- An MSI controller (GICv2M or ITS). GICv2M is selected by default along
-  with GICv2. ITS is selected by default with GICv3 (>= virt-2.7). Note
-  that ITS is not modeled in TCG mode.
+- An MSI controller (GICv2m or ITS).
+  - When using GICv3, ITS is selected by default when available on the platform.
+  - If using GICv2, a GICv2m is provided by default instead.
+  - When ITS is not available on a GICv3 platform, a GICv2m is provided by default.
 - 32 virtio-mmio transport devices
 - running guests using the KVM accelerator on aarch64 hardware
 - large amounts of RAM (at least 255GB, and more if using highmem)
@@ -167,9 +168,11 @@ gic-version
     with TCG this is currently ``3`` if ``virtualization`` is ``off`` and
     ``4`` if ``virtualization`` is ``on``, but this may change in future)
 
+msi
+  Set ``auto``/``gicv2m``/``its``/``none`` to control MSI controller configuration. The default is ``auto``.
+
 its
-  Set ``on``/``off`` to enable/disable ITS instantiation. The default is ``on``
-  for machine types later than ``virt-2.7``.
+  Set ``on``/``off`` to control ITS instantiation. This is a deprecated option, use ``msi`` instead.
 
 iommu
   Set the IOMMU type to create for the guest. Valid values are:
Signature · 1 line
2.50.1 (Apple Git-155)
#14 lore raw

PATCH v17 6/21 whpx: Move around files before introducing AArch64 support

In reply to Mohamed Mediouni

Switch to a design where we can share whpx code between x86 and AArch64 when it makes sense to do so.

---
 MAINTAINERS                                           | 2 ++
 accel/meson.build                                     | 1 +
 accel/whpx/meson.build                                | 6 ++++++
 {target/i386 => accel}/whpx/whpx-accel-ops.c          | 4 ++--
 {target/i386/whpx => include/system}/whpx-accel-ops.h | 4 ++--
 {target/i386/whpx => include/system}/whpx-internal.h  | 6 ++++--
 target/i386/whpx/meson.build                          | 1 -
 target/i386/whpx/whpx-all.c                           | 4 ++--
 target/i386/whpx/whpx-apic.c                          | 2 +-
 9 files changed, 20 insertions(+), 10 deletions(-)
 create mode 100644 accel/whpx/meson.build
 rename {target/i386 => accel}/whpx/whpx-accel-ops.c (97%)
 rename {target/i386/whpx => include/system}/whpx-accel-ops.h (92%)
 rename {target/i386/whpx => include/system}/whpx-internal.h (97%)
MAINTAINERS +2 -0
index 36a2be3ddb..7365094cd7 100644
@@ -565,9 +565,11 @@ WHPX CPUs
 M: Pedro Barbuda <[email protected]>
 M: Mohamed Mediouni <[email protected]>
 S: Supported
+F: accel/whpx/
 F: target/i386/whpx/
 F: accel/stubs/whpx-stub.c
 F: include/system/whpx.h
+F: include/system/whpx-accel-ops.h
 
 MSHV
 M: Magnus Kulke <[email protected]>
accel/meson.build +1 -0
index 983dfd0bd5..289b7420ff 100644
@@ -6,6 +6,7 @@ user_ss.add(files('accel-user.c'))
 subdir('tcg')
 if have_system
   subdir('hvf')
+  subdir('whpx')
   subdir('qtest')
   subdir('kvm')
   subdir('xen')
accel/whpx/meson.build new file +6 -0
new file mode 100644
index 0000000000..7b3d6f1c1c
@@ -0,0 +1,6 @@
+whpx_ss = ss.source_set()
+whpx_ss.add(files(
+  'whpx-accel-ops.c',
+))
+
+specific_ss.add_all(when: 'CONFIG_WHPX', if_true: whpx_ss)
accel/whpx/whpx-accel-ops.c renamed from target/i386/whpx/whpx-accel-ops.c +2 -2
similarity index 97%
rename from target/i386/whpx/whpx-accel-ops.c
rename to accel/whpx/whpx-accel-ops.c
index f75886128d..c84a25c273 100644
@@ -16,8 +16,8 @@
 #include "qemu/guest-random.h"
 
 #include "system/whpx.h"
-#include "whpx-internal.h"
-#include "whpx-accel-ops.h"
+#include "system/whpx-internal.h"
+#include "system/whpx-accel-ops.h"
 
 static void *whpx_cpu_thread_fn(void *arg)
 {
include/system/whpx-accel-ops.h renamed from target/i386/whpx/whpx-accel-ops.h +2 -2
similarity index 92%
rename from target/i386/whpx/whpx-accel-ops.h
rename to include/system/whpx-accel-ops.h
index 54cfc25a14..ed9d4c49f4 100644
@@ -7,8 +7,8 @@
  * See the COPYING file in the top-level directory.
  */
 
-#ifndef TARGET_I386_WHPX_ACCEL_OPS_H
-#define TARGET_I386_WHPX_ACCEL_OPS_H
+#ifndef SYSTEM_WHPX_ACCEL_OPS_H
+#define SYSTEM_WHPX_ACCEL_OPS_H
 
 #include "system/cpus.h"
 
include/system/whpx-internal.h renamed from target/i386/whpx/whpx-internal.h +4 -2
similarity index 97%
rename from target/i386/whpx/whpx-internal.h
rename to include/system/whpx-internal.h
index 2dcad1f565..041fa958b4 100644
@@ -1,11 +1,13 @@
-#ifndef TARGET_I386_WHPX_INTERNAL_H
-#define TARGET_I386_WHPX_INTERNAL_H
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+#ifndef SYSTEM_WHPX_INTERNAL_H
+#define SYSTEM_WHPX_INTERNAL_H
 
 #include <windows.h>
 #include <winhvplatform.h>
 #include <winhvemulation.h>
 
 #include "hw/i386/apic.h"
+#include "exec/vaddr.h"
 
 typedef enum WhpxBreakpointState {
     WHPX_BP_CLEARED = 0,
target/i386/whpx/meson.build +0 -1
index 9c54aaad39..c3aaaff9fd 100644
@@ -1,5 +1,4 @@
 i386_system_ss.add(when: 'CONFIG_WHPX', if_true: files(
   'whpx-all.c',
   'whpx-apic.c',
-  'whpx-accel-ops.c',
 ))
target/i386/whpx/whpx-all.c +2 -2
index db184e1b0d..cef31fc1a8 100644
@@ -31,8 +31,8 @@
 #include "accel/accel-cpu-target.h"
 #include <winerror.h>
 
-#include "whpx-internal.h"
-#include "whpx-accel-ops.h"
+#include "system/whpx-internal.h"
+#include "system/whpx-accel-ops.h"
 
 #include <winhvplatform.h>
 #include <winhvemulation.h>
target/i386/whpx/whpx-apic.c +1 -1
index afcb25843b..b934fdcbe1 100644
@@ -18,7 +18,7 @@
 #include "hw/pci/msi.h"
 #include "system/hw_accel.h"
 #include "system/whpx.h"
-#include "whpx-internal.h"
+#include "system/whpx-internal.h"
 
 struct whpx_lapic_state {
     struct {
Signature · 1 line
2.50.1 (Apple Git-155)
#15 lore raw

PATCH v17 7/21 whpx: reshuffle common code

In reply to Mohamed Mediouni

Some code can be shared between x86_64 and arm64 WHPX. Do so as much as reasonable.

---
 MAINTAINERS                  |   2 +
 accel/whpx/meson.build       |   1 +
 accel/whpx/whpx-common.c     | 558 +++++++++++++++++++++++++++++++++++
 include/system/whpx-all.h    |  20 ++
 include/system/whpx-common.h |  21 ++
 target/i386/whpx/whpx-all.c  | 551 +---------------------------------
 6 files changed, 612 insertions(+), 541 deletions(-)
 create mode 100644 accel/whpx/whpx-common.c
 create mode 100644 include/system/whpx-all.h
 create mode 100644 include/system/whpx-common.h
MAINTAINERS +2 -0
index 7365094cd7..d9eff7bed3 100644
@@ -570,6 +570,8 @@ F: target/i386/whpx/
 F: accel/stubs/whpx-stub.c
 F: include/system/whpx.h
 F: include/system/whpx-accel-ops.h
+F: include/system/whpx-common.h
+F: include/system/whpx-internal.h
 
 MSHV
 M: Magnus Kulke <[email protected]>
accel/whpx/meson.build +1 -0
index 7b3d6f1c1c..fad28dddcb 100644
@@ -1,6 +1,7 @@
 whpx_ss = ss.source_set()
 whpx_ss.add(files(
   'whpx-accel-ops.c',
+  'whpx-common.c'
 ))
 
 specific_ss.add_all(when: 'CONFIG_WHPX', if_true: whpx_ss)
accel/whpx/whpx-common.c new file +558 -0

558 lines not on this page · raw message

include/system/whpx-all.h new file +20 -0
new file mode 100644
index 0000000000..f13cdf7f66
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+#ifndef SYSTEM_WHPX_ALL_H
+#define SYSTEM_WHPX_ALL_H
+
+/* Called by whpx-common */
+int whpx_vcpu_run(CPUState *cpu);
+void whpx_get_registers(CPUState *cpu);
+void whpx_set_registers(CPUState *cpu, int level);
+int whpx_accel_init(AccelState *as, MachineState *ms);
+void whpx_cpu_instance_init(CPUState *cs);
+HRESULT whpx_set_exception_exit_bitmap(UINT64 exceptions);
+void whpx_apply_breakpoints(
+struct whpx_breakpoint_collection *breakpoints,
+    CPUState *cpu,
+    bool resuming);
+void whpx_translate_cpu_breakpoints(
+    struct whpx_breakpoints *breakpoints,
+    CPUState *cpu,
+    int cpu_breakpoint_count);
+#endif
include/system/whpx-common.h new file +21 -0
new file mode 100644
index 0000000000..e549c7539c
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+#ifndef SYSTEM_WHPX_COMMON_H
+#define SYSTEM_WHPX_COMMON_H
+
+struct AccelCPUState {
+    WHV_EMULATOR_HANDLE emulator;
+    bool window_registered;
+    bool interruptable;
+    bool ready_for_pic_interrupt;
+    uint64_t tpr;
+    uint64_t apic_base;
+    bool interruption_pending;
+    /* Must be the last field as it may have a tail */
+    WHV_RUN_VP_EXIT_CONTEXT exit_ctx;
+};
+
+int whpx_first_vcpu_starting(CPUState *cpu);
+int whpx_last_vcpu_stopping(CPUState *cpu);
+void whpx_memory_init(void);
+struct whpx_breakpoint *whpx_lookup_breakpoint_by_addr(uint64_t address);
+#endif
target/i386/whpx/whpx-all.c +10 -541

639 lines not on this page · raw message

Signature · 1 line
2.50.1 (Apple Git-155)
#16 lore raw

PATCH v17 8/21 whpx: ifdef out winhvemulation on non-x86_64

In reply to Mohamed Mediouni

winhvemulation is x86_64 only.

In the future, we might want to get rid of winhvemulation usage entirely.

---
 accel/whpx/whpx-common.c       | 14 ++++++++++++--
 include/system/whpx-common.h   |  2 ++
 include/system/whpx-internal.h |  7 ++++++-
 3 files changed, 20 insertions(+), 3 deletions(-)
accel/whpx/whpx-common.c +12 -2
index 0a6068fdde..c58344cb61 100644
@@ -37,7 +37,9 @@
 bool whpx_allowed;
 static bool whp_dispatch_initialized;
 static HMODULE hWinHvPlatform;
+#ifdef HOST_X86_64
 static HMODULE hWinHvEmulation;
+#endif
 
 struct whpx_state whpx_global;
 struct WHPDispatch whp_dispatch;
@@ -232,8 +234,10 @@ void whpx_destroy_vcpu(CPUState *cpu)
     struct whpx_state *whpx = &whpx_global;
 
     whp_dispatch.WHvDeleteVirtualProcessor(whpx->partition, cpu->cpu_index);
+#ifdef HOST_X86_64
     AccelCPUState *vcpu = cpu->accel;
     whp_dispatch.WHvEmulatorDestroyEmulator(vcpu->emulator);
+#endif
     g_free(cpu->accel);
 }
 
@@ -408,8 +412,12 @@ static bool load_whp_dispatch_fns(HMODULE *handle,
         LIST_WINHVPLATFORM_FUNCTIONS(WHP_LOAD_FIELD)
         break;
     case WINHV_EMULATION_FNS_DEFAULT:
+#ifdef HOST_X86_64
         WHP_LOAD_LIB(WINHV_EMULATION_DLL, hLib)
         LIST_WINHVEMULATION_FUNCTIONS(WHP_LOAD_FIELD)
+#else
+        g_assert_not_reached();
+#endif
         break;
     case WINHV_PLATFORM_FNS_SUPPLEMENTAL:
         WHP_LOAD_LIB(WINHV_PLATFORM_DLL, hLib)
@@ -535,11 +543,11 @@ bool init_whp_dispatch(void)
     if (!load_whp_dispatch_fns(&hWinHvPlatform, WINHV_PLATFORM_FNS_DEFAULT)) {
         goto error;
     }
-
+#ifdef HOST_X86_64
     if (!load_whp_dispatch_fns(&hWinHvEmulation, WINHV_EMULATION_FNS_DEFAULT)) {
         goto error;
     }
-
+#endif
     assert(load_whp_dispatch_fns(&hWinHvPlatform,
         WINHV_PLATFORM_FNS_SUPPLEMENTAL));
     whp_dispatch_initialized = true;
@@ -549,9 +557,11 @@ error:
     if (hWinHvPlatform) {
         FreeLibrary(hWinHvPlatform);
     }
+#ifdef HOST_X86_64
     if (hWinHvEmulation) {
         FreeLibrary(hWinHvEmulation);
     }
+#endif
     return false;
 }
 
include/system/whpx-common.h +2 -0
index e549c7539c..8f171d1397 100644
@@ -3,7 +3,9 @@
 #define SYSTEM_WHPX_COMMON_H
 
 struct AccelCPUState {
+#ifdef HOST_X86_64
     WHV_EMULATOR_HANDLE emulator;
+#endif
     bool window_registered;
     bool interruptable;
     bool ready_for_pic_interrupt;
include/system/whpx-internal.h +6 -1
index 041fa958b4..609d0e1c08 100644
@@ -4,8 +4,9 @@
 
 #include <windows.h>
 #include <winhvplatform.h>
+#ifdef HOST_X86_64
 #include <winhvemulation.h>
-
+#endif
 #include "hw/i386/apic.h"
 #include "exec/vaddr.h"
 
@@ -101,12 +102,16 @@ void whpx_apic_get(APICCommonState *s);
 
 /* Define function typedef */
 LIST_WINHVPLATFORM_FUNCTIONS(WHP_DEFINE_TYPE)
+#ifdef HOST_X86_64
 LIST_WINHVEMULATION_FUNCTIONS(WHP_DEFINE_TYPE)
+#endif
 LIST_WINHVPLATFORM_FUNCTIONS_SUPPLEMENTAL(WHP_DEFINE_TYPE)
 
 struct WHPDispatch {
     LIST_WINHVPLATFORM_FUNCTIONS(WHP_DECLARE_MEMBER)
+#ifdef HOST_X86_64
     LIST_WINHVEMULATION_FUNCTIONS(WHP_DECLARE_MEMBER)
+#endif
     LIST_WINHVPLATFORM_FUNCTIONS_SUPPLEMENTAL(WHP_DECLARE_MEMBER)
 };
 
Signature · 1 line
2.50.1 (Apple Git-155)
#17 lore raw

PATCH v17 9/21 whpx: common: add WHPX_INTERCEPT_DEBUG_TRAPS define

In reply to Mohamed Mediouni

As of why: WHPX on arm64 doesn't have debug trap support as of today.

Keep the exception bitmap interface for now - despite that being entirely unavailable on arm64 too.

---
 accel/whpx/whpx-common.c     | 2 +-
 include/system/whpx-common.h | 3 +++
 2 files changed, 4 insertions(+), 1 deletion(-)
accel/whpx/whpx-common.c +1 -1
index c58344cb61..c0610815d9 100644
@@ -119,7 +119,7 @@ int whpx_first_vcpu_starting(CPUState *cpu)
          * have one or more breakpoints enabled. Both require intercepting
          * the WHvX64ExceptionTypeBreakpointTrap exception.
          */
-        exception_mask = 1UL << WHvX64ExceptionTypeDebugTrapOrFault;
+        exception_mask = 1UL << WHPX_INTERCEPT_DEBUG_TRAPS;
     } else {
         /* Let the guest handle all exceptions. */
         exception_mask = 0;
include/system/whpx-common.h +3 -0
index 8f171d1397..b86fe9db6e 100644
@@ -20,4 +20,7 @@ int whpx_first_vcpu_starting(CPUState *cpu);
 int whpx_last_vcpu_stopping(CPUState *cpu);
 void whpx_memory_init(void);
 struct whpx_breakpoint *whpx_lookup_breakpoint_by_addr(uint64_t address);
+
+/* On x64: same as WHvX64ExceptionTypeDebugTrapOrFault */
+#define WHPX_INTERCEPT_DEBUG_TRAPS 1
 #endif
Signature · 1 line
2.50.1 (Apple Git-155)
#18 lore raw

PATCH v17 10/21 hw, target, accel: whpx: change apic_in_platform to kernel_irqchip

In reply to Mohamed Mediouni

Change terminology to match the KVM one, as APIC is x86-specific.

And move out whpx_irqchip_in_kernel() to make it usable from common code even when not compiling with WHPX support.

---
 accel/stubs/whpx-stub.c        |  1 +
 accel/whpx/whpx-accel-ops.c    |  2 +-
 accel/whpx/whpx-common.c       | 10 +---------
 hw/i386/x86-cpu.c              |  4 ++--
 include/system/whpx-internal.h |  1 -
 include/system/whpx.h          |  5 +++--
 target/i386/cpu-apic.c         |  2 +-
 target/i386/whpx/whpx-all.c    | 14 +++++++-------
 8 files changed, 16 insertions(+), 23 deletions(-)
accel/stubs/whpx-stub.c +1 -0
index c564c89fd0..4529dc4f78 100644
@@ -10,3 +10,4 @@
 #include "system/whpx.h"
 
 bool whpx_allowed;
+bool whpx_irqchip_in_kernel;
accel/whpx/whpx-accel-ops.c +1 -1
index c84a25c273..50fadea0fd 100644
@@ -78,7 +78,7 @@ static void whpx_kick_vcpu_thread(CPUState *cpu)
 
 static bool whpx_vcpu_thread_is_idle(CPUState *cpu)
 {
-    return !whpx_apic_in_platform();
+    return !whpx_irqchip_in_kernel();
 }
 
 static void whpx_accel_ops_class_init(ObjectClass *oc, const void *data)
accel/whpx/whpx-common.c +1 -9
index c0610815d9..05f9e520b7 100644
@@ -35,6 +35,7 @@
 #include <winhvplatformdefs.h>
 
 bool whpx_allowed;
+bool whpx_irqchip_in_kernel;
 static bool whp_dispatch_initialized;
 static HMODULE hWinHvPlatform;
 #ifdef HOST_X86_64
@@ -488,15 +489,6 @@ static const TypeInfo whpx_cpu_accel_type = {
     .abstract = true,
 };
 
-/*
- * Partition support
- */
-
-bool whpx_apic_in_platform(void)
-{
-    return whpx_global.apic_in_platform;
-}
-
 static void whpx_accel_class_init(ObjectClass *oc, const void *data)
 {
     AccelClass *ac = ACCEL_CLASS(oc);
hw/i386/x86-cpu.c +2 -2
index 276f2b0cdf..95e08e3c2a 100644
@@ -45,7 +45,7 @@ static void pic_irq_request(void *opaque, int irq, int level)
 
     trace_x86_pic_interrupt(irq, level);
     if (cpu_is_apic_enabled(cpu->apic_state) && !kvm_irqchip_in_kernel() &&
-        !whpx_apic_in_platform()) {
+        !whpx_irqchip_in_kernel()) {
         CPU_FOREACH(cs) {
             cpu = X86_CPU(cs);
             if (apic_accept_pic_intr(cpu->apic_state)) {
@@ -71,7 +71,7 @@ int cpu_get_pic_interrupt(CPUX86State *env)
     X86CPU *cpu = env_archcpu(env);
     int intno;
 
-    if (!kvm_irqchip_in_kernel() && !whpx_apic_in_platform()) {
+    if (!kvm_irqchip_in_kernel() && !whpx_irqchip_in_kernel()) {
         intno = apic_get_interrupt(cpu->apic_state);
         if (intno >= 0) {
             return intno;
include/system/whpx-internal.h +0 -1
index 609d0e1c08..8ded54a39b 100644
@@ -45,7 +45,6 @@ struct whpx_state {
 
     bool kernel_irqchip_allowed;
     bool kernel_irqchip_required;
-    bool apic_in_platform;
 };
 
 extern struct whpx_state whpx_global;
include/system/whpx.h +3 -2
index 00f6a3e523..4217a27e91 100644
@@ -25,11 +25,12 @@
 
 #ifdef CONFIG_WHPX_IS_POSSIBLE
 extern bool whpx_allowed;
+extern bool whpx_irqchip_in_kernel;
 #define whpx_enabled() (whpx_allowed)
-bool whpx_apic_in_platform(void);
+#define whpx_irqchip_in_kernel() (whpx_irqchip_in_kernel)
 #else /* !CONFIG_WHPX_IS_POSSIBLE */
 #define whpx_enabled() 0
-#define whpx_apic_in_platform() (0)
+#define whpx_irqchip_in_kernel() (0)
 #endif /* !CONFIG_WHPX_IS_POSSIBLE */
 
 #endif /* QEMU_WHPX_H */
target/i386/cpu-apic.c +1 -1
index eeee62b52a..50a2b8ae86 100644
@@ -32,7 +32,7 @@ APICCommonClass *apic_get_class(Error **errp)
         apic_type = "kvm-apic";
     } else if (xen_enabled()) {
         apic_type = "xen-apic";
-    } else if (whpx_apic_in_platform()) {
+    } else if (whpx_irqchip_in_kernel()) {
         apic_type = "whpx-apic";
     }
 
target/i386/whpx/whpx-all.c +7 -7
index 052cda42bf..8210250dc3 100644
@@ -607,7 +607,7 @@ void whpx_get_registers(CPUState *cpu)
                      hr);
     }
 
-    if (whpx_apic_in_platform()) {
+    if (whpx_irqchip_in_kernel()) {
         /*
          * Fetch the TPR value from the emulated APIC. It may get overwritten
          * below with the value from CR8 returned by
@@ -749,7 +749,7 @@ void whpx_get_registers(CPUState *cpu)
 
     assert(idx == RTL_NUMBER_OF(whpx_register_names));
 
-    if (whpx_apic_in_platform()) {
+    if (whpx_irqchip_in_kernel()) {
         whpx_apic_get(x86_cpu->apic_state);
     }
 
@@ -1379,7 +1379,7 @@ static void whpx_vcpu_pre_run(CPUState *cpu)
     }
 
     /* Get pending hard interruption or replay one that was overwritten */
-    if (!whpx_apic_in_platform()) {
+    if (!whpx_irqchip_in_kernel()) {
         if (!vcpu->interruption_pending &&
             vcpu->interruptable && (env->eflags & IF_MASK)) {
             assert(!new_int.InterruptionPending);
@@ -1553,7 +1553,7 @@ int whpx_vcpu_run(CPUState *cpu)
 
     if (exclusive_step_mode == WHPX_STEP_NONE) {
         whpx_vcpu_process_async_events(cpu);
-        if (cpu->halted && !whpx_apic_in_platform()) {
+        if (cpu->halted && !whpx_irqchip_in_kernel()) {
             cpu->exception_index = EXCP_HLT;
             qatomic_set(&cpu->exit_request, false);
             return 0;
@@ -1642,7 +1642,7 @@ int whpx_vcpu_run(CPUState *cpu)
             break;
 
         case WHvRunVpExitReasonX64ApicEoi:
-            assert(whpx_apic_in_platform());
+            assert(whpx_irqchip_in_kernel());
             ioapic_eoi_broadcast(vcpu->exit_ctx.ApicEoi.InterruptVector);
             break;
 
@@ -2187,7 +2187,7 @@ int whpx_accel_init(AccelState *as, MachineState *ms)
                 goto error;
             }
         } else {
-            whpx->apic_in_platform = true;
+            whpx_irqchip_in_kernel = true;
         }
     }
 
@@ -2196,7 +2196,7 @@ int whpx_accel_init(AccelState *as, MachineState *ms)
     prop.ExtendedVmExits.X64MsrExit = 1;
     prop.ExtendedVmExits.X64CpuidExit = 1;
     prop.ExtendedVmExits.ExceptionExit = 1;
-    if (whpx_apic_in_platform()) {
+    if (whpx_irqchip_in_kernel()) {
         prop.ExtendedVmExits.X64ApicInitSipiExitTrap = 1;
     }
 
Signature · 1 line
2.50.1 (Apple Git-155)
#19 lore raw

PATCH v17 11/21 whpx: interrupt controller support

In reply to Mohamed Mediouni
---
 MAINTAINERS                        |   1 +
 hw/arm/virt.c                      |  10 ++
 hw/intc/arm_gicv3_common.c         |   3 +
 hw/intc/arm_gicv3_whpx.c           | 238 +++++++++++++++++++++++++++++
 hw/intc/meson.build                |   1 +
 include/hw/intc/arm_gicv3_common.h |   3 +
 6 files changed, 256 insertions(+)
 create mode 100644 hw/intc/arm_gicv3_whpx.c
MAINTAINERS +1 -0
index d9eff7bed3..90c3b585e3 100644
@@ -567,6 +567,7 @@ M: Mohamed Mediouni <[email protected]>
 S: Supported
 F: accel/whpx/
 F: target/i386/whpx/
+F: hw/intc/arm_gicv3_whpx.c
 F: accel/stubs/whpx-stub.c
 F: include/system/whpx.h
 F: include/system/whpx-accel-ops.h
hw/arm/virt.c +10 -0
index 3f62247f94..3b45a6c788 100644
@@ -49,6 +49,7 @@
 #include "system/tcg.h"
 #include "system/kvm.h"
 #include "system/hvf.h"
+#include "system/whpx.h"
 #include "system/qtest.h"
 #include "system/system.h"
 #include "hw/core/loader.h"
@@ -2110,6 +2111,8 @@ static void finalize_gic_version(VirtMachineState *vms)
         /* KVM w/o kernel irqchip can only deal with GICv2 */
         gics_supported |= VIRT_GIC_VERSION_2_MASK;
         accel_name = "KVM with kernel-irqchip=off";
+    } else if (whpx_enabled()) {
+        gics_supported |= VIRT_GIC_VERSION_3_MASK;
     } else if (tcg_enabled() || hvf_enabled() || qtest_enabled())  {
         gics_supported |= VIRT_GIC_VERSION_2_MASK;
         if (module_object_class_by_name("arm-gicv3")) {
@@ -2143,6 +2146,9 @@ static void finalize_msi_controller(VirtMachineState *vms)
         }
     }
     if (vms->msi_controller == VIRT_MSI_CTRL_AUTO) {
+        if (whpx_enabled() && whpx_irqchip_in_kernel()) {
+            vms->msi_controller = VIRT_MSI_CTRL_GICV2M;
+        }
         if (vms->gic_version == VIRT_GIC_VERSION_2) {
             vms->msi_controller = VIRT_MSI_CTRL_GICV2M;
         }
@@ -2157,6 +2163,10 @@ static void finalize_msi_controller(VirtMachineState *vms)
             error_report("GICv2 + ITS is an invalid configuration.");
             exit(1);
         }
+        if (whpx_enabled()) {
+            error_report("ITS not supported on WHPX.");
+            exit(1);
+        }
     }
 
     assert(vms->msi_controller != VIRT_MSI_CTRL_AUTO);
hw/intc/arm_gicv3_common.c +3 -0
index 0a2e5a3e2f..9054143ea7 100644
@@ -32,6 +32,7 @@
 #include "gicv3_internal.h"
 #include "hw/arm/linux-boot-if.h"
 #include "system/kvm.h"
+#include "system/whpx.h"
 
 
 static void gicv3_gicd_no_migration_shift_bug_post_load(GICv3State *cs)
@@ -663,6 +664,8 @@ const char *gicv3_class_name(void)
 {
     if (kvm_irqchip_in_kernel()) {
         return "kvm-arm-gicv3";
+    } else if (whpx_enabled()) {
+        return TYPE_WHPX_GICV3;
     } else {
         if (kvm_enabled()) {
             error_report("Userspace GICv3 is not supported with KVM");
hw/intc/arm_gicv3_whpx.c new file +238 -0

238 lines not on this page · raw message

hw/intc/meson.build +1 -0
index faae20b93d..96742df090 100644
@@ -41,6 +41,7 @@ specific_ss.add(when: 'CONFIG_APIC', if_true: files('apic.c', 'apic_common.c'))
 arm_common_ss.add(when: 'CONFIG_ARM_GIC', if_true: files('arm_gicv3_cpuif_common.c'))
 arm_common_ss.add(when: 'CONFIG_ARM_GICV3', if_true: files('arm_gicv3_cpuif.c'))
 specific_ss.add(when: 'CONFIG_ARM_GIC_KVM', if_true: files('arm_gic_kvm.c'))
+specific_ss.add(when: ['CONFIG_WHPX', 'TARGET_AARCH64'], if_true: files('arm_gicv3_whpx.c'))
 specific_ss.add(when: ['CONFIG_ARM_GIC_KVM', 'TARGET_AARCH64'], if_true: files('arm_gicv3_kvm.c', 'arm_gicv3_its_kvm.c'))
 arm_common_ss.add(when: 'CONFIG_ARM_V7M', if_true: files('armv7m_nvic.c'))
 specific_ss.add(when: 'CONFIG_GRLIB', if_true: files('grlib_irqmp.c'))
include/hw/intc/arm_gicv3_common.h +3 -0
index 3d24ad22d2..c55cf18120 100644
@@ -313,6 +313,9 @@ typedef struct ARMGICv3CommonClass ARMGICv3CommonClass;
 DECLARE_OBJ_CHECKERS(GICv3State, ARMGICv3CommonClass,
                      ARM_GICV3_COMMON, TYPE_ARM_GICV3_COMMON)
 
+/* Types for GICv3 kernel-irqchip */
+#define TYPE_WHPX_GICV3 "whpx-arm-gicv3"
+
 struct ARMGICv3CommonClass {
     /*< private >*/
     SysBusDeviceClass parent_class;
Signature · 1 line
2.50.1 (Apple Git-155)
#21 lore raw

Re: [PATCH v17 11/21] whpx: interrupt controller support

In reply to Mohamed Mediouni
Quoted 32 lines · Mohamed Mediouni
---
 MAINTAINERS                        |   1 +
 hw/arm/virt.c                      |  10 ++
 hw/intc/arm_gicv3_common.c         |   3 +
 hw/intc/arm_gicv3_whpx.c           | 238 +++++++++++++++++++++++++++++
 hw/intc/meson.build                |   1 +
 include/hw/intc/arm_gicv3_common.h |   3 +
 6 files changed, 256 insertions(+)
 create mode 100644 hw/intc/arm_gicv3_whpx.c
hw/intc/arm_gicv3_whpx.c new file +20 -0
+static void whpx_gicv3_icc_reset(CPUARMState *env, const ARMCPRegInfo *ri)
+{
+    GICv3State *s;
+    GICv3CPUState *c;
+
+    c = env->gicv3state;
+    s = c->gic;
+
+    c->icc_pmr_el1 = 0;
+    /*
+     * Architecturally the reset value of the ICC_BPR registers
+     * is UNKNOWN. We set them all to 0 here; when the kernel
+     * uses these values to program the ICH_VMCR_EL2 fields that
+     * determine the guest-visible ICC_BPR register values, the
+     * hardware's "writing a value less than the minimum sets
+     * the field to the minimum value" behaviour will result in
+     * them effectively resetting to the correct minimum value
+     * for the host GIC.
+     */
+    c->icc_bpr[GICV3_G0] = 0;

hw/intc/arm_gicv3_whpx.c line 98

Quoted 10 lines
hw/intc/arm_gicv3_whpx.c new file +10 -0
+    c->icc_bpr[GICV3_G1] = 0;
+    c->icc_bpr[GICV3_G1NS] = 0;
+
+    c->icc_sre_el1 = 0x7;
+    memset(c->icc_apr, 0, sizeof(c->icc_apr));
+    memset(c->icc_igrpen, 0, sizeof(c->icc_igrpen));
+
+    if (s->migration_blocker) {
+        return;
+    }

Looking a bit more closely at this, this is odd. Why the early return? This looks like it was copied from the KVM GICv3 from prior to commit 59ad421d6. But in the KVM code before that commit, the reason for the early return is to avoid the "read the ICC_CTLR value from KVM" call that won't work (the migration blocker is for the "no KVM API to read/write GIC registers" case). And in the KVM code after that commit, we restructured the reset to not read the register in the reset function, so we don't need to test s->migration_blocker at all then.

Probably you want to follow the pattern the newer KVM code uses, but either way, this early return doesn't make much sense for this whpx code as it stands, especially as the whpx code isn't currently setting s->migration_blocker...

hw/intc/arm_gicv3_whpx.c line 101

Quoted 3 lines
hw/intc/arm_gicv3_whpx.c new file +3 -0
+
+    c->icc_ctlr_el1[GICV3_S] = c->icc_ctlr_el1[GICV3_NS];
+}

thanks
-- PMM

#22 lore raw

Re: [PATCH v17 11/21] whpx: interrupt controller support

Quoted 43 lines · Peter Maydell
Quoted 38 lines · Mohamed Mediouni

---

MAINTAINERS                        |   1 +
hw/arm/virt.c                      |  10 ++
hw/intc/arm_gicv3_common.c         |   3 +
hw/intc/arm_gicv3_whpx.c           | 238 +++++++++++++++++++++++++++++
hw/intc/meson.build                |   1 +
include/hw/intc/arm_gicv3_common.h |   3 +
6 files changed, 256 insertions(+)

create mode 100644 hw/intc/arm_gicv3_whpx.c

hw/intc/arm_gicv3_whpx.c new file +26 -0
+static void whpx_gicv3_icc_reset(CPUARMState *env, const ARMCPRegInfo *ri)
+{
+    GICv3State *s;
+    GICv3CPUState *c;
+
+    c = env->gicv3state;
+    s = c->gic;
+
+    c->icc_pmr_el1 = 0;
+    /*
+     * Architecturally the reset value of the ICC_BPR registers
+     * is UNKNOWN. We set them all to 0 here; when the kernel
+     * uses these values to program the ICH_VMCR_EL2 fields that
+     * determine the guest-visible ICC_BPR register values, the
+     * hardware's "writing a value less than the minimum sets
+     * the field to the minimum value" behaviour will result in
+     * them effectively resetting to the correct minimum value
+     * for the host GIC.
+     */
+    c->icc_bpr[GICV3_G0] = 0;
+    c->icc_bpr[GICV3_G1] = 0;
+    c->icc_bpr[GICV3_G1NS] = 0;
+
+    c->icc_sre_el1 = 0x7;
+    memset(c->icc_apr, 0, sizeof(c->icc_apr));
+    memset(c->icc_igrpen, 0, sizeof(c->icc_igrpen));
Quoted 8 lines

hw/intc/arm_gicv3_whpx.c line 98

Quoted 4 lines
hw/intc/arm_gicv3_whpx.c new file +4 -0
+
+    if (s->migration_blocker) {
+        return;
+    }

Looking a bit more closely at this, this is odd. Why the early return? This looks like it was copied from the KVM GICv3 from prior to commit 59ad421d6. But in the KVM code before

Hello,

Indeed it was, will remove this from the HVF nested virt series too…
And update to use s->migration_blocker for "whpx: arm64: gicv3: add migration blocker” (which I’ll squash into “whpx: interrupt controller support”)

Thank you,

Quoted 20 lines

that commit, the reason for the early return is to avoid the "read the ICC_CTLR value from KVM" call that won't work (the migration blocker is for the "no KVM API to read/write GIC registers" case). And in the KVM code after that commit, we restructured the reset to not read the register in the reset function, so we don't need to test s->migration_blocker at all then.

Probably you want to follow the pattern the newer KVM code uses, but either way, this early return doesn't make much sense for this whpx code as it stands, especially as the whpx code isn't currently setting s->migration_blocker...

hw/intc/arm_gicv3_whpx.c line 101

Quoted 3 lines
hw/intc/arm_gicv3_whpx.c new file +3 -0
+
+    c->icc_ctlr_el1[GICV3_S] = c->icc_ctlr_el1[GICV3_NS];
+}

thanks
-- PMM

#23 lore raw

PATCH v17 12/21 whpx: add arm64 support

In reply to Mohamed Mediouni
---
 MAINTAINERS                 |   1 +
 accel/whpx/whpx-common.c    |   1 +
 target/arm/meson.build      |   1 +
 target/arm/whpx/meson.build |   3 +
 target/arm/whpx/whpx-all.c  | 810 ++++++++++++++++++++++++++++++++++++
 5 files changed, 816 insertions(+)
 create mode 100644 target/arm/whpx/meson.build
 create mode 100644 target/arm/whpx/whpx-all.c
MAINTAINERS +1 -0
index 90c3b585e3..c9c8ecc0d9 100644
@@ -567,6 +567,7 @@ M: Mohamed Mediouni <[email protected]>
 S: Supported
 F: accel/whpx/
 F: target/i386/whpx/
+F: target/arm/whpx/
 F: hw/intc/arm_gicv3_whpx.c
 F: accel/stubs/whpx-stub.c
 F: include/system/whpx.h
accel/whpx/whpx-common.c +1 -0
index 05f9e520b7..827f50f3e0 100644
@@ -12,6 +12,7 @@
 #include "gdbstub/helpers.h"
 #include "qemu/accel.h"
 #include "accel/accel-ops.h"
+#include "system/memory.h"
 #include "system/whpx.h"
 #include "system/cpus.h"
 #include "system/runstate.h"
target/arm/meson.build +1 -0
index 462c71148d..ce155ba9b4 100644
@@ -59,6 +59,7 @@ arm_common_system_ss.add(files(
 ))
 
 subdir('hvf')
+subdir('whpx')
 
 if 'CONFIG_TCG' in config_all_accel
    subdir('tcg')
target/arm/whpx/meson.build new file +3 -0
new file mode 100644
index 0000000000..1de2ef0283
@@ -0,0 +1,3 @@
+arm_system_ss.add(when: 'CONFIG_WHPX', if_true: files(
+  'whpx-all.c',
+))
target/arm/whpx/whpx-all.c new file +810 -0

810 lines not on this page · raw message

Signature · 1 line
2.50.1 (Apple Git-155)
#24 lore raw

PATCH v17 13/21 whpx: change memory management logic

In reply to Mohamed Mediouni

This allows edk2 to work on Arm, although u-boot is still not functional.

---
 accel/whpx/whpx-common.c | 97 +++++++++++++++-------------------------
 1 file changed, 36 insertions(+), 61 deletions(-)
accel/whpx/whpx-common.c +36 -61
index 827f50f3e0..f018a8f5c7 100644
@@ -255,89 +255,64 @@ void whpx_vcpu_kick(CPUState *cpu)
  * Memory support.
  */
 
-static void whpx_update_mapping(hwaddr start_pa, ram_addr_t size,
-                                void *host_va, int add, int rom,
-                                const char *name)
+static void whpx_set_phys_mem(MemoryRegionSection *section, bool add)
 {
     struct whpx_state *whpx = &whpx_global;
+    MemoryRegion *area = section->mr;
+    bool writable = !area->readonly && !area->rom_device;
+    WHV_MAP_GPA_RANGE_FLAGS flags;
+    uint64_t page_size = qemu_real_host_page_size();
+    uint64_t gva = section->offset_within_address_space;
+    uint64_t size = int128_get64(section->size);
     HRESULT hr;
+    void *mem;
 
-    /*
-    if (add) {
-        printf("WHPX: ADD PA:%p Size:%p, Host:%p, %s, '%s'\n",
-               (void*)start_pa, (void*)size, host_va,
-               (rom ? "ROM" : "RAM"), name);
-    } else {
-        printf("WHPX: DEL PA:%p Size:%p, Host:%p,      '%s'\n",
-               (void*)start_pa, (void*)size, host_va, name);
-    }
-    */
-
-    if (add) {
-        hr = whp_dispatch.WHvMapGpaRange(whpx->partition,
-                                         host_va,
-                                         start_pa,
-                                         size,
-                                         (WHvMapGpaRangeFlagRead |
-                                          WHvMapGpaRangeFlagExecute |
-                                          (rom ? 0 : WHvMapGpaRangeFlagWrite)));
-    } else {
-        hr = whp_dispatch.WHvUnmapGpaRange(whpx->partition,
-                                           start_pa,
-                                           size);
-    }
-
-    if (FAILED(hr)) {
-        error_report("WHPX: Failed to %s GPA range '%s' PA:%p, Size:%p bytes,"
-                     " Host:%p, hr=%08lx",
-                     (add ? "MAP" : "UNMAP"), name,
-                     (void *)(uintptr_t)start_pa, (void *)size, host_va, hr);
+    if (!memory_region_is_ram(area)) {
+        if (writable) {
+            return;
+        } else if (!memory_region_is_romd(area)) {
+            add = false;
+        }
     }
-}
-
-static void whpx_process_section(MemoryRegionSection *section, int add)
-{
-    MemoryRegion *mr = section->mr;
-    hwaddr start_pa = section->offset_within_address_space;
-    ram_addr_t size = int128_get64(section->size);
-    unsigned int delta;
-    uint64_t host_va;
 
-    if (!memory_region_is_ram(mr)) {
-        return;
+    if (!QEMU_IS_ALIGNED(size, page_size) ||
+        !QEMU_IS_ALIGNED(gva, page_size)) {
+        /* Not page aligned, so we can not map as RAM */
+        add = false;
     }
 
-    delta = qemu_real_host_page_size() - (start_pa & ~qemu_real_host_page_mask());
-    delta &= ~qemu_real_host_page_mask();
-    if (delta > size) {
-        return;
-    }
-    start_pa += delta;
-    size -= delta;
-    size &= qemu_real_host_page_mask();
-    if (!size || (start_pa & ~qemu_real_host_page_mask())) {
+    if (!add) {
+        hr = whp_dispatch.WHvUnmapGpaRange(whpx->partition,
+                gva, size);
+        if (FAILED(hr)) {
+            error_report("WHPX: failed to unmap GPA range");
+            abort();
+        }
         return;
     }
 
-    host_va = (uintptr_t)memory_region_get_ram_ptr(mr)
-            + section->offset_within_region + delta;
+    flags = WHvMapGpaRangeFlagRead | WHvMapGpaRangeFlagExecute
+     | (writable ? WHvMapGpaRangeFlagWrite : 0);
+    mem = memory_region_get_ram_ptr(area) + section->offset_within_region;
 
-    whpx_update_mapping(start_pa, size, (void *)(uintptr_t)host_va, add,
-                        memory_region_is_rom(mr), mr->name);
+    hr = whp_dispatch.WHvMapGpaRange(whpx->partition,
+         mem, gva, size, flags);
+    if (FAILED(hr)) {
+        error_report("WHPX: failed to map GPA range");
+        abort();
+    }
 }
 
 static void whpx_region_add(MemoryListener *listener,
                            MemoryRegionSection *section)
 {
-    memory_region_ref(section->mr);
-    whpx_process_section(section, 1);
+    whpx_set_phys_mem(section, true);
 }
 
 static void whpx_region_del(MemoryListener *listener,
                            MemoryRegionSection *section)
 {
-    whpx_process_section(section, 0);
-    memory_region_unref(section->mr);
+    whpx_set_phys_mem(section, false);
 }
 
 static void whpx_transaction_begin(MemoryListener *listener)
Signature · 1 line
2.50.1 (Apple Git-155)
#25 lore raw

PATCH v17 14/21 target/arm: cpu: mark WHPX as supporting PSCI 1.3

In reply to Mohamed Mediouni

Hyper-V supports PSCI 1.3, and that implementation is exposed through WHPX.

---
 target/arm/cpu.c | 3 +++
 1 file changed, 3 insertions(+)
target/arm/cpu.c +3 -0
index 05fa3339b1..fd1e3197ad 100644
@@ -23,6 +23,7 @@
 #include "qemu/timer.h"
 #include "qemu/log.h"
 #include "exec/page-vary.h"
+#include "system/whpx.h"
 #include "target/arm/idau.h"
 #include "qemu/module.h"
 #include "qapi/error.h"
@@ -1143,6 +1144,8 @@ static void arm_cpu_initfn(Object *obj)
     if (tcg_enabled() || hvf_enabled()) {
         /* TCG and HVF implement PSCI 1.1 */
         cpu->psci_version = QEMU_PSCI_VERSION_1_1;
+    } else if (whpx_enabled()) {
+        cpu->psci_version = QEMU_PSCI_VERSION_1_3;
     }
 }
 
Signature · 1 line
2.50.1 (Apple Git-155)
#26 lore raw

PATCH v17 15/21 whpx: arm64: clamp down IPA size

In reply to Mohamed Mediouni

Code taken from HVF and adapted for WHPX use. Note that WHPX doesn't have a default vs maximum IPA distinction.

---
 hw/arm/virt.c               | 32 ++++++++++++++++++++++++++
 include/hw/core/boards.h    |  1 +
 target/arm/whpx/meson.build |  2 ++
 target/arm/whpx/whpx-all.c  | 45 +++++++++++++++++++++++++++++++++++++
 target/arm/whpx/whpx-stub.c | 15 +++++++++++++
 target/arm/whpx_arm.h       | 16 +++++++++++++
 6 files changed, 111 insertions(+)
 create mode 100644 target/arm/whpx/whpx-stub.c
 create mode 100644 target/arm/whpx_arm.h
hw/arm/virt.c +32 -0
index 3b45a6c788..d132a9e995 100644
@@ -72,6 +72,7 @@
 #include "hw/core/irq.h"
 #include "kvm_arm.h"
 #include "hvf_arm.h"
+#include "whpx_arm.h"
 #include "hw/firmware/smbios.h"
 #include "qapi/visitor.h"
 #include "qapi/qapi-visit-common.h"
@@ -3354,6 +3355,36 @@ static int virt_kvm_type(MachineState *ms, const char *type_str)
     return fixed_ipa ? 0 : requested_pa_size;
 }
 
+static int virt_whpx_get_physical_address_range(MachineState *ms)
+{
+    VirtMachineState *vms = VIRT_MACHINE(ms);
+
+    int max_ipa_size = whpx_arm_get_ipa_bit_size();
+
+    /* We freeze the memory map to compute the highest gpa */
+    virt_set_memmap(vms, max_ipa_size);
+
+    int requested_ipa_size = 64 - clz64(vms->highest_gpa);
+
+    /*
+     * If we're <= the default IPA size just use the default.
+     * If we're above the default but below the maximum, round up to
+     * the maximum. whpx_arm_get_max_ipa_bit_size() conveniently only
+     * returns values that are valid ARM PARange values.
+     */
+    if (requested_ipa_size <= max_ipa_size) {
+        requested_ipa_size = max_ipa_size;
+    } else {
+        error_report("-m and ,maxmem option values "
+                     "require an IPA range (%d bits) larger than "
+                     "the one supported by the host (%d bits)",
+                     requested_ipa_size, max_ipa_size);
+        return -1;
+    }
+
+    return requested_ipa_size;
+}
+
 static int virt_hvf_get_physical_address_range(MachineState *ms)
 {
     VirtMachineState *vms = VIRT_MACHINE(ms);
@@ -3453,6 +3484,7 @@ static void virt_machine_class_init(ObjectClass *oc, const void *data)
     mc->get_default_cpu_node_id = virt_get_default_cpu_node_id;
     mc->kvm_type = virt_kvm_type;
     mc->hvf_get_physical_address_range = virt_hvf_get_physical_address_range;
+    mc->whpx_get_physical_address_range = virt_whpx_get_physical_address_range;
     assert(!mc->get_hotplug_handler);
     mc->get_hotplug_handler = virt_machine_get_hotplug_handler;
     hc->pre_plug = virt_machine_device_pre_plug_cb;
include/hw/core/boards.h +1 -0
index 07f8938752..0b2aefb126 100644
@@ -278,6 +278,7 @@ struct MachineClass {
     void (*wakeup)(MachineState *state);
     int (*kvm_type)(MachineState *machine, const char *arg);
     int (*hvf_get_physical_address_range)(MachineState *machine);
+    int (*whpx_get_physical_address_range)(MachineState *machine);
 
     BlockInterfaceType block_default_type;
     int units_per_default_bus;
target/arm/whpx/meson.build +2 -0
index 1de2ef0283..3df632c9d3 100644
@@ -1,3 +1,5 @@
 arm_system_ss.add(when: 'CONFIG_WHPX', if_true: files(
   'whpx-all.c',
 ))
+
+arm_common_system_ss.add(when: 'CONFIG_WHPX', if_false: files('whpx-stub.c'))
target/arm/whpx/whpx-all.c +45 -0

75 lines not on this page · raw message

target/arm/whpx/whpx-stub.c new file +15 -0
new file mode 100644
index 0000000000..32e434a5f6
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * WHPX stubs for ARM
+ *
+ *  Copyright (c) 2025 Mohamed Mediouni
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "whpx_arm.h"
+
+uint32_t whpx_arm_get_ipa_bit_size(void)
+{
+    g_assert_not_reached();
+}
target/arm/whpx_arm.h new file +16 -0
new file mode 100644
index 0000000000..de7406b66f
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * WHPX support -- ARM specifics
+ *
+ * Copyright (c) 2025 Mohamed Mediouni
+ *
+ */
+
+#ifndef QEMU_WHPX_ARM_H
+#define QEMU_WHPX_ARM_H
+
+#include "target/arm/cpu-qom.h"
+
+uint32_t whpx_arm_get_ipa_bit_size(void);
+
+#endif
Signature · 1 line
2.50.1 (Apple Git-155)
#27 lore raw

PATCH v17 16/21 hw/arm, accel/hvf, whpx: unify get_physical_address_range between WHPX and HVF

In reply to Mohamed Mediouni
---
 accel/hvf/hvf-all.c        |  7 +++++--
 hw/arm/virt.c              | 43 +++++---------------------------------
 include/hw/core/boards.h   |  4 ++--
 include/system/hvf_int.h   |  4 ++++
 target/arm/hvf-stub.c      | 20 ------------------
 target/arm/hvf/hvf.c       |  6 +++---
 target/arm/hvf_arm.h       |  3 ---
 target/arm/meson.build     |  1 -
 target/arm/whpx/whpx-all.c |  5 +++--
 target/i386/hvf/hvf.c      | 11 ++++++++++
 10 files changed, 33 insertions(+), 71 deletions(-)
 delete mode 100644 target/arm/hvf-stub.c
accel/hvf/hvf-all.c +5 -2
index 0fbe27dfa2..033c677b6f 100644
@@ -18,6 +18,7 @@
 #include "system/hvf_int.h"
 #include "hw/core/cpu.h"
 #include "hw/core/boards.h"
+#include "target/arm/hvf_arm.h"
 #include "trace.h"
 
 bool hvf_allowed;
@@ -186,8 +187,10 @@ static int hvf_accel_init(AccelState *as, MachineState *ms)
     int pa_range = 36;
     MachineClass *mc = MACHINE_GET_CLASS(ms);
 
-    if (mc->hvf_get_physical_address_range) {
-        pa_range = mc->hvf_get_physical_address_range(ms);
+
+    if (mc->get_physical_address_range) {
+        pa_range = mc->get_physical_address_range(ms,
+            hvf_arch_get_default_ipa_bit_size(), hvf_arch_get_max_ipa_bit_size());
         if (pa_range < 0) {
             return -EINVAL;
         }
hw/arm/virt.c +5 -38

64 lines not on this page · raw message

include/hw/core/boards.h +2 -2
index 0b2aefb126..26e0879e1a 100644
@@ -277,8 +277,8 @@ struct MachineClass {
     void (*reset)(MachineState *state, ResetType type);
     void (*wakeup)(MachineState *state);
     int (*kvm_type)(MachineState *machine, const char *arg);
-    int (*hvf_get_physical_address_range)(MachineState *machine);
-    int (*whpx_get_physical_address_range)(MachineState *machine);
+    int (*get_physical_address_range)(MachineState *machine,
+        int default_ipa_size, int max_ipa_size);
 
     BlockInterfaceType block_default_type;
     int units_per_default_bus;
include/system/hvf_int.h +4 -0
index 96790b4938..2621164cb2 100644
@@ -57,6 +57,8 @@ void assert_hvf_ok_impl(hv_return_t ret, const char *file, unsigned int line,
 const char *hvf_return_string(hv_return_t ret);
 int hvf_arch_init(void);
 hv_return_t hvf_arch_vm_create(MachineState *ms, uint32_t pa_range);
+uint32_t hvf_arch_get_default_ipa_bit_size(void);
+uint32_t hvf_arch_get_max_ipa_bit_size(void);
 void hvf_kick_vcpu_thread(CPUState *cpu);
 
 /* Must be called by the owning thread */
@@ -107,5 +109,7 @@ int hvf_update_guest_debug(CPUState *cpu);
 bool hvf_arch_supports_guest_debug(void);
 
 bool hvf_arch_cpu_realize(CPUState *cpu, Error **errp);
+uint32_t hvf_arch_get_default_ipa_bit_size(void);
+uint32_t hvf_arch_get_max_ipa_bit_size(void);
 
 #endif
target/arm/hvf-stub.c deleted +0 -20
deleted file mode 100644
index ff137267a0..0000000000
@@ -1,20 +0,0 @@
-/*
- * QEMU Hypervisor.framework (HVF) stubs for ARM
- *
- *  Copyright (c) Linaro
- *
- * SPDX-License-Identifier: GPL-2.0-or-later
- */
-
-#include "qemu/osdep.h"
-#include "hvf_arm.h"
-
-uint32_t hvf_arm_get_default_ipa_bit_size(void)
-{
-    g_assert_not_reached();
-}
-
-uint32_t hvf_arm_get_max_ipa_bit_size(void)
-{
-    g_assert_not_reached();
-}
target/arm/hvf/hvf.c +3 -3
index e4c0d936f1..74b6f5e7db 100644
@@ -729,7 +729,7 @@ static uint64_t hvf_get_reg(CPUState *cpu, int rt)
 static void clamp_id_aa64mmfr0_parange_to_ipa_size(ARMISARegisters *isar)
 {
     uint32_t ipa_size = chosen_ipa_bit_size ?
-            chosen_ipa_bit_size : hvf_arm_get_max_ipa_bit_size();
+            chosen_ipa_bit_size : hvf_arch_get_max_ipa_bit_size();
     uint64_t id_aa64mmfr0;
 
     /* Clamp down the PARange to the IPA size the kernel supports. */
@@ -825,7 +825,7 @@ static bool hvf_arm_get_host_cpu_features(ARMHostCPUFeatures *ahcf)
     return r == HV_SUCCESS;
 }
 
-uint32_t hvf_arm_get_default_ipa_bit_size(void)
+uint32_t hvf_arch_get_default_ipa_bit_size(void)
 {
     uint32_t default_ipa_size;
     hv_return_t ret = hv_vm_config_get_default_ipa_size(&default_ipa_size);
@@ -834,7 +834,7 @@ uint32_t hvf_arm_get_default_ipa_bit_size(void)
     return default_ipa_size;
 }
 
-uint32_t hvf_arm_get_max_ipa_bit_size(void)
+uint32_t hvf_arch_get_max_ipa_bit_size(void)
 {
     uint32_t max_ipa_size;
     hv_return_t ret = hv_vm_config_get_max_ipa_size(&max_ipa_size);
target/arm/hvf_arm.h +0 -3
index ea82f2691d..5d19d82e5d 100644
@@ -22,7 +22,4 @@ void hvf_arm_init_debug(void);
 
 void hvf_arm_set_cpu_features_from_host(ARMCPU *cpu);
 
-uint32_t hvf_arm_get_default_ipa_bit_size(void);
-uint32_t hvf_arm_get_max_ipa_bit_size(void);
-
 #endif
target/arm/meson.build +0 -1
index ce155ba9b4..1ddc4b2a54 100644
@@ -41,7 +41,6 @@ arm_common_system_ss.add(files('cpu.c'))
 arm_common_system_ss.add(when: 'TARGET_AARCH64', if_false: files(
   'cpu32-stubs.c'))
 arm_common_system_ss.add(when: 'CONFIG_KVM', if_false: files('kvm-stub.c'))
-arm_common_system_ss.add(when: 'CONFIG_HVF', if_false: files('hvf-stub.c'))
 arm_common_system_ss.add(when: 'CONFIG_ARM_COMPATIBLE_SEMIHOSTING',
 		                 if_true: files('common-semi-target.c'))
 arm_common_system_ss.add(files(
target/arm/whpx/whpx-all.c +3 -2
index 850f6ec81f..6067918b27 100644
@@ -770,8 +770,9 @@ int whpx_accel_init(AccelState *as, MachineState *ms)
         goto error;
     }
 
-    if (mc->whpx_get_physical_address_range) {
-        pa_range = mc->whpx_get_physical_address_range(ms);
+    if (mc->get_physical_address_range) {
+        pa_range = mc->get_physical_address_range(ms,
+            whpx_arm_get_ipa_bit_size(), whpx_arm_get_ipa_bit_size());
         if (pa_range < 0) {
             return -EINVAL;
         }
target/i386/hvf/hvf.c +11 -0
index 7cfaee389e..ce54020f00 100644
@@ -228,6 +228,17 @@ int hvf_arch_init(void)
     return 0;
 }
 
+/* 48-bit on all Intel Macs. Function currently unused. */
+uint32_t hvf_arch_get_default_ipa_bit_size(void)
+{
+    g_assert_not_reached();
+}
+
+uint32_t hvf_arch_get_max_ipa_bit_size(void)
+{
+    g_assert_not_reached();
+}
+
 hv_return_t hvf_arch_vm_create(MachineState *ms, uint32_t pa_range)
 {
     return hv_vm_create(HV_VM_DEFAULT);
Signature · 1 line
2.50.1 (Apple Git-155)
#28 lore raw

PATCH v17 17/21 whpx: arm64: implement -cpu host

In reply to Mohamed Mediouni

Logic to fetch MIDR_EL1 for cpu 0 adapted from:
https://github.com/FEX-Emu/FEX/blob/e6de17e72ef03aa88ba14fa0ec13163061608c74/Source/Windows/Common/CPUFeatures.cpp#L62

---
 target/arm/cpu64.c         |  17 +++---
 target/arm/whpx/whpx-all.c | 104 +++++++++++++++++++++++++++++++++++++
 target/arm/whpx_arm.h      |   1 +
 3 files changed, 116 insertions(+), 6 deletions(-)
target/arm/cpu64.c +11 -6
index bf30381370..689babe822 100644
@@ -26,10 +26,13 @@
 #include "qemu/units.h"
 #include "system/kvm.h"
 #include "system/hvf.h"
+#include "system/whpx.h"
+#include "system/hw_accel.h"
 #include "system/qtest.h"
 #include "system/tcg.h"
 #include "kvm_arm.h"
 #include "hvf_arm.h"
+#include "whpx_arm.h"
 #include "qapi/visitor.h"
 #include "hw/core/qdev-properties.h"
 #include "internals.h"
@@ -521,7 +524,7 @@ void arm_cpu_pauth_finalize(ARMCPU *cpu, Error **errp)
     isar2 = FIELD_DP64(isar2, ID_AA64ISAR2, APA3, 0);
     isar2 = FIELD_DP64(isar2, ID_AA64ISAR2, GPA3, 0);
 
-    if (kvm_enabled() || hvf_enabled()) {
+    if (hwaccel_enabled()) {
         /*
          * Exit early if PAuth is enabled and fall through to disable it.
          * The algorithm selection properties are not present.
@@ -598,10 +601,10 @@ void aarch64_add_pauth_properties(Object *obj)
 
     /* Default to PAUTH on, with the architected algorithm on TCG. */
     qdev_property_add_static(DEVICE(obj), &arm_cpu_pauth_property);
-    if (kvm_enabled() || hvf_enabled()) {
+    if (hwaccel_enabled()) {
         /*
          * Mirror PAuth support from the probed sysregs back into the
-         * property for KVM or hvf. Is it just a bit backward? Yes it is!
+         * property for HW accel. Is it just a bit backward? Yes it is!
          * Note that prop_pauth is true whether the host CPU supports the
          * architected QARMA5 algorithm or the IMPDEF one. We don't
          * provide the separate pauth-impdef property for KVM or hvf,
@@ -769,6 +772,8 @@ static void aarch64_host_initfn(Object *obj)
     }
 #elif defined(CONFIG_HVF)
     hvf_arm_set_cpu_features_from_host(cpu);
+#elif defined(CONFIG_WHPX)
+    whpx_arm_set_cpu_features_from_host(cpu);
 #else
     g_assert_not_reached();
 #endif
@@ -779,8 +784,8 @@ static void aarch64_host_initfn(Object *obj)
 
 static void aarch64_max_initfn(Object *obj)
 {
-    if (kvm_enabled() || hvf_enabled()) {
-        /* With KVM or HVF, '-cpu max' is identical to '-cpu host' */
+    if (hwaccel_enabled()) {
+        /* When hardware acceleration enabled, '-cpu max' is identical to '-cpu host' */
         aarch64_host_initfn(obj);
         return;
     }
@@ -799,7 +804,7 @@ static const ARMCPUInfo aarch64_cpus[] = {
     { .name = "cortex-a57",         .initfn = aarch64_a57_initfn },
     { .name = "cortex-a53",         .initfn = aarch64_a53_initfn },
     { .name = "max",                .initfn = aarch64_max_initfn },
-#if defined(CONFIG_KVM) || defined(CONFIG_HVF)
+#if defined(CONFIG_KVM) || defined(CONFIG_HVF) || defined(CONFIG_WHPX)
     { .name = "host",               .initfn = aarch64_host_initfn },
 #endif
 };
target/arm/whpx/whpx-all.c +104 -0

116 lines not on this page · raw message

target/arm/whpx_arm.h +1 -0
index de7406b66f..df65fd753c 100644
@@ -12,5 +12,6 @@
 #include "target/arm/cpu-qom.h"
 
 uint32_t whpx_arm_get_ipa_bit_size(void);
+void whpx_arm_set_cpu_features_from_host(ARMCPU *cpu);
 
 #endif
Signature · 1 line
2.50.1 (Apple Git-155)
#29 lore raw

PATCH v17 18/21 target/arm: whpx: instantiate GIC early

In reply to Mohamed Mediouni

While figuring out a better spot for it, put it in whpx_accel_init.

Needs to be done before WHvSetupPartition.

---
 target/arm/whpx/whpx-all.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
target/arm/whpx/whpx-all.c +23 -0
index c88c67a9e2..55773555b7 100644
@@ -939,6 +939,29 @@ int whpx_accel_init(AccelState *as, MachineState *ms)
 
     memset(&prop, 0, sizeof(prop));
 
+    WHV_ARM64_IC_PARAMETERS ic_params = {
+        .EmulationMode = WHvArm64IcEmulationModeGicV3,
+        .GicV3Parameters = {
+            .GicdBaseAddress = 0x08000000,
+            .GitsTranslaterBaseAddress = 0x08080000,
+            .GicLpiIntIdBits = 0,
+            .GicPpiPerformanceMonitorsInterrupt = VIRTUAL_PMU_IRQ,
+            .GicPpiOverflowInterruptFromCntv = ARCH_TIMER_VIRT_IRQ
+        }
+    };
+    prop.Arm64IcParameters = ic_params;
+
+    hr = whp_dispatch.WHvSetPartitionProperty(
+            whpx->partition,
+            WHvPartitionPropertyCodeArm64IcParameters,
+            &prop,
+            sizeof(prop));
+    if (FAILED(hr)) {
+        error_report("WHPX: Failed to enable GICv3 interrupt controller, hr=%08lx", hr);
+        ret = -EINVAL;
+        goto error;
+    }
+
     hr = whp_dispatch.WHvSetupPartition(whpx->partition);
     if (FAILED(hr)) {
         error_report("WHPX: Failed to setup partition, hr=%08lx", hr);
Signature · 1 line
2.50.1 (Apple Git-155)
#30 lore raw

PATCH v17 19/21 whpx: arm64: gicv3: add migration blocker

In reply to Mohamed Mediouni

GICv3 state save-restore is currently not implemented yet.

---
 hw/intc/arm_gicv3_whpx.c | 9 +++++++++
 1 file changed, 9 insertions(+)
hw/intc/arm_gicv3_whpx.c +9 -0
index 9bc036d1ac..a2f8e3c5fb 100644
@@ -17,6 +17,7 @@
 #include "system/whpx-internal.h"
 #include "gicv3_internal.h"
 #include "vgic_common.h"
+#include "migration/blocker.h"
 #include "qom/object.h"
 #include "target/arm/cpregs.h"
 
@@ -204,6 +205,14 @@ static void whpx_gicv3_realize(DeviceState *dev, Error **errp)
         error_setg(errp, "Nested virtualisation not currently supported by WHPX.");
         return;
     }
+
+    Error *whpx_migration_blocker = NULL;
+
+    error_setg(&whpx_migration_blocker,
+        "Live migration disabled because GIC state save/restore not supported on WHPX");
+    if (migrate_add_blocker(&whpx_migration_blocker, errp) < 0) {
+        error_report_err(*errp);
+    }
 }
 
 static void whpx_gicv3_class_init(ObjectClass *klass, const void *data)
Signature · 1 line
2.50.1 (Apple Git-155)
Reviewed-by on v17 19/21
#31 lore raw

Re: [PATCH v17 19/21] whpx: arm64: gicv3: add migration blocker

Quoted 26 lines · Mohamed Mediouni

GICv3 state save-restore is currently not implemented yet.

---
 hw/intc/arm_gicv3_whpx.c | 9 +++++++++
 1 file changed, 9 insertions(+)
hw/intc/arm_gicv3_whpx.c +3 -0
index 9bc036d1ac..a2f8e3c5fb 100644
@@ -17,6 +17,7 @@
 #include "system/whpx-internal.h"
 #include "gicv3_internal.h"
 #include "vgic_common.h"
+#include "migration/blocker.h"
 #include "qom/object.h"
 #include "target/arm/cpregs.h"
 
@@ -204,6 +205,14 @@ static void whpx_gicv3_realize(DeviceState *dev, Error **errp)
         error_setg(errp, "Nested virtualisation not currently supported by WHPX.");
         return;
     }
+
+    Error *whpx_migration_blocker = NULL;

hw/intc/arm_gicv3_whpx.c static void whpx_gicv3_realize(DeviceState *dev, Error **errp)

Quoted 10 lines
hw/intc/arm_gicv3_whpx.c +6 -1
+
+    error_setg(&whpx_migration_blocker,
+        "Live migration disabled because GIC state save/restore not supported on WHPX");
+    if (migrate_add_blocker(&whpx_migration_blocker, errp) < 0) {
+        error_report_err(*errp);
+    }
 }
 
 static void whpx_gicv3_class_init(ObjectClass *klass, const void *data)
--

Thank you.
---
- Prasad

#32 lore raw

Re: [PATCH v17 19/21] whpx: arm64: gicv3: add migration blocker

In reply to Mohamed Mediouni
Quoted 25 lines · Mohamed Mediouni

GICv3 state save-restore is currently not implemented yet.

---
 hw/intc/arm_gicv3_whpx.c | 9 +++++++++
 1 file changed, 9 insertions(+)
hw/intc/arm_gicv3_whpx.c +1 -0
index 9bc036d1ac..a2f8e3c5fb 100644
@@ -17,6 +17,7 @@
 #include "system/whpx-internal.h"
 #include "gicv3_internal.h"
 #include "vgic_common.h"
+#include "migration/blocker.h"
 #include "qom/object.h"
 #include "target/arm/cpregs.h"
 
@@ -204,6 +205,14 @@ static void whpx_gicv3_realize(DeviceState *dev, Error **errp)
         error_setg(errp, "Nested virtualisation not currently supported by WHPX.");
         return;

hw/intc/arm_gicv3_whpx.c static void whpx_gicv3_realize(DeviceState *dev, Error **errp) · line 216

Quoted 10 lines
hw/intc/arm_gicv3_whpx.c +8 -0
     }
+
+    Error *whpx_migration_blocker = NULL;
+
+    error_setg(&whpx_migration_blocker,
+        "Live migration disabled because GIC state save/restore not supported on WHPX");
+    if (migrate_add_blocker(&whpx_migration_blocker, errp) < 0) {
+        error_report_err(*errp);
+    }
 }

I think you can just fold this in to patch 11 where you create this device.

PS: QEMU style says no declarations of variables in the middle of a code block. The GIC base class has an s->migration_blocker field that you should use for this (and indeed you're already testing that in your whpx_gicv3_icc_reset()...)

thanks
-- PMM

#33 lore raw

PATCH v17 20/21 whpx: enable arm64 builds

In reply to Mohamed Mediouni
---
 meson.build | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)
meson.build +13 -7
index a84f14258b..286afca4bb 100644
@@ -300,7 +300,8 @@ accelerator_targets += { 'CONFIG_XEN': xen_targets }
 
 if cpu == 'aarch64'
   accelerator_targets += {
-    'CONFIG_HVF': ['aarch64-softmmu']
+    'CONFIG_HVF': ['aarch64-softmmu'],
+    'CONFIG_WHPX': ['aarch64-softmmu']
   }
 elif cpu == 'x86_64'
   accelerator_targets += {
@@ -856,13 +857,18 @@ if get_option('mshv').allowed() and host_os == 'linux'
 endif
 
 if get_option('whpx').allowed() and host_os == 'windows'
-  if get_option('whpx').enabled() and host_machine.cpu() != 'x86_64'
-    error('WHPX requires 64-bit host')
-  elif cc.has_header('winhvplatform.h', required: get_option('whpx')) and \
-       cc.has_header('winhvemulation.h', required: get_option('whpx'))
-    accelerators += 'CONFIG_WHPX'
+  if cpu == 'i386'
+    if get_option('whpx').enabled()
+     error('WHPX requires 64-bit host')
+    endif
+   # Leave CONFIG_WHPX disabled
+  else
+    if cc.has_header('winhvplatform.h', required: get_option('whpx')) and \
+      cc.has_header('winhvemulation.h', required: get_option('whpx'))
+      accelerators += 'CONFIG_WHPX'
+    endif
   endif
-endif
+ endif
 
 hvf = not_found
 if get_option('hvf').allowed()
Signature · 1 line
2.50.1 (Apple Git-155)
#34 lore raw

PATCH v17 21/21 whpx: arm64: add partition-wide reset on the reboot path

In reply to Mohamed Mediouni

This resets non-architectural state to allow for reboots to succeed.

---
 include/system/whpx-internal.h | 2 ++
 target/arm/whpx/whpx-all.c     | 2 ++
 2 files changed, 4 insertions(+)
include/system/whpx-internal.h +2 -0
index 8ded54a39b..ad6ade223e 100644
@@ -86,6 +86,8 @@ void whpx_apic_get(APICCommonState *s);
   X(HRESULT, WHvSetVirtualProcessorInterruptControllerState2, \
         (WHV_PARTITION_HANDLE Partition, UINT32 VpIndex, PVOID State, \
          UINT32 StateSize)) \
+  X(HRESULT, WHvResetPartition, \
+        (WHV_PARTITION_HANDLE Partition)) \
 
 #define LIST_WINHVEMULATION_FUNCTIONS(X) \
   X(HRESULT, WHvEmulatorCreateEmulator, (const WHV_EMULATOR_CALLBACKS* Callbacks, WHV_EMULATOR_HANDLE* Emulator)) \
target/arm/whpx/whpx-all.c +2 -0
index 55773555b7..444f8c3d42 100644
@@ -497,6 +497,8 @@ int whpx_vcpu_run(CPUState *cpu)
             if (arm_cpu->power_state != PSCI_OFF) {
                 whpx_psci_cpu_off(arm_cpu);
             }
+            /* Partition-wide reset, to reset state for reboots to succeed. */
+            whp_dispatch.WHvResetPartition(whpx->partition);
             bql_unlock();
             break;
         case WHvRunVpExitReasonNone:
Signature · 1 line
2.50.1 (Apple Git-155)
#35 lore raw
In reply to Mohamed Mediouni
Quoted 10 lines · Mohamed Mediouni

Link to branch: https://github.com/mediouni-m/qemu whpx (tag for this submission: whpx-v18)

Missing features:
- VM save-restore: interrupt controller state notably
- SVE register sync: I didn't have the time to test this on pre-release hardware with SVE2 support yet. So SVE2 is currently masked for VMs when running this.

Known bugs:
- U-Boot still doesn't work (hangs when trying to parse firmware) but EDK2 does.

Hi; I left a couple of review comments, but overall I think this is very close to being ready to go in.

thanks
-- PMM