Skip to contentSkip to conversation

Add migration support to MSHV accelerator, Part 2

PATCH In review Magnus Kulke 2 participants 25 messages Jul 10 – Jul 23 Arm/Amlogic Meson SoC support

lore
Share & tools
lore
#0 lore raw

cover Add migration support to MSHV accelerator, Part 2

Hey all,

this is a follow up to a previous series "Add migration support to the MSHV accelerator".

There were 2 contested changes that have been reworked in this series

- target/i386/mshv: migrate LAPIC state
- accel/mshv: introduce SaveVMHandler

"Introduce SaveVMHandler" was adding lifecycle hooks that froze the timer at the start of the migration on the source, which effectively paused any workload that would depend on timer progress (like an os) during the migration window. This is undesired for live migration and has been split out into a dedicated clock device that has its own migration lifecycle in "hw/i386/mshv: migrate REFERENCE_TIME".

"Migrate LAPIC state" changed in 2 ways: 1) for mshv-specific fields there is now an "MshvAPICState" object that captures auxiliary fields that exist in the MSHV LAPIC state. Initially we also shadowed the lvt[APIC_LVT_LINT0/1] fields, since a write to those fields was triggering a userpace PIC delivery path that MSHV doesn't support.
2) We add a guard in "pic_irq_request()" to short-circuit the userland delivery for MSHV for PIC interrupts, that means can use the LINT0/1 fields from the CommonAPICState.

The other migration steps have been carried over more or less verbatim from the first patch series.

3 drive-by fixes:

- disable AMX TILE features: this is required on because we do not not support XSAVE buffers beyond 4k in the accelerator yet
- drop initial msi vector 0: this has produced a warning at launch, after tracing the source we can drop it silently with confidence.
- toggle fpu/xsave migration: a subtle bug where fpu state would be clobbered by xsave legacy space when applied in the wrong order.

best regards,

magnus

Magnus Kulke (12):
target/i386/mshv: disable AMX TILE features accel/mshv: introduce SaveVMHandler hw/i386/mshv: migrate REFERENCE_TIME accel/mshv: install dummy handler for SIG_IPI target/i386/mshv: migrate LAPIC state target/i386/mshv: migrate Synic SINT MSRs target/i386/mshv: migrate SIMP and SIEFP state target/i386/mshv: migrate STIMER state accel/mshv: write synthetic MSRs after migration target/i386/mshv: migrate MP_STATE target/i386/mshv: toggle fpu/xsave migration hw/i386/mshv: drop initial msi vector 0

 MAINTAINERS                    |   1 +
 accel/mshv/mshv-all.c          |  98 ++++----
 accel/mshv/trace-events        |   1 +
 hw/i386/meson.build            |   1 +
 hw/i386/mshv/apic.c            | 405 +++++++++++++++++++++++++++++++++
 hw/i386/mshv/clock.c           | 189 +++++++++++++++
 hw/i386/mshv/meson.build       |   5 +
 hw/i386/x86-cpu.c              |   6 +
 include/hw/hyperv/hvgdk_mini.h |   5 +
 include/hw/i386/apic-msidef.h  |   1 +
 include/system/mshv.h          |   5 +
 include/system/mshv_int.h      |  19 +-
 target/i386/cpu-apic.c         |   3 +
 target/i386/cpu.h              |  10 +
 target/i386/machine.c          |  46 ++++
 target/i386/mshv/meson.build   |   1 +
 target/i386/mshv/mshv-cpu.c    | 305 +++++++++++++++++--------
 target/i386/mshv/msr.c         |  40 ++++
 target/i386/mshv/synic.c       | 206 +++++++++++++++++
 19 files changed, 1196 insertions(+), 151 deletions(-)
 create mode 100644 hw/i386/mshv/apic.c
 create mode 100644 hw/i386/mshv/clock.c
 create mode 100644 hw/i386/mshv/meson.build
 create mode 100644 target/i386/mshv/synic.c
Signature · 1 line
2.34.1
#1 lore raw

PATCH 1/12 target/i386/mshv: disable AMX TILE features

For the time being we disable AMX TILE in partition processor features and CPUID b/c AMX TILE XSAVE state (XTILE_DATA) is 8KB, which exceeds the current fixed 4KB XSAVE buffer size.

For now we filter it until buffer sizing is computed dynamically from CPUID.

---
 accel/mshv/mshv-all.c       |  8 ++++++++
 target/i386/mshv/mshv-cpu.c | 16 ++++++++++++++++
 2 files changed, 24 insertions(+)
accel/mshv/mshv-all.c +8 -0
index 72721d0f0d..5e8756ede2 100644
@@ -203,6 +203,14 @@ static int create_partition(int mshv_fd, int *vm_fd)
 
     /* enable all */
     disabled_xsave_features.as_uint64 = 0;
+    /*
+     * AMX TILE XSAVE state (XTILE_DATA) is 8KB, which exceeds the
+     * current fixed 4KB XSAVE buffer size.
+     */
+    disabled_xsave_features.amx_tile_support = 1;
+    disabled_xsave_features.amx_bf16_support = 1;
+    disabled_xsave_features.amx_int8_support = 1;
+    disabled_xsave_features.amx_fp16_support = 1;
 
     /*
      * query host for supported processor features and disable unsupported
target/i386/mshv/mshv-cpu.c +16 -0
index 1c433c408c..8eca01a8fa 100644
@@ -2156,6 +2156,22 @@ uint32_t mshv_get_supported_cpuid(uint32_t func, uint32_t idx, int reg)
          */
         ret &= ~CPUID_7_0_ECX_LA57;
     }
+    if (func == 0x07 && idx == 0 && reg == R_EDX) {
+        /*
+         * AMX TILE XSAVE state (XTILE_DATA) is 8KB, which exceeds the
+         * current fixed 4KB XSAVE buffer size. Filter until buffer
+         * sizing is computed dynamically from CPUID.
+         */
+        ret &= ~CPUID_7_0_EDX_AMX_TILE;
+        ret &= ~CPUID_7_0_EDX_AMX_BF16;
+        ret &= ~CPUID_7_0_EDX_AMX_INT8;
+    }
+    if (func == 0x07 && idx == 1 && reg == R_EAX) {
+        ret &= ~CPUID_7_1_EAX_AMX_FP16;
+    }
+    if (func == 0x07 && idx == 1 && reg == R_EDX) {
+        ret &= ~CPUID_7_1_EDX_AMX_COMPLEX;
+    }
 
     return ret;
 }
Signature · 1 line
2.34.1
Reviewed-by on v1 1/12
#2 lore raw

Re: [PATCH 01/12] target/i386/mshv: disable AMX TILE features

Quoted 60 lines · Magnus Kulke

For the time being we disable AMX TILE in partition processor features and CPUID b/c AMX TILE XSAVE state (XTILE_DATA) is 8KB, which exceeds the current fixed 4KB XSAVE buffer size.

For now we filter it until buffer sizing is computed dynamically from CPUID.

---
 accel/mshv/mshv-all.c       |  8 ++++++++
 target/i386/mshv/mshv-cpu.c | 16 ++++++++++++++++
 2 files changed, 24 insertions(+)
accel/mshv/mshv-all.c +8 -0
index 72721d0f0d..5e8756ede2 100644
@@ -203,6 +203,14 @@ static int create_partition(int mshv_fd, int *vm_fd)
 
     /* enable all */
     disabled_xsave_features.as_uint64 = 0;
+    /*
+     * AMX TILE XSAVE state (XTILE_DATA) is 8KB, which exceeds the
+     * current fixed 4KB XSAVE buffer size.
+     */
+    disabled_xsave_features.amx_tile_support = 1;
+    disabled_xsave_features.amx_bf16_support = 1;
+    disabled_xsave_features.amx_int8_support = 1;
+    disabled_xsave_features.amx_fp16_support = 1;
 
     /*
      * query host for supported processor features and disable unsupported
target/i386/mshv/mshv-cpu.c +16 -0
index 1c433c408c..8eca01a8fa 100644
@@ -2156,6 +2156,22 @@ uint32_t mshv_get_supported_cpuid(uint32_t func, uint32_t idx, int reg)
          */
         ret &= ~CPUID_7_0_ECX_LA57;
     }
+    if (func == 0x07 && idx == 0 && reg == R_EDX) {
+        /*
+         * AMX TILE XSAVE state (XTILE_DATA) is 8KB, which exceeds the
+         * current fixed 4KB XSAVE buffer size. Filter until buffer
+         * sizing is computed dynamically from CPUID.
+         */
+        ret &= ~CPUID_7_0_EDX_AMX_TILE;
+        ret &= ~CPUID_7_0_EDX_AMX_BF16;
+        ret &= ~CPUID_7_0_EDX_AMX_INT8;
+    }
+    if (func == 0x07 && idx == 1 && reg == R_EAX) {
+        ret &= ~CPUID_7_1_EAX_AMX_FP16;
+    }
+    if (func == 0x07 && idx == 1 && reg == R_EDX) {
+        ret &= ~CPUID_7_1_EDX_AMX_COMPLEX;
+    }
 
     return ret;
 }
Signature · 1 line
2.34.1
#3 lore raw

PATCH 2/12 accel/mshv: introduce SaveVMHandler

In reply to Magnus Kulke

This mechanism is used to handle more imperative partition-wide steps that have to be taken as part of a migration routine. Currently it's just a skeleton.

---
 accel/mshv/mshv-all.c | 7 +++++++
 1 file changed, 7 insertions(+)
accel/mshv/mshv-all.c +7 -0
index 5e8756ede2..b34fb34d05 100644
@@ -39,6 +39,8 @@
 #include "system/mshv.h"
 #include "system/mshv_int.h"
 #include "system/reset.h"
+#include "migration/qemu-file-types.h"
+#include "migration/register.h"
 #include "trace.h"
 #include <err.h>
 #include <sys/ioctl.h>
@@ -544,6 +546,9 @@ static int mshv_init_vcpu(CPUState *cpu)
     return 0;
 }
 
+static SaveVMHandlers savevm_mshv = {
+};
+
 static int mshv_init(AccelState *as, MachineState *ms)
 {
     MshvState *s;
@@ -599,6 +604,8 @@ static int mshv_init(AccelState *as, MachineState *ms)
                                   0, "mshv-memory");
     memory_listener_register(&mshv_io_listener, &address_space_io);
 
+    register_savevm_live("mshv", 0, 1, &savevm_mshv, s);
+
     return 0;
 }
 
Signature · 1 line
2.34.1
Reviewed-by on v1 2/12
#4 lore raw

Re: [PATCH 02/12] accel/mshv: introduce SaveVMHandler

Quoted 43 lines · Magnus Kulke

This mechanism is used to handle more imperative partition-wide steps that have to be taken as part of a migration routine. Currently it's just a skeleton.

---
 accel/mshv/mshv-all.c | 7 +++++++
 1 file changed, 7 insertions(+)
accel/mshv/mshv-all.c +7 -0
index 5e8756ede2..b34fb34d05 100644
@@ -39,6 +39,8 @@
 #include "system/mshv.h"
 #include "system/mshv_int.h"
 #include "system/reset.h"
+#include "migration/qemu-file-types.h"
+#include "migration/register.h"
 #include "trace.h"
 #include <err.h>
 #include <sys/ioctl.h>
@@ -544,6 +546,9 @@ static int mshv_init_vcpu(CPUState *cpu)
     return 0;
 }
 
+static SaveVMHandlers savevm_mshv = {
+};
+
 static int mshv_init(AccelState *as, MachineState *ms)
 {
     MshvState *s;
@@ -599,6 +604,8 @@ static int mshv_init(AccelState *as, MachineState *ms)
                                   0, "mshv-memory");
     memory_listener_register(&mshv_io_listener, &address_space_io);
 
+    register_savevm_live("mshv", 0, 1, &savevm_mshv, s);
+
     return 0;
 }
 
Signature · 1 line
2.34.1
#5 lore raw

PATCH 3/12 hw/i386/mshv: migrate REFERENCE_TIME

In reply to Magnus Kulke

This is a partition-wide state for which we use a dedicated hw clock facility, similar to KVM. We have to freeze the time for a partition before we are allowed to set it. We register a state change handler for the clock device and a post-load handler for migration state. In the post-load handler we toggle a flag that will set the reference time state on next state to "running" on the partition.

We can move the time freeze and reference-time ioctls/hvcalls to the clock module.

---
 MAINTAINERS              |   1 +
 accel/mshv/mshv-all.c    |  67 ++------------
 accel/mshv/trace-events  |   1 +
 hw/i386/meson.build      |   1 +
 hw/i386/mshv/clock.c     | 189 +++++++++++++++++++++++++++++++++++++++
 hw/i386/mshv/meson.build |   4 +
 include/system/mshv.h    |   3 +
 7 files changed, 208 insertions(+), 58 deletions(-)
 create mode 100644 hw/i386/mshv/clock.c
 create mode 100644 hw/i386/mshv/meson.build
MAINTAINERS +1 -0
index 6171cc7494..269876db3e 100644
@@ -613,6 +613,7 @@ R: Wei Liu <[email protected]>
 R: Doru Blânzeanu <[email protected]>
 S: Supported
 F: target/i386/mshv/
+F: hw/i386/mshv/
 
 X86 Instruction Emulator
 M: Roman Bolshakov <[email protected]>
accel/mshv/mshv-all.c +9 -58

97 lines not on this page · raw message

accel/mshv/trace-events +1 -0
index a4dffeb24a..859e8bfb0f 100644
@@ -4,6 +4,7 @@
 # SPDX-License-Identifier: GPL-2.0-or-later
 
 mshv_start_vcpu_thread(const char* thread, uint32_t cpu) "thread=%s cpu_index=%d"
+mshv_interrupt_exit_request(uint32_t cpu) "cpu_index=%d"
 
 mshv_set_memory(bool add, uint64_t gpa, uint64_t size, uint64_t user_addr, bool readonly, int ret) "add=%d gpa=0x%" PRIx64 " size=0x%" PRIx64 " user=0x%" PRIx64 " readonly=%d result=%d"
 mshv_mem_ioeventfd_add(uint64_t addr, uint32_t size, uint32_t data) "addr=0x%" PRIx64 " size=%d data=0x%x"
hw/i386/meson.build +1 -0
index b611fbb5a7..f4d3122863 100644
@@ -39,6 +39,7 @@ i386_ss.add(when: 'CONFIG_TDX', if_true: files('tdvf.c', 'tdvf-hob.c'))
 
 subdir('kvm')
 subdir('xen')
+subdir('mshv')
 
 i386_ss.add_all(xenpv_ss)
 
hw/i386/mshv/clock.c new file +189 -0

189 lines not on this page · raw message

hw/i386/mshv/meson.build new file +4 -0
new file mode 100644
index 0000000000..0e556851b6
@@ -0,0 +1,4 @@
+i386_mshv_ss = ss.source_set()
+i386_mshv_ss.add(files('clock.c'))
+
+i386_ss.add_all(when: 'CONFIG_MSHV', if_true: i386_mshv_ss)
include/system/mshv.h +3 -0
index 46fe3fcebc..a6f815b822 100644
@@ -55,6 +55,9 @@ DECLARE_INSTANCE_CHECKER(MshvState, MSHV_STATE,
 
 extern MshvState *mshv_state;
 
+/* clock (partition reference time) */
+void mshv_clock_init(void);
+
 /* interrupt */
 int mshv_request_interrupt(MshvState *mshv_state, uint32_t interrupt_type, uint32_t vector,
                            uint32_t vp_index, bool logical_destination_mode,
Signature · 1 line
2.34.1
Reviewed-by on v1 3/12
#6 lore raw

Re: [PATCH 03/12] hw/i386/mshv: migrate REFERENCE_TIME

Quoted 386 lines · Magnus Kulke

This is a partition-wide state for which we use a dedicated hw clock facility, similar to KVM. We have to freeze the time for a partition before we are allowed to set it. We register a state change handler for the clock device and a post-load handler for migration state. In the post-load handler we toggle a flag that will set the reference time state on next state to "running" on the partition.

We can move the time freeze and reference-time ioctls/hvcalls to the clock module.

---
 MAINTAINERS              |   1 +
 accel/mshv/mshv-all.c    |  67 ++------------
 accel/mshv/trace-events  |   1 +
 hw/i386/meson.build      |   1 +
 hw/i386/mshv/clock.c     | 189 +++++++++++++++++++++++++++++++++++++++
 hw/i386/mshv/meson.build |   4 +
 include/system/mshv.h    |   3 +
 7 files changed, 208 insertions(+), 58 deletions(-)
 create mode 100644 hw/i386/mshv/clock.c
 create mode 100644 hw/i386/mshv/meson.build
MAINTAINERS +1 -0
index 6171cc7494..269876db3e 100644
@@ -613,6 +613,7 @@ R: Wei Liu <[email protected]>
 R: Doru Blânzeanu <[email protected]>
 S: Supported
 F: target/i386/mshv/
+F: hw/i386/mshv/
 
 X86 Instruction Emulator
 M: Roman Bolshakov <[email protected]>
accel/mshv/mshv-all.c +9 -58
index b34fb34d05..7b3c13cf1e 100644
@@ -60,54 +60,6 @@ static int init_mshv(int *mshv_fd)
     return 0;
 }
 
-/* freeze 1 to pause, 0 to resume */
-static int set_time_freeze(int vm_fd, int freeze)
-{
-    int ret;
-    struct hv_input_set_partition_property in = {0};
-    in.property_code = HV_PARTITION_PROPERTY_TIME_FREEZE;
-    in.property_value = freeze;
-
-    struct mshv_root_hvcall args = {0};
-    args.code = HVCALL_SET_PARTITION_PROPERTY;
-    args.in_sz = sizeof(in);
-    args.in_ptr = (uint64_t)&in;
-
-    ret = mshv_hvcall(vm_fd, &args);
-    if (ret < 0) {
-        error_report("Failed to set time freeze");
-        return -1;
-    }
-
-    return 0;
-}
-
-static int pause_vm(int vm_fd)
-{
-    int ret;
-
-    ret = set_time_freeze(vm_fd, 1);
-    if (ret < 0) {
-        error_report("Failed to pause partition: %s", strerror(errno));
-        return -1;
-    }
-
-    return 0;
-}
-
-static int resume_vm(int vm_fd)
-{
-    int ret;
-
-    ret = set_time_freeze(vm_fd, 0);
-    if (ret < 0) {
-        error_report("Failed to resume partition: %s", strerror(errno));
-        return -1;
-    }
-
-    return 0;
-}
-
 static int get_host_partition_property(int mshv_fd, uint32_t property_code,
                                        uint64_t *value)
 {
@@ -330,9 +282,6 @@ static int create_vm(int mshv_fd, int *vm_fd)
         return -1;
     }
 
-    /* Always create a frozen partition */
-    pause_vm(*vm_fd);
-
     return 0;
 }
 
@@ -578,13 +527,6 @@ static int mshv_init(AccelState *as, MachineState *ms)
         return -1;
     }
 
-    ret = resume_vm(vm_fd);
-    if (ret < 0) {
-        close(mshv_fd);
-        close(vm_fd);
-        return -1;
-    }
-
     s->vm = vm_fd;
     s->fd = mshv_fd;
 
@@ -606,6 +548,8 @@ static int mshv_init(AccelState *as, MachineState *ms)
 
     register_savevm_live("mshv", 0, 1, &savevm_mshv, s);
 
+    mshv_clock_init();
+
     return 0;
 }
 
@@ -643,6 +587,13 @@ static int mshv_cpu_exec(CPUState *cpu)
             cpu->vcpu_dirty = false;
         }
 
+        /* Corresponding store-release is in cpu_exit. */
+        if (qatomic_load_acquire(&cpu->exit_request)) {
+            trace_mshv_interrupt_exit_request(cpu->cpu_index);
+            ret = EXCP_INTERRUPT;
+            break;
+        }
+
         ret = mshv_run_vcpu(mshv_state->vm, cpu, &mshv_msg, &exit_reason);
         if (ret < 0) {
             error_report("Failed to run on vcpu %d", cpu->cpu_index);
accel/mshv/trace-events +1 -0
index a4dffeb24a..859e8bfb0f 100644
@@ -4,6 +4,7 @@
 # SPDX-License-Identifier: GPL-2.0-or-later
 
 mshv_start_vcpu_thread(const char* thread, uint32_t cpu) "thread=%s cpu_index=%d"
+mshv_interrupt_exit_request(uint32_t cpu) "cpu_index=%d"
 
 mshv_set_memory(bool add, uint64_t gpa, uint64_t size, uint64_t user_addr, bool readonly, int ret) "add=%d gpa=0x%" PRIx64 " size=0x%" PRIx64 " user=0x%" PRIx64 " readonly=%d result=%d"
 mshv_mem_ioeventfd_add(uint64_t addr, uint32_t size, uint32_t data) "addr=0x%" PRIx64 " size=%d data=0x%x"
hw/i386/meson.build +1 -0
index b611fbb5a7..f4d3122863 100644
@@ -39,6 +39,7 @@ i386_ss.add(when: 'CONFIG_TDX', if_true: files('tdvf.c', 'tdvf-hob.c'))
 
 subdir('kvm')
 subdir('xen')
+subdir('mshv')
 
 i386_ss.add_all(xenpv_ss)
 
hw/i386/mshv/clock.c new file +189 -0
new file mode 100644
index 0000000000..02c586503a
@@ -0,0 +1,189 @@
+/*
+ * MSHV partition reference clock
+ *
+ * Copyright Microsoft, Corp. 2026
+ *
+ * Authors: Magnus Kulke <[email protected]>
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/error-report.h"
+#include "migration/vmstate.h"
+#include "system/runstate.h"
+#include "hw/hyperv/hvhdk.h"
+#include "hw/hyperv/hvhdk_mini.h"
+#include "hw/hyperv/hvgdk.h"
+#include "hw/hyperv/hvgdk_mini.h"
+#include "linux/mshv.h"
+#include "system/mshv.h"
+#include "system/mshv_int.h"
+
+/*
+ * Partition reference clock (HV_PARTITION_PROPERTY_REFERENCE_TIME), captured
+ * when the VM is stopped and re-applied when it resumes.
+ *
+ * Mirrors hw/i386/kvm/clock.c.
+ */
+typedef struct MshvClockState {
+    uint64_t ref_time;
+    bool ref_time_pending;
+} MshvClockState;
+
+static MshvClockState mshv_clock;
+
+static int mshv_get_reference_time(int vm_fd, uint64_t *ref_time)
+{
+    struct hv_input_get_partition_property in = { 0 };
+    struct hv_output_get_partition_property out = { 0 };
+    struct mshv_root_hvcall args = { 0 };
+    int ret;
+
+    in.property_code = HV_PARTITION_PROPERTY_REFERENCE_TIME;
+
+    args.code        = HVCALL_GET_PARTITION_PROPERTY;
+    args.in_sz       = sizeof(in);
+    args.in_ptr      = (uint64_t)&in;
+    args.out_sz      = sizeof(out);
+    args.out_ptr     = (uint64_t)&out;
+
+    ret = mshv_hvcall(vm_fd, &args);
+    if (ret < 0) {
+        error_report("Failed to get reference time");
+        return -1;
+    }
+
+    *ref_time = out.property_value;
+    return 0;
+}
+
+static int mshv_set_reference_time(int vm_fd, uint64_t ref_time)
+{
+    struct hv_input_set_partition_property in = { 0 };
+    struct mshv_root_hvcall args = { 0 };
+    int ret;
+
+    in.property_code  = HV_PARTITION_PROPERTY_REFERENCE_TIME;
+    in.property_value = ref_time;
+
+    args.code         = HVCALL_SET_PARTITION_PROPERTY;
+    args.in_sz        = sizeof(in);
+    args.in_ptr       = (uint64_t)&in;
+
+    ret = mshv_hvcall(vm_fd, &args);
+    if (ret < 0) {
+        error_report("Failed to set reference time");
+        return -1;
+    }
+
+    return 0;
+}
+
+/*
+ * Freeze (freeze=1) or unfreeze (freeze=0) time for a partition. This will not
+ * pause/eject vCPU execution. It is assumed that the caller already has stopped
+ * the partition's vCPUs.
+ *
+ * NB: a partition's reference clock can only be written while the time is
+ * frozen.
+ */
+static int mshv_set_time_freeze(int vm_fd, int freeze)
+{
+    struct hv_input_set_partition_property in = { 0 };
+    struct mshv_root_hvcall args = { 0 };
+    int ret;
+
+    in.property_code  = HV_PARTITION_PROPERTY_TIME_FREEZE;
+    in.property_value = freeze;
+
+    args.code         = HVCALL_SET_PARTITION_PROPERTY;
+    args.in_sz        = sizeof(in);
+    args.in_ptr       = (uint64_t)&in;
+
+    ret = mshv_hvcall(vm_fd, &args);
+    if (ret < 0) {
+        error_report("Failed to set time freeze");
+        return -1;
+    }
+
+    return 0;
+}
+
+static void mshv_clock_vm_state_change(void *opaque, bool running,
+                                       RunState state)
+{
+    MshvClockState *s = opaque;
+    int vm_fd = mshv_state->vm;
+    int ret;
+
+    if (running) {
+        /* Skip if we have nothing to restore, e.g. on initial boot. */
+        if (!s->ref_time_pending) {
+            return;
+        }
+
+        ret = mshv_set_time_freeze(vm_fd, 1);
+        if (ret < 0) {
+            error_report("Failed to freeze partition time on resume");
+            abort();
+        }
+
+        /* INVARIANT: reference time can only be written if time is frozen. */
+        ret = mshv_set_reference_time(vm_fd, s->ref_time);
+        if (ret < 0) {
+            error_report("Failed to restore reference time on resume");
+            abort();
+        }
+
+        if (mshv_set_time_freeze(vm_fd, 0) < 0) {
+            error_report("Failed to unfreeze partition time on resume");
+            abort();
+        }
+
+        s->ref_time_pending = false;
+    } else {
+        /* Skip if we already have a to-be-set ref time */
+        if (s->ref_time_pending) {
+            return;
+        }
+
+        ret = mshv_get_reference_time(vm_fd, &s->ref_time);
+        if (ret < 0) {
+            error_report("Failed to capture reference time on stop");
+            abort();
+        }
+
+        s->ref_time_pending = true;
+    }
+}
+
+/*
+ * The incoming reference time should be applied on the next resume before
+ * vCPUs start executing.
+ */
+static int mshv_clock_post_load(void *opaque, int version_id)
+{
+    MshvClockState *s = opaque;
+
+    s->ref_time_pending = true;
+
+    return 0;
+}
+
+static const VMStateDescription vmstate_mshv_clock = {
+    .name = "mshv-clock",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .post_load = mshv_clock_post_load,
+    .fields = (const VMStateField[]) {
+        VMSTATE_UINT64(ref_time, MshvClockState),
+        VMSTATE_END_OF_LIST()
+    },
+};
+
+void mshv_clock_init(void)
+{
+    vmstate_register(NULL, 0, &vmstate_mshv_clock, &mshv_clock);
+    qemu_add_vm_change_state_handler(mshv_clock_vm_state_change, &mshv_clock);
+}
hw/i386/mshv/meson.build new file +4 -0
new file mode 100644
index 0000000000..0e556851b6
@@ -0,0 +1,4 @@
+i386_mshv_ss = ss.source_set()
+i386_mshv_ss.add(files('clock.c'))
+
+i386_ss.add_all(when: 'CONFIG_MSHV', if_true: i386_mshv_ss)
include/system/mshv.h +3 -0
index 46fe3fcebc..a6f815b822 100644
@@ -55,6 +55,9 @@ DECLARE_INSTANCE_CHECKER(MshvState, MSHV_STATE,
 
 extern MshvState *mshv_state;
 
+/* clock (partition reference time) */
+void mshv_clock_init(void);
+
 /* interrupt */
 int mshv_request_interrupt(MshvState *mshv_state, uint32_t interrupt_type, uint32_t vector,
                            uint32_t vp_index, bool logical_destination_mode,
Signature · 1 line
2.34.1
#7 lore raw

PATCH 4/12 accel/mshv: install dummy handler for SIG_IPI

In reply to Magnus Kulke

This is similar to HVF's implementation. We want to interrupt the blocking vcpu run. The self-kick was effectively a no-op for mshv.

---
 accel/mshv/mshv-all.c | 16 ++++++----------
 1 file changed, 6 insertions(+), 10 deletions(-)
accel/mshv/mshv-all.c +6 -10
index 7b3c13cf1e..f9511ca050 100644
@@ -621,17 +621,13 @@ static int mshv_cpu_exec(CPUState *cpu)
 }
 
 /*
- * The signal handler is triggered when QEMU's main thread receives a SIG_IPI
- * (SIGUSR1). This signal causes the current CPU thread to be kicked, forcing a
- * VM exit on the CPU. The VM exit generates an exit reason that breaks the loop
- * (see mshv_cpu_exec). If the exit is due to a Ctrl+A+x command, the system
- * will shut down. For other cases, the system will continue running.
+ * We need a dummy handler to make SIG_IPI a deliverable signal. The kernel
+ * handler will be woken up by the caught signal and instruct the hypervisor
+ * to suspend execution (the concrete mechanism differs between schedulers)
+ * and return to userspace.
  */
-static void sa_ipi_handler(int sig)
+static void dummy_handler(int sig)
 {
-    /* TODO: call IOCTL to set_immediate_exit, once implemented. */
-
-    qemu_cpu_kick_self();
 }
 
 static void init_signal(CPUState *cpu)
@@ -641,7 +637,7 @@ static void init_signal(CPUState *cpu)
     sigset_t set;
 
     memset(&sigact, 0, sizeof(sigact));
-    sigact.sa_handler = sa_ipi_handler;
+    sigact.sa_handler = dummy_handler;
     sigaction(SIG_IPI, &sigact, NULL);
 
     pthread_sigmask(SIG_BLOCK, NULL, &set);
Signature · 1 line
2.34.1
Reviewed-by on v1 4/12
#8 lore raw

Re: [PATCH 04/12] accel/mshv: install dummy handler for SIG_IPI

Quoted 46 lines · Magnus Kulke

This is similar to HVF's implementation. We want to interrupt the blocking vcpu run. The self-kick was effectively a no-op for mshv.

---
 accel/mshv/mshv-all.c | 16 ++++++----------
 1 file changed, 6 insertions(+), 10 deletions(-)
accel/mshv/mshv-all.c +6 -10
index 7b3c13cf1e..f9511ca050 100644
@@ -621,17 +621,13 @@ static int mshv_cpu_exec(CPUState *cpu)
 }
 
 /*
- * The signal handler is triggered when QEMU's main thread receives a SIG_IPI
- * (SIGUSR1). This signal causes the current CPU thread to be kicked, forcing a
- * VM exit on the CPU. The VM exit generates an exit reason that breaks the loop
- * (see mshv_cpu_exec). If the exit is due to a Ctrl+A+x command, the system
- * will shut down. For other cases, the system will continue running.
+ * We need a dummy handler to make SIG_IPI a deliverable signal. The kernel
+ * handler will be woken up by the caught signal and instruct the hypervisor
+ * to suspend execution (the concrete mechanism differs between schedulers)
+ * and return to userspace.
  */
-static void sa_ipi_handler(int sig)
+static void dummy_handler(int sig)
 {
-    /* TODO: call IOCTL to set_immediate_exit, once implemented. */
-
-    qemu_cpu_kick_self();
 }
 
 static void init_signal(CPUState *cpu)
@@ -641,7 +637,7 @@ static void init_signal(CPUState *cpu)
     sigset_t set;
 
     memset(&sigact, 0, sizeof(sigact));
-    sigact.sa_handler = sa_ipi_handler;
+    sigact.sa_handler = dummy_handler;
     sigaction(SIG_IPI, &sigact, NULL);
 
     pthread_sigmask(SIG_BLOCK, NULL, &set);
Signature · 1 line
2.34.1
#9 lore raw

PATCH 5/12 target/i386/mshv: migrate LAPIC state

In reply to Magnus Kulke

This change implements loading and storing the hyperv lapic state as part of the load/store routines for a vcpu.

The HyperV LAPIC is similar to the the split-irqchip in KVM. MSHV currently keeps PIC/IOAPIC emulation in userspace, while LAPIC interrupt injection is handled through hypercalls.

We introduced dedicated apic infra in hw/i386/mshv to handle the migration and move lapic related functions from target/i386/mshv there. References have been the WHPX's whpx-apic implemenation and the mshv-ioctls crate's get_/set_lapic() impl for the mapping between MSHV/QEMU lapic state.

We are mapping the lapic state that we receive from the hypervisor to fields in APICCommonState. Common fields are used where feasible, with an mshv-specific MshvAPICState object that carries mshv-specific fields.

We have introduced a guard in pic_irq_request() that will early exit for the mshv accelerator, because mshv cannot take part in the userland path for legacy PIC interrupt injection.

The TSC_DEADLINE MSR is also migrated as part of LAPIC migration.

---
 accel/mshv/mshv-all.c          |   1 +
 hw/i386/meson.build            |   2 +-
 hw/i386/mshv/apic.c            | 395 +++++++++++++++++++++++++++++++++
 hw/i386/mshv/meson.build       |   1 +
 hw/i386/x86-cpu.c              |   6 +
 include/hw/hyperv/hvgdk_mini.h |   2 +
 include/hw/i386/apic-msidef.h  |   1 +
 include/system/mshv.h          |   2 +
 include/system/mshv_int.h      |   8 +-
 target/i386/cpu-apic.c         |   3 +
 target/i386/mshv/mshv-cpu.c    | 151 ++++---------
 target/i386/mshv/msr.c         |   2 +
 12 files changed, 460 insertions(+), 114 deletions(-)
 create mode 100644 hw/i386/mshv/apic.c
accel/mshv/mshv-all.c +1 -0
index f9511ca050..1516475f34 100644
@@ -813,6 +813,7 @@ static void mshv_accel_ops_class_init(ObjectClass *oc, const void *data)
     ops->synchronize_state = mshv_cpu_synchronize;
     ops->synchronize_pre_loadvm = mshv_cpu_synchronize_pre_loadvm;
     ops->cpus_are_resettable = mshv_cpus_are_resettable;
+    ops->cpu_thread_is_idle = mshv_vcpu_thread_is_idle;
     ops->handle_interrupt = generic_handle_interrupt;
 }
 
hw/i386/meson.build +1 -1
index f4d3122863..39ac8c9edc 100644
@@ -38,8 +38,8 @@ i386_ss.add(when: 'CONFIG_X86_FW_OVMF', if_true: files('pc_sysfw_ovmf.c'),
 i386_ss.add(when: 'CONFIG_TDX', if_true: files('tdvf.c', 'tdvf-hob.c'))
 
 subdir('kvm')
-subdir('xen')
 subdir('mshv')
+subdir('xen')
 
 i386_ss.add_all(xenpv_ss)
 
hw/i386/mshv/apic.c new file +395 -0

395 lines not on this page · raw message

hw/i386/mshv/meson.build +1 -0
index 0e556851b6..c631ee1302 100644
@@ -1,4 +1,5 @@
 i386_mshv_ss = ss.source_set()
 i386_mshv_ss.add(files('clock.c'))
+i386_mshv_ss.add(when: 'CONFIG_APIC', if_true: files('apic.c'))
 
 i386_ss.add_all(when: 'CONFIG_MSHV', if_true: i386_mshv_ss)
hw/i386/x86-cpu.c +6 -0
index 95e08e3c2a..fba313376c 100644
@@ -22,6 +22,7 @@
  */
 #include "qemu/osdep.h"
 #include "system/whpx.h"
+#include "system/mshv.h"
 #include "system/cpu-timers.h"
 #include "trace.h"
 
@@ -44,6 +45,11 @@ static void pic_irq_request(void *opaque, int irq, int level)
     X86CPU *cpu = X86_CPU(cs);
 
     trace_x86_pic_interrupt(irq, level);
+
+    if (mshv_irqchip_in_kernel()) {
+        return;
+    }
+
     if (cpu_is_apic_enabled(cpu->apic_state) && !kvm_irqchip_in_kernel() &&
         !whpx_irqchip_in_kernel()) {
         CPU_FOREACH(cs) {
include/hw/hyperv/hvgdk_mini.h +2 -0
index f8838a31bb..0602a7c6cc 100644
@@ -168,6 +168,7 @@ typedef enum hv_register_name {
     /* Available */
 
     HV_X64_REGISTER_SPEC_CTRL       = 0x00080084,
+    HV_X64_REGISTER_TSC_DEADLINE    = 0x00080095,
     HV_X64_REGISTER_TSC_ADJUST      = 0x00080096,
 
     /* CET / Shadow Stack */
@@ -930,6 +931,7 @@ struct hv_cpuid {
 #define IA32_MSR_DEBUG_CTL        0x1D9
 #define IA32_MSR_SPEC_CTRL        0x00000048
 #define IA32_MSR_TSC_ADJUST       0x0000003b
+#define IA32_MSR_TSC_DEADLINE     0x000006e0
 
 #define IA32_MSR_MISC_ENABLE 0x000001a0
 
include/hw/i386/apic-msidef.h +1 -0
index 420b41167d..6b860b5807 100644
@@ -13,6 +13,7 @@
 #define  MSI_DATA_VECTOR_MASK           0x000000ff
 
 #define MSI_DATA_DELIVERY_MODE_SHIFT    8
+#define MSI_DATA_DELIVERY_MODE_MASK     7
 #define MSI_DATA_LEVEL_SHIFT            14
 #define MSI_DATA_TRIGGER_SHIFT          15
 
include/system/mshv.h +2 -0
index a6f815b822..f98ddff9b5 100644
@@ -41,9 +41,11 @@
 extern bool mshv_allowed;
 #define mshv_enabled() (mshv_allowed)
 #define mshv_msi_via_irqfd_enabled() mshv_enabled()
+#define mshv_irqchip_in_kernel() mshv_enabled()
 #else /* CONFIG_MSHV_IS_POSSIBLE */
 #define mshv_enabled() false
 #define mshv_msi_via_irqfd_enabled() mshv_enabled()
+#define mshv_irqchip_in_kernel() mshv_enabled()
 #endif
 
 #define TYPE_MSHV_ACCEL ACCEL_CLASS_NAME("mshv")
include/system/mshv_int.h +7 -1
index b91c4d661a..c2f13c0194 100644
@@ -105,10 +105,16 @@ void mshv_arch_amend_proc_features(
 void mshv_arch_disable_partition_proc_features(
      union hv_partition_processor_features *disabled_features);
 int mshv_arch_post_init_vm(int vm_fd);
-
+int mshv_get_vp_state(int cpu_fd, struct mshv_get_set_vp_state *state);
+int mshv_set_vp_state(int cpu_fd, const struct mshv_get_set_vp_state *state);
 typedef struct mshv_root_hvcall mshv_root_hvcall;
 int mshv_hvcall(int fd, const mshv_root_hvcall *args);
 
+/* apic */
+int mshv_init_lint(CPUState *cpu);
+int mshv_set_lapic(const CPUState *cpu);
+int mshv_get_lapic(CPUState *cpu);
+
 /* memory */
 typedef struct MshvMemoryRegion {
     uint64_t guest_phys_addr;
target/i386/cpu-apic.c +3 -0
index 04b7257ad1..b4cf048a7c 100644
@@ -14,6 +14,7 @@
 #include "system/hw_accel.h"
 #include "system/kvm.h"
 #include "system/xen.h"
+#include "system/mshv.h"
 #include "system/address-spaces.h"
 #include "hw/core/qdev-properties.h"
 #include "hw/i386/apic_internal.h"
@@ -34,6 +35,8 @@ APICCommonClass *apic_get_class(Error **errp)
         apic_type = "xen-apic";
     } else if (whpx_irqchip_in_kernel()) {
         apic_type = "whpx-apic";
+    } else if (mshv_enabled()) {
+        apic_type = "mshv-apic";
     }
 
     return APIC_COMMON_CLASS(object_class_by_name(apic_type));
target/i386/mshv/mshv-cpu.c +39 -112

187 lines not on this page · raw message

target/i386/mshv/msr.c +2 -0
index 8c220a9942..a5f639c3ca 100644
@@ -60,6 +60,8 @@ static const MshvMsrEnvMap msr_env_map[] = {
                              offsetof(CPUX86State, tsc_aux) },
     { IA32_MSR_TSC_ADJUST,   HV_X64_REGISTER_TSC_ADJUST,
                              offsetof(CPUX86State, tsc_adjust) },
+    { IA32_MSR_TSC_DEADLINE, HV_X64_REGISTER_TSC_DEADLINE,
+                             offsetof(CPUX86State, tsc_deadline) },
 
     /* Hyper-V per-partition MSRs */
     { HV_X64_MSR_HYPERCALL,     HV_X64_REGISTER_HYPERCALL,
Signature · 1 line
2.34.1
Reviewed-by on v1 5/12
#10 lore raw

Re: [PATCH 05/12] target/i386/mshv: migrate LAPIC state

Quoted 805 lines · Magnus Kulke

This change implements loading and storing the hyperv lapic state as part of the load/store routines for a vcpu.

The HyperV LAPIC is similar to the the split-irqchip in KVM. MSHV currently keeps PIC/IOAPIC emulation in userspace, while LAPIC interrupt injection is handled through hypercalls.

We introduced dedicated apic infra in hw/i386/mshv to handle the migration and move lapic related functions from target/i386/mshv there. References have been the WHPX's whpx-apic implemenation and the mshv-ioctls crate's get_/set_lapic() impl for the mapping between MSHV/QEMU lapic state.

We are mapping the lapic state that we receive from the hypervisor to fields in APICCommonState. Common fields are used where feasible, with an mshv-specific MshvAPICState object that carries mshv-specific fields.

We have introduced a guard in pic_irq_request() that will early exit for the mshv accelerator, because mshv cannot take part in the userland path for legacy PIC interrupt injection.

The TSC_DEADLINE MSR is also migrated as part of LAPIC migration.

---
 accel/mshv/mshv-all.c          |   1 +
 hw/i386/meson.build            |   2 +-
 hw/i386/mshv/apic.c            | 395 +++++++++++++++++++++++++++++++++
 hw/i386/mshv/meson.build       |   1 +
 hw/i386/x86-cpu.c              |   6 +
 include/hw/hyperv/hvgdk_mini.h |   2 +
 include/hw/i386/apic-msidef.h  |   1 +
 include/system/mshv.h          |   2 +
 include/system/mshv_int.h      |   8 +-
 target/i386/cpu-apic.c         |   3 +
 target/i386/mshv/mshv-cpu.c    | 151 ++++---------
 target/i386/mshv/msr.c         |   2 +
 12 files changed, 460 insertions(+), 114 deletions(-)
 create mode 100644 hw/i386/mshv/apic.c
accel/mshv/mshv-all.c +1 -0
index f9511ca050..1516475f34 100644
@@ -813,6 +813,7 @@ static void mshv_accel_ops_class_init(ObjectClass *oc, const void *data)
     ops->synchronize_state = mshv_cpu_synchronize;
     ops->synchronize_pre_loadvm = mshv_cpu_synchronize_pre_loadvm;
     ops->cpus_are_resettable = mshv_cpus_are_resettable;
+    ops->cpu_thread_is_idle = mshv_vcpu_thread_is_idle;
     ops->handle_interrupt = generic_handle_interrupt;
 }
 
hw/i386/meson.build +1 -1
index f4d3122863..39ac8c9edc 100644
@@ -38,8 +38,8 @@ i386_ss.add(when: 'CONFIG_X86_FW_OVMF', if_true: files('pc_sysfw_ovmf.c'),
 i386_ss.add(when: 'CONFIG_TDX', if_true: files('tdvf.c', 'tdvf-hob.c'))
 
 subdir('kvm')
-subdir('xen')
 subdir('mshv')
+subdir('xen')
 
 i386_ss.add_all(xenpv_ss)
 
hw/i386/mshv/apic.c new file +395 -0
new file mode 100644
index 0000000000..9188a93b2f
@@ -0,0 +1,395 @@
+/*
+ * MSHV in-kernel APIC support
+ *
+ * Copyright Microsoft, Corp. 2026
+ *
+ * Authors: Magnus Kulke <[email protected]>
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/module.h"
+#include "qemu/memalign.h"
+#include "qemu/error-report.h"
+#include "hw/i386/apic_internal.h"
+#include "hw/i386/apic-msidef.h"
+#include "hw/pci/msi.h"
+#include "migration/vmstate.h"
+#include "qemu/typedefs.h"
+#include "system/hw_accel.h"
+#include "system/mshv.h"
+#include "system/mshv_int.h"
+
+typedef struct hv_local_interrupt_controller_state
+               hv_local_interrupt_controller_state;
+
+#define TYPE_MSHV_APIC "mshv-apic"
+OBJECT_DECLARE_SIMPLE_TYPE(MshvAPICState, MSHV_APIC)
+
+struct MshvAPICState {
+    APICCommonState parent_obj;
+
+    uint32_t apic_version;
+    uint32_t apic_lvt_cmci;
+    uint32_t apic_error_status;
+    uint32_t apic_counter_value;
+    uint32_t apic_remote_read;
+};
+
+static int get_lapic(int cpu_fd,
+                     struct hv_local_interrupt_controller_state *state)
+{
+    int ret;
+    size_t size = 4096;
+    /* buffer aligned to 4k, as *state requires that */
+    void *buffer = qemu_memalign(size, size);
+    struct mshv_get_set_vp_state mshv_state = { 0 };
+
+    mshv_state.buf_ptr = (uint64_t) buffer;
+    mshv_state.buf_sz = size;
+    mshv_state.type = MSHV_VP_STATE_LAPIC;
+
+    ret = mshv_get_vp_state(cpu_fd, &mshv_state);
+    if (ret == 0) {
+        memcpy(state, buffer, sizeof(*state));
+    }
+    qemu_vfree(buffer);
+    if (ret < 0) {
+        error_report("failed to get lapic");
+        return -1;
+    }
+
+    return 0;
+}
+
+static int set_lapic(int cpu_fd,
+                     const struct hv_local_interrupt_controller_state *state)
+{
+    int ret;
+    size_t size = 4096;
+    /* buffer aligned to 4k, as *state requires that */
+    void *buffer = qemu_memalign(size, size);
+    struct mshv_get_set_vp_state mshv_state = { 0 };
+
+    if (!state) {
+        error_report("lapic state is NULL");
+        return -1;
+    }
+    memcpy(buffer, state, sizeof(*state));
+
+    mshv_state.buf_ptr = (uint64_t) buffer;
+    mshv_state.buf_sz = size;
+    mshv_state.type = MSHV_VP_STATE_LAPIC;
+
+    ret = mshv_set_vp_state(cpu_fd, &mshv_state);
+    qemu_vfree(buffer);
+    if (ret < 0) {
+        error_report("failed to set lapic: %s", strerror(errno));
+        return -1;
+    }
+
+    return 0;
+}
+
+static void populate_apic_state(CPUState *cpu,
+                                const hv_local_interrupt_controller_state *hv)
+{
+    X86CPU *x86cpu = X86_CPU(cpu);
+    MshvAPICState *ms = MSHV_APIC(x86cpu->apic_state);
+    APICCommonState *s = &ms->parent_obj;
+    size_t i;
+
+    /*
+     * x2APIC:
+     * - APIC ID is the full 32-bit initial_apic_id
+     * - LDR is read-only, architecturally derived from the ID
+     * - DFR does not exist in x2APIC mode
+     */
+    if (is_x2apic_mode(s)) {
+        s->initial_apic_id = hv->apic_id;
+    } else {
+        s->id        = hv->apic_id  >> 24;
+        s->log_dest  = hv->apic_ldr >> 24;
+        s->dest_mode = hv->apic_dfr >> 28;
+    }
+    ms->apic_version = hv->apic_version;
+    s->spurious_vec = hv->apic_spurious;
+    for (i = 0; i < 8; i++) {
+        s->isr[i] = hv->apic_isr[i];
+        s->tmr[i] = hv->apic_tmr[i];
+        s->irr[i] = hv->apic_irr[i];
+    }
+    s->esr = hv->apic_esr;
+    s->icr[1] = hv->apic_icr_high;
+    s->icr[0] = hv->apic_icr_low;
+
+    s->lvt[APIC_LVT_TIMER]   = hv->apic_lvt_timer;
+    s->lvt[APIC_LVT_THERMAL] = hv->apic_lvt_thermal;
+    s->lvt[APIC_LVT_PERFORM] = hv->apic_lvt_perfmon;
+    s->lvt[APIC_LVT_LINT0]   = hv->apic_lvt_lint0;
+    s->lvt[APIC_LVT_LINT1]   = hv->apic_lvt_lint1;
+    s->lvt[APIC_LVT_ERROR]   = hv->apic_lvt_error;
+    ms->apic_lvt_cmci        = hv->apic_lvt_cmci;
+
+    ms->apic_error_status  = hv->apic_error_status;
+    s->initial_count       = hv->apic_initial_count;
+    ms->apic_counter_value = hv->apic_counter_value;
+    s->divide_conf         = hv->apic_divide_configuration;
+    ms->apic_remote_read   = hv->apic_remote_read;
+}
+
+static uint32_t set_apic_delivery_mode(uint32_t reg, uint32_t mode)
+{
+    return ((reg) & ~0x700) | ((mode) << 8);
+}
+
+int mshv_init_lint(CPUState *cpu)
+{
+    uint32_t *lvt_lint0, *lvt_lint1;
+    int cpu_fd = mshv_vcpufd(cpu);
+    int ret;
+    struct hv_local_interrupt_controller_state lapic_state = { 0 };
+
+    ret = get_lapic(cpu_fd, &lapic_state);
+    if (ret < 0) {
+        return ret;
+    }
+
+    lvt_lint0 = &lapic_state.apic_lvt_lint0;
+    *lvt_lint0 = set_apic_delivery_mode(*lvt_lint0, APIC_DM_EXTINT);
+
+    lvt_lint1 = &lapic_state.apic_lvt_lint1;
+    *lvt_lint1 = set_apic_delivery_mode(*lvt_lint1, APIC_DM_NMI);
+
+    /* TODO: should we skip setting lapic if the values are the same? */
+
+    ret = set_lapic(cpu_fd, &lapic_state);
+    if (ret < 0) {
+        return -1;
+    }
+
+    populate_apic_state(cpu, &lapic_state);
+
+    return 0;
+}
+
+static void populate_hv_lapic_state(hv_local_interrupt_controller_state *hv,
+                                    const CPUState *cpu)
+{
+    uint32_t x2apic_id;
+    X86CPU *x86cpu = X86_CPU(cpu);
+    MshvAPICState *ms = MSHV_APIC(x86cpu->apic_state);
+    APICCommonState *s = &ms->parent_obj;
+    size_t i;
+
+    /*
+     * x2APIC:
+     * - APIC ID is the full 32-bit initial_apic_id
+     * - LDR is read-only, architecturally derived from the ID
+     * - DFR does not exist in x2APIC mode
+     */
+    if (is_x2apic_mode(s)) {
+        x2apic_id = s->initial_apic_id;
+
+        hv->apic_id  = x2apic_id;
+        hv->apic_ldr = ((x2apic_id >> 4) << 16) | (1 << (x2apic_id & 0xf));
+        hv->apic_dfr = 0;
+    } else {
+        hv->apic_id  = s->id << 24;
+        hv->apic_ldr = s->log_dest << 24;
+        hv->apic_dfr = s->dest_mode << 28 | 0x0fffffff;
+    }
+    hv->apic_version = ms->apic_version;
+    hv->apic_spurious = s->spurious_vec;
+    for (i = 0; i < 8; i++) {
+        hv->apic_isr[i] = s->isr[i];
+        hv->apic_tmr[i] = s->tmr[i];
+        hv->apic_irr[i] = s->irr[i];
+    }
+    hv->apic_esr      = s->esr;
+    hv->apic_icr_high = s->icr[1];
+    hv->apic_icr_low  = s->icr[0];
+
+    hv->apic_lvt_timer   = s->lvt[APIC_LVT_TIMER];
+    hv->apic_lvt_thermal = s->lvt[APIC_LVT_THERMAL];
+    hv->apic_lvt_perfmon = s->lvt[APIC_LVT_PERFORM];
+    hv->apic_lvt_lint0   = s->lvt[APIC_LVT_LINT0];
+    hv->apic_lvt_lint1   = s->lvt[APIC_LVT_LINT1];
+    hv->apic_lvt_error   = s->lvt[APIC_LVT_ERROR];
+    hv->apic_lvt_cmci    = ms->apic_lvt_cmci;
+
+    hv->apic_error_status         = ms->apic_error_status;
+    hv->apic_initial_count        = s->initial_count;
+    hv->apic_counter_value        = ms->apic_counter_value;
+    hv->apic_divide_configuration = s->divide_conf;
+    hv->apic_remote_read          = ms->apic_remote_read;
+}
+
+int mshv_set_lapic(const CPUState *cpu)
+{
+    int cpu_fd = mshv_vcpufd(cpu);
+    struct hv_local_interrupt_controller_state lapic_state = { 0 };
+
+    populate_hv_lapic_state(&lapic_state, cpu);
+
+    return set_lapic(cpu_fd, &lapic_state);
+}
+
+int mshv_get_lapic(CPUState *cpu)
+{
+    int cpu_fd = mshv_vcpufd(cpu);
+    int ret;
+    struct hv_local_interrupt_controller_state lapic_state = { 0 };
+
+    ret = get_lapic(cpu_fd, &lapic_state);
+    if (ret < 0) {
+        return -1;
+    }
+
+    populate_apic_state(cpu, &lapic_state);
+
+    return 0;
+}
+
+static int mshv_apic_set_base(APICCommonState *s, uint64_t val)
+{
+    s->apicbase = val;
+
+    return 0;
+}
+
+static void mshv_apic_set_tpr(APICCommonState *s, uint8_t val)
+{
+    s->tpr = (val & APIC_PR_SUB_CLASS) << APIC_PR_CLASS_SHIFT;
+}
+
+static uint8_t mshv_apic_get_tpr(APICCommonState *s)
+{
+    return s->tpr >> APIC_PR_CLASS_SHIFT;
+}
+
+static void mshv_apic_external_nmi(APICCommonState *s)
+{
+}
+
+static void mshv_apic_vapic_base_update(APICCommonState *s)
+{
+}
+
+static void mshv_send_msi(MSIMessage *msi)
+{
+    uint64_t addr;
+    uint32_t data, dest;
+    uint8_t vector, dest_mode, trigger_mode, delivery;
+
+    addr         = msi->address;
+    data         = msi->data;
+    dest         = (addr & MSI_ADDR_DEST_ID_MASK) >> MSI_ADDR_DEST_ID_SHIFT |
+                   (addr >> 32);
+    vector       = (data & MSI_DATA_VECTOR_MASK) >> MSI_DATA_VECTOR_SHIFT;
+    dest_mode    = (addr >> MSI_ADDR_DEST_MODE_SHIFT) & 0x1;
+    trigger_mode = (data >> MSI_DATA_TRIGGER_SHIFT) & 0x1;
+    delivery     = (data >> MSI_DATA_DELIVERY_MODE_SHIFT) &
+                   MSI_DATA_DELIVERY_MODE_MASK;
+
+    mshv_request_interrupt(mshv_state, delivery, vector, dest, dest_mode,
+                           trigger_mode);
+}
+
+static uint64_t mshv_apic_mem_read(void *opaque, hwaddr addr,
+                                   unsigned size)
+{
+    return UINT64_MAX;
+}
+
+static void mshv_apic_mem_write(void *opaque, hwaddr addr,
+                                uint64_t data, unsigned size)
+{
+    MSIMessage msg = { .address = addr, .data = data };
+
+    mshv_send_msi(&msg);
+}
+
+static const MemoryRegionOps mshv_apic_io_ops = {
+    .read = mshv_apic_mem_read,
+    .write = mshv_apic_mem_write,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+};
+
+static void mshv_apic_reset(APICCommonState *s)
+{
+    s->wait_for_sipi = 0;
+}
+
+static const VMStateDescription vmstate_mshv_apic = {
+    .name = "mshv-apic",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .fields = (const VMStateField[]) {
+        VMSTATE_UINT32(apic_version, MshvAPICState),
+        VMSTATE_UINT32(apic_lvt_cmci, MshvAPICState),
+        VMSTATE_UINT32(apic_error_status, MshvAPICState),
+        VMSTATE_UINT32(apic_counter_value, MshvAPICState),
+        VMSTATE_UINT32(apic_remote_read, MshvAPICState),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+static void mshv_apic_realize(DeviceState *dev, Error **errp)
+{
+    APICCommonState *s = APIC_COMMON(dev);
+    MshvAPICState *ms = MSHV_APIC(dev);
+
+    memory_region_init_io(&s->io_memory, OBJECT(s), &mshv_apic_io_ops, s,
+                          "mshv-apic-msi", APIC_SPACE_SIZE);
+
+    msi_nonbroken = true;
+
+    /*
+     * We register this state explicity, rather than going via dc->vmsd.
+     * The auto-wiring would register the state with
+     * instance_id == VMSTATE_INSTANCE_ID_ANY, which for the APIC doesn't
+     * work, b/c the ID carries semantic meaning for restoring the state
+     * on the destination (which vcpu it belongs to).
+     */
+    vmstate_register_with_alias_id(NULL,
+                                   s->initial_apic_id, &vmstate_mshv_apic, ms,
+                                   -1, 0, NULL);
+}
+
+static void mshv_apic_unrealize(DeviceState *dev)
+{
+    MshvAPICState *ms = MSHV_APIC(dev);
+
+    vmstate_unregister(NULL, &vmstate_mshv_apic, ms);
+}
+
+static void mshv_apic_class_init(ObjectClass *klass, const void *data)
+{
+    APICCommonClass *k = APIC_COMMON_CLASS(klass);
+
+    k->realize = mshv_apic_realize;
+    k->unrealize = mshv_apic_unrealize;
+    k->reset = mshv_apic_reset;
+    k->set_base = mshv_apic_set_base;
+    k->set_tpr = mshv_apic_set_tpr;
+    k->get_tpr = mshv_apic_get_tpr;
+    k->external_nmi = mshv_apic_external_nmi;
+    k->vapic_base_update = mshv_apic_vapic_base_update;
+    k->send_msi = mshv_send_msi;
+}
+
+static const TypeInfo mshv_apic_info = {
+    .name = TYPE_MSHV_APIC,
+    .parent = TYPE_APIC_COMMON,
+    .instance_size = sizeof(MshvAPICState),
+    .class_init = mshv_apic_class_init,
+};
+
+static void mshv_apic_register_types(void)
+{
+    type_register_static(&mshv_apic_info);
+}
+
+type_init(mshv_apic_register_types)
hw/i386/mshv/meson.build +1 -0
index 0e556851b6..c631ee1302 100644
@@ -1,4 +1,5 @@
 i386_mshv_ss = ss.source_set()
 i386_mshv_ss.add(files('clock.c'))
+i386_mshv_ss.add(when: 'CONFIG_APIC', if_true: files('apic.c'))
 
 i386_ss.add_all(when: 'CONFIG_MSHV', if_true: i386_mshv_ss)
hw/i386/x86-cpu.c +6 -0
index 95e08e3c2a..fba313376c 100644
@@ -22,6 +22,7 @@
  */
 #include "qemu/osdep.h"
 #include "system/whpx.h"
+#include "system/mshv.h"
 #include "system/cpu-timers.h"
 #include "trace.h"
 
@@ -44,6 +45,11 @@ static void pic_irq_request(void *opaque, int irq, int level)
     X86CPU *cpu = X86_CPU(cs);
 
     trace_x86_pic_interrupt(irq, level);
+
+    if (mshv_irqchip_in_kernel()) {
+        return;
+    }
+
     if (cpu_is_apic_enabled(cpu->apic_state) && !kvm_irqchip_in_kernel() &&
         !whpx_irqchip_in_kernel()) {
         CPU_FOREACH(cs) {
include/hw/hyperv/hvgdk_mini.h +2 -0
index f8838a31bb..0602a7c6cc 100644
@@ -168,6 +168,7 @@ typedef enum hv_register_name {
     /* Available */
 
     HV_X64_REGISTER_SPEC_CTRL       = 0x00080084,
+    HV_X64_REGISTER_TSC_DEADLINE    = 0x00080095,
     HV_X64_REGISTER_TSC_ADJUST      = 0x00080096,
 
     /* CET / Shadow Stack */
@@ -930,6 +931,7 @@ struct hv_cpuid {
 #define IA32_MSR_DEBUG_CTL        0x1D9
 #define IA32_MSR_SPEC_CTRL        0x00000048
 #define IA32_MSR_TSC_ADJUST       0x0000003b
+#define IA32_MSR_TSC_DEADLINE     0x000006e0
 
 #define IA32_MSR_MISC_ENABLE 0x000001a0
 
include/hw/i386/apic-msidef.h +1 -0
index 420b41167d..6b860b5807 100644
@@ -13,6 +13,7 @@
 #define  MSI_DATA_VECTOR_MASK           0x000000ff
 
 #define MSI_DATA_DELIVERY_MODE_SHIFT    8
+#define MSI_DATA_DELIVERY_MODE_MASK     7
 #define MSI_DATA_LEVEL_SHIFT            14
 #define MSI_DATA_TRIGGER_SHIFT          15
 
include/system/mshv.h +2 -0
index a6f815b822..f98ddff9b5 100644
@@ -41,9 +41,11 @@
 extern bool mshv_allowed;
 #define mshv_enabled() (mshv_allowed)
 #define mshv_msi_via_irqfd_enabled() mshv_enabled()
+#define mshv_irqchip_in_kernel() mshv_enabled()
 #else /* CONFIG_MSHV_IS_POSSIBLE */
 #define mshv_enabled() false
 #define mshv_msi_via_irqfd_enabled() mshv_enabled()
+#define mshv_irqchip_in_kernel() mshv_enabled()
 #endif
 
 #define TYPE_MSHV_ACCEL ACCEL_CLASS_NAME("mshv")
include/system/mshv_int.h +7 -1
index b91c4d661a..c2f13c0194 100644
@@ -105,10 +105,16 @@ void mshv_arch_amend_proc_features(
 void mshv_arch_disable_partition_proc_features(
      union hv_partition_processor_features *disabled_features);
 int mshv_arch_post_init_vm(int vm_fd);
-
+int mshv_get_vp_state(int cpu_fd, struct mshv_get_set_vp_state *state);
+int mshv_set_vp_state(int cpu_fd, const struct mshv_get_set_vp_state *state);
 typedef struct mshv_root_hvcall mshv_root_hvcall;
 int mshv_hvcall(int fd, const mshv_root_hvcall *args);
 
+/* apic */
+int mshv_init_lint(CPUState *cpu);
+int mshv_set_lapic(const CPUState *cpu);
+int mshv_get_lapic(CPUState *cpu);
+
 /* memory */
 typedef struct MshvMemoryRegion {
     uint64_t guest_phys_addr;
target/i386/cpu-apic.c +3 -0
index 04b7257ad1..b4cf048a7c 100644
@@ -14,6 +14,7 @@
 #include "system/hw_accel.h"
 #include "system/kvm.h"
 #include "system/xen.h"
+#include "system/mshv.h"
 #include "system/address-spaces.h"
 #include "hw/core/qdev-properties.h"
 #include "hw/i386/apic_internal.h"
@@ -34,6 +35,8 @@ APICCommonClass *apic_get_class(Error **errp)
         apic_type = "xen-apic";
     } else if (whpx_irqchip_in_kernel()) {
         apic_type = "whpx-apic";
+    } else if (mshv_enabled()) {
+        apic_type = "mshv-apic";
     }
 
     return APIC_COMMON_CLASS(object_class_by_name(apic_type));
target/i386/mshv/mshv-cpu.c +39 -112
index 8eca01a8fa..333ee35e72 100644
@@ -21,7 +21,6 @@
 #include "hw/hyperv/hvgdk.h"
 #include "hw/hyperv/hvgdk_mini.h"
 #include "hw/hyperv/hvhdk_mini.h"
-#include "hw/i386/apic_internal.h"
 
 #include "cpu.h"
 #include "host-cpu.h"
@@ -955,6 +954,11 @@ int mshv_arch_load_vcpu_state(CPUState *cpu)
         return ret;
     }
 
+    ret = mshv_get_lapic(cpu);
+    if (ret < 0) {
+        return ret;
+    }
+
     ret = mshv_get_msrs(cpu);
     if (ret < 0) {
         return ret;
@@ -1377,116 +1381,6 @@ static int set_xc_reg(const CPUState *cpu)
     return 0;
 }
 
-static int get_vp_state(int cpu_fd, struct mshv_get_set_vp_state *state)
-{
-    int ret;
-
-    ret = ioctl(cpu_fd, MSHV_GET_VP_STATE, state);
-    if (ret < 0) {
-        error_report("failed to get partition state: %s", strerror(errno));
-        return -1;
-    }
-
-    return 0;
-}
-
-static int get_lapic(const CPUState *cpu,
-                     struct hv_local_interrupt_controller_state *state)
-{
-    int ret;
-    size_t size = 4096;
-    /* buffer aligned to 4k, as *state requires that */
-    void *buffer = qemu_memalign(size, size);
-    struct mshv_get_set_vp_state mshv_state = { 0 };
-    int cpu_fd = mshv_vcpufd(cpu);
-
-    mshv_state.buf_ptr = (uint64_t) buffer;
-    mshv_state.buf_sz = size;
-    mshv_state.type = MSHV_VP_STATE_LAPIC;
-
-    ret = get_vp_state(cpu_fd, &mshv_state);
-    if (ret == 0) {
-        memcpy(state, buffer, sizeof(*state));
-    }
-    qemu_vfree(buffer);
-    if (ret < 0) {
-        error_report("failed to get lapic");
-        return -1;
-    }
-
-    return 0;
-}
-
-static uint32_t set_apic_delivery_mode(uint32_t reg, uint32_t mode)
-{
-    return ((reg) & ~0x700) | ((mode) << 8);
-}
-
-static int set_vp_state(int cpu_fd, const struct mshv_get_set_vp_state *state)
-{
-    int ret;
-
-    ret = ioctl(cpu_fd, MSHV_SET_VP_STATE, state);
-    if (ret < 0) {
-        error_report("failed to set partition state: %s", strerror(errno));
-        return -1;
-    }
-
-    return 0;
-}
-
-static int set_lapic(const CPUState *cpu,
-                     const struct hv_local_interrupt_controller_state *state)
-{
-    int ret;
-    size_t size = 4096;
-    /* buffer aligned to 4k, as *state requires that */
-    void *buffer = qemu_memalign(size, size);
-    struct mshv_get_set_vp_state mshv_state = { 0 };
-    int cpu_fd = mshv_vcpufd(cpu);
-
-    if (!state) {
-        error_report("lapic state is NULL");
-        return -1;
-    }
-    memcpy(buffer, state, sizeof(*state));
-
-    mshv_state.buf_ptr = (uint64_t) buffer;
-    mshv_state.buf_sz = size;
-    mshv_state.type = MSHV_VP_STATE_LAPIC;
-
-    ret = set_vp_state(cpu_fd, &mshv_state);
-    qemu_vfree(buffer);
-    if (ret < 0) {
-        error_report("failed to set lapic: %s", strerror(errno));
-        return -1;
-    }
-
-    return 0;
-}
-
-static int init_lint(const CPUState *cpu)
-{
-    int ret;
-    uint32_t *lvt_lint0, *lvt_lint1;
-
-    struct hv_local_interrupt_controller_state lapic_state = { 0 };
-    ret = get_lapic(cpu, &lapic_state);
-    if (ret < 0) {
-        return ret;
-    }
-
-    lvt_lint0 = &lapic_state.apic_lvt_lint0;
-    *lvt_lint0 = set_apic_delivery_mode(*lvt_lint0, APIC_DM_EXTINT);
-
-    lvt_lint1 = &lapic_state.apic_lvt_lint1;
-    *lvt_lint1 = set_apic_delivery_mode(*lvt_lint1, APIC_DM_NMI);
-
-    /* TODO: should we skip setting lapic if the values are the same? */
-
-    return set_lapic(cpu, &lapic_state);
-}
-
 int mshv_arch_store_vcpu_state(const CPUState *cpu)
 {
     int ret;
@@ -1511,6 +1405,12 @@ int mshv_arch_store_vcpu_state(const CPUState *cpu)
         return ret;
     }
 
+    /* INVARIANT: special regs (APIC_BASE) must be restored before LAPIC */
+    ret = mshv_set_lapic(cpu);
+    if (ret < 0) {
+        return ret;
+    }
+
     ret = mshv_set_msrs(cpu);
     if (ret < 0) {
         return ret;
@@ -2099,7 +1999,7 @@ void mshv_arch_init_vcpu(CPUState *cpu)
     ret = mshv_init_msrs(cpu);
     assert(ret == 0);
 
-    ret = init_lint(cpu);
+    ret = mshv_init_lint(cpu);
     assert(ret == 0);
 }
 
@@ -2246,6 +2146,33 @@ static void mshv_cpu_xsave_init(void)
     }
 }
 
+int mshv_set_vp_state(int cpu_fd, const struct mshv_get_set_vp_state *state)
+{
+    int ret;
+
+    ret = ioctl(cpu_fd, MSHV_SET_VP_STATE, state);
+    if (ret < 0) {
+        error_report("failed to set partition state: %s", strerror(errno));
+        return -1;
+    }
+
+    return 0;
+}
+
+
+int mshv_get_vp_state(int cpu_fd, struct mshv_get_set_vp_state *state)
+{
+    int ret;
+
+    ret = ioctl(cpu_fd, MSHV_GET_VP_STATE, state);
+    if (ret < 0) {
+        error_report("failed to get partition state: %s", strerror(errno));
+        return -1;
+    }
+
+    return 0;
+}
+
 static void mshv_cpu_instance_init(CPUState *cs)
 {
     X86CPU *cpu = X86_CPU(cs);
target/i386/mshv/msr.c +2 -0
index 8c220a9942..a5f639c3ca 100644
@@ -60,6 +60,8 @@ static const MshvMsrEnvMap msr_env_map[] = {
                              offsetof(CPUX86State, tsc_aux) },
     { IA32_MSR_TSC_ADJUST,   HV_X64_REGISTER_TSC_ADJUST,
                              offsetof(CPUX86State, tsc_adjust) },
+    { IA32_MSR_TSC_DEADLINE, HV_X64_REGISTER_TSC_DEADLINE,
+                             offsetof(CPUX86State, tsc_deadline) },
 
     /* Hyper-V per-partition MSRs */
     { HV_X64_MSR_HYPERCALL,     HV_X64_REGISTER_HYPERCALL,
Signature · 1 line
2.34.1
#11 lore raw

PATCH 6/12 target/i386/mshv: migrate Synic SINT MSRs

In reply to Magnus Kulke

Migrate HyperV SynIC SINT MSRs. We can only read/write those if SCONTROL is enabled in the guest, hence we have to split the SINT MSR out and make reading/writing them dependent on that MSR.

---
 target/i386/mshv/msr.c | 41 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)
target/i386/mshv/msr.c +41 -0
index a5f639c3ca..5288e32b6d 100644
@@ -331,6 +331,8 @@ int mshv_get_msrs(CPUState *cpu)
     struct hv_register_assoc assocs[MSHV_MSR_TOTAL_COUNT];
     size_t i, j;
     uint32_t name;
+    X86CPU *x86cpu = X86_CPU(cpu);
+    bool synic_enabled;
 
     set_hv_name_in_assocs(assocs, n_assocs);
 
@@ -357,6 +359,27 @@ int mshv_get_msrs(CPUState *cpu)
 
     store_in_env(cpu, assocs, n_assocs);
 
+    /* Read SINT MSRs only if SynIC is enabled */
+    synic_enabled = x86cpu->env.msr_hv_synic_control & 1;
+    if (synic_enabled) {
+        QEMU_BUILD_BUG_ON(MSHV_MSR_TOTAL_COUNT < HV_SINT_COUNT);
+
+        for (i = 0; i < HV_SINT_COUNT; i++) {
+            assocs[i].name = HV_REGISTER_SINT0 + i;
+        }
+
+        ret = mshv_get_generic_regs(cpu, assocs, HV_SINT_COUNT);
+        if (ret < 0) {
+            error_report("Failed to get SynIC SINT MSRs");
+            return -errno;
+        }
+
+        for (i = 0; i < HV_SINT_COUNT; i++) {
+            uint64_t hv_sint_value = assocs[i].value.reg64;
+            x86cpu->env.msr_hv_synic_sint[i] = hv_sint_value;
+        }
+    }
+
     return 0;
 }
 
@@ -391,6 +414,8 @@ int mshv_set_msrs(const CPUState *cpu)
     struct hv_register_assoc assocs[MSHV_MSR_TOTAL_COUNT];
     int ret;
     size_t i, j;
+    X86CPU *x86cpu = X86_CPU(cpu);
+    bool synic_enabled = x86cpu->env.msr_hv_synic_control & 1;
 
     load_from_env(cpu, assocs, n_assocs);
 
@@ -423,5 +448,21 @@ int mshv_set_msrs(const CPUState *cpu)
         return -errno;
     }
 
+    /* SINT MSRs can only be written if SCONTROL has been set, so we split */
+    if (synic_enabled) {
+        QEMU_BUILD_BUG_ON(MSHV_MSR_TOTAL_COUNT < HV_SINT_COUNT);
+
+        for (i = 0; i < HV_SINT_COUNT; i++) {
+            assocs[i].name        = HV_REGISTER_SINT0 + i;
+            assocs[i].value.reg64 = x86cpu->env.msr_hv_synic_sint[i];
+        }
+
+        ret = mshv_set_generic_regs(cpu, assocs, HV_SINT_COUNT);
+        if (ret < 0) {
+            error_report("Failed to set SynIC SINT MSRs");
+            return -errno;
+        }
+    }
+
     return 0;
 }
Signature · 1 line
2.34.1
Reviewed-by on v1 6/12
#12 lore raw

Re: [PATCH 06/12] target/i386/mshv: migrate Synic SINT MSRs

Quoted 83 lines · Magnus Kulke

Migrate HyperV SynIC SINT MSRs. We can only read/write those if SCONTROL is enabled in the guest, hence we have to split the SINT MSR out and make reading/writing them dependent on that MSR.

---
 target/i386/mshv/msr.c | 41 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)
target/i386/mshv/msr.c +41 -0
index a5f639c3ca..5288e32b6d 100644
@@ -331,6 +331,8 @@ int mshv_get_msrs(CPUState *cpu)
     struct hv_register_assoc assocs[MSHV_MSR_TOTAL_COUNT];
     size_t i, j;
     uint32_t name;
+    X86CPU *x86cpu = X86_CPU(cpu);
+    bool synic_enabled;
 
     set_hv_name_in_assocs(assocs, n_assocs);
 
@@ -357,6 +359,27 @@ int mshv_get_msrs(CPUState *cpu)
 
     store_in_env(cpu, assocs, n_assocs);
 
+    /* Read SINT MSRs only if SynIC is enabled */
+    synic_enabled = x86cpu->env.msr_hv_synic_control & 1;
+    if (synic_enabled) {
+        QEMU_BUILD_BUG_ON(MSHV_MSR_TOTAL_COUNT < HV_SINT_COUNT);
+
+        for (i = 0; i < HV_SINT_COUNT; i++) {
+            assocs[i].name = HV_REGISTER_SINT0 + i;
+        }
+
+        ret = mshv_get_generic_regs(cpu, assocs, HV_SINT_COUNT);
+        if (ret < 0) {
+            error_report("Failed to get SynIC SINT MSRs");
+            return -errno;
+        }
+
+        for (i = 0; i < HV_SINT_COUNT; i++) {
+            uint64_t hv_sint_value = assocs[i].value.reg64;
+            x86cpu->env.msr_hv_synic_sint[i] = hv_sint_value;
+        }
+    }
+
     return 0;
 }
 
@@ -391,6 +414,8 @@ int mshv_set_msrs(const CPUState *cpu)
     struct hv_register_assoc assocs[MSHV_MSR_TOTAL_COUNT];
     int ret;
     size_t i, j;
+    X86CPU *x86cpu = X86_CPU(cpu);
+    bool synic_enabled = x86cpu->env.msr_hv_synic_control & 1;
 
     load_from_env(cpu, assocs, n_assocs);
 
@@ -423,5 +448,21 @@ int mshv_set_msrs(const CPUState *cpu)
         return -errno;
     }
 
+    /* SINT MSRs can only be written if SCONTROL has been set, so we split */
+    if (synic_enabled) {
+        QEMU_BUILD_BUG_ON(MSHV_MSR_TOTAL_COUNT < HV_SINT_COUNT);
+
+        for (i = 0; i < HV_SINT_COUNT; i++) {
+            assocs[i].name        = HV_REGISTER_SINT0 + i;
+            assocs[i].value.reg64 = x86cpu->env.msr_hv_synic_sint[i];
+        }
+
+        ret = mshv_set_generic_regs(cpu, assocs, HV_SINT_COUNT);
+        if (ret < 0) {
+            error_report("Failed to set SynIC SINT MSRs");
+            return -errno;
+        }
+    }
+
     return 0;
 }
Signature · 1 line
2.34.1
#13 lore raw

PATCH 7/12 target/i386/mshv: migrate SIMP and SIEFP state

In reply to Magnus Kulke

This part SynIC state is retrieved from the hypervisor via aligned state pages:

- Add new synic source file
- Centralize the synic_enabled() check
- r/w pages from the hyper via aligned pages
- only handle pages when synic is enabled
- add buffers for migration to VM state

---
 include/system/mshv_int.h    |   7 ++
 target/i386/cpu.h            |   5 ++
 target/i386/machine.c        |  26 ++++++
 target/i386/mshv/meson.build |   1 +
 target/i386/mshv/mshv-cpu.c  |  64 +++++++++++++++
 target/i386/mshv/msr.c       |   7 +-
 target/i386/mshv/synic.c     | 155 +++++++++++++++++++++++++++++++++++
 7 files changed, 260 insertions(+), 5 deletions(-)
 create mode 100644 target/i386/mshv/synic.c
include/system/mshv_int.h +7 -0
index c2f13c0194..bc023d3535 100644
@@ -138,4 +138,11 @@ int mshv_init_msrs(const CPUState *cpu);
 int mshv_get_msrs(CPUState *cpu);
 int mshv_set_msrs(const CPUState *cpu);
 
+/* synic */
+int mshv_get_simp(int cpu_fd, uint8_t *page);
+int mshv_set_simp(int cpu_fd, const uint8_t *page);
+int mshv_get_siefp(int cpu_fd, uint8_t *page);
+int mshv_set_siefp(int cpu_fd, const uint8_t *page);
+bool mshv_synic_enabled(const CPUState *cpu);
+
 #endif
target/i386/cpu.h +5 -0
index e6a197602d..9270ae95d0 100644
@@ -33,6 +33,7 @@
 #include "qemu/cpu-float.h"
 #include "qemu/timer.h"
 #include "standard-headers/asm-x86/kvm_para.h"
+#include "hw/hyperv/hvgdk_mini.h"
 
 #define XEN_NR_VIRQS 24
 
@@ -2299,6 +2300,10 @@ typedef struct CPUArchState {
 #if defined(CONFIG_HVF) || defined(CONFIG_MSHV) || defined(CONFIG_WHPX)
     void *emu_mmio_buf;
 #endif
+#if defined(CONFIG_MSHV)
+    uint8_t hv_simp_page[HV_HYP_PAGE_SIZE];
+    uint8_t hv_siefp_page[HV_HYP_PAGE_SIZE];
+#endif
 
     uint64_t mcg_cap;
     uint64_t mcg_ctl;
target/i386/machine.c +26 -0
index df0e0c178e..023a397ab8 100644
@@ -951,6 +951,29 @@ static const VMStateDescription vmstate_msr_hyperv_reenlightenment = {
     }
 };
 
+#ifdef CONFIG_MSHV
+static bool mshv_synic_vp_state_needed(void *opaque)
+{
+    X86CPU *cpu = opaque;
+    CPUX86State *env = &cpu->env;
+
+    /* Only migrate SIMP/SIEFP if SynIC is enabled */
+    return env->msr_hv_synic_control & 1;
+}
+
+static const VMStateDescription vmstate_mshv_synic_vp_state = {
+    .name = "cpu/mshv_synic_vp_state",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .needed = mshv_synic_vp_state_needed,
+    .fields = (const VMStateField[]) {
+        VMSTATE_BUFFER(env.hv_simp_page, X86CPU),
+        VMSTATE_BUFFER(env.hv_siefp_page, X86CPU),
+        VMSTATE_END_OF_LIST()
+    }
+};
+#endif
+
 static bool avx512_needed(void *opaque)
 {
     X86CPU *cpu = opaque;
@@ -1915,6 +1938,9 @@ const VMStateDescription vmstate_x86_cpu = {
         &vmstate_cet,
 #ifdef TARGET_X86_64
         &vmstate_apx,
+#endif
+#ifdef CONFIG_MSHV
+        &vmstate_mshv_synic_vp_state,
 #endif
         NULL
     }
target/i386/mshv/meson.build +1 -0
index 6091c21887..31ff4cc995 100644
@@ -3,6 +3,7 @@ i386_mshv_ss = ss.source_set()
 i386_mshv_ss.add(files(
   'mshv-cpu.c',
   'msr.c',
+  'synic.c',
 ))
 
 i386_system_ss.add_all(when: 'CONFIG_MSHV', if_true: i386_mshv_ss)
target/i386/mshv/mshv-cpu.c +64 -0

88 lines not on this page · raw message

target/i386/mshv/msr.c +2 -5
index 5288e32b6d..d3788d7715 100644
@@ -332,7 +332,6 @@ int mshv_get_msrs(CPUState *cpu)
     size_t i, j;
     uint32_t name;
     X86CPU *x86cpu = X86_CPU(cpu);
-    bool synic_enabled;
 
     set_hv_name_in_assocs(assocs, n_assocs);
 
@@ -360,8 +359,7 @@ int mshv_get_msrs(CPUState *cpu)
     store_in_env(cpu, assocs, n_assocs);
 
     /* Read SINT MSRs only if SynIC is enabled */
-    synic_enabled = x86cpu->env.msr_hv_synic_control & 1;
-    if (synic_enabled) {
+    if (mshv_synic_enabled(cpu)) {
         QEMU_BUILD_BUG_ON(MSHV_MSR_TOTAL_COUNT < HV_SINT_COUNT);
 
         for (i = 0; i < HV_SINT_COUNT; i++) {
@@ -415,7 +413,6 @@ int mshv_set_msrs(const CPUState *cpu)
     int ret;
     size_t i, j;
     X86CPU *x86cpu = X86_CPU(cpu);
-    bool synic_enabled = x86cpu->env.msr_hv_synic_control & 1;
 
     load_from_env(cpu, assocs, n_assocs);
 
@@ -449,7 +446,7 @@ int mshv_set_msrs(const CPUState *cpu)
     }
 
     /* SINT MSRs can only be written if SCONTROL has been set, so we split */
-    if (synic_enabled) {
+    if (mshv_synic_enabled(cpu)) {
         QEMU_BUILD_BUG_ON(MSHV_MSR_TOTAL_COUNT < HV_SINT_COUNT);
 
         for (i = 0; i < HV_SINT_COUNT; i++) {
target/i386/mshv/synic.c new file +155 -0

155 lines not on this page · raw message

Signature · 1 line
2.34.1
Reviewed-by on v1 7/12
#14 lore raw

Re: [PATCH 07/12] target/i386/mshv: migrate SIMP and SIEFP state

Quoted 414 lines · Magnus Kulke

This part SynIC state is retrieved from the hypervisor via aligned state pages:

- Add new synic source file
- Centralize the synic_enabled() check
- r/w pages from the hyper via aligned pages
- only handle pages when synic is enabled
- add buffers for migration to VM state

---
 include/system/mshv_int.h    |   7 ++
 target/i386/cpu.h            |   5 ++
 target/i386/machine.c        |  26 ++++++
 target/i386/mshv/meson.build |   1 +
 target/i386/mshv/mshv-cpu.c  |  64 +++++++++++++++
 target/i386/mshv/msr.c       |   7 +-
 target/i386/mshv/synic.c     | 155 +++++++++++++++++++++++++++++++++++
 7 files changed, 260 insertions(+), 5 deletions(-)
 create mode 100644 target/i386/mshv/synic.c
include/system/mshv_int.h +7 -0
index c2f13c0194..bc023d3535 100644
@@ -138,4 +138,11 @@ int mshv_init_msrs(const CPUState *cpu);
 int mshv_get_msrs(CPUState *cpu);
 int mshv_set_msrs(const CPUState *cpu);
 
+/* synic */
+int mshv_get_simp(int cpu_fd, uint8_t *page);
+int mshv_set_simp(int cpu_fd, const uint8_t *page);
+int mshv_get_siefp(int cpu_fd, uint8_t *page);
+int mshv_set_siefp(int cpu_fd, const uint8_t *page);
+bool mshv_synic_enabled(const CPUState *cpu);
+
 #endif
target/i386/cpu.h +5 -0
index e6a197602d..9270ae95d0 100644
@@ -33,6 +33,7 @@
 #include "qemu/cpu-float.h"
 #include "qemu/timer.h"
 #include "standard-headers/asm-x86/kvm_para.h"
+#include "hw/hyperv/hvgdk_mini.h"
 
 #define XEN_NR_VIRQS 24
 
@@ -2299,6 +2300,10 @@ typedef struct CPUArchState {
 #if defined(CONFIG_HVF) || defined(CONFIG_MSHV) || defined(CONFIG_WHPX)
     void *emu_mmio_buf;
 #endif
+#if defined(CONFIG_MSHV)
+    uint8_t hv_simp_page[HV_HYP_PAGE_SIZE];
+    uint8_t hv_siefp_page[HV_HYP_PAGE_SIZE];
+#endif
 
     uint64_t mcg_cap;
     uint64_t mcg_ctl;
target/i386/machine.c +26 -0
index df0e0c178e..023a397ab8 100644
@@ -951,6 +951,29 @@ static const VMStateDescription vmstate_msr_hyperv_reenlightenment = {
     }
 };
 
+#ifdef CONFIG_MSHV
+static bool mshv_synic_vp_state_needed(void *opaque)
+{
+    X86CPU *cpu = opaque;
+    CPUX86State *env = &cpu->env;
+
+    /* Only migrate SIMP/SIEFP if SynIC is enabled */
+    return env->msr_hv_synic_control & 1;
+}
+
+static const VMStateDescription vmstate_mshv_synic_vp_state = {
+    .name = "cpu/mshv_synic_vp_state",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .needed = mshv_synic_vp_state_needed,
+    .fields = (const VMStateField[]) {
+        VMSTATE_BUFFER(env.hv_simp_page, X86CPU),
+        VMSTATE_BUFFER(env.hv_siefp_page, X86CPU),
+        VMSTATE_END_OF_LIST()
+    }
+};
+#endif
+
 static bool avx512_needed(void *opaque)
 {
     X86CPU *cpu = opaque;
@@ -1915,6 +1938,9 @@ const VMStateDescription vmstate_x86_cpu = {
         &vmstate_cet,
 #ifdef TARGET_X86_64
         &vmstate_apx,
+#endif
+#ifdef CONFIG_MSHV
+        &vmstate_mshv_synic_vp_state,
 #endif
         NULL
     }
target/i386/mshv/meson.build +1 -0
index 6091c21887..31ff4cc995 100644
@@ -3,6 +3,7 @@ i386_mshv_ss = ss.source_set()
 i386_mshv_ss.add(files(
   'mshv-cpu.c',
   'msr.c',
+  'synic.c',
 ))
 
 i386_system_ss.add_all(when: 'CONFIG_MSHV', if_true: i386_mshv_ss)
target/i386/mshv/mshv-cpu.c +64 -0
index 333ee35e72..57244bc667 100644
@@ -111,6 +111,33 @@ static enum hv_register_name FPU_REGISTER_NAMES[26] = {
 
 static int set_special_regs(const CPUState *cpu);
 
+static int get_synic_state(CPUState *cpu)
+{
+    X86CPU *x86cpu = X86_CPU(cpu);
+    CPUX86State *env = &x86cpu->env;
+    int cpu_fd = mshv_vcpufd(cpu);
+    int ret;
+
+    /* SIMP/SIEFP can only be read when SynIC is enabled */
+    if (!mshv_synic_enabled(cpu)) {
+        return 0;
+    }
+
+    ret = mshv_get_simp(cpu_fd, env->hv_simp_page);
+    if (ret < 0) {
+        error_report("failed to get simp state");
+        return -1;
+    }
+
+    ret = mshv_get_siefp(cpu_fd, env->hv_siefp_page);
+    if (ret < 0) {
+        error_report("failed to get siefp state");
+        return -1;
+    }
+
+    return 0;
+}
+
 static int get_xsave_state(CPUState *cpu)
 {
     X86CPU *x86cpu = X86_CPU(cpu);
@@ -969,6 +996,11 @@ int mshv_arch_load_vcpu_state(CPUState *cpu)
         return ret;
     }
 
+    ret = get_synic_state(cpu);
+    if (ret < 0) {
+        return ret;
+    }
+
     ret = get_vcpu_events(cpu);
     if (ret < 0) {
         return ret;
@@ -1381,6 +1413,33 @@ static int set_xc_reg(const CPUState *cpu)
     return 0;
 }
 
+static int set_synic_state(const CPUState *cpu)
+{
+    X86CPU *x86cpu = X86_CPU(cpu);
+    CPUX86State *env = &x86cpu->env;
+    int cpu_fd = mshv_vcpufd(cpu);
+    int ret;
+
+    /* SIMP/SIEFP can only be written when SynIC is enabled */
+    if (!mshv_synic_enabled(cpu)) {
+        return 0;
+    }
+
+    ret = mshv_set_simp(cpu_fd, env->hv_simp_page);
+    if (ret < 0) {
+        error_report("failed to set simp state");
+        return -1;
+    }
+
+    ret = mshv_set_siefp(cpu_fd, env->hv_siefp_page);
+    if (ret < 0) {
+        error_report("failed to set siefp state");
+        return -1;
+    }
+
+    return 0;
+}
+
 int mshv_arch_store_vcpu_state(const CPUState *cpu)
 {
     int ret;
@@ -1421,6 +1480,11 @@ int mshv_arch_store_vcpu_state(const CPUState *cpu)
         return ret;
     }
 
+    ret = set_synic_state(cpu);
+    if (ret < 0) {
+        return ret;
+    }
+
     ret = set_vcpu_events(cpu);
     if (ret < 0) {
         return ret;
target/i386/mshv/msr.c +2 -5
index 5288e32b6d..d3788d7715 100644
@@ -332,7 +332,6 @@ int mshv_get_msrs(CPUState *cpu)
     size_t i, j;
     uint32_t name;
     X86CPU *x86cpu = X86_CPU(cpu);
-    bool synic_enabled;
 
     set_hv_name_in_assocs(assocs, n_assocs);
 
@@ -360,8 +359,7 @@ int mshv_get_msrs(CPUState *cpu)
     store_in_env(cpu, assocs, n_assocs);
 
     /* Read SINT MSRs only if SynIC is enabled */
-    synic_enabled = x86cpu->env.msr_hv_synic_control & 1;
-    if (synic_enabled) {
+    if (mshv_synic_enabled(cpu)) {
         QEMU_BUILD_BUG_ON(MSHV_MSR_TOTAL_COUNT < HV_SINT_COUNT);
 
         for (i = 0; i < HV_SINT_COUNT; i++) {
@@ -415,7 +413,6 @@ int mshv_set_msrs(const CPUState *cpu)
     int ret;
     size_t i, j;
     X86CPU *x86cpu = X86_CPU(cpu);
-    bool synic_enabled = x86cpu->env.msr_hv_synic_control & 1;
 
     load_from_env(cpu, assocs, n_assocs);
 
@@ -449,7 +446,7 @@ int mshv_set_msrs(const CPUState *cpu)
     }
 
     /* SINT MSRs can only be written if SCONTROL has been set, so we split */
-    if (synic_enabled) {
+    if (mshv_synic_enabled(cpu)) {
         QEMU_BUILD_BUG_ON(MSHV_MSR_TOTAL_COUNT < HV_SINT_COUNT);
 
         for (i = 0; i < HV_SINT_COUNT; i++) {
target/i386/mshv/synic.c new file +155 -0
new file mode 100644
index 0000000000..8f9fee6ed7
@@ -0,0 +1,155 @@
+/*
+ * QEMU MSHV SynIC support
+ *
+ * Copyright Microsoft, Corp. 2026
+ *
+ * Authors: Magnus Kulke  <[email protected]>
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/memalign.h"
+#include "qemu/error-report.h"
+
+#include "system/mshv.h"
+#include "system/mshv_int.h"
+
+#include "linux/mshv.h"
+#include "hw/hyperv/hvgdk_mini.h"
+#include "cpu.h"
+
+#include <sys/ioctl.h>
+
+bool mshv_synic_enabled(const CPUState *cpu)
+{
+    X86CPU *x86cpu = X86_CPU(cpu);
+
+    return x86cpu->env.msr_hv_synic_control & 1;
+}
+
+static int get_vp_state(int cpu_fd, struct mshv_get_set_vp_state *state)
+{
+    int ret;
+
+    ret = ioctl(cpu_fd, MSHV_GET_VP_STATE, state);
+    if (ret < 0) {
+        error_report("failed to get vp state: %s", strerror(errno));
+        return -1;
+    }
+
+    return 0;
+}
+
+static int set_vp_state(int cpu_fd, const struct mshv_get_set_vp_state *state)
+{
+    int ret;
+
+    ret = ioctl(cpu_fd, MSHV_SET_VP_STATE, state);
+    if (ret < 0) {
+        error_report("failed to set vp state: %s", strerror(errno));
+        return -1;
+    }
+
+    return 0;
+}
+
+int mshv_get_simp(int cpu_fd, uint8_t *page)
+{
+    int ret;
+    void *buffer;
+    struct mshv_get_set_vp_state args = {0};
+
+    buffer = qemu_memalign(HV_HYP_PAGE_SIZE, HV_HYP_PAGE_SIZE);
+    args.buf_ptr = (uint64_t)buffer;
+    args.buf_sz = HV_HYP_PAGE_SIZE;
+    args.type = MSHV_VP_STATE_SIMP;
+
+    ret = get_vp_state(cpu_fd, &args);
+
+    if (ret < 0) {
+        qemu_vfree(buffer);
+        error_report("failed to get simp");
+        return -1;
+    }
+
+    memcpy(page, buffer, HV_HYP_PAGE_SIZE);
+    qemu_vfree(buffer);
+
+    return 0;
+}
+
+int mshv_set_simp(int cpu_fd, const uint8_t *page)
+{
+    int ret;
+    void *buffer;
+    struct mshv_get_set_vp_state args = {0};
+
+    buffer = qemu_memalign(HV_HYP_PAGE_SIZE, HV_HYP_PAGE_SIZE);
+    args.buf_ptr = (uint64_t)buffer;
+    args.buf_sz = HV_HYP_PAGE_SIZE;
+    args.type = MSHV_VP_STATE_SIMP;
+
+    assert(page);
+    memcpy(buffer, page, HV_HYP_PAGE_SIZE);
+
+    ret = set_vp_state(cpu_fd, &args);
+    qemu_vfree(buffer);
+
+    if (ret < 0) {
+        error_report("failed to set simp");
+        return -1;
+    }
+
+    return 0;
+}
+
+int mshv_get_siefp(int cpu_fd, uint8_t *page)
+{
+    int ret;
+    void *buffer;
+    struct mshv_get_set_vp_state args = {0};
+
+    buffer = qemu_memalign(HV_HYP_PAGE_SIZE, HV_HYP_PAGE_SIZE);
+    args.buf_ptr = (uint64_t)buffer;
+    args.buf_sz = HV_HYP_PAGE_SIZE;
+    args.type = MSHV_VP_STATE_SIEFP,
+
+    ret = get_vp_state(cpu_fd, &args);
+
+    if (ret < 0) {
+        qemu_vfree(buffer);
+        error_report("failed to get siefp");
+        return -1;
+    }
+
+    memcpy(page, buffer, HV_HYP_PAGE_SIZE);
+    qemu_vfree(buffer);
+
+    return 0;
+}
+
+int mshv_set_siefp(int cpu_fd, const uint8_t *page)
+{
+    int ret;
+    void *buffer;
+    struct mshv_get_set_vp_state args = {0};
+
+    buffer = qemu_memalign(HV_HYP_PAGE_SIZE, HV_HYP_PAGE_SIZE);
+    args.buf_ptr = (uint64_t)buffer;
+    args.buf_sz = HV_HYP_PAGE_SIZE;
+    args.type = MSHV_VP_STATE_SIEFP,
+
+    assert(page);
+    memcpy(buffer, page, HV_HYP_PAGE_SIZE);
+
+    ret = set_vp_state(cpu_fd, &args);
+    qemu_vfree(buffer);
+
+    if (ret < 0) {
+        error_report("failed to set simp");
+        return -1;
+    }
+
+    return 0;
+}
Signature · 1 line
2.34.1
#15 lore raw

PATCH 8/12 target/i386/mshv: migrate STIMER state

In reply to Magnus Kulke

This part of Synic state is retrieved via a mem-aligned page. We declare the required space (size reference: rust-vmm/mshv) as a buffer on the VM state struct for inclusion in a migration.

Other than other SynIC features, STIMER doesn't depend on SCONTROL being set.

---
 include/system/mshv_int.h   |  2 ++
 target/i386/cpu.h           |  5 ++++
 target/i386/machine.c       | 20 +++++++++++++++
 target/i386/mshv/mshv-cpu.c | 12 +++++++++
 target/i386/mshv/synic.c    | 51 +++++++++++++++++++++++++++++++++++++
 5 files changed, 90 insertions(+)
include/system/mshv_int.h +2 -0
index bc023d3535..063852115e 100644
@@ -144,5 +144,7 @@ int mshv_set_simp(int cpu_fd, const uint8_t *page);
 int mshv_get_siefp(int cpu_fd, uint8_t *page);
 int mshv_set_siefp(int cpu_fd, const uint8_t *page);
 bool mshv_synic_enabled(const CPUState *cpu);
+int mshv_get_synthetic_timers(int cpu_fd, uint8_t *state);
+int mshv_set_synthetic_timers(int cpu_fd, const uint8_t *state);
 
 #endif
target/i386/cpu.h +5 -0
index 9270ae95d0..6c9e674e81 100644
@@ -45,6 +45,10 @@
 #define ELF_MACHINE_UNAME "i686"
 #endif
 
+#ifdef CONFIG_MSHV
+#define MSHV_STIMERS_STATE_SIZE 200
+#endif
+
 enum {
     R_EAX = 0,
     R_ECX = 1,
@@ -2303,6 +2307,7 @@ typedef struct CPUArchState {
 #if defined(CONFIG_MSHV)
     uint8_t hv_simp_page[HV_HYP_PAGE_SIZE];
     uint8_t hv_siefp_page[HV_HYP_PAGE_SIZE];
+    uint8_t hv_synthetic_timers_state[MSHV_STIMERS_STATE_SIZE];
 #endif
 
     uint64_t mcg_cap;
target/i386/machine.c +20 -0
index 023a397ab8..8d69d7e25e 100644
@@ -10,6 +10,7 @@
 #include "exec/watchpoint.h"
 #include "system/kvm.h"
 #include "system/kvm_xen.h"
+#include "system/mshv.h"
 #include "system/tcg.h"
 
 #include "qemu/error-report.h"
@@ -952,6 +953,24 @@ static const VMStateDescription vmstate_msr_hyperv_reenlightenment = {
 };
 
 #ifdef CONFIG_MSHV
+
+static bool mshv_synthetic_timers_needed(void *opaque)
+{
+    /* Always migrate synthetic timers */
+    return mshv_enabled();
+}
+
+static const VMStateDescription vmstate_mshv_synthetic_timers = {
+    .name = "cpu/mshv_synthetic_timers",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .needed = mshv_synthetic_timers_needed,
+    .fields = (const VMStateField[]) {
+        VMSTATE_BUFFER(env.hv_synthetic_timers_state, X86CPU),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 static bool mshv_synic_vp_state_needed(void *opaque)
 {
     X86CPU *cpu = opaque;
@@ -1941,6 +1960,7 @@ const VMStateDescription vmstate_x86_cpu = {
 #endif
 #ifdef CONFIG_MSHV
         &vmstate_mshv_synic_vp_state,
+        &vmstate_mshv_synthetic_timers,
 #endif
         NULL
     }
target/i386/mshv/mshv-cpu.c +12 -0
index 57244bc667..ce6cfccc0a 100644
@@ -118,6 +118,12 @@ static int get_synic_state(CPUState *cpu)
     int cpu_fd = mshv_vcpufd(cpu);
     int ret;
 
+    ret = mshv_get_synthetic_timers(cpu_fd, env->hv_synthetic_timers_state);
+    if (ret < 0) {
+        error_report("failed to get synthetic timers");
+        return -1;
+    }
+
     /* SIMP/SIEFP can only be read when SynIC is enabled */
     if (!mshv_synic_enabled(cpu)) {
         return 0;
@@ -1420,6 +1426,12 @@ static int set_synic_state(const CPUState *cpu)
     int cpu_fd = mshv_vcpufd(cpu);
     int ret;
 
+    ret = mshv_set_synthetic_timers(cpu_fd, env->hv_synthetic_timers_state);
+    if (ret < 0) {
+        error_report("failed to set synthetic timers state");
+        return -1;
+    }
+
     /* SIMP/SIEFP can only be written when SynIC is enabled */
     if (!mshv_synic_enabled(cpu)) {
         return 0;
target/i386/mshv/synic.c +51 -0

57 lines not on this page · raw message

Signature · 1 line
2.34.1
Reviewed-by on v1 8/12
#16 lore raw

Re: [PATCH 08/12] target/i386/mshv: migrate STIMER state

Quoted 190 lines · Magnus Kulke

This part of Synic state is retrieved via a mem-aligned page. We declare the required space (size reference: rust-vmm/mshv) as a buffer on the VM state struct for inclusion in a migration.

Other than other SynIC features, STIMER doesn't depend on SCONTROL being set.

---
 include/system/mshv_int.h   |  2 ++
 target/i386/cpu.h           |  5 ++++
 target/i386/machine.c       | 20 +++++++++++++++
 target/i386/mshv/mshv-cpu.c | 12 +++++++++
 target/i386/mshv/synic.c    | 51 +++++++++++++++++++++++++++++++++++++
 5 files changed, 90 insertions(+)
include/system/mshv_int.h +2 -0
index bc023d3535..063852115e 100644
@@ -144,5 +144,7 @@ int mshv_set_simp(int cpu_fd, const uint8_t *page);
 int mshv_get_siefp(int cpu_fd, uint8_t *page);
 int mshv_set_siefp(int cpu_fd, const uint8_t *page);
 bool mshv_synic_enabled(const CPUState *cpu);
+int mshv_get_synthetic_timers(int cpu_fd, uint8_t *state);
+int mshv_set_synthetic_timers(int cpu_fd, const uint8_t *state);
 
 #endif
target/i386/cpu.h +5 -0
index 9270ae95d0..6c9e674e81 100644
@@ -45,6 +45,10 @@
 #define ELF_MACHINE_UNAME "i686"
 #endif
 
+#ifdef CONFIG_MSHV
+#define MSHV_STIMERS_STATE_SIZE 200
+#endif
+
 enum {
     R_EAX = 0,
     R_ECX = 1,
@@ -2303,6 +2307,7 @@ typedef struct CPUArchState {
 #if defined(CONFIG_MSHV)
     uint8_t hv_simp_page[HV_HYP_PAGE_SIZE];
     uint8_t hv_siefp_page[HV_HYP_PAGE_SIZE];
+    uint8_t hv_synthetic_timers_state[MSHV_STIMERS_STATE_SIZE];
 #endif
 
     uint64_t mcg_cap;
target/i386/machine.c +20 -0
index 023a397ab8..8d69d7e25e 100644
@@ -10,6 +10,7 @@
 #include "exec/watchpoint.h"
 #include "system/kvm.h"
 #include "system/kvm_xen.h"
+#include "system/mshv.h"
 #include "system/tcg.h"
 
 #include "qemu/error-report.h"
@@ -952,6 +953,24 @@ static const VMStateDescription vmstate_msr_hyperv_reenlightenment = {
 };
 
 #ifdef CONFIG_MSHV
+
+static bool mshv_synthetic_timers_needed(void *opaque)
+{
+    /* Always migrate synthetic timers */
+    return mshv_enabled();
+}
+
+static const VMStateDescription vmstate_mshv_synthetic_timers = {
+    .name = "cpu/mshv_synthetic_timers",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .needed = mshv_synthetic_timers_needed,
+    .fields = (const VMStateField[]) {
+        VMSTATE_BUFFER(env.hv_synthetic_timers_state, X86CPU),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 static bool mshv_synic_vp_state_needed(void *opaque)
 {
     X86CPU *cpu = opaque;
@@ -1941,6 +1960,7 @@ const VMStateDescription vmstate_x86_cpu = {
 #endif
 #ifdef CONFIG_MSHV
         &vmstate_mshv_synic_vp_state,
+        &vmstate_mshv_synthetic_timers,
 #endif
         NULL
     }
target/i386/mshv/mshv-cpu.c +12 -0
index 57244bc667..ce6cfccc0a 100644
@@ -118,6 +118,12 @@ static int get_synic_state(CPUState *cpu)
     int cpu_fd = mshv_vcpufd(cpu);
     int ret;
 
+    ret = mshv_get_synthetic_timers(cpu_fd, env->hv_synthetic_timers_state);
+    if (ret < 0) {
+        error_report("failed to get synthetic timers");
+        return -1;
+    }
+
     /* SIMP/SIEFP can only be read when SynIC is enabled */
     if (!mshv_synic_enabled(cpu)) {
         return 0;
@@ -1420,6 +1426,12 @@ static int set_synic_state(const CPUState *cpu)
     int cpu_fd = mshv_vcpufd(cpu);
     int ret;
 
+    ret = mshv_set_synthetic_timers(cpu_fd, env->hv_synthetic_timers_state);
+    if (ret < 0) {
+        error_report("failed to set synthetic timers state");
+        return -1;
+    }
+
     /* SIMP/SIEFP can only be written when SynIC is enabled */
     if (!mshv_synic_enabled(cpu)) {
         return 0;
target/i386/mshv/synic.c +51 -0
index 8f9fee6ed7..4c629adc3a 100644
@@ -54,6 +54,57 @@ static int set_vp_state(int cpu_fd, const struct mshv_get_set_vp_state *state)
     return 0;
 }
 
+int mshv_get_synthetic_timers(int cpu_fd, uint8_t *state)
+{
+    int ret;
+    void *buffer;
+    struct mshv_get_set_vp_state args = {0};
+
+    buffer = qemu_memalign(HV_HYP_PAGE_SIZE, HV_HYP_PAGE_SIZE);
+    args.buf_ptr = (uint64_t)buffer;
+    args.buf_sz = HV_HYP_PAGE_SIZE;
+    args.type = MSHV_VP_STATE_SYNTHETIC_TIMERS;
+
+    ret = get_vp_state(cpu_fd, &args);
+
+    if (ret < 0) {
+        qemu_vfree(buffer);
+        error_report("failed to get synthetic timers");
+        return -1;
+    }
+
+    memcpy(state, buffer, MSHV_STIMERS_STATE_SIZE);
+    qemu_vfree(buffer);
+
+    return 0;
+}
+
+int mshv_set_synthetic_timers(int cpu_fd, const uint8_t *state)
+{
+    int ret;
+    void *buffer;
+    struct mshv_get_set_vp_state args = {0};
+
+    buffer = qemu_memalign(HV_HYP_PAGE_SIZE, HV_HYP_PAGE_SIZE);
+    memset(buffer, 0, HV_HYP_PAGE_SIZE);
+    args.buf_ptr = (uint64_t)buffer;
+    args.buf_sz = HV_HYP_PAGE_SIZE;
+    args.type = MSHV_VP_STATE_SYNTHETIC_TIMERS;
+
+    assert(state);
+    memcpy(buffer, state, MSHV_STIMERS_STATE_SIZE);
+
+    ret = set_vp_state(cpu_fd, &args);
+    qemu_vfree(buffer);
+
+    if (ret < 0) {
+        error_report("failed to set synthetic timers");
+        return -1;
+    }
+
+    return 0;
+}
+
 int mshv_get_simp(int cpu_fd, uint8_t *page)
 {
     int ret;
Signature · 1 line
2.34.1
#17 lore raw

PATCH 9/12 accel/mshv: write synthetic MSRs after migration

In reply to Magnus Kulke

Write partition-wide synthetic MSRs. This ensures the hypercall page and SynIC facilities are set up before vCPUs attempt to use it.

---
 accel/mshv/mshv-all.c          | 15 +++++++++++++++
 include/hw/hyperv/hvgdk_mini.h |  3 +++
 include/system/mshv_int.h      |  1 +
 target/i386/mshv/mshv-cpu.c    | 15 +++++++++++++++
 4 files changed, 34 insertions(+)
accel/mshv/mshv-all.c +15 -0
index 1516475f34..1ca1d4b54f 100644
@@ -60,6 +60,20 @@ static int init_mshv(int *mshv_fd)
     return 0;
 }
 
+static int mshv_load_cleanup(void *opaque)
+{
+    int ret;
+
+    ret = mshv_arch_set_partition_msrs(first_cpu);
+    if (ret < 0) {
+        error_report("Failed to set partition MSRs: %s", strerror(-ret));
+        return -1;
+    }
+
+    return 0;
+}
+
+
 static int get_host_partition_property(int mshv_fd, uint32_t property_code,
                                        uint64_t *value)
 {
@@ -496,6 +510,7 @@ static int mshv_init_vcpu(CPUState *cpu)
 }
 
 static SaveVMHandlers savevm_mshv = {
+    .load_cleanup = mshv_load_cleanup,
 };
 
 static int mshv_init(AccelState *as, MachineState *ms)
include/hw/hyperv/hvgdk_mini.h +3 -0
index 0602a7c6cc..a5527d49cf 100644
@@ -23,6 +23,9 @@
 #define HV_X64_MSR_APIC_FREQUENCY   0x40000023
 
 typedef enum hv_register_name {
+    /* VP Management Registers */
+    HV_REGISTER_INTERNAL_ACTIVITY_STATE = 0x00000004,
+
     /* Pending Interruption Register */
     HV_REGISTER_PENDING_INTERRUPTION = 0x00010002,
     HV_REGISTER_INTERRUPT_STATE      = 0x00010003,
include/system/mshv_int.h +1 -0
index 063852115e..cbfcb8611b 100644
@@ -98,6 +98,7 @@ int mshv_get_generic_regs(CPUState *cpu, hv_register_assoc *assocs,
                           size_t n_regs);
 int mshv_arch_store_vcpu_state(const CPUState *cpu);
 int mshv_arch_load_vcpu_state(CPUState *cpu);
+int mshv_arch_set_partition_msrs(const CPUState *cpu);
 void mshv_arch_init_vcpu(CPUState *cpu);
 void mshv_arch_destroy_vcpu(CPUState *cpu);
 void mshv_arch_amend_proc_features(
target/i386/mshv/mshv-cpu.c +15 -0
index ce6cfccc0a..1485f6a1ef 100644
@@ -1505,6 +1505,21 @@ int mshv_arch_store_vcpu_state(const CPUState *cpu)
     return 0;
 }
 
+int mshv_arch_set_partition_msrs(const CPUState *cpu)
+{
+    CPUX86State *env = &X86_CPU(cpu)->env;
+    struct hv_register_assoc assocs[] = {
+        { .name = HV_REGISTER_GUEST_OS_ID,
+          .value.reg64 = env->msr_hv_guest_os_id },
+        { .name = HV_REGISTER_REFERENCE_TSC,
+          .value.reg64 = env->msr_hv_tsc },
+        { .name = HV_X64_REGISTER_HYPERCALL,
+          .value.reg64 = env->msr_hv_hypercall },
+    };
+
+    return mshv_set_generic_regs(cpu, assocs, ARRAY_SIZE(assocs));
+}
+
 void mshv_arch_amend_proc_features(
     union hv_partition_synthetic_processor_features *features)
 {
Signature · 1 line
2.34.1
Reviewed-by on v1 9/12
#18 lore raw

Re: [PATCH 09/12] accel/mshv: write synthetic MSRs after migration

Quoted 98 lines · Magnus Kulke

Write partition-wide synthetic MSRs. This ensures the hypercall page and SynIC facilities are set up before vCPUs attempt to use it.

---
 accel/mshv/mshv-all.c          | 15 +++++++++++++++
 include/hw/hyperv/hvgdk_mini.h |  3 +++
 include/system/mshv_int.h      |  1 +
 target/i386/mshv/mshv-cpu.c    | 15 +++++++++++++++
 4 files changed, 34 insertions(+)
accel/mshv/mshv-all.c +15 -0
index 1516475f34..1ca1d4b54f 100644
@@ -60,6 +60,20 @@ static int init_mshv(int *mshv_fd)
     return 0;
 }
 
+static int mshv_load_cleanup(void *opaque)
+{
+    int ret;
+
+    ret = mshv_arch_set_partition_msrs(first_cpu);
+    if (ret < 0) {
+        error_report("Failed to set partition MSRs: %s", strerror(-ret));
+        return -1;
+    }
+
+    return 0;
+}
+
+
 static int get_host_partition_property(int mshv_fd, uint32_t property_code,
                                        uint64_t *value)
 {
@@ -496,6 +510,7 @@ static int mshv_init_vcpu(CPUState *cpu)
 }
 
 static SaveVMHandlers savevm_mshv = {
+    .load_cleanup = mshv_load_cleanup,
 };
 
 static int mshv_init(AccelState *as, MachineState *ms)
include/hw/hyperv/hvgdk_mini.h +3 -0
index 0602a7c6cc..a5527d49cf 100644
@@ -23,6 +23,9 @@
 #define HV_X64_MSR_APIC_FREQUENCY   0x40000023
 
 typedef enum hv_register_name {
+    /* VP Management Registers */
+    HV_REGISTER_INTERNAL_ACTIVITY_STATE = 0x00000004,
+
     /* Pending Interruption Register */
     HV_REGISTER_PENDING_INTERRUPTION = 0x00010002,
     HV_REGISTER_INTERRUPT_STATE      = 0x00010003,
include/system/mshv_int.h +1 -0
index 063852115e..cbfcb8611b 100644
@@ -98,6 +98,7 @@ int mshv_get_generic_regs(CPUState *cpu, hv_register_assoc *assocs,
                           size_t n_regs);
 int mshv_arch_store_vcpu_state(const CPUState *cpu);
 int mshv_arch_load_vcpu_state(CPUState *cpu);
+int mshv_arch_set_partition_msrs(const CPUState *cpu);
 void mshv_arch_init_vcpu(CPUState *cpu);
 void mshv_arch_destroy_vcpu(CPUState *cpu);
 void mshv_arch_amend_proc_features(
target/i386/mshv/mshv-cpu.c +15 -0
index ce6cfccc0a..1485f6a1ef 100644
@@ -1505,6 +1505,21 @@ int mshv_arch_store_vcpu_state(const CPUState *cpu)
     return 0;
 }
 
+int mshv_arch_set_partition_msrs(const CPUState *cpu)
+{
+    CPUX86State *env = &X86_CPU(cpu)->env;
+    struct hv_register_assoc assocs[] = {
+        { .name = HV_REGISTER_GUEST_OS_ID,
+          .value.reg64 = env->msr_hv_guest_os_id },
+        { .name = HV_REGISTER_REFERENCE_TSC,
+          .value.reg64 = env->msr_hv_tsc },
+        { .name = HV_X64_REGISTER_HYPERCALL,
+          .value.reg64 = env->msr_hv_hypercall },
+    };
+
+    return mshv_set_generic_regs(cpu, assocs, ARRAY_SIZE(assocs));
+}
+
 void mshv_arch_amend_proc_features(
     union hv_partition_synthetic_processor_features *features)
 {
Signature · 1 line
2.34.1
#19 lore raw

PATCH 10/12 target/i386/mshv: migrate MP_STATE

In reply to Magnus Kulke

MSHV's "internal activity state" roughly maps to QEMU's env->mp_state and cpu->halted states that describe state of APs in a guest.

We don't invoke set_mp_state as part of store_vcpu_state() b/c we would put all BSP + APs in a RUNNABLE (0) state immediately, breaking SMP boot

Instead we store the mp state as part of the load_cleanup() routine after a migration.

---
 accel/mshv/mshv-all.c       | 10 +++++
 include/system/mshv_int.h   |  1 +
 target/i386/mshv/mshv-cpu.c | 80 +++++++++++++++++++++++++++++++++++++
 3 files changed, 91 insertions(+)
accel/mshv/mshv-all.c +10 -0
index 1ca1d4b54f..5921ce693e 100644
@@ -62,6 +62,7 @@ static int init_mshv(int *mshv_fd)
 
 static int mshv_load_cleanup(void *opaque)
 {
+    CPUState *cpu;
     int ret;
 
     ret = mshv_arch_set_partition_msrs(first_cpu);
@@ -70,6 +71,15 @@ static int mshv_load_cleanup(void *opaque)
         return -1;
     }
 
+    CPU_FOREACH(cpu) {
+        ret = mshv_arch_set_mp_state(cpu);
+        if (ret < 0) {
+            error_report("Failed to set mp state for vCPU %d: %s",
+                         cpu->cpu_index, strerror(-ret));
+            return -1;
+        }
+    }
+
     return 0;
 }
 
include/system/mshv_int.h +1 -0
index cbfcb8611b..3dffe3c5fb 100644
@@ -99,6 +99,7 @@ int mshv_get_generic_regs(CPUState *cpu, hv_register_assoc *assocs,
 int mshv_arch_store_vcpu_state(const CPUState *cpu);
 int mshv_arch_load_vcpu_state(CPUState *cpu);
 int mshv_arch_set_partition_msrs(const CPUState *cpu);
+int mshv_arch_set_mp_state(const CPUState *cpu);
 void mshv_arch_init_vcpu(CPUState *cpu);
 void mshv_arch_destroy_vcpu(CPUState *cpu);
 void mshv_arch_amend_proc_features(
target/i386/mshv/mshv-cpu.c +80 -0

98 lines not on this page · raw message

Signature · 1 line
2.34.1
Reviewed-by on v1 10/12
#20 lore raw

Re: [PATCH 10/12] target/i386/mshv: migrate MP_STATE

Quoted 163 lines · Magnus Kulke

MSHV's "internal activity state" roughly maps to QEMU's env->mp_state and cpu->halted states that describe state of APs in a guest.

We don't invoke set_mp_state as part of store_vcpu_state() b/c we would put all BSP + APs in a RUNNABLE (0) state immediately, breaking SMP boot

Instead we store the mp state as part of the load_cleanup() routine after a migration.

---
 accel/mshv/mshv-all.c       | 10 +++++
 include/system/mshv_int.h   |  1 +
 target/i386/mshv/mshv-cpu.c | 80 +++++++++++++++++++++++++++++++++++++
 3 files changed, 91 insertions(+)
accel/mshv/mshv-all.c +10 -0
index 1ca1d4b54f..5921ce693e 100644
@@ -62,6 +62,7 @@ static int init_mshv(int *mshv_fd)
 
 static int mshv_load_cleanup(void *opaque)
 {
+    CPUState *cpu;
     int ret;
 
     ret = mshv_arch_set_partition_msrs(first_cpu);
@@ -70,6 +71,15 @@ static int mshv_load_cleanup(void *opaque)
         return -1;
     }
 
+    CPU_FOREACH(cpu) {
+        ret = mshv_arch_set_mp_state(cpu);
+        if (ret < 0) {
+            error_report("Failed to set mp state for vCPU %d: %s",
+                         cpu->cpu_index, strerror(-ret));
+            return -1;
+        }
+    }
+
     return 0;
 }
 
include/system/mshv_int.h +1 -0
index cbfcb8611b..3dffe3c5fb 100644
@@ -99,6 +99,7 @@ int mshv_get_generic_regs(CPUState *cpu, hv_register_assoc *assocs,
 int mshv_arch_store_vcpu_state(const CPUState *cpu);
 int mshv_arch_load_vcpu_state(CPUState *cpu);
 int mshv_arch_set_partition_msrs(const CPUState *cpu);
+int mshv_arch_set_mp_state(const CPUState *cpu);
 void mshv_arch_init_vcpu(CPUState *cpu);
 void mshv_arch_destroy_vcpu(CPUState *cpu);
 void mshv_arch_amend_proc_features(
target/i386/mshv/mshv-cpu.c +80 -0
index 1485f6a1ef..bff1ac9d17 100644
@@ -35,6 +35,11 @@
 
 #include <sys/ioctl.h>
 
+#define MSHV_MP_STATE_RUNNABLE       0
+#define MSHV_MP_STATE_UNINITIALIZED  1
+#define MSHV_MP_STATE_INIT_RECEIVED  2
+#define MSHV_MP_STATE_HALTED         3
+
 #define MAX_REGISTER_COUNT (MAX_CONST(ARRAY_SIZE(STANDARD_REGISTER_NAMES), \
                             MAX_CONST(ARRAY_SIZE(SPECIAL_REGISTER_NAMES), \
                                       ARRAY_SIZE(FPU_REGISTER_NAMES))))
@@ -950,6 +955,76 @@ static int set_vcpu_events(const CPUState *cpu)
     return 0;
 }
 
+static int get_mp_state(CPUState *cpu)
+{
+    X86CPU *x86cpu = X86_CPU(cpu);
+    CPUX86State *env = &x86cpu->env;
+    struct hv_register_assoc assoc = {
+        .name = HV_REGISTER_INTERNAL_ACTIVITY_STATE,
+    };
+    union hv_internal_activity_register activity;
+    int ret;
+
+    ret = mshv_get_generic_regs(cpu, &assoc, 1);
+    if (ret < 0) {
+        error_report("failed to get internal activity state");
+        return -1;
+    }
+
+    activity.as_uint64 = assoc.value.reg64;
+
+    /*
+     * map MSHV activity state to KVM mp_state values, which are used as the
+     * shared representation in env->mp_state and serialized by vmstate_x86_cpu.
+     */
+
+    if (activity.startup_suspend) {
+        env->mp_state = MSHV_MP_STATE_UNINITIALIZED;
+    } else if (activity.halt_suspend) {
+        env->mp_state = MSHV_MP_STATE_HALTED;
+    } else {
+        env->mp_state = MSHV_MP_STATE_RUNNABLE;
+    }
+
+    cpu->halted = (env->mp_state == MSHV_MP_STATE_HALTED);
+
+    return 0;
+}
+
+int mshv_arch_set_mp_state(const CPUState *cpu)
+{
+    X86CPU *x86cpu = X86_CPU(cpu);
+    CPUX86State *env = &x86cpu->env;
+    union hv_internal_activity_register activity = { 0 };
+    struct hv_register_assoc assoc = {
+        .name = HV_REGISTER_INTERNAL_ACTIVITY_STATE,
+    };
+    int ret;
+
+    switch (env->mp_state) {
+    case MSHV_MP_STATE_HALTED:
+        activity.halt_suspend = 1;
+        break;
+    case MSHV_MP_STATE_UNINITIALIZED:
+    case MSHV_MP_STATE_INIT_RECEIVED:
+        activity.startup_suspend = 1;
+        break;
+    case MSHV_MP_STATE_RUNNABLE:
+    default:
+        break;
+    }
+
+    assoc.value.reg64 = activity.as_uint64;
+
+    ret = mshv_set_generic_regs(cpu, &assoc, 1);
+    if (ret < 0) {
+        error_report("failed to set internal activity state");
+        return -1;
+    }
+
+    return 0;
+}
+
 static int update_hflags(CPUState *cpu)
 {
     X86CPU *x86cpu = X86_CPU(cpu);
@@ -1012,6 +1087,11 @@ int mshv_arch_load_vcpu_state(CPUState *cpu)
         return ret;
     }
 
+    ret = get_mp_state(cpu);
+    if (ret < 0) {
+        return ret;
+    }
+
     return 0;
 }
 
Signature · 1 line
2.34.1
#21 lore raw

PATCH 11/12 target/i386/mshv: toggle fpu/xsave migration

In reply to Magnus Kulke

MSHV exposes overlapping legacy FP/SSE state through two paths:
explicit Hyper-V FPU/XMM + registers and VP XSAVE state. There can be subtle inconsistencies across migrations when XSAVE is written after FPU state.

---
 target/i386/mshv/mshv-cpu.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
target/i386/mshv/mshv-cpu.c +5 -4
index bff1ac9d17..f528dd2b9a 100644
@@ -1057,7 +1057,7 @@ int mshv_arch_load_vcpu_state(CPUState *cpu)
         return ret;
     }
 
-    ret = get_fpu(cpu);
+    ret = get_xsave_state(cpu);
     if (ret < 0) {
         return ret;
     }
@@ -1072,7 +1072,7 @@ int mshv_arch_load_vcpu_state(CPUState *cpu)
         return ret;
     }
 
-    ret = get_xsave_state(cpu);
+    ret = get_fpu(cpu);
     if (ret < 0) {
         return ret;
     }
@@ -1551,7 +1551,7 @@ int mshv_arch_store_vcpu_state(const CPUState *cpu)
         return ret;
     }
 
-    ret = set_fpu(cpu);
+    ret = set_xsave_state(cpu);
     if (ret < 0) {
         return ret;
     }
@@ -1567,7 +1567,8 @@ int mshv_arch_store_vcpu_state(const CPUState *cpu)
         return ret;
     }
 
-    ret = set_xsave_state(cpu);
+    /* INVARIANT: legacy FPU state must be restored after XSAVE */
+    ret = set_fpu(cpu);
     if (ret < 0) {
         return ret;
     }
Signature · 1 line
2.34.1
Reviewed-by on v1 11/12
#22 lore raw

Re: [PATCH 11/12] target/i386/mshv: toggle fpu/xsave migration

Quoted 53 lines · Magnus Kulke

MSHV exposes overlapping legacy FP/SSE state through two paths:
explicit Hyper-V FPU/XMM + registers and VP XSAVE state. There can be subtle inconsistencies across migrations when XSAVE is written after FPU state.

---
 target/i386/mshv/mshv-cpu.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
target/i386/mshv/mshv-cpu.c +5 -4
index bff1ac9d17..f528dd2b9a 100644
@@ -1057,7 +1057,7 @@ int mshv_arch_load_vcpu_state(CPUState *cpu)
         return ret;
     }
 
-    ret = get_fpu(cpu);
+    ret = get_xsave_state(cpu);
     if (ret < 0) {
         return ret;
     }
@@ -1072,7 +1072,7 @@ int mshv_arch_load_vcpu_state(CPUState *cpu)
         return ret;
     }
 
-    ret = get_xsave_state(cpu);
+    ret = get_fpu(cpu);
     if (ret < 0) {
         return ret;
     }
@@ -1551,7 +1551,7 @@ int mshv_arch_store_vcpu_state(const CPUState *cpu)
         return ret;
     }
 
-    ret = set_fpu(cpu);
+    ret = set_xsave_state(cpu);
     if (ret < 0) {
         return ret;
     }
@@ -1567,7 +1567,8 @@ int mshv_arch_store_vcpu_state(const CPUState *cpu)
         return ret;
     }
 
-    ret = set_xsave_state(cpu);
+    /* INVARIANT: legacy FPU state must be restored after XSAVE */
+    ret = set_fpu(cpu);
     if (ret < 0) {
         return ret;
     }
Signature · 1 line
2.34.1
#23 lore raw

PATCH 12/12 hw/i386/mshv: drop initial msi vector 0

In reply to Magnus Kulke

This has been a warning before that was always raised if the machine has a hpet. hpet_reset() will eventually result in mshv_send_msi called w/ vector 0, which we can safely drop.

---
 hw/i386/mshv/apic.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
hw/i386/mshv/apic.c +10 -0
index 9188a93b2f..ad326030bf 100644
@@ -293,6 +293,16 @@ static void mshv_send_msi(MSIMessage *msi)
     delivery     = (data >> MSI_DATA_DELIVERY_MODE_SHIFT) &
                    MSI_DATA_DELIVERY_MODE_MASK;
 
+    /*
+     * Vector 0 is not a valid interrupt vector (0-15 are reserved for CPU
+     * exceptions). This can trigger during machine reset, if hpet_reset()
+     * forces the PIT to pulse GSI 2 before IOAPIC's own reset has masked its
+     * redirection entries.
+     */
+    if (vector == 0) {
+        return;
+    }
+
     mshv_request_interrupt(mshv_state, delivery, vector, dest, dest_mode,
                            trigger_mode);
 }
Signature · 1 line
2.34.1
Reviewed-by on v1 12/12
#24 lore raw

Re: [PATCH 12/12] hw/i386/mshv: drop initial msi vector 0

Quoted 32 lines · Magnus Kulke

This has been a warning before that was always raised if the machine has a hpet. hpet_reset() will eventually result in mshv_send_msi called w/ vector 0, which we can safely drop.

---
 hw/i386/mshv/apic.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
hw/i386/mshv/apic.c +10 -0
index 9188a93b2f..ad326030bf 100644
@@ -293,6 +293,16 @@ static void mshv_send_msi(MSIMessage *msi)
     delivery     = (data >> MSI_DATA_DELIVERY_MODE_SHIFT) &
                    MSI_DATA_DELIVERY_MODE_MASK;
 
+    /*
+     * Vector 0 is not a valid interrupt vector (0-15 are reserved for CPU
+     * exceptions). This can trigger during machine reset, if hpet_reset()
+     * forces the PIT to pulse GSI 2 before IOAPIC's own reset has masked its
+     * redirection entries.
+     */
+    if (vector == 0) {
+        return;
+    }
+
     mshv_request_interrupt(mshv_state, delivery, vector, dest, dest_mode,
                            trigger_mode);
 }
Signature · 1 line
2.34.1