Skip to contentSkip to conversation

i386: KVM: expand Hyper-V features early

PATCH In review Vitaly Kuznetsov 4 participants 19 messages Jun 8, 2021 – Jul 19, 2021 Arm/Amlogic Meson SoC support

lore
Share & tools
lore
#0 lore raw

v8 cover i386: KVM: expand Hyper-V features early

Changes since v7:
- Make eVMCS version check future proof [Eduardo]
- Collect R-b tags [Eduardo]
- Drop 'if (!strcmp(arch, "i386") || !strcmp(arch, "x86_64"))' check from qtest [Eduardo]
- s/priviliges/privileges/ [Eric]

The last two functional patches are inspired by 'Fine-grained access check to Hyper-V hypercalls and MSRs' work for KVM:
https://lore.kernel.org/kvm/[email protected]/

Original description:

Upper layer tools like libvirt want to figure out which Hyper-V features are supported by the underlying stack (QEMU/KVM) but currently they are unable to do so. We have a nice 'hv_passthrough' CPU flag supported by QEMU but it has no effect on e.g. QMP's

query-cpu-model-expansion type=full model={"name":"host","props":{"hv-passthrough":true}}

command as we parse Hyper-V features after creating KVM vCPUs and not at feature expansion time. To support the use-case we first need to make KVM_GET_SUPPORTED_HV_CPUID ioctl a system-wide ioctl as the existing vCPU version can't be used that early. This is what KVM part does. With that done, we can make early Hyper-V feature expansion (this series).

Vitaly Kuznetsov (9):
i386: avoid hardcoding '12' as 'hyperv_vendor_id' length
i386: clarify 'hv-passthrough' behavior
i386: hardcode supported eVMCS version to '1'
i386: make hyperv_expand_features() return bool
i386: expand Hyper-V features during CPU feature expansion time
i386: kill off hv_cpuid_check_and_set()
i386: HV_HYPERCALL_AVAILABLE privilege bit is always needed
i386: Hyper-V SynIC requires POST_MESSAGES/SIGNAL_EVENTS privileges qtest/hyperv: Introduce a simple hyper-v test

 MAINTAINERS                    |   1 +
 docs/hyperv.txt                |   9 +-
 target/i386/cpu.c              |  13 +-
 target/i386/kvm/hyperv-proto.h |   6 +
 target/i386/kvm/kvm-stub.c     |   5 +
 target/i386/kvm/kvm.c          | 189 +++++++++++++++-------------
 target/i386/kvm/kvm_i386.h     |   1 +
 tests/qtest/hyperv-test.c      | 221 +++++++++++++++++++++++++++++++++
 tests/qtest/meson.build        |   3 +-
 9 files changed, 357 insertions(+), 91 deletions(-)
 create mode 100644 tests/qtest/hyperv-test.c
Signature · 1 line
2.31.1
#1 lore raw

PATCH v8 1/9 i386: avoid hardcoding '12' as 'hyperv_vendor_id' length

While this is very unlikely to change, let's avoid hardcoding '12' as 'hyperv_vendor_id' length.

No functional change intended.

---
 target/i386/cpu.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
target/i386/cpu.c +5 -4
index a9fe1662d392..f8ae45be0d53 100644
@@ -6057,11 +6057,12 @@ static void x86_cpu_hyperv_realize(X86CPU *cpu)
                                 &error_abort);
     }
     len = strlen(cpu->hyperv_vendor);
-    if (len > 12) {
-        warn_report("hv-vendor-id truncated to 12 characters");
-        len = 12;
+    if (len > sizeof(cpu->hyperv_vendor_id)) {
+        warn_report("hv-vendor-id truncated to %ld characters",
+                    sizeof(cpu->hyperv_vendor_id));
+        len = sizeof(cpu->hyperv_vendor_id);
     }
-    memset(cpu->hyperv_vendor_id, 0, 12);
+    memset(cpu->hyperv_vendor_id, 0, sizeof(cpu->hyperv_vendor_id));
     memcpy(cpu->hyperv_vendor_id, cpu->hyperv_vendor, len);
 
     /* 'Hv#1' interface identification*/
Signature · 1 line
2.31.1
Reviewed-by on v8 1/9
#2 lore raw

Re: [PATCH v8 1/9] i386: avoid hardcoding '12' as 'hyperv_vendor_id' length

Quoted 10 lines · Vitaly Kuznetsov

While this is very unlikely to change, let's avoid hardcoding '12' as 'hyperv_vendor_id' length.

No functional change intended.

---
 target/i386/cpu.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
#3 lore raw

PATCH v8 2/9 i386: clarify 'hv-passthrough' behavior

In reply to Vitaly Kuznetsov

Clarify the fact that 'hv-passthrough' only enables features which are already known to QEMU and that it overrides all other 'hv-*' settings.

---
 docs/hyperv.txt | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
docs/hyperv.txt +5 -2
index e53c581f4586..a51953daa833 100644
@@ -209,8 +209,11 @@ In some cases (e.g. during development) it may make sense to use QEMU in
 'pass-through' mode and give Windows guests all enlightenments currently
 supported by KVM. This pass-through mode is enabled by "hv-passthrough" CPU
 flag.
-Note: enabling this flag effectively prevents migration as supported features
-may differ between target and destination.
+Note: "hv-passthrough" flag only enables enlightenments which are known to QEMU
+(have corresponding "hv-*" flag) and copies "hv-spinlocks="/"hv-vendor-id="
+values from KVM to QEMU. "hv-passthrough" overrides all other "hv-*" settings on
+the command line. Also, enabling this flag effectively prevents migration as the
+list of enabled enlightenments may differ between target and destination hosts.
 
 
 4. Useful links
Signature · 1 line
2.31.1
#4 lore raw

PATCH v8 3/9 i386: hardcode supported eVMCS version to '1'

In reply to Vitaly Kuznetsov

Currently, the only eVMCS version, supported by KVM (and described in TLFS) is '1'. When Enlightened VMCS feature is enabled, QEMU takes the supported eVMCS version range (from KVM_CAP_HYPERV_ENLIGHTENED_VMCS enablement) and puts it to guest visible CPUIDs. When (and if) eVMCS ver.2 appears a problem on migration is expected: it doesn't seem to be possible to migrate from a host supporting eVMCS ver.2 to a host, which only support eVMCS ver.1.

Hardcode eVMCS ver.1 as the result of 'hv-evmcs' enablement for now. Newer eVMCS versions will have to have their own enablement options (e.g. 'hv-evmcs=2').

---
 docs/hyperv.txt       |  2 +-
 target/i386/kvm/kvm.c | 39 +++++++++++++++++++++++++++++++++++----
 2 files changed, 36 insertions(+), 5 deletions(-)
docs/hyperv.txt +1 -1
index a51953daa833..000638a2fd38 100644
@@ -170,7 +170,7 @@ Recommended: hv-frequencies
 3.16. hv-evmcs
 ===============
 The enlightenment is nested specific, it targets Hyper-V on KVM guests. When
-enabled, it provides Enlightened VMCS feature to the guest. The feature
+enabled, it provides Enlightened VMCS version 1 feature to the guest. The feature
 implements paravirtualized protocol between L0 (KVM) and L1 (Hyper-V)
 hypervisors making L2 exits to the hypervisor faster. The feature is Intel-only.
 Note: some virtualization features (e.g. Posted Interrupts) are disabled when
target/i386/kvm/kvm.c +35 -4
index c676ee8b38a7..13d63f576b88 100644
@@ -1406,6 +1406,21 @@ static int hyperv_fill_cpuids(CPUState *cs,
 static Error *hv_passthrough_mig_blocker;
 static Error *hv_no_nonarch_cs_mig_blocker;
 
+/* Checks that the exposed eVMCS version range is supported by KVM */
+static bool evmcs_version_supported(uint16_t evmcs_version,
+                                    uint16_t supported_evmcs_version)
+{
+    uint8_t min_version = evmcs_version & 0xff;
+    uint8_t max_version = evmcs_version >> 8;
+    uint8_t min_supported_version = supported_evmcs_version & 0xff;
+    uint8_t max_supported_version = supported_evmcs_version >> 8;
+
+    return (min_version >= min_supported_version) &&
+        (max_version <= max_supported_version);
+}
+
+#define DEFAULT_EVMCS_VERSION ((1 << 8) | 1)
+
 static int hyperv_init_vcpu(X86CPU *cpu)
 {
     CPUState *cs = CPU(cpu);
@@ -1485,17 +1500,33 @@ static int hyperv_init_vcpu(X86CPU *cpu)
     }
 
     if (hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS)) {
-        uint16_t evmcs_version;
+        uint16_t evmcs_version = DEFAULT_EVMCS_VERSION;
+        uint16_t supported_evmcs_version;
 
         ret = kvm_vcpu_enable_cap(cs, KVM_CAP_HYPERV_ENLIGHTENED_VMCS, 0,
-                                  (uintptr_t)&evmcs_version);
+                                  (uintptr_t)&supported_evmcs_version);
 
+        /*
+         * KVM is required to support EVMCS ver.1. as that's what 'hv-evmcs'
+         * option sets. Note: we hardcode the maximum supported eVMCS version
+         * to '1' as well so 'hv-evmcs' feature is migratable even when (and if)
+         * ver.2 is implemented. A new option (e.g. 'hv-evmcs=2') will then have
+         * to be added.
+         */
         if (ret < 0) {
-            fprintf(stderr, "Hyper-V %s is not supported by kernel\n",
-                    kvm_hyperv_properties[HYPERV_FEAT_EVMCS].desc);
+            error_report("Hyper-V %s is not supported by kernel",
+                         kvm_hyperv_properties[HYPERV_FEAT_EVMCS].desc);
             return ret;
         }
 
+        if (!evmcs_version_supported(evmcs_version, supported_evmcs_version)) {
+            error_report("eVMCS version range [%d..%d] is not supported by "
+                         "kernel (supported: [%d..%d])", evmcs_version & 0xff,
+                         evmcs_version >> 8, supported_evmcs_version & 0xff,
+                         supported_evmcs_version >> 8);
+            return -ENOTSUP;
+        }
+
         cpu->hyperv_nested[0] = evmcs_version;
     }
 
Signature · 1 line
2.31.1
Reviewed-by on v8 3/9
#5 lore raw

Re: [PATCH v8 3/9] i386: hardcode supported eVMCS version to '1'

Quoted 7 lines · Vitaly Kuznetsov

Currently, the only eVMCS version, supported by KVM (and described in TLFS) is '1'. When Enlightened VMCS feature is enabled, QEMU takes the supported eVMCS version range (from KVM_CAP_HYPERV_ENLIGHTENED_VMCS enablement) and puts it to guest visible CPUIDs. When (and if) eVMCS ver.2 appears a problem on migration is expected: it doesn't seem to be possible to migrate from a host supporting eVMCS ver.2 to a host, which only support eVMCS ver.1.

Should we rewrite this as "it wouldn't be possible to migrate", as this patch fixes the problem and makes it possible?

Quoted 6 lines

Hardcode eVMCS ver.1 as the result of 'hv-evmcs' enablement for now. Newer eVMCS versions will have to have their own enablement options (e.g. 'hv-evmcs=2').

Signature · 1 line
Eduardo
#6 lore raw

Re: [PATCH v8 3/9] i386: hardcode supported eVMCS version to '1'

Quoted 11 lines · Eduardo Habkost
Quoted 7 lines · Vitaly Kuznetsov

Currently, the only eVMCS version, supported by KVM (and described in TLFS) is '1'. When Enlightened VMCS feature is enabled, QEMU takes the supported eVMCS version range (from KVM_CAP_HYPERV_ENLIGHTENED_VMCS enablement) and puts it to guest visible CPUIDs. When (and if) eVMCS ver.2 appears a problem on migration is expected: it doesn't seem to be possible to migrate from a host supporting eVMCS ver.2 to a host, which only support eVMCS ver.1.

Should we rewrite this as "it wouldn't be possible to migrate", as this patch fixes the problem and makes it possible?

Yes, no problem with such amendment. Currently, there's no issue as EVMCSv2 just doesn't exist. We, however, expect it to appear some time in the future and this change allows us to re-use KVM_CAP_HYPERV_ENLIGHTENED_VMCS in KVM without (potentially) breaking migrations. Note: the migration will only be broken when we migrate to KVM/QEMU which does not support EVMCSv2 *and* when the guest is already using it. As we expose the range of supported versions, it is possible that guests (esp. older Hyper-V versions) will stick to 'v1' even when 'v2' is supported.

Quoted 9 lines
Quoted 7 lines

Hardcode eVMCS ver.1 as the result of 'hv-evmcs' enablement for now. Newer eVMCS versions will have to have their own enablement options (e.g. 'hv-evmcs=2').

Thanks! Please let me know if expect v9 with amended commit message or if you're able to alter it upon commit.

Signature · 1 line
Vitaly
#7 lore raw

PATCH v8 4/9 i386: make hyperv_expand_features() return bool

In reply to Vitaly Kuznetsov

Return 'false' when hyperv_expand_features() sets an error.

No functional change intended.

---
 target/i386/kvm/kvm.c | 40 +++++++++++++++++++++-------------------
 1 file changed, 21 insertions(+), 19 deletions(-)
target/i386/kvm/kvm.c +21 -19
index 13d63f576b88..1e6f3c483e28 100644
@@ -1217,12 +1217,12 @@ static uint32_t hv_build_cpuid_leaf(CPUState *cs, uint32_t func, int reg)
  * of 'hv_passthrough' mode and fills the environment with all supported
  * Hyper-V features.
  */
-static void hyperv_expand_features(CPUState *cs, Error **errp)
+static bool hyperv_expand_features(CPUState *cs, Error **errp)
 {
     X86CPU *cpu = X86_CPU(cs);
 
     if (!hyperv_enabled(cpu))
-        return;
+        return true;
 
     if (cpu->hyperv_passthrough) {
         cpu->hyperv_vendor_id[0] =
@@ -1270,49 +1270,49 @@ static void hyperv_expand_features(CPUState *cs, Error **errp)
 
     /* Features */
     if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_RELAXED, errp)) {
-        return;
+        return false;
     }
     if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_VAPIC, errp)) {
-        return;
+        return false;
     }
     if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_TIME, errp)) {
-        return;
+        return false;
     }
     if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_CRASH, errp)) {
-        return;
+        return false;
     }
     if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_RESET, errp)) {
-        return;
+        return false;
     }
     if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_VPINDEX, errp)) {
-        return;
+        return false;
     }
     if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_RUNTIME, errp)) {
-        return;
+        return false;
     }
     if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_SYNIC, errp)) {
-        return;
+        return false;
     }
     if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_STIMER, errp)) {
-        return;
+        return false;
     }
     if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_FREQUENCIES, errp)) {
-        return;
+        return false;
     }
     if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_REENLIGHTENMENT, errp)) {
-        return;
+        return false;
     }
     if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_TLBFLUSH, errp)) {
-        return;
+        return false;
     }
     if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_EVMCS, errp)) {
-        return;
+        return false;
     }
     if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_IPI, errp)) {
-        return;
+        return false;
     }
     if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_STIMER_DIRECT, errp)) {
-        return;
+        return false;
     }
 
     /* Additional dependencies not covered by kvm_hyperv_properties[] */
@@ -1322,7 +1322,10 @@ static void hyperv_expand_features(CPUState *cs, Error **errp)
         error_setg(errp, "Hyper-V %s requires Hyper-V %s",
                    kvm_hyperv_properties[HYPERV_FEAT_SYNIC].desc,
                    kvm_hyperv_properties[HYPERV_FEAT_VPINDEX].desc);
+        return false;
     }
+
+    return true;
 }
 
 /*
@@ -1588,8 +1591,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
     env->apic_bus_freq = KVM_APIC_BUS_FREQUENCY;
 
     /* Paravirtualization CPUIDs */
-    hyperv_expand_features(cs, &local_err);
-    if (local_err) {
+    if (!hyperv_expand_features(cs, &local_err)) {
         error_report_err(local_err);
         return -ENOSYS;
     }
Signature · 1 line
2.31.1
#8 lore raw

PATCH v8 5/9 i386: expand Hyper-V features during CPU feature expansion time

In reply to Vitaly Kuznetsov

To make Hyper-V features appear in e.g. QMP query-cpu-model-expansion we need to expand and set the corresponding CPUID leaves early. Modify x86_cpu_get_supported_feature_word() to call newly intoduced Hyper-V specific kvm_hv_get_supported_cpuid() instead of kvm_arch_get_supported_cpuid(). We can't use kvm_arch_get_supported_cpuid() as Hyper-V specific CPUID leaves intersect with KVM's.

Note, early expansion will only happen when KVM supports system wide KVM_GET_SUPPORTED_HV_CPUID ioctl (KVM_CAP_SYS_HYPERV_CPUID).

---
 target/i386/cpu.c          |  4 ++++
 target/i386/kvm/kvm-stub.c |  5 +++++
 target/i386/kvm/kvm.c      | 24 ++++++++++++++++++++----
 target/i386/kvm/kvm_i386.h |  1 +
 4 files changed, 30 insertions(+), 4 deletions(-)
target/i386/cpu.c +4 -0
index f8ae45be0d53..c5d19216787c 100644
@@ -5990,6 +5990,10 @@ void x86_cpu_expand_features(X86CPU *cpu, Error **errp)
     if (env->cpuid_xlevel2 == UINT32_MAX) {
         env->cpuid_xlevel2 = env->cpuid_min_xlevel2;
     }
+
+    if (kvm_enabled()) {
+        kvm_hyperv_expand_features(cpu, errp);
+    }
 }
 
 /*
target/i386/kvm/kvm-stub.c +5 -0
index 92f49121b8fa..f6e7e4466e1a 100644
@@ -39,3 +39,8 @@ bool kvm_hv_vpindex_settable(void)
 {
     return false;
 }
+
+bool kvm_hyperv_expand_features(X86CPU *cpu, Error **errp)
+{
+    abort();
+}
target/i386/kvm/kvm.c +20 -4
index 1e6f3c483e28..b679dfdfc655 100644
@@ -1217,13 +1217,22 @@ static uint32_t hv_build_cpuid_leaf(CPUState *cs, uint32_t func, int reg)
  * of 'hv_passthrough' mode and fills the environment with all supported
  * Hyper-V features.
  */
-static bool hyperv_expand_features(CPUState *cs, Error **errp)
+bool kvm_hyperv_expand_features(X86CPU *cpu, Error **errp)
 {
-    X86CPU *cpu = X86_CPU(cs);
+    CPUState *cs = CPU(cpu);
 
     if (!hyperv_enabled(cpu))
         return true;
 
+    /*
+     * When kvm_hyperv_expand_features is called at CPU feature expansion
+     * time per-CPU kvm_state is not available yet so we can only proceed
+     * when KVM_CAP_SYS_HYPERV_CPUID is supported.
+     */
+    if (!cs->kvm_state &&
+        !kvm_check_extension(kvm_state, KVM_CAP_SYS_HYPERV_CPUID))
+        return true;
+
     if (cpu->hyperv_passthrough) {
         cpu->hyperv_vendor_id[0] =
             hv_cpuid_get_host(cs, HV_CPUID_VENDOR_AND_MAX_FUNCTIONS, R_EBX);
@@ -1590,8 +1599,15 @@ int kvm_arch_init_vcpu(CPUState *cs)
 
     env->apic_bus_freq = KVM_APIC_BUS_FREQUENCY;
 
-    /* Paravirtualization CPUIDs */
-    if (!hyperv_expand_features(cs, &local_err)) {
+    /*
+     * kvm_hyperv_expand_features() is called here for the second time in case
+     * KVM_CAP_SYS_HYPERV_CPUID is not supported. While we can't possibly handle
+     * 'query-cpu-model-expansion' in this case as we don't have a KVM vCPU to
+     * check which Hyper-V enlightenments are supported and which are not, we
+     * can still proceed and check/expand Hyper-V enlightenments here so legacy
+     * behavior is preserved.
+     */
+    if (!kvm_hyperv_expand_features(cpu, &local_err)) {
         error_report_err(local_err);
         return -ENOSYS;
     }
target/i386/kvm/kvm_i386.h +1 -0
index dc725083891c..54667b35f09c 100644
@@ -47,6 +47,7 @@ bool kvm_has_x2apic_api(void);
 bool kvm_has_waitpkg(void);
 
 bool kvm_hv_vpindex_settable(void);
+bool kvm_hyperv_expand_features(X86CPU *cpu, Error **errp);
 
 uint64_t kvm_swizzle_msi_ext_dest_id(uint64_t address);
 
Signature · 1 line
2.31.1
#9 lore raw

PATCH v8 6/9 i386: kill off hv_cpuid_check_and_set()

In reply to Vitaly Kuznetsov

hv_cpuid_check_and_set() does too much:
- Checks if the feature is supported by KVM;
- Checks if all dependencies are enabled;
- Sets the feature bit in cpu->hyperv_features for 'passthrough' mode.

To reduce the complexity, move all the logic except for dependencies check out of it. Also, in 'passthrough' mode we don't really need to check dependencies because KVM is supposed to provide a consistent set anyway.

---
 target/i386/kvm/kvm.c | 104 +++++++++++++++---------------------------
 1 file changed, 36 insertions(+), 68 deletions(-)
target/i386/kvm/kvm.c +36 -68
index b679dfdfc655..1cce0969067e 100644
@@ -1145,16 +1145,12 @@ static bool hyperv_feature_supported(CPUState *cs, int feature)
     return true;
 }
 
-static int hv_cpuid_check_and_set(CPUState *cs, int feature, Error **errp)
+/* Checks that all feature dependencies are enabled */
+static bool hv_feature_check_deps(X86CPU *cpu, int feature, Error **errp)
 {
-    X86CPU *cpu = X86_CPU(cs);
     uint64_t deps;
     int dep_feat;
 
-    if (!hyperv_feat_enabled(cpu, feature) && !cpu->hyperv_passthrough) {
-        return 0;
-    }
-
     deps = kvm_hyperv_properties[feature].dependencies;
     while (deps) {
         dep_feat = ctz64(deps);
@@ -1162,26 +1158,12 @@ static int hv_cpuid_check_and_set(CPUState *cs, int feature, Error **errp)
             error_setg(errp, "Hyper-V %s requires Hyper-V %s",
                        kvm_hyperv_properties[feature].desc,
                        kvm_hyperv_properties[dep_feat].desc);
-            return 1;
+            return false;
         }
         deps &= ~(1ull << dep_feat);
     }
 
-    if (!hyperv_feature_supported(cs, feature)) {
-        if (hyperv_feat_enabled(cpu, feature)) {
-            error_setg(errp, "Hyper-V %s is not supported by kernel",
-                       kvm_hyperv_properties[feature].desc);
-            return 1;
-        } else {
-            return 0;
-        }
-    }
-
-    if (cpu->hyperv_passthrough) {
-        cpu->hyperv_features |= BIT(feature);
-    }
-
-    return 0;
+    return true;
 }
 
 static uint32_t hv_build_cpuid_leaf(CPUState *cs, uint32_t func, int reg)
@@ -1220,6 +1202,8 @@ static uint32_t hv_build_cpuid_leaf(CPUState *cs, uint32_t func, int reg)
 bool kvm_hyperv_expand_features(X86CPU *cpu, Error **errp)
 {
     CPUState *cs = CPU(cpu);
+    Error *local_err = NULL;
+    int feat;
 
     if (!hyperv_enabled(cpu))
         return true;
@@ -1275,53 +1259,37 @@ bool kvm_hyperv_expand_features(X86CPU *cpu, Error **errp)
 
         cpu->hyperv_spinlock_attempts =
             hv_cpuid_get_host(cs, HV_CPUID_ENLIGHTMENT_INFO, R_EBX);
-    }
 
-    /* Features */
-    if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_RELAXED, errp)) {
-        return false;
-    }
-    if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_VAPIC, errp)) {
-        return false;
-    }
-    if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_TIME, errp)) {
-        return false;
-    }
-    if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_CRASH, errp)) {
-        return false;
-    }
-    if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_RESET, errp)) {
-        return false;
-    }
-    if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_VPINDEX, errp)) {
-        return false;
-    }
-    if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_RUNTIME, errp)) {
-        return false;
-    }
-    if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_SYNIC, errp)) {
-        return false;
-    }
-    if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_STIMER, errp)) {
-        return false;
-    }
-    if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_FREQUENCIES, errp)) {
-        return false;
-    }
-    if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_REENLIGHTENMENT, errp)) {
-        return false;
-    }
-    if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_TLBFLUSH, errp)) {
-        return false;
-    }
-    if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_EVMCS, errp)) {
-        return false;
-    }
-    if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_IPI, errp)) {
-        return false;
-    }
-    if (hv_cpuid_check_and_set(cs, HYPERV_FEAT_STIMER_DIRECT, errp)) {
-        return false;
+        /*
+         * Mark feature as enabled in 'cpu->hyperv_features' as
+         * hv_build_cpuid_leaf() uses this info to build guest CPUIDs.
+         */
+        for (feat = 0; feat < ARRAY_SIZE(kvm_hyperv_properties); feat++) {
+            if (hyperv_feature_supported(cs, feat)) {
+                cpu->hyperv_features |= BIT(feat);
+            }
+        }
+    } else {
+        /* Check features availability and dependencies */
+        for (feat = 0; feat < ARRAY_SIZE(kvm_hyperv_properties); feat++) {
+            /* If the feature was not requested skip it. */
+            if (!hyperv_feat_enabled(cpu, feat)) {
+                continue;
+            }
+
+            /* Check if the feature is supported by KVM */
+            if (!hyperv_feature_supported(cs, feat)) {
+                error_setg(errp, "Hyper-V %s is not supported by kernel",
+                           kvm_hyperv_properties[feat].desc);
+                return false;
+            }
+
+            /* Check dependencies */
+            if (!hv_feature_check_deps(cpu, feat, &local_err)) {
+                error_propagate(errp, local_err);
+                return false;
+            }
+        }
     }
 
     /* Additional dependencies not covered by kvm_hyperv_properties[] */
Signature · 1 line
2.31.1
#10 lore raw

PATCH v8 7/9 i386: HV_HYPERCALL_AVAILABLE privilege bit is always needed

In reply to Vitaly Kuznetsov

According to TLFS, Hyper-V guest is supposed to check HV_HYPERCALL_AVAILABLE privilege bit before accessing HV_X64_MSR_GUEST_OS_ID/HV_X64_MSR_HYPERCALL MSRs but at least some Windows versions ignore that. As KVM is very permissive and allows accessing these MSRs unconditionally, no issue is observed. We may, however, want to tighten the checks eventually. Conforming to the spec is probably also a good idea.

Enable HV_HYPERCALL_AVAILABLE bit unconditionally.

---
 target/i386/kvm/kvm.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
target/i386/kvm/kvm.c +5 -5
index 1cce0969067e..33830117fa31 100644
@@ -810,8 +810,6 @@ static struct {
     [HYPERV_FEAT_RELAXED] = {
         .desc = "relaxed timing (hv-relaxed)",
         .flags = {
-            {.func = HV_CPUID_FEATURES, .reg = R_EAX,
-             .bits = HV_HYPERCALL_AVAILABLE},
             {.func = HV_CPUID_ENLIGHTMENT_INFO, .reg = R_EAX,
              .bits = HV_RELAXED_TIMING_RECOMMENDED}
         }
@@ -820,7 +818,7 @@ static struct {
         .desc = "virtual APIC (hv-vapic)",
         .flags = {
             {.func = HV_CPUID_FEATURES, .reg = R_EAX,
-             .bits = HV_HYPERCALL_AVAILABLE | HV_APIC_ACCESS_AVAILABLE},
+             .bits = HV_APIC_ACCESS_AVAILABLE},
             {.func = HV_CPUID_ENLIGHTMENT_INFO, .reg = R_EAX,
              .bits = HV_APIC_ACCESS_RECOMMENDED}
         }
@@ -829,8 +827,7 @@ static struct {
         .desc = "clocksources (hv-time)",
         .flags = {
             {.func = HV_CPUID_FEATURES, .reg = R_EAX,
-             .bits = HV_HYPERCALL_AVAILABLE | HV_TIME_REF_COUNT_AVAILABLE |
-             HV_REFERENCE_TSC_AVAILABLE}
+             .bits = HV_TIME_REF_COUNT_AVAILABLE | HV_REFERENCE_TSC_AVAILABLE}
         }
     },
     [HYPERV_FEAT_CRASH] = {
@@ -1343,6 +1340,9 @@ static int hyperv_fill_cpuids(CPUState *cs,
     c->ebx = hv_build_cpuid_leaf(cs, HV_CPUID_FEATURES, R_EBX);
     c->edx = hv_build_cpuid_leaf(cs, HV_CPUID_FEATURES, R_EDX);
 
+    /* Unconditionally required with any Hyper-V enlightenment */
+    c->eax |= HV_HYPERCALL_AVAILABLE;
+
     /* Not exposed by KVM but needed to make CPU hotplug in Windows work */
     c->edx |= HV_CPU_DYNAMIC_PARTITIONING_AVAILABLE;
 
Signature · 1 line
2.31.1
#11 lore raw

PATCH v8 8/9 i386: Hyper-V SynIC requires POST_MESSAGES/SIGNAL_EVENTS privileges

In reply to Vitaly Kuznetsov

When Hyper-V SynIC is enabled, we may need to allow Windows guests to make hypercalls (POST_MESSAGES/SIGNAL_EVENTS). No issue is currently observed because KVM is very permissive, allowing these hypercalls regarding of guest visible CPUid bits.

---
 target/i386/kvm/hyperv-proto.h | 6 ++++++
 target/i386/kvm/kvm.c          | 6 ++++++
 2 files changed, 12 insertions(+)
target/i386/kvm/hyperv-proto.h +6 -0
index e30d64b4ade4..5fbb385cc136 100644
@@ -38,6 +38,12 @@
 #define HV_ACCESS_FREQUENCY_MSRS     (1u << 11)
 #define HV_ACCESS_REENLIGHTENMENTS_CONTROL  (1u << 13)
 
+/*
+ * HV_CPUID_FEATURES.EBX bits
+ */
+#define HV_POST_MESSAGES             (1u << 4)
+#define HV_SIGNAL_EVENTS             (1u << 5)
+
 /*
  * HV_CPUID_FEATURES.EDX bits
  */
target/i386/kvm/kvm.c +6 -0
index 33830117fa31..260c563d59a3 100644
@@ -1343,6 +1343,12 @@ static int hyperv_fill_cpuids(CPUState *cs,
     /* Unconditionally required with any Hyper-V enlightenment */
     c->eax |= HV_HYPERCALL_AVAILABLE;
 
+    /* SynIC and Vmbus devices require messages/signals hypercalls */
+    if (hyperv_feat_enabled(cpu, HYPERV_FEAT_SYNIC) &&
+        !cpu->hyperv_synic_kvm_only) {
+        c->ebx |= HV_POST_MESSAGES | HV_SIGNAL_EVENTS;
+    }
+
     /* Not exposed by KVM but needed to make CPU hotplug in Windows work */
     c->edx |= HV_CPU_DYNAMIC_PARTITIONING_AVAILABLE;
 
Signature · 1 line
2.31.1
#12 lore raw

PATCH v8 9/9 qtest/hyperv: Introduce a simple hyper-v test

In reply to Vitaly Kuznetsov

For the beginning, just test 'hv-passthrough' and a couple of custom Hyper-V enlightenments configurations through QMP. Later, it would be great to complement this by checking CPUID values from within the guest.

---
 MAINTAINERS               |   1 +
 tests/qtest/hyperv-test.c | 221 ++++++++++++++++++++++++++++++++++++++
 tests/qtest/meson.build   |   3 +-
 3 files changed, 224 insertions(+), 1 deletion(-)
 create mode 100644 tests/qtest/hyperv-test.c
MAINTAINERS +1 -0
index 7d9cd2904264..6345bad461e8 100644
@@ -1545,6 +1545,7 @@ F: hw/isa/apm.c
 F: include/hw/isa/apm.h
 F: tests/unit/test-x86-cpuid.c
 F: tests/qtest/test-x86-cpuid-compat.c
+F: tests/qtest/hyperv-test.c
 
 PC Chipset
 M: Michael S. Tsirkin <[email protected]>
tests/qtest/hyperv-test.c new file +221 -0

221 lines not on this page · raw message

tests/qtest/meson.build +2 -1
index c3a223a83d6a..958a88d0c8b4 100644
@@ -83,7 +83,8 @@ qtests_i386 = \
    'vmgenid-test',
    'migration-test',
    'test-x86-cpuid-compat',
-   'numa-test']
+   'numa-test',
+   'hyperv-test']
 
 dbus_daemon = find_program('dbus-daemon', required: false)
 if dbus_daemon.found() and config_host.has_key('GDBUS_CODEGEN')
Signature · 1 line
2.31.1
Acked-by on v8 9/9
#13 lore raw

Re: [PATCH v8 9/9] qtest/hyperv: Introduce a simple hyper-v test

Quoted 6 lines · Vitaly Kuznetsov

For the beginning, just test 'hv-passthrough' and a couple of custom Hyper-V enlightenments configurations through QMP. Later, it would be great to complement this by checking CPUID values from within the guest.

Signature · 1 line
Eduardo
#14 lore raw

Re: [PATCH v8 9/9] qtest/hyperv: Introduce a simple hyper-v test

In reply to Vitaly Kuznetsov
Quoted 6 lines · Vitaly Kuznetsov

For the beginning, just test 'hv-passthrough' and a couple of custom Hyper-V enlightenments configurations through QMP. Later, it would be great to complement this by checking CPUID values from within the guest.

[...]

tests/qtest/hyperv-test.c line 45

Quoted 6 lines
tests/qtest/hyperv-test.c new file +6 -0
+static bool kvm_has_sys_hyperv_cpuid(void)
+{
+    int fd = open("/dev/kvm", O_RDWR);
+    int ret;
+
+    g_assert(fd > 0);

This crashes when /dev/kvm doesn't exist. See:
https://gitlab.com/ehabkost/qemu/-/jobs/1404084459

I'm removing it from the queue.

Signature · 1 line
Eduardo
#15 lore raw

Re: [PATCH v8 9/9] qtest/hyperv: Introduce a simple hyper-v test

Quoted 8 lines · On Thu, 8 Jul 2021 17:02:22 -0400 Eduardo Habkost <[email protected]> wrote:
Quoted 6 lines · Vitaly Kuznetsov

For the beginning, just test 'hv-passthrough' and a couple of custom Hyper-V enlightenments configurations through QMP. Later, it would be great to complement this by checking CPUID values from within the guest.

[...]

Quoted 9 lines

tests/qtest/hyperv-test.c line 45

Quoted 6 lines
tests/qtest/hyperv-test.c new file +6 -0
+static bool kvm_has_sys_hyperv_cpuid(void)
+{
+    int fd = open("/dev/kvm", O_RDWR);
+    int ret;
+
+    g_assert(fd > 0);  

This crashes when /dev/kvm doesn't exist. See:
https://gitlab.com/ehabkost/qemu/-/jobs/1404084459

maybe reuse qtest_has_accel()
https://lists.gnu.org/archive/html/qemu-devel/2021-06/msg06864.html

instead of op encoding it.

Quoted 2 lines

I'm removing it from the queue.

#16 lore raw

Re: [PATCH v8 9/9] qtest/hyperv: Introduce a simple hyper-v test

Quoted 10 lines · Igor Mammedov
Quoted 7 lines · On Thu, 8 Jul 2021 17:02:22 -0400 Eduardo Habkost <[email protected]> wrote:
Quoted 6 lines · Vitaly Kuznetsov

For the beginning, just test 'hv-passthrough' and a couple of custom Hyper-V enlightenments configurations through QMP. Later, it would be great to complement this by checking CPUID values from within the guest.

Quoted 8 lines
Quoted 8 lines

[...]

tests/qtest/hyperv-test.c line 45

Quoted 6 lines
tests/qtest/hyperv-test.c new file +6 -0
+static bool kvm_has_sys_hyperv_cpuid(void)
+{
+    int fd = open("/dev/kvm", O_RDWR);
+    int ret;
+
+    g_assert(fd > 0);  

g_assert() was an overkill, just 'return false' would do.

Quoted 7 lines
Quoted 2 lines

This crashes when /dev/kvm doesn't exist. See:
https://gitlab.com/ehabkost/qemu/-/jobs/1404084459

maybe reuse qtest_has_accel()
https://lists.gnu.org/archive/html/qemu-devel/2021-06/msg06864.html

instead of op encoding it.

The purpose of this function is to check if KVM_CAP_SYS_HYPERV_CPUID is supported by KVM. It is certainly unsupported when KVM is not present :-) but an ioctl() is needed when it is.

We already have a similar check in tests/qtest/migration-test.c where we test for KVM_CAP_DIRTY_LOG_RING, maybe we can create a library function but we don't seem to have any KVM-specific stuff in qtest at this moment
...

Quoted 1 line
Quoted 1 line

I'm removing it from the queue.

I'll fix g_assert() and send as a separate patch if it's fine.

Signature · 1 line
Vitaly
#17 lore raw

Re: [PATCH v8 9/9] qtest/hyperv: Introduce a simple hyper-v test

Quoted 31 lines · On Fri, 16 Jul 2021 14:12:06 +0200 Vitaly Kuznetsov <[email protected]> wrote:
Quoted 10 lines · Igor Mammedov
Quoted 7 lines · On Thu, 8 Jul 2021 17:02:22 -0400 Eduardo Habkost <[email protected]> wrote:
Quoted 6 lines · Vitaly Kuznetsov

For the beginning, just test 'hv-passthrough' and a couple of custom Hyper-V enlightenments configurations through QMP. Later, it would be great to complement this by checking CPUID values from within the guest.

Quoted 8 lines
Quoted 8 lines

[...]

tests/qtest/hyperv-test.c line 45

Quoted 6 lines
tests/qtest/hyperv-test.c new file +6 -0
+static bool kvm_has_sys_hyperv_cpuid(void)
+{
+    int fd = open("/dev/kvm", O_RDWR);
+    int ret;
+
+    g_assert(fd > 0);    

g_assert() was an overkill, just 'return false' would do.

Quoted 7 lines
Quoted 2 lines

This crashes when /dev/kvm doesn't exist. See:
https://gitlab.com/ehabkost/qemu/-/jobs/1404084459

maybe reuse qtest_has_accel()
https://lists.gnu.org/archive/html/qemu-devel/2021-06/msg06864.html

instead of op encoding it.

Quoted 7 lines

The purpose of this function is to check if KVM_CAP_SYS_HYPERV_CPUID is supported by KVM. It is certainly unsupported when KVM is not present :-) but an ioctl() is needed when it is.

We already have a similar check in tests/qtest/migration-test.c where we test for KVM_CAP_DIRTY_LOG_RING, maybe we can create a library function but we don't seem to have any KVM-specific stuff in qtest at this moment

qtest_has_accel() is a such library function
in the same series see https://lists.gnu.org/archive/html/qemu-devel/2021-06/msg06878.html
which replaces custom kvm probing in tests/qtest/migration-test.c

Quoted 6 lines

...

Quoted 1 line
Quoted 1 line

I'm removing it from the queue.

I'll fix g_assert() and send as a separate patch if it's fine.

#18 lore raw
In reply to Vitaly Kuznetsov
Quoted 18 lines · Vitaly Kuznetsov

Changes since v7:
- Make eVMCS version check future proof [Eduardo]
- Collect R-b tags [Eduardo]
- Drop 'if (!strcmp(arch, "i386") || !strcmp(arch, "x86_64"))' check from qtest [Eduardo]
- s/priviliges/privileges/ [Eric]

The last two functional patches are inspired by 'Fine-grained access check to Hyper-V hypercalls and MSRs' work for KVM:
https://lore.kernel.org/kvm/[email protected]/

Original description:

Upper layer tools like libvirt want to figure out which Hyper-V features are supported by the underlying stack (QEMU/KVM) but currently they are unable to do so. We have a nice 'hv_passthrough' CPU flag supported by QEMU but it has no effect on e.g. QMP's

Quoted 7 lines

query-cpu-model-expansion type=full model={"name":"host","props":{"hv-passthrough":true}}

command as we parse Hyper-V features after creating KVM vCPUs and not at feature expansion time. To support the use-case we first need to make KVM_GET_SUPPORTED_HV_CPUID ioctl a system-wide ioctl as the existing vCPU version can't be used that early. This is what KVM part does. With that done, we can make early Hyper-V feature expansion (this series).

I'm finally queueing this (please ignore my reply to v7). Thanks, and sorry for the delay!

Signature · 1 line
Eduardo