Skip to contentSkip to conversation

target/i386: Restrict system-specific features from user emulation

PATCH In review Philippe Mathieu-Daudé 3 participants 10 messages Sep 13, 2023 KVM

lore
Share & tools
lore
#0 lore raw

v5 cover target/i386: Restrict system-specific features from user emulation

Since v4:
- Addressed Paolo's suggestions (clearly better)

Too many system-specific code (and in particular KVM related) is pulled in user-only build. This led to adding unjustified stubs as kludge to unagressive linker non-optimizations.

This series restrict x86 system-specific features to sysemu, so we don't require any stub, and remove all x86 KVM declarations from user emulation code (to trigger compile failure instead of link one).

Philippe Mathieu-Daudé (6):
target/i386: Check kvm_hyperv_expand_features() return value target/i386: Drop accel_uses_host_cpuid before
x86_cpu_get_supported_cpuid
target/i386: Call accel-agnostic x86_cpu_get_supported_cpuid() target/i386: Move x86_cpu_get_migratable_flags() around RFC target/i386: Restrict system-specific code from user emulation target/i386: Prohibit target specific KVM prototypes on user emulation

 target/i386/kvm/kvm_i386.h |   4 ++
 target/i386/cpu.c          | 134 +++++++++++++++++++++++--------------
 2 files changed, 88 insertions(+), 50 deletions(-)
Signature · 1 line
2.41.0
#1 lore raw

PATCH v5 1/6 target/i386: Check kvm_hyperv_expand_features() return value

In case more code is added after the kvm_hyperv_expand_features() call, check its return value (since it can fail).

Fixes 071ce4b03b ("i386: expand Hyper-V features during CPU feature expansion time")
---
 target/i386/cpu.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
target/i386/cpu.c +2 -2
index 24ee67b42d..bd6a932d08 100644
@@ -7121,8 +7121,8 @@ void x86_cpu_expand_features(X86CPU *cpu, Error **errp)
         env->cpuid_xlevel2 = env->cpuid_min_xlevel2;
     }
 
-    if (kvm_enabled()) {
-        kvm_hyperv_expand_features(cpu, errp);
+    if (kvm_enabled() && !kvm_hyperv_expand_features(cpu, errp)) {
+        return;
     }
 }
 
Signature · 1 line
2.41.0
#2 lore raw

PATCH v5 2/6 target/i386: Drop accel_uses_host_cpuid before x86_cpu_get_supported_cpuid

In reply to Philippe Mathieu-Daudé

x86_cpu_get_supported_cpuid() already checks for KVM/HVF accelerators, so it is not needed to manually check it via a call to accel_uses_host_cpuid() before calling it.

---
 target/i386/cpu.c | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)
target/i386/cpu.c +4 -6
index bd6a932d08..94b1ba0cf1 100644
@@ -6207,7 +6207,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
         break;
     case 0xA:
         /* Architectural Performance Monitoring Leaf */
-        if (accel_uses_host_cpuid() && cpu->enable_pmu) {
+        if (cpu->enable_pmu) {
             x86_cpu_get_supported_cpuid(0xA, count, eax, ebx, ecx, edx);
         } else {
             *eax = 0;
@@ -6247,8 +6247,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
         *ebx &= 0xffff; /* The count doesn't need to be reliable. */
         break;
     case 0x1C:
-        if (accel_uses_host_cpuid() && cpu->enable_pmu &&
-            (env->features[FEAT_7_0_EDX] & CPUID_7_0_EDX_ARCH_LBR)) {
+        if (cpu->enable_pmu && (env->features[FEAT_7_0_EDX] & CPUID_7_0_EDX_ARCH_LBR)) {
             x86_cpu_get_supported_cpuid(0x1C, 0, eax, ebx, ecx, edx);
             *edx = 0;
         }
@@ -6322,9 +6321,8 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
             } else {
                 *ecx &= ~XSTATE_ARCH_LBR_MASK;
             }
-        } else if (count == 0xf &&
-                   accel_uses_host_cpuid() && cpu->enable_pmu &&
-                   (env->features[FEAT_7_0_EDX] & CPUID_7_0_EDX_ARCH_LBR)) {
+        } else if (count == 0xf && cpu->enable_pmu
+                   && (env->features[FEAT_7_0_EDX] & CPUID_7_0_EDX_ARCH_LBR)) {
             x86_cpu_get_supported_cpuid(0xD, count, eax, ebx, ecx, edx);
         } else if (count < ARRAY_SIZE(x86_ext_save_areas)) {
             const ExtSaveArea *esa = &x86_ext_save_areas[count];
Signature · 1 line
2.41.0
#3 lore raw

PATCH v5 3/6 target/i386: Call accel-agnostic x86_cpu_get_supported_cpuid()

In reply to Philippe Mathieu-Daudé

x86_cpu_get_supported_cpuid() is generic and handles the different accelerators. Use it instead of kvm_arch_get_supported_cpuid().

That fixes a link failure introduced by commit 3adce820cf ("target/i386: Remove unused KVM stubs") when QEMU is configured as:

  $ ./configure --cc=clang \
    --target-list=x86_64-linux-user,x86_64-softmmu \
    --enable-debug

We were getting:

[71/71] Linking target qemu-x86_64
FAILED: qemu-x86_64
/usr/bin/ld: libqemu-x86_64-linux-user.fa.p/target_i386_cpu.c.o: in function `cpu_x86_cpuid':
cpu.c:(.text+0x1374): undefined reference to `kvm_arch_get_supported_cpuid'
/usr/bin/ld: libqemu-x86_64-linux-user.fa.p/target_i386_cpu.c.o: in function `x86_cpu_filter_features':
cpu.c:(.text+0x81c2): undefined reference to `kvm_arch_get_supported_cpuid'
/usr/bin/ld: cpu.c:(.text+0x81da): undefined reference to `kvm_arch_get_supported_cpuid'
/usr/bin/ld: cpu.c:(.text+0x81f2): undefined reference to `kvm_arch_get_supported_cpuid'
/usr/bin/ld: cpu.c:(.text+0x820a): undefined reference to `kvm_arch_get_supported_cpuid'
/usr/bin/ld: libqemu-x86_64-linux-user.fa.p/target_i386_cpu.c.o:cpu.c:(.text+0x8225): more undefined references to `kvm_arch_get_supported_cpuid' follow
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

For the record, this is because '--enable-debug' disables optimizations (CFLAGS=-O0).

While at this (un)optimization level GCC eliminate the following dead code (CPP output of mentioned build):

static void x86_cpu_get_supported_cpuid(uint32_t func, uint32_t index,

                                         uint32_t *eax, uint32_t *ebx,
                                         uint32_t *ecx, uint32_t *edx)
 {
     if ((0)) {
         *eax = kvm_arch_get_supported_cpuid(kvm_state, func, index, R_EAX);
         *ebx = kvm_arch_get_supported_cpuid(kvm_state, func, index, R_EBX);
         *ecx = kvm_arch_get_supported_cpuid(kvm_state, func, index, R_ECX);
         *edx = kvm_arch_get_supported_cpuid(kvm_state, func, index, R_EDX);
     } else if (0) {
         *eax = 0;
         *ebx = 0;
         *ecx = 0;
         *edx = 0;
     } else {
         *eax = 0;
         *ebx = 0;
         *ecx = 0;
         *edx = 0;
     }

Clang does not (see commit 2140cfa51d "i386: Fix build by providing stub kvm_arch_get_supported_cpuid()").

Fixes 3adce820cf ("target/i386: Remove unused KVM stubs")
---
 target/i386/cpu.c | 32 ++++++++++++++++----------------
 1 file changed, 16 insertions(+), 16 deletions(-)
target/i386/cpu.c +16 -16
index 94b1ba0cf1..b2a20365e1 100644
@@ -6154,6 +6154,8 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
     case 7:
         /* Structured Extended Feature Flags Enumeration Leaf */
         if (count == 0) {
+            uint32_t eax_0_unused, ebx_0, ecx_0, edx_0_unused;
+
             /* Maximum ECX value for sub-leaves */
             *eax = env->cpuid_level_func7;
             *ebx = env->features[FEAT_7_0_EBX]; /* Feature flags */
@@ -6168,17 +6170,15 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
              * support enabling SGX and/or SGX flexible launch control,
              * then we need to update the VM's CPUID values accordingly.
              */
-            if ((*ebx & CPUID_7_0_EBX_SGX) &&
-                (!kvm_enabled() ||
-                 !(kvm_arch_get_supported_cpuid(cs->kvm_state, 0x7, 0, R_EBX) &
-                    CPUID_7_0_EBX_SGX))) {
+            x86_cpu_get_supported_cpuid(0x7, 0,
+                                        &eax_0_unused, &ebx_0,
+                                        &ecx_0, &edx_0_unused);
+            if ((*ebx & CPUID_7_0_EBX_SGX) && !(ebx_0 & CPUID_7_0_EBX_SGX)) {
                 *ebx &= ~CPUID_7_0_EBX_SGX;
             }
 
-            if ((*ecx & CPUID_7_0_ECX_SGX_LC) &&
-                (!(*ebx & CPUID_7_0_EBX_SGX) || !kvm_enabled() ||
-                 !(kvm_arch_get_supported_cpuid(cs->kvm_state, 0x7, 0, R_ECX) &
-                    CPUID_7_0_ECX_SGX_LC))) {
+            if ((*ecx & CPUID_7_0_ECX_SGX_LC)
+                    && (!(*ebx & CPUID_7_0_EBX_SGX) || !(ecx_0 & CPUID_7_0_ECX_SGX_LC))) {
                 *ecx &= ~CPUID_7_0_ECX_SGX_LC;
             }
         } else if (count == 1) {
@@ -7150,14 +7150,14 @@ static void x86_cpu_filter_features(X86CPU *cpu, bool verbose)
         mark_unavailable_features(cpu, w, unavailable_features, prefix);
     }
 
-    if ((env->features[FEAT_7_0_EBX] & CPUID_7_0_EBX_INTEL_PT) &&
-        kvm_enabled()) {
-        KVMState *s = CPU(cpu)->kvm_state;
-        uint32_t eax_0 = kvm_arch_get_supported_cpuid(s, 0x14, 0, R_EAX);
-        uint32_t ebx_0 = kvm_arch_get_supported_cpuid(s, 0x14, 0, R_EBX);
-        uint32_t ecx_0 = kvm_arch_get_supported_cpuid(s, 0x14, 0, R_ECX);
-        uint32_t eax_1 = kvm_arch_get_supported_cpuid(s, 0x14, 1, R_EAX);
-        uint32_t ebx_1 = kvm_arch_get_supported_cpuid(s, 0x14, 1, R_EBX);
+    if (env->features[FEAT_7_0_EBX] & CPUID_7_0_EBX_INTEL_PT) {
+        uint32_t eax_0, ebx_0, ecx_0, edx_0_unused;
+        uint32_t eax_1, ebx_1, ecx_1_unused, edx_1_unused;
+
+        x86_cpu_get_supported_cpuid(0x14, 0,
+                                    &eax_0, &ebx_0, &ecx_0, &edx_0_unused);
+        x86_cpu_get_supported_cpuid(0x14, 1,
+                                    &eax_1, &ebx_1, &ecx_1_unused, &edx_1_unused);
 
         if (!eax_0 ||
            ((ebx_0 & INTEL_PT_MINIMAL_EBX) != INTEL_PT_MINIMAL_EBX) ||
Signature · 1 line
2.41.0
#4 lore raw

Re: [PATCH v5 3/6] target/i386: Call accel-agnostic x86_cpu_get_supported_cpuid()

13.09.2023 12:30, Philippe Mathieu-Daudé:

Quoted 6 lines

x86_cpu_get_supported_cpuid() is generic and handles the different accelerators. Use it instead of kvm_arch_get_supported_cpuid().

That fixes a link failure introduced by commit 3adce820cf ("target/i386: Remove unused KVM stubs") when QEMU is configured as:

..

Quoted 2 lines
Fixes 3adce820cf ("target/i386: Remove unused KVM stubs")

This hopefully should not go to -stable as the issue has been introduced in v8.1.0-291-g3adce820cf which is past latest release. That's okay though, - just a small side-note.

/mjt

#5 lore raw

Re: [PATCH v5 3/6] target/i386: Call accel-agnostic x86_cpu_get_supported_cpuid()

Quoted 7 lines · Michael Tokarev

13.09.2023 12:30, Philippe Mathieu-Daudé:

Quoted 6 lines

x86_cpu_get_supported_cpuid() is generic and handles the different accelerators. Use it instead of kvm_arch_get_supported_cpuid().

That fixes a link failure introduced by commit 3adce820cf ("target/i386: Remove unused KVM stubs") when QEMU is configured as:

Quoted 7 lines

..

Quoted 2 lines
Fixes 3adce820cf ("target/i386: Remove unused KVM stubs")

This hopefully should not go to -stable as the issue has been introduced in v8.1.0-291-g3adce820cf which is past latest release. That's okay though, - just a small side-note.

Oh indeed. Paolo, do you mind removing the tag?

Thanks both!

#6 lore raw

PATCH v5 4/6 target/i386: Move x86_cpu_get_migratable_flags() around

In reply to Philippe Mathieu-Daudé

x86_cpu_get_migratable_flags() is only used once in x86_cpu_get_supported_feature_word(). Move it the code just before its caller, to reduce #ifdef'ry in the next commit, when we restrict both functions to system emulation.

---
 target/i386/cpu.c | 46 +++++++++++++++++++++++-----------------------
 1 file changed, 23 insertions(+), 23 deletions(-)
target/i386/cpu.c +23 -23
index b2a20365e1..3df85a6347 100644
@@ -1664,29 +1664,6 @@ static inline uint64_t x86_cpu_xsave_xss_components(X86CPU *cpu)
            cpu->env.features[FEAT_XSAVE_XSS_LO];
 }
 
-/*
- * Returns the set of feature flags that are supported and migratable by
- * QEMU, for a given FeatureWord.
- */
-static uint64_t x86_cpu_get_migratable_flags(FeatureWord w)
-{
-    FeatureWordInfo *wi = &feature_word_info[w];
-    uint64_t r = 0;
-    int i;
-
-    for (i = 0; i < 64; i++) {
-        uint64_t f = 1ULL << i;
-
-        /* If the feature name is known, it is implicitly considered migratable,
-         * unless it is explicitly set in unmigratable_flags */
-        if ((wi->migratable_flags & f) ||
-            (wi->feat_names[i] && !(wi->unmigratable_flags & f))) {
-            r |= f;
-        }
-    }
-    return r;
-}
-
 void host_cpuid(uint32_t function, uint32_t count,
                 uint32_t *eax, uint32_t *ebx, uint32_t *ecx, uint32_t *edx)
 {
@@ -5679,6 +5656,29 @@ CpuDefinitionInfoList *qmp_query_cpu_definitions(Error **errp)
 
 #endif /* !CONFIG_USER_ONLY */
 
+/*
+ * Returns the set of feature flags that are supported and migratable by
+ * QEMU, for a given FeatureWord.
+ */
+static uint64_t x86_cpu_get_migratable_flags(FeatureWord w)
+{
+    FeatureWordInfo *wi = &feature_word_info[w];
+    uint64_t r = 0;
+    int i;
+
+    for (i = 0; i < 64; i++) {
+        uint64_t f = 1ULL << i;
+
+        /* If the feature name is known, it is implicitly considered migratable,
+         * unless it is explicitly set in unmigratable_flags */
+        if ((wi->migratable_flags & f) ||
+            (wi->feat_names[i] && !(wi->unmigratable_flags & f))) {
+            r |= f;
+        }
+    }
+    return r;
+}
+
 uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
                                             bool migratable_only)
 {
Signature · 1 line
2.41.0
#7 lore raw

RFC PATCH v5 5/6 target/i386: Restrict system-specific code from user emulation

In reply to Philippe Mathieu-Daudé

Restrict calls to:
- kvm_arch_get_supported_cpuid()
- kvm_arch_get_supported_msr_feature()
- kvm_request_xsave_components()
- kvm_hyperv_expand_features()
so we can remove restrict "kvm/kvm_i386.h" and all its declarations to system emulation (see the next commit).

---
 target/i386/cpu.c | 41 ++++++++++++++++++++++++++++++++++++++---
 1 file changed, 38 insertions(+), 3 deletions(-)
target/i386/cpu.c +38 -3
index 3df85a6347..c201ff26bd 100644
@@ -5654,8 +5654,6 @@ CpuDefinitionInfoList *qmp_query_cpu_definitions(Error **errp)
     return cpu_list;
 }
 
-#endif /* !CONFIG_USER_ONLY */
-
 /*
  * Returns the set of feature flags that are supported and migratable by
  * QEMU, for a given FeatureWord.
@@ -5781,6 +5779,38 @@ static void x86_cpu_get_cache_cpuid(uint32_t func, uint32_t index,
     }
 }
 
+#else /* CONFIG_USER_ONLY */
+
+uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
+                                            bool migratable_only)
+{
+    FeatureWordInfo *wi = &feature_word_info[w];
+
+    return wi->tcg_features;
+}
+
+static void x86_cpu_get_supported_cpuid(uint32_t func, uint32_t index,
+                                        uint32_t *eax, uint32_t *ebx,
+                                        uint32_t *ecx, uint32_t *edx)
+{
+    *eax = 0;
+    *ebx = 0;
+    *ecx = 0;
+    *edx = 0;
+}
+
+static void x86_cpu_get_cache_cpuid(uint32_t func, uint32_t index,
+                                    uint32_t *eax, uint32_t *ebx,
+                                    uint32_t *ecx, uint32_t *edx)
+{
+    *eax = 0;
+    *ebx = 0;
+    *ecx = 0;
+    *edx = 0;
+}
+
+#endif /* !CONFIG_USER_ONLY */
+
 /*
  * Only for builtin_x86_defs models initialized with x86_register_cpudef_types.
  */
@@ -6918,7 +6948,6 @@ static void x86_cpu_enable_xsave_components(X86CPU *cpu)
     CPUX86State *env = &cpu->env;
     int i;
     uint64_t mask;
-    static bool request_perm;
 
     if (!(env->features[FEAT_1_ECX] & CPUID_EXT_XSAVE)) {
         env->features[FEAT_XSAVE_XCR0_LO] = 0;
@@ -6934,11 +6963,15 @@ static void x86_cpu_enable_xsave_components(X86CPU *cpu)
         }
     }
 
+#ifndef CONFIG_USER_ONLY
+    static bool request_perm;
+
     /* Only request permission for first vcpu */
     if (kvm_enabled() && !request_perm) {
         kvm_request_xsave_components(cpu, mask);
         request_perm = true;
     }
+#endif /* !CONFIG_USER_ONLY */
 
     env->features[FEAT_XSAVE_XCR0_LO] = mask & CPUID_XSTATE_XCR0_MASK;
     env->features[FEAT_XSAVE_XCR0_HI] = mask >> 32;
@@ -7119,9 +7152,11 @@ void x86_cpu_expand_features(X86CPU *cpu, Error **errp)
         env->cpuid_xlevel2 = env->cpuid_min_xlevel2;
     }
 
+#ifndef CONFIG_USER_ONLY
     if (kvm_enabled() && !kvm_hyperv_expand_features(cpu, errp)) {
         return;
     }
+#endif /* !CONFIG_USER_ONLY */
 }
 
 /*
Signature · 1 line
2.41.0
#8 lore raw

PATCH v5 6/6 target/i386: Prohibit target specific KVM prototypes on user emulation

In reply to Philippe Mathieu-Daudé

None of these target-specific prototypes should be used by user emulation. Remove their declaration there, so we get a compile failure if ever used (instead of having to deal with linker and its possible optimizations, such dead code removal).

---
 target/i386/kvm/kvm_i386.h | 4 ++++
 target/i386/cpu.c          | 3 ++-
 2 files changed, 6 insertions(+), 1 deletion(-)
target/i386/kvm/kvm_i386.h +4 -0
index 55d4e68c34..5ef73f0a1c 100644
@@ -13,6 +13,10 @@
 
 #include "sysemu/kvm.h"
 
+#ifdef CONFIG_USER_ONLY
+#error Cannot include kvm_i386.h from user emulation
+#endif
+
 #ifdef CONFIG_KVM
 
 #define kvm_pit_in_kernel() \
target/i386/cpu.c +2 -1
index c201ff26bd..db8ed6284d 100644
@@ -26,7 +26,7 @@
 #include "tcg/helper-tcg.h"
 #include "sysemu/reset.h"
 #include "sysemu/hvf.h"
-#include "kvm/kvm_i386.h"
+#include "sysemu/kvm.h"
 #include "sev.h"
 #include "qapi/error.h"
 #include "qemu/error-report.h"
@@ -40,6 +40,7 @@
 #include "exec/address-spaces.h"
 #include "hw/boards.h"
 #include "hw/i386/sgx-epc.h"
+#include "kvm/kvm_i386.h"
 #endif
 
 #include "disas/capstone.h"
Signature · 1 line
2.41.0
maintainer
#9 lore raw
In reply to Philippe Mathieu-Daudé
Quoted 11 lines · Philippe Mathieu-Daudé

Since v4:
- Addressed Paolo's suggestions (clearly better)

Too many system-specific code (and in particular KVM related) is pulled in user-only build. This led to adding unjustified stubs as kludge to unagressive linker non-optimizations.

This series restrict x86 system-specific features to sysemu, so we don't require any stub, and remove all x86 KVM declarations from user emulation code (to trigger compile failure instead of link one).

I'm still not sure about patch 5, though I'd like to have something like patch 6. But fortunately patches 1-3 are enough to placate clang, so I have queued them.

Thanks Philippe!

Paolo