Skip to contentSkip to conversation

lib{xc,xl}: support for guest MSR features

PATCH Superseded Roger Pau Monne 3 participants 21 messages Jun 16, 2023 – Jul 6, 2023

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

cover lib{xc,xl}: support for guest MSR features

Hello,

The following series adds support for handling guest MSR features as defined in arch-x86/cpufeatureset.h.

The end result is the user being able to use such features with the xl.cfg(5) cpuid option. This also involves adding support to all the underlying layers, so both libxl and libxc also get new functionality in order to properly parse those.

In order for such support to be as transparent as possible for existing users of libxl, both libxl_cpuid_policy_list and libxl_cpuid_policy are modified, so that the libxl_cpuid_policy_list type is no longer an array anymore, and libxl_cpuid_policy is converted into a structure with two fields to hold both a CPUID and MSR arrays. It's my thinking that current users of libxl had no business in poking at libxl_cpuid_policy_list, since the underlying type (struct xc_xend_cpuid) is opaque in that context. Also the format of the array (with a terminating empty element) is not documented in the public headers.

Some of the patches had been salvaged from a previous series of mine to introduce better cpu_policy management support in libxc and libxl, and hence contain some version notes about changes, or are already reviewed.

Thanks, Roger.

Roger Pau Monne (13):
libs/guest: simplify xc_cpuid_apply_policy()
libx86: introduce helper to fetch cpuid leaf libs/guest: allow fetching a specific CPUID leaf from a cpu policy
libx86: introduce helper to fetch msr entry
libs/guest: allow fetching a specific MSR entry from a cpu policy libs/guest: replace usage of host featureset in
xc_cpuid_apply_policy()
libs/guest: rework xc_cpuid_xend_policy
libs/guest: introduce support for setting guest MSRs
libxl: change the type of libxl_cpuid_policy_list
libxl: introduce MSR data in libxl_cpuid_policy
libxl: split logic to parse user provided CPUID features
libxl: use the cpuid feature names from cpufeatureset.h
libxl: add support for parsing MSR features

 docs/man/xl.cfg.5.pod.in                 |  24 +-
 tools/include/libxl.h                    |   7 +-
 tools/include/xenctrl.h                  |  21 +-
 tools/include/xenguest.h                 |  13 +
 tools/libs/guest/xg_cpuid_x86.c          | 383 +++++++++--------
 tools/libs/light/gentest.py              |   2 +-
 tools/libs/light/libxl_cpuid.c           | 504 +++++++++++------------
 tools/tests/cpu-policy/test-cpu-policy.c | 220 +++++++++-
 tools/xl/xl_parse.c                      |   3 +
 xen/arch/x86/cpuid.c                     |  55 +--
 xen/include/xen/lib/x86/cpu-policy.h     |  37 +-
 xen/lib/x86/cpuid.c                      |  52 +++
 xen/lib/x86/msr.c                        |  41 +-
 13 files changed, 821 insertions(+), 541 deletions(-)
Signature · 1 line
2.40.0
#1 lore raw

PATCH 1/13 libs/guest: simplify xc_cpuid_apply_policy()

Introduce a local xc_cpu_policy_t and use it to simplify some of the logic in the function:

* Populate the introduced xc_cpu_policy_t with the current domain policy, which will already be the default for the given domain type. This avoids fetching and processing any default policy.

* Use xc_cpu_policy_set_domain() to apply the adjusted policy to the domain, thus avoiding open-coding the policy serialization.

Note that some error messages are removed, as the newly used xc_cpu_policy_ functions already print an error message themselves.

No functional change intended.

---
 tools/libs/guest/xg_cpuid_x86.c | 51 ++++++---------------------------
 1 file changed, 9 insertions(+), 42 deletions(-)
tools/libs/guest/xg_cpuid_x86.c +9 -42
index 5b035223f4f5..67e0dc9b4ad2 100644
@@ -431,10 +431,9 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
     int rc;
     bool hvm;
     xc_domaininfo_t di;
-    unsigned int i, nr_leaves, nr_msrs;
-    xen_cpuid_leaf_t *leaves = NULL;
-    struct cpu_policy *p = NULL;
-    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
+    unsigned int i;
+    xc_cpu_policy_t *policy = NULL;
+    struct cpu_policy *p;
     uint32_t host_featureset[FEATURESET_NR_ENTRIES] = {};
     uint32_t len = ARRAY_SIZE(host_featureset);
 
@@ -446,17 +445,8 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
     }
     hvm = di.flags & XEN_DOMINF_hvm_guest;
 
-    rc = xc_cpu_policy_get_size(xch, &nr_leaves, &nr_msrs);
-    if ( rc )
-    {
-        PERROR("Failed to obtain policy info size");
-        rc = -errno;
-        goto out;
-    }
-
     rc = -ENOMEM;
-    if ( (leaves = calloc(nr_leaves, sizeof(*leaves))) == NULL ||
-         (p = calloc(1, sizeof(*p))) == NULL )
+    if ( (policy = xc_cpu_policy_init()) == NULL )
         goto out;
 
     /* Get the host policy. */
@@ -470,26 +460,14 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
         goto out;
     }
 
-    /* Get the domain's default policy. */
-    nr_msrs = 0;
-    rc = get_system_cpu_policy(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
-                                        : XEN_SYSCTL_cpu_policy_pv_default,
-                               &nr_leaves, leaves, &nr_msrs, NULL);
+    /* Get the domain's policy. */
+    rc = xc_cpu_policy_get_domain(xch, domid, policy);
     if ( rc )
     {
-        PERROR("Failed to obtain %s default policy", hvm ? "hvm" : "pv");
         rc = -errno;
         goto out;
     }
-
-    rc = x86_cpuid_copy_from_buffer(p, leaves, nr_leaves,
-                                    &err_leaf, &err_subleaf);
-    if ( rc )
-    {
-        ERROR("Failed to deserialise CPUID (err leaf %#x, subleaf %#x) (%d = %s)",
-              err_leaf, err_subleaf, -rc, strerror(-rc));
-        goto out;
-    }
+    p = &policy->policy;
 
     if ( restore )
     {
@@ -643,19 +621,9 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
         }
     }
 
-    rc = x86_cpuid_copy_to_buffer(p, leaves, &nr_leaves);
-    if ( rc )
-    {
-        ERROR("Failed to serialise CPUID (%d = %s)", -rc, strerror(-rc));
-        goto out;
-    }
-
-    rc = xc_set_domain_cpu_policy(xch, domid, nr_leaves, leaves, 0, NULL,
-                                  &err_leaf, &err_subleaf, &err_msr);
+    rc = xc_cpu_policy_set_domain(xch, domid, policy);
     if ( rc )
     {
-        PERROR("Failed to set d%d's policy (err leaf %#x, subleaf %#x, msr %#x)",
-               domid, err_leaf, err_subleaf, err_msr);
         rc = -errno;
         goto out;
     }
@@ -666,8 +634,7 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
     rc = 0;
 
 out:
-    free(p);
-    free(leaves);
+    xc_cpu_policy_destroy(policy);
 
     return rc;
 }
Signature · 1 line
2.40.0
#2 lore raw

Re: [PATCH 01/13] libs/guest: simplify xc_cpuid_apply_policy()

Quoted 6 lines · Roger Pau Monne

Introduce a local xc_cpu_policy_t and use it to simplify some of the logic in the function:

* Populate the introduced xc_cpu_policy_t with the current domain policy, which will already be the default for the given domain type. This avoids fetching and processing any default policy.

I'm afraid this isn't accurate.

Right now, xc_cpuid_apply_policy() is idempotent on a domain, but after this change, it's only true for the first call after domain creation.

I can't remember if we depend on this behaviour right now, but my gut feeling is that it's going to become more important as we gain more opt-in features.

~Andrew

#3 lore raw

PATCH 2/13 libx86: introduce helper to fetch cpuid leaf

In reply to Roger Pau Monne

Introduce a helper based on the current Xen guest_cpuid code in order to fetch a cpuid leaf from a policy. The newly introduced function in cpuid.c should not be directly called and instead the provided x86_cpuid_get_leaf macro should be used that will properly deal with const and non-const inputs.

Also add a test to check that the introduced helper doesn't go over the bounds of the policy.

Note the code in x86_cpuid_copy_from_buffer is not switched to use the new function because of the boundary checks against the max fields of the policy, which might not be properly set at the point where x86_cpuid_copy_from_buffer get called, for example when filling an empty policy from scratch.

---
Changes since v6:
- Add more tests.
- Drop Jan R-b.

Changes since v4:
- Rename _x86_cpuid_get_leaf to x86_cpuid_get_leaf_const.

Changes since v3:
- New in this version.

---
 tools/tests/cpu-policy/test-cpu-policy.c | 177 +++++++++++++++++++++++
 xen/arch/x86/cpuid.c                     |  55 +------
 xen/include/xen/lib/x86/cpu-policy.h     |  19 +++
 xen/lib/x86/cpuid.c                      |  52 +++++++
 4 files changed, 255 insertions(+), 48 deletions(-)
tools/tests/cpu-policy/test-cpu-policy.c +177 -0
index 301df2c00285..a11c8f067aad 100644
@@ -1,6 +1,7 @@
 #include <assert.h>
 #include <errno.h>
 #include <stdbool.h>
+#include <stddef.h>
 #include <stdint.h>
 #include <stdio.h>
 #include <stdlib.h>
@@ -565,6 +566,180 @@ static void test_cpuid_out_of_range_clearing(void)
     }
 }
 
+static void test_cpuid_get_leaf_failure(void)
+{
+    static const struct test {
+        struct cpu_policy p;
+        const char *name;
+        uint32_t leaf, subleaf;
+    } tests[] = {
+        /* Test for invalid configurations in the object itself. */
+        {
+            .name = "Basic max leaf >= array size",
+            .p = {
+                .basic.max_leaf = CPUID_GUEST_NR_BASIC,
+            },
+        },
+        {
+            .name = "Feature max leaf >= array size",
+            .p = {
+                .basic.max_leaf = CPUID_GUEST_NR_BASIC - 1,
+                .feat.max_subleaf = CPUID_GUEST_NR_FEAT,
+            },
+            .leaf = 0x7,
+        },
+        {
+            .name = "Extended max leaf >= array size",
+            .p = {
+                .extd.max_leaf = 0x80000000 + CPUID_GUEST_NR_EXTD,
+            },
+            .leaf = 0x80000000,
+        },
+
+        /* Test out-of-bounds checks in the accessor. */
+        {
+            .name = "Basic leaf >= max leaf",
+            .p = {
+                .basic.max_leaf = CPUID_GUEST_NR_BASIC - 1,
+            },
+            .leaf = CPUID_GUEST_NR_BASIC,
+        },
+        {
+            .name = "Cache leaf >= cache array size",
+            .p = {
+                .basic.max_leaf = CPUID_GUEST_NR_BASIC - 1,
+            },
+            .leaf = 0x4,
+            .subleaf = CPUID_GUEST_NR_CACHE,
+        },
+        {
+            .name = "Feature leaf >= max leaf",
+            .p = {
+                .basic.max_leaf = CPUID_GUEST_NR_BASIC - 1,
+                .feat.max_subleaf = CPUID_GUEST_NR_FEAT - 1,
+            },
+            .leaf = 0x7,
+            .subleaf = CPUID_GUEST_NR_FEAT,
+        },
+        {
+            .name = "Extended Topo leaf >= cache array size",
+            .p = {
+                .basic.max_leaf = CPUID_GUEST_NR_BASIC - 1,
+            },
+            .leaf = 0xb,
+            .subleaf = CPUID_GUEST_NR_TOPO,
+        },
+        {
+            .name = "Xstate leaf >= cache array size",
+            .p = {
+                .basic.max_leaf = CPUID_GUEST_NR_BASIC - 1,
+            },
+            .leaf = 0xd,
+            .subleaf = CPUID_GUEST_NR_XSTATE,
+        },
+        {
+            .name = "Extended leaf >= max leaf",
+            .p = {
+                .extd.max_leaf = 0x80000000 + CPUID_GUEST_NR_EXTD - 1,
+            },
+            .leaf = 0x80000000 + CPUID_GUEST_NR_EXTD,
+        },
+
+        /* Test fetching Xsave without present. */
+        {
+            .name = "Fetch Xsave without present",
+            .p = {
+                .basic = {
+                    .max_leaf = CPUID_GUEST_NR_BASIC - 1,
+                    .xsave = false,
+                },
+            },
+            .leaf = 0xd,
+        },
+
+    };
+    const struct cpu_policy pc = {};
+    const struct cpuid_leaf *lc;
+    struct cpu_policy p = {};
+    struct cpuid_leaf *l;
+
+    /* Constness build test. */
+    lc = x86_cpuid_get_leaf(&pc, 0, 0);
+    l = x86_cpuid_get_leaf(&p, 0, 0);
+
+    printf("Testing CPUID get leaf bound checking:\n");
+
+    for ( size_t i = 0; i < ARRAY_SIZE(tests); ++i )
+    {
+        const struct test *t = &tests[i];
+        const struct cpu_policy *p = memdup(&t->p);
+
+        if ( x86_cpuid_get_leaf_const(p, t->leaf, t->subleaf) )
+            fail("  Test %s get leaf fail\n", t->name);
+    }
+}
+
+static void test_cpuid_get_leaf(void)
+{
+    static const struct cpu_policy policy = {
+        .basic = {
+            .max_leaf = CPUID_GUEST_NR_BASIC - 1,
+            .xsave = true,
+        },
+        .feat.max_subleaf = CPUID_GUEST_NR_FEAT - 1,
+        .extd.max_leaf = 0x80000000 + CPUID_GUEST_NR_EXTD - 1,
+    };
+    static const struct test {
+        const char *name;
+        uint32_t leaf, subleaf;
+        size_t offset;
+    } tests[] = {
+        /* Test fetch different leaves. */
+        {
+            .name = "Fetch basic leaf",
+            .offset = offsetof(struct cpu_policy, basic.raw[0]),
+        },
+        {
+            .name = "Fetch cache leaf",
+            .leaf = 0x4,
+            .offset = offsetof(struct cpu_policy, cache.raw[0]),
+        },
+        {
+            .name = "Fetch feature leaf",
+            .leaf = 0x7,
+            .offset = offsetof(struct cpu_policy, feat.raw[0]),
+        },
+        {
+            .name = "Fetch Topo leaf",
+            .leaf = 0xb,
+            .offset = offsetof(struct cpu_policy, topo.raw[0]),
+        },
+        {
+            .name = "Fetch Xstate leaf",
+            .leaf = 0xd,
+            .offset = offsetof(struct cpu_policy, xstate.raw[0]),
+        },
+        {
+            .name = "Fetch extended leaf",
+            .leaf = 0x80000000,
+            .offset = offsetof(struct cpu_policy, extd.raw[0]),
+        },
+    };
+    const struct cpu_policy *p = memdup(&policy);
+
+    printf("Testing CPUID get leaf:\n");
+
+    for ( size_t i = 0; i < ARRAY_SIZE(tests); ++i )
+    {
+        const struct test *t = &tests[i];
+        const struct cpuid_leaf *l = x86_cpuid_get_leaf_const(p, t->leaf,
+                                                              t->subleaf);
+
+        if ( l != (const void *)p + t->offset )
+            fail("  Test %s get leaf fail\n", t->name);
+    }
+}
+
 static void test_is_compatible_success(void)
 {
     static struct test {
@@ -660,6 +835,8 @@ int main(int argc, char **argv)
     test_cpuid_serialise_success();
     test_cpuid_deserialise_failure();
     test_cpuid_out_of_range_clearing();
+    test_cpuid_get_leaf_failure();
+    test_cpuid_get_leaf();
 
     test_msr_serialise_success();
     test_msr_deserialise_failure();
xen/arch/x86/cpuid.c +7 -48

70 lines not on this page · raw message

xen/include/xen/lib/x86/cpu-policy.h +19 -0
index 6d5e9edd269b..3fcc02c729db 100644
@@ -540,6 +540,25 @@ int x86_cpu_policies_are_compatible(const struct cpu_policy *host,
                                     const struct cpu_policy *guest,
                                     struct cpu_policy_errors *err);
 
+/**
+ * Get a cpuid leaf from a policy object.
+ *
+ * @param policy      The cpuid_policy object.
+ * @param leaf        The leaf index.
+ * @param subleaf     The subleaf index.
+ * @returns a pointer to the requested leaf or NULL in case of error.
+ *
+ * The function will perform out of bound checks. Do not call this function
+ * directly and instead use x86_cpuid_get_leaf that will deal with both const
+ * and non-const policies returning a pointer with constness matching that of
+ * the input.
+ */
+const struct cpuid_leaf *x86_cpuid_get_leaf_const(const struct cpu_policy *p,
+                                                  uint32_t leaf,
+                                                  uint32_t subleaf);
+#define x86_cpuid_get_leaf(p, l, s) \
+    ((__typeof__(&(p)->basic.raw[0]))x86_cpuid_get_leaf_const(p, l, s))
+
 #endif /* !XEN_LIB_X86_POLICIES_H */
 
 /*
xen/lib/x86/cpuid.c +52 -0

58 lines not on this page · raw message

Signature · 1 line
2.40.0
#4 lore raw

Re: [PATCH 02/13] libx86: introduce helper to fetch cpuid leaf

Quoted 15 lines · Roger Pau Monne

Introduce a helper based on the current Xen guest_cpuid code in order to fetch a cpuid leaf from a policy. The newly introduced function in cpuid.c should not be directly called and instead the provided x86_cpuid_get_leaf macro should be used that will properly deal with const and non-const inputs.

Also add a test to check that the introduced helper doesn't go over the bounds of the policy.

Note the code in x86_cpuid_copy_from_buffer is not switched to use the new function because of the boundary checks against the max fields of the policy, which might not be properly set at the point where x86_cpuid_copy_from_buffer get called, for example when filling an empty policy from scratch.

Quoted 5 lines

---
Changes since v6:
- Add more tests.
- Drop Jan R-b.

Feel free to re-instate.

Jan

#5 lore raw

PATCH 3/13 libs/guest: allow fetching a specific CPUID leaf from a cpu policy

In reply to Roger Pau Monne

Introduce an interface that returns a specific leaf/subleaf from a cpu policy in xen_cpuid_leaf_t format.

This is useful to callers can peek data from the opaque xc_cpu_policy_t type.

No caller of the interface introduced on this patch.

---
Changes since v6:
- Add newline before return.

Changes since v5:
- Zero out parameter.

Changes since v3:
- Use x86_cpuid_get_leaf.

Changes since v1:
- Use find leaf.

---
 tools/include/xenguest.h        |  3 +++
 tools/libs/guest/xg_cpuid_x86.c | 26 ++++++++++++++++++++++++++
 2 files changed, 29 insertions(+)
tools/include/xenguest.h +3 -0
index e01f494b772a..0a6fd9930627 100644
@@ -807,6 +807,9 @@ int xc_cpu_policy_update_cpuid(xc_interface *xch, xc_cpu_policy_t *policy,
                                uint32_t nr);
 int xc_cpu_policy_update_msrs(xc_interface *xch, xc_cpu_policy_t *policy,
                               const xen_msr_entry_t *msrs, uint32_t nr);
+int xc_cpu_policy_get_cpuid(xc_interface *xch, const xc_cpu_policy_t *policy,
+                            uint32_t leaf, uint32_t subleaf,
+                            xen_cpuid_leaf_t *out);
 
 /* Compatibility calculations. */
 bool xc_cpu_policy_is_compatible(xc_interface *xch, xc_cpu_policy_t *host,
tools/libs/guest/xg_cpuid_x86.c +26 -0
index 67e0dc9b4ad2..630d0018529f 100644
@@ -828,6 +828,32 @@ int xc_cpu_policy_update_msrs(xc_interface *xch, xc_cpu_policy_t *policy,
     return rc;
 }
 
+int xc_cpu_policy_get_cpuid(xc_interface *xch, const xc_cpu_policy_t *policy,
+                            uint32_t leaf, uint32_t subleaf,
+                            xen_cpuid_leaf_t *out)
+{
+    const struct cpuid_leaf *tmp;
+
+    *out = (xen_cpuid_leaf_t){};
+
+    tmp = x86_cpuid_get_leaf(&policy->policy, leaf, subleaf);
+    if ( !tmp )
+    {
+        /* Unable to find a matching leaf. */
+        errno = ENOENT;
+        return -1;
+    }
+
+    out->leaf = leaf;
+    out->subleaf = subleaf;
+    out->a = tmp->a;
+    out->b = tmp->b;
+    out->c = tmp->c;
+    out->d = tmp->d;
+
+    return 0;
+}
+
 bool xc_cpu_policy_is_compatible(xc_interface *xch, xc_cpu_policy_t *host,
                                  xc_cpu_policy_t *guest)
 {
Signature · 1 line
2.40.0
#6 lore raw

PATCH 4/13 libx86: introduce helper to fetch msr entry

In reply to Roger Pau Monne

Use such helper in order to replace the code in x86_msr_copy_from_buffer. Note the introduced helper should not be directly called and instead x86_msr_get_entry should be used that will properly deal with const and non-const inputs.

Note this requires making the raw fields uint64_t so that it can accommodate the maximum size of MSRs values, and in turn removing the truncation tests.

---
Changes since v4:
- Rename _x86_msr_get_entry to x86_msr_get_entry_const.
- Add newline before endif.

Changes since v3:
- New in this version.

---
 tools/tests/cpu-policy/test-cpu-policy.c | 43 +++++++++++++++++++++---
 xen/include/xen/lib/x86/cpu-policy.h     | 18 +++++++++-
 xen/lib/x86/msr.c                        | 41 +++++++++++-----------
 3 files changed, 75 insertions(+), 27 deletions(-)
tools/tests/cpu-policy/test-cpu-policy.c +38 -5
index a11c8f067aad..2ce6ee96f91f 100644
@@ -387,11 +387,6 @@ static void test_msr_deserialise_failure(void)
             .msr = { .idx = 0xce, .flags = 1 },
             .rc = -EINVAL,
         },
-        {
-            .name = "truncated val",
-            .msr = { .idx = 0xce, .val = ~0ull },
-            .rc = -EOVERFLOW,
-        },
     };
 
     printf("Testing MSR deserialise failure:\n");
@@ -740,6 +735,43 @@ static void test_cpuid_get_leaf(void)
     }
 }
 
+static void test_msr_get_entry(void)
+{
+    static const struct test {
+        const char *name;
+        unsigned int idx;
+        bool success;
+    } tests[] = {
+        {
+            .name = "bad msr index",
+            .idx = -1,
+        },
+        {
+            .name = "good msr index",
+            .idx = 0xce,
+            .success = true,
+        },
+    };
+    const struct cpu_policy pc = { };
+    const uint64_t *ec;
+    struct cpu_policy p = { };
+    uint64_t *e;
+
+    /* Constness build test. */
+    ec = x86_msr_get_entry(&pc, 0);
+    e = x86_msr_get_entry(&p, 0);
+
+    printf("Testing MSR get leaf:\n");
+
+    for ( size_t i = 0; i < ARRAY_SIZE(tests); ++i )
+    {
+        const struct test *t = &tests[i];
+
+        if ( !!x86_msr_get_entry(&pc, t->idx) != t->success )
+            fail("  Test %s failed\n", t->name);
+    }
+}
+
 static void test_is_compatible_success(void)
 {
     static struct test {
@@ -840,6 +872,7 @@ int main(int argc, char **argv)
 
     test_msr_serialise_success();
     test_msr_deserialise_failure();
+    test_msr_get_entry();
 
     test_is_compatible_success();
     test_is_compatible_failure();
xen/include/xen/lib/x86/cpu-policy.h +17 -1
index 3fcc02c729db..877879fc8c16 100644
@@ -337,7 +337,7 @@ struct cpu_policy
      * is dependent on real hardware support.
      */
     union {
-        uint32_t raw;
+        uint64_t raw;
         struct {
             uint32_t :31;
             bool cpuid_faulting:1;
@@ -559,6 +559,22 @@ const struct cpuid_leaf *x86_cpuid_get_leaf_const(const struct cpu_policy *p,
 #define x86_cpuid_get_leaf(p, l, s) \
     ((__typeof__(&(p)->basic.raw[0]))x86_cpuid_get_leaf_const(p, l, s))
 
+/**
+ * Get a MSR entry from a policy object.
+ *
+ * @param policy      The msr_policy object.
+ * @param idx         The index.
+ * @returns a pointer to the requested leaf or NULL in case of error.
+ *
+ * Do not call this function directly and instead use x86_msr_get_entry that
+ * will deal with both const and non-const policies returning a pointer with
+ * constness matching that of the input.
+ */
+const uint64_t *x86_msr_get_entry_const(const struct cpu_policy *p,
+                                        uint32_t idx);
+#define x86_msr_get_entry(p, i) \
+    ((__typeof__(&(p)->platform_info.raw))x86_msr_get_entry_const(p, i))
+
 #endif /* !XEN_LIB_X86_POLICIES_H */
 
 /*
xen/lib/x86/msr.c +20 -21

63 lines not on this page · raw message

Signature · 1 line
2.40.0
#7 lore raw

Re: [PATCH 04/13] libx86: introduce helper to fetch msr entry

xen/lib/x86/msr.c int x86_msr_copy_from_buffer(struct cpu_policy *p, · line 107

Quoted 6 lines · Roger Pau Monne
xen/lib/x86/msr.c +2 -0
@@ -119,6 +103,21 @@ int x86_msr_copy_from_buffer(struct cpu_policy *p,
     return rc;
 }
 
+const uint64_t *x86_msr_get_entry_const(const struct cpu_policy *p,
+                                        uint32_t idx)

I don't think idx needs to be of a fixed width type; unsigned int ought to be quite fine.

xen/lib/x86/msr.c int x86_msr_copy_from_buffer(struct cpu_policy *p, · line 115

Quoted 8 lines
xen/lib/x86/msr.c +8 -0
+{
+    switch ( idx )
+    {
+    case MSR_INTEL_PLATFORM_INFO:
+        return &p->platform_info.raw;
+
+    case MSR_ARCH_CAPABILITIES:
+        return &p->arch_caps.raw;

Earlier on (also for the CPUID counterpart) a risky aspect of this didn't occur to me: The validity of the returned pointer is tied to the scope of the object the address of which was passed in. I'm afraid this is an aspect which is easy to miss, and hence wants calling out in the comments in cpu-policy.h (provided we're okay to accept this risk in the first place).

Jan

#8 lore raw

PATCH 5/13 libs/guest: allow fetching a specific MSR entry from a cpu policy

In reply to Roger Pau Monne

Introduce an interface that returns a specific MSR entry from a cpu policy in xen_msr_entry_t format.

This is useful to callers can peek data from the opaque xc_cpu_policy_t type.

No caller of the interface introduced on this patch.

---
Changes since v3:
- Use x86_msr_get_entry.

Changes since v1:
- Introduce a helper to perform a binary search of the MSR entries array.

---
 tools/include/xenguest.h        |  2 ++
 tools/libs/guest/xg_cpuid_x86.c | 20 ++++++++++++++++++++
 2 files changed, 22 insertions(+)
tools/include/xenguest.h +2 -0
index 0a6fd9930627..2672fd043cf9 100644
@@ -810,6 +810,8 @@ int xc_cpu_policy_update_msrs(xc_interface *xch, xc_cpu_policy_t *policy,
 int xc_cpu_policy_get_cpuid(xc_interface *xch, const xc_cpu_policy_t *policy,
                             uint32_t leaf, uint32_t subleaf,
                             xen_cpuid_leaf_t *out);
+int xc_cpu_policy_get_msr(xc_interface *xch, const xc_cpu_policy_t *policy,
+                          uint32_t msr, xen_msr_entry_t *out);
 
 /* Compatibility calculations. */
 bool xc_cpu_policy_is_compatible(xc_interface *xch, xc_cpu_policy_t *host,
tools/libs/guest/xg_cpuid_x86.c +20 -0
index 630d0018529f..c67e8c458f24 100644
@@ -854,6 +854,26 @@ int xc_cpu_policy_get_cpuid(xc_interface *xch, const xc_cpu_policy_t *policy,
     return 0;
 }
 
+int xc_cpu_policy_get_msr(xc_interface *xch, const xc_cpu_policy_t *policy,
+                          uint32_t msr, xen_msr_entry_t *out)
+{
+    const uint64_t *val;
+
+    *out = (xen_msr_entry_t){};
+
+    val = x86_msr_get_entry(&policy->policy, msr);
+    if ( !val )
+    {
+        errno = ENOENT;
+        return -1;
+    }
+
+    out->idx = msr;
+    out->val = *val;
+
+    return 0;
+}
+
 bool xc_cpu_policy_is_compatible(xc_interface *xch, xc_cpu_policy_t *host,
                                  xc_cpu_policy_t *guest)
 {
Signature · 1 line
2.40.0
#9 lore raw

PATCH 6/13 libs/guest: replace usage of host featureset in xc_cpuid_apply_policy()

In reply to Roger Pau Monne

Further changes to the function will require a host policy, hence switch usages now in order to avoid having both a host featureset and a host policy in the same context.

No functional change intended.

---
 tools/libs/guest/xg_cpuid_x86.c | 27 ++++++++++++---------------
 1 file changed, 12 insertions(+), 15 deletions(-)
tools/libs/guest/xg_cpuid_x86.c +12 -15
index c67e8c458f24..1e532b255c21 100644
@@ -432,10 +432,8 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
     bool hvm;
     xc_domaininfo_t di;
     unsigned int i;
-    xc_cpu_policy_t *policy = NULL;
+    xc_cpu_policy_t *policy = NULL, *host = NULL;
     struct cpu_policy *p;
-    uint32_t host_featureset[FEATURESET_NR_ENTRIES] = {};
-    uint32_t len = ARRAY_SIZE(host_featureset);
 
     if ( (rc = xc_domain_getinfo_single(xch, domid, &di)) < 0 )
     {
@@ -446,16 +444,14 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
     hvm = di.flags & XEN_DOMINF_hvm_guest;
 
     rc = -ENOMEM;
-    if ( (policy = xc_cpu_policy_init()) == NULL )
+    if ( (policy = xc_cpu_policy_init()) == NULL ||
+         (host = xc_cpu_policy_init()) == NULL )
         goto out;
 
     /* Get the host policy. */
-    rc = xc_get_cpu_featureset(xch, XEN_SYSCTL_cpu_featureset_host,
-                               &len, host_featureset);
-    /* Tolerate "buffer too small", as we've got the bits we need. */
-    if ( rc && errno != ENOBUFS )
+    rc = xc_cpu_policy_get_system(xch, XEN_SYSCTL_cpu_policy_host, host);
+    if ( rc )
     {
-        PERROR("Failed to obtain host featureset");
         rc = -errno;
         goto out;
     }
@@ -485,13 +481,13 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
          * - Re-enable features which have become (possibly) off by default.
          */
 
-        p->basic.rdrand = test_bit(X86_FEATURE_RDRAND, host_featureset);
-        p->feat.hle = test_bit(X86_FEATURE_HLE, host_featureset);
-        p->feat.rtm = test_bit(X86_FEATURE_RTM, host_featureset);
+        p->basic.rdrand = host->policy.basic.rdrand;
+        p->feat.hle = host->policy.feat.hle;
+        p->feat.rtm = host->policy.feat.rtm;
 
         if ( hvm )
         {
-            p->feat.mpx = test_bit(X86_FEATURE_MPX, host_featureset);
+            p->feat.mpx = host->policy.feat.mpx;
         }
 
         p->basic.max_leaf = min(p->basic.max_leaf, 0xdu);
@@ -560,8 +556,8 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
          * On hardware without CPUID Faulting, PV guests see real topology.
          * As a consequence, they also need to see the host htt/cmp fields.
          */
-        p->basic.htt       = test_bit(X86_FEATURE_HTT, host_featureset);
-        p->extd.cmp_legacy = test_bit(X86_FEATURE_CMP_LEGACY, host_featureset);
+        p->basic.htt       = host->policy.basic.htt;
+        p->extd.cmp_legacy = host->policy.extd.cmp_legacy;
     }
     else
     {
@@ -635,6 +631,7 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
 
 out:
     xc_cpu_policy_destroy(policy);
+    xc_cpu_policy_destroy(host);
 
     return rc;
 }
Signature · 1 line
2.40.0
#10 lore raw

PATCH 7/13 libs/guest: rework xc_cpuid_xend_policy

In reply to Roger Pau Monne

Rename xc_cpuid_xend_policy to xc_cpu_policy_apply_cpuid and make it public. Modify the function internally to use the new xc_cpu_policy_* set of functions. Also don't apply the passed policy to a domain directly, and instead modify the provided xc_cpu_policy_t. The caller will be responsible of applying the modified cpu policy to the domain.

The find_leaf helper and related comparison function is also removed, as it's no longer needed to search for cpuid leafs as finding the matching leaves is now done using xc_cpu_policy_get_cpuid.

For the purpose of the 'x' modifier, assume that the passed xc_cpu_policy_t 'policy' parameter contains the default policy, and hence just skip making any modifications in that case. This simplifies some of the logic and avoids having to fetch the default policy for the domain.

No functional change intended.

---
Changes since v6:
- Pass a host policy to xc_cpuid_apply_policy.

Changes since v3:
- Drop find_leaf and comparison helper.

---
 tools/include/xenctrl.h         |   2 +-
 tools/include/xenguest.h        |   5 +
 tools/libs/guest/xg_cpuid_x86.c | 201 +++++++++-----------------------
 3 files changed, 64 insertions(+), 144 deletions(-)
tools/include/xenctrl.h +1 -1
index dba33d5d0f39..45f05a2d3d7e 100644
@@ -1839,7 +1839,7 @@ int xc_cpuid_apply_policy(xc_interface *xch,
                           uint32_t domid, bool restore,
                           const uint32_t *featureset,
                           unsigned int nr_features, bool pae, bool itsc,
-                          bool nested_virt, const struct xc_xend_cpuid *xend);
+                          bool nested_virt, const struct xc_xend_cpuid *cpuid);
 int xc_mca_op(xc_interface *xch, struct xen_mc *mc);
 int xc_mca_op_inject_v2(xc_interface *xch, unsigned int flags,
                         xc_cpumap_t cpumap, unsigned int nr_cpus);
tools/include/xenguest.h +5 -0
index 2672fd043cf9..705a93a058fb 100644
@@ -817,6 +817,11 @@ int xc_cpu_policy_get_msr(xc_interface *xch, const xc_cpu_policy_t *policy,
 bool xc_cpu_policy_is_compatible(xc_interface *xch, xc_cpu_policy_t *host,
                                  xc_cpu_policy_t *guest);
 
+/* Apply an array of xc_xend_cpuid leafs to the policy. */
+int xc_cpu_policy_apply_cpuid(xc_interface *xch, xc_cpu_policy_t *policy,
+                              const struct xc_xend_cpuid *cpuid,
+                              const xc_cpu_policy_t *host);
+
 int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
 int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
                           uint32_t *nr_features, uint32_t *featureset);
tools/libs/guest/xg_cpuid_x86.c +58 -143

251 lines not on this page · raw message

Signature · 1 line
2.40.0
#11 lore raw

PATCH 8/13 libs/guest: introduce support for setting guest MSRs

In reply to Roger Pau Monne

Like it's done with CPUID, introduce support for passing MSR values to xc_cpuid_apply_policy(). The chosen format for expressing MSR policy data matches the current one used for CPUID. Note that existing callers of xc_cpuid_apply_policy() can pass NULL as the value for the newly introduced 'msr' parameter in order to preserve the same functionality, and in fact that's done in libxl on this patch.

---
 tools/include/xenctrl.h         | 21 ++++++++-
 tools/include/xenguest.h        |  5 ++-
 tools/libs/guest/xg_cpuid_x86.c | 76 ++++++++++++++++++++++++++++++++-
 tools/libs/light/libxl_cpuid.c  |  2 +-
 4 files changed, 99 insertions(+), 5 deletions(-)
tools/include/xenctrl.h +19 -2
index 45f05a2d3d7e..786061282c91 100644
@@ -1822,6 +1822,21 @@ struct xc_xend_cpuid {
     char *policy[4];
 };
 
+/*
+ * MSR policy data.
+ *
+ * The format of the policy string is the following:
+ *   '1' -> force to 1
+ *   '0' -> force to 0
+ *   'x' -> we don't care (use default)
+ *   'k' -> pass through host value
+ */
+struct xc_msr {
+    uint32_t index;
+    char policy[65];
+};
+#define XC_MSR_INPUT_UNUSED 0xffffffffu
+
 /*
  * Make adjustments to the CPUID settings for a domain.
  *
@@ -1833,13 +1848,15 @@ struct xc_xend_cpuid {
  * Either pass a full new @featureset (and @nr_features), or adjust individual
  * features (@pae, @itsc, @nested_virt).
  *
- * Then (optionally) apply legacy XEND overrides (@xend) to the result.
+ * Then (optionally) apply legacy XEND CPUID overrides (@cpuid) or MSR (@msr)
+ * to the result.
  */
 int xc_cpuid_apply_policy(xc_interface *xch,
                           uint32_t domid, bool restore,
                           const uint32_t *featureset,
                           unsigned int nr_features, bool pae, bool itsc,
-                          bool nested_virt, const struct xc_xend_cpuid *cpuid);
+                          bool nested_virt, const struct xc_xend_cpuid *cpuid,
+                          const struct xc_msr *msr);
 int xc_mca_op(xc_interface *xch, struct xen_mc *mc);
 int xc_mca_op_inject_v2(xc_interface *xch, unsigned int flags,
                         xc_cpumap_t cpumap, unsigned int nr_cpus);
tools/include/xenguest.h +4 -1
index 705a93a058fb..be61ff0af7fe 100644
@@ -817,10 +817,13 @@ int xc_cpu_policy_get_msr(xc_interface *xch, const xc_cpu_policy_t *policy,
 bool xc_cpu_policy_is_compatible(xc_interface *xch, xc_cpu_policy_t *host,
                                  xc_cpu_policy_t *guest);
 
-/* Apply an array of xc_xend_cpuid leafs to the policy. */
+/* Apply an array of xc_xend_cpuid leafs or xc_msrs to the policy. */
 int xc_cpu_policy_apply_cpuid(xc_interface *xch, xc_cpu_policy_t *policy,
                               const struct xc_xend_cpuid *cpuid,
                               const xc_cpu_policy_t *host);
+int xc_cpu_policy_apply_msr(xc_interface *xch, xc_cpu_policy_t *policy,
+                            const struct xc_msr *msr,
+                            const xc_cpu_policy_t *host);
 
 int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
 int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
tools/libs/guest/xg_cpuid_x86.c +75 -1

91 lines not on this page · raw message

tools/libs/light/libxl_cpuid.c +1 -1
index f5ce9f97959c..c96aeb3bce46 100644
@@ -502,7 +502,7 @@ int libxl__cpuid_legacy(libxl_ctx *ctx, uint32_t domid, bool restore,
             info->tsc_mode == LIBXL_TSC_MODE_ALWAYS_EMULATE);
 
     r = xc_cpuid_apply_policy(ctx->xch, domid, restore, NULL, 0,
-                              pae, itsc, nested_virt, info->cpuid);
+                              pae, itsc, nested_virt, info->cpuid, NULL);
     if (r)
         LOGEVD(ERROR, -r, domid, "Failed to apply CPUID policy");
 
Signature · 1 line
2.40.0
#12 lore raw

Re: [PATCH 08/13] libs/guest: introduce support for setting guest MSRs

Quoted 19 lines · Roger Pau Monne
tools/libs/guest/xg_cpuid_x86.c +13 -0
@@ -331,10 +331,74 @@ int xc_cpu_policy_apply_cpuid(xc_interface *xch, xc_cpu_policy_t *policy,
     return 0;
 }
 
+int xc_cpu_policy_apply_msr(xc_interface *xch, xc_cpu_policy_t *policy,
+                            const struct xc_msr *msr,
+                            const xc_cpu_policy_t *host)
+{
+    for ( ; msr->index != XC_MSR_INPUT_UNUSED; ++msr )
+    {
+        xen_msr_entry_t cur_msr, host_msr;
+        int rc;
+
+        rc = xc_cpu_policy_get_msr(xch, policy, msr->index, &cur_msr);
+        if ( rc )
+        {
+            ERROR("Failed to get current MSR %#x", msr->index);

tools/libs/guest/xg_cpuid_x86.c int xc_cpu_policy_apply_cpuid(xc_interface *xch, xc_cpu_policy_t *policy, · line 356

Quoted 10 lines
tools/libs/guest/xg_cpuid_x86.c +10 -0
+            return rc;
+        }
+        rc = xc_cpu_policy_get_msr(xch, host, msr->index, &host_msr);
+        if ( rc )
+        {
+            ERROR("Failed to get host policy MSR %#x", msr->index);
+            return rc;
+        }
+
+        for ( unsigned int i = 0; i < ARRAY_SIZE(msr->policy) - 1; i++ )

While correct, the "- 1" struck me as odd. Could we deviate a little from the CPUID side and permit an early nul in the string, implying all 'x' in subsequent slots? Then you could drop the -1 here and simply bail from the loop when finding a nul char.

Jan

#13 lore raw

PATCH 9/13 libxl: change the type of libxl_cpuid_policy_list

In reply to Roger Pau Monne

Currently libxl_cpuid_policy_list is an opaque type to the users of libxl, and internally it's an array of xc_xend_cpuid objects.

Change the type to instead be a structure that contains one array for CPUID policies, in preparation for it also holding another array for MSR policies.

---
 tools/include/libxl.h          |  6 +++--
 tools/libs/light/gentest.py    |  2 +-
 tools/libs/light/libxl_cpuid.c | 49 +++++++++++++++++++---------------
 3 files changed, 32 insertions(+), 25 deletions(-)
tools/include/libxl.h +4 -2
index cac641a7eba2..41e19f2af7f5 100644
@@ -1459,8 +1459,10 @@ void libxl_bitmap_dispose(libxl_bitmap *map);
  * libxl_cpuid_policy is opaque in the libxl ABI.  Users of both libxl and
  * libxc may not make assumptions about xc_xend_cpuid.
  */
-typedef struct xc_xend_cpuid libxl_cpuid_policy;
-typedef libxl_cpuid_policy * libxl_cpuid_policy_list;
+typedef struct libxl_cpu_policy {
+    struct xc_xend_cpuid *cpuid;
+} libxl_cpuid_policy;
+typedef libxl_cpuid_policy libxl_cpuid_policy_list;
 void libxl_cpuid_dispose(libxl_cpuid_policy_list *cpuid_list);
 int libxl_cpuid_policy_list_length(const libxl_cpuid_policy_list *l);
 void libxl_cpuid_policy_list_copy(libxl_ctx *ctx,
tools/libs/light/gentest.py +1 -1
index 1cc7eebc826d..207f988a741d 100644
@@ -194,7 +194,7 @@ static void libxl_cpuid_policy_list_rand_init(libxl_cpuid_policy_list *pp)
     };
     const int nr_options = sizeof(options)/sizeof(options[0]);
     char buf[64];
-    libxl_cpuid_policy_list p = NULL;
+    libxl_cpuid_policy_list p = { };
 
     for (i = 0; i < nr_policies; i++) {
         int opt = test_rand(nr_options);
tools/libs/light/libxl_cpuid.c +27 -22

147 lines not on this page · raw message

Signature · 1 line
2.40.0
#14 lore raw

Re: [PATCH 09/13] libxl: change the type of libxl_cpuid_policy_list

Quoted 19 lines · Roger Pau Monne

Currently libxl_cpuid_policy_list is an opaque type to the users of libxl, and internally it's an array of xc_xend_cpuid objects.

Change the type to instead be a structure that contains one array for CPUID policies, in preparation for it also holding another array for MSR policies.

---
 tools/include/libxl.h          |  6 +++--
 tools/libs/light/gentest.py    |  2 +-
 tools/libs/light/libxl_cpuid.c | 49 +++++++++++++++++++---------------
 3 files changed, 32 insertions(+), 25 deletions(-)
tools/include/libxl.h +0 -0
index cac641a7eba2..41e19f2af7f5 100644

tools/include/libxl.h void libxl_bitmap_dispose(libxl_bitmap *map); · line 1465

Quoted 9 lines
tools/include/libxl.h +4 -2
@@ -1459,8 +1459,10 @@ void libxl_bitmap_dispose(libxl_bitmap *map);
  * libxl_cpuid_policy is opaque in the libxl ABI.  Users of both libxl and
  * libxc may not make assumptions about xc_xend_cpuid.
  */
-typedef struct xc_xend_cpuid libxl_cpuid_policy;
-typedef libxl_cpuid_policy * libxl_cpuid_policy_list;
+typedef struct libxl_cpu_policy {
+    struct xc_xend_cpuid *cpuid;
+} libxl_cpuid_policy;
+typedef libxl_cpuid_policy libxl_cpuid_policy_list;

I don't think we can get away with doing this.  It makes the type non-opaque, and you'll also break the libxl ABI in the next patch when you change the size of this object.

For better or worse, I think

typedef libxl_cpuid_policy * libxl_cpuid_policy_list;

needs to stay here, and libxl_cpuid_policy get moved into libxl_internal.h where it can then be altered.

~Andrew

#15 lore raw

PATCH 10/13 libxl: introduce MSR data in libxl_cpuid_policy

In reply to Roger Pau Monne

Add a new array field to libxl_cpuid_policy in order to store the MSR policies.

Note that libxl_cpuid_policy_list_{copy,length,parse_json,gen_json} are not adjusted to deal with the new MSR array now part of libxl_cpuid_policy_list.

Adding the MSR data in the libxl_cpuid_policy_list type is done so that existing users can seamlessly pass MSR features as part of the CPUID data, without requiring the introduction of a separate domain_build_info field, and a new set of handlers functions.

---
 tools/include/libxl.h          |  1 +
 tools/libs/light/libxl_cpuid.c | 31 +++++++++++++++++++------------
 2 files changed, 20 insertions(+), 12 deletions(-)
tools/include/libxl.h +1 -0
index 41e19f2af7f5..4e7b08ab5027 100644
@@ -1461,6 +1461,7 @@ void libxl_bitmap_dispose(libxl_bitmap *map);
  */
 typedef struct libxl_cpu_policy {
     struct xc_xend_cpuid *cpuid;
+    struct xc_msr *msr;
 } libxl_cpuid_policy;
 typedef libxl_cpuid_policy libxl_cpuid_policy_list;
 void libxl_cpuid_dispose(libxl_cpuid_policy_list *cpuid_list);
tools/libs/light/libxl_cpuid.c +19 -12
index ded0d0b8bc15..7261c1f1fd82 100644
@@ -21,20 +21,26 @@ int libxl__cpuid_policy_is_empty(libxl_cpuid_policy_list *pl)
 
 void libxl_cpuid_dispose(libxl_cpuid_policy_list *policy)
 {
-    int i, j;
     struct xc_xend_cpuid *cpuid_list = policy->cpuid;
 
-    if (cpuid_list == NULL)
-        return;
-    for (i = 0; cpuid_list[i].input[0] != XEN_CPUID_INPUT_UNUSED; i++) {
-        for (j = 0; j < 4; j++)
-            if (cpuid_list[i].policy[j] != NULL) {
-                free(cpuid_list[i].policy[j]);
-                cpuid_list[i].policy[j] = NULL;
-            }
+    if (cpuid_list) {
+        unsigned int i, j;
+
+        for (i = 0; cpuid_list[i].input[0] != XEN_CPUID_INPUT_UNUSED; i++) {
+            for (j = 0; j < 4; j++)
+                if (cpuid_list[i].policy[j] != NULL) {
+                    free(cpuid_list[i].policy[j]);
+                    cpuid_list[i].policy[j] = NULL;
+                }
+        }
+        free(policy->cpuid);
+        policy->cpuid = NULL;
+    }
+
+    if (policy->msr) {
+        free(policy->msr);
+        policy->msr = NULL;
     }
-    free(policy->cpuid);
-    policy->cpuid = NULL;
     return;
 }
 
@@ -503,7 +509,8 @@ int libxl__cpuid_legacy(libxl_ctx *ctx, uint32_t domid, bool restore,
             info->tsc_mode == LIBXL_TSC_MODE_ALWAYS_EMULATE);
 
     r = xc_cpuid_apply_policy(ctx->xch, domid, restore, NULL, 0,
-                              pae, itsc, nested_virt, info->cpuid.cpuid, NULL);
+                              pae, itsc, nested_virt, info->cpuid.cpuid,
+                              info->cpuid.msr);
     if (r)
         LOGEVD(ERROR, -r, domid, "Failed to apply CPUID policy");
 
Signature · 1 line
2.40.0
#16 lore raw

PATCH 11/13 libxl: split logic to parse user provided CPUID features

In reply to Roger Pau Monne

Move the CPUID value parsers out of libxl_cpuid_parse_config() into a newly created cpuid_add() local helper. This is in preparation for also adding MSR feature parsing support.

No functional change intended.

---
 tools/libs/light/libxl_cpuid.c | 120 +++++++++++++++++----------------
 1 file changed, 63 insertions(+), 57 deletions(-)
tools/libs/light/libxl_cpuid.c +63 -57
index 7261c1f1fd82..d0ac5b2bc102 100644
@@ -88,6 +88,66 @@ static struct xc_xend_cpuid *cpuid_find_match(libxl_cpuid_policy *policy,
     return *list + i;
 }
 
+static int cpuid_add(libxl_cpuid_policy *policy, const struct cpuid_flags *flag,
+                     const char *val)
+{
+    struct xc_xend_cpuid *entry = cpuid_find_match(policy, flag->leaf,
+                                                   flag->subleaf);
+    unsigned long num;
+    char flags[33], *resstr, *endptr;
+    unsigned int i;
+
+    resstr = entry->policy[flag->reg - 1];
+    num = strtoull(val, &endptr, 0);
+    flags[flag->length] = 0;
+    if (endptr != val) {
+        /* if this was a valid number, write the binary form into the string */
+        for (i = 0; i < flag->length; i++) {
+            flags[flag->length - 1 - i] = "01"[!!(num & (1 << i))];
+        }
+    } else {
+        switch(val[0]) {
+        case 'x': case 'k': case 's':
+            memset(flags, val[0], flag->length);
+            break;
+        default:
+            return 3;
+        }
+    }
+
+    if (resstr == NULL) {
+        resstr = strdup("xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx");
+    }
+
+    /* the family and model entry is potentially split up across
+     * two fields in Fn0000_0001_EAX, so handle them here separately.
+     */
+    if (!strcmp(flag->name, "family")) {
+        if (num < 16) {
+            memcpy(resstr + (32 - 4) - flag->bit, flags + 4, 4);
+            memcpy(resstr + (32 - 8) - 20, "00000000", 8);
+        } else {
+            num -= 15;
+            memcpy(resstr + (32 - 4) - flag->bit, "1111", 4);
+            for (i = 0; i < 7; i++) {
+                flags[7 - i] = "01"[num & 1];
+                num >>= 1;
+            }
+            memcpy(resstr + (32 - 8) - 20, flags, 8);
+        }
+    } else if (!strcmp(flag->name, "model")) {
+        memcpy(resstr + (32 - 4) - 16, flags, 4);
+        memcpy(resstr + (32 - 4) - flag->bit, flags + 4, 4);
+    } else {
+        memcpy(resstr + (32 - flag->length) - flag->bit, flags,
+               flag->length);
+    }
+    entry->policy[flag->reg - 1] = resstr;
+
+    return 0;
+
+}
+
 /* parse a single key=value pair and translate it into the libxc
  * used interface using 32-characters strings for each register.
  * Will overwrite earlier entries and thus can be called multiple
@@ -332,12 +392,8 @@ int libxl_cpuid_parse_config(libxl_cpuid_policy_list *policy, const char* str)
         {NULL, 0, NA, CPUID_REG_INV, 0, 0}
     };
 #undef NA
-    char *sep, *val, *endptr;
-    int i;
+    const char *sep, *val;
     const struct cpuid_flags *flag;
-    struct xc_xend_cpuid *entry;
-    unsigned long num;
-    char flags[33], *resstr;
 
     sep = strchr(str, '=');
     if (sep == NULL) {
@@ -347,60 +403,10 @@ int libxl_cpuid_parse_config(libxl_cpuid_policy_list *policy, const char* str)
     }
     for (flag = cpuid_flags; flag->name != NULL; flag++) {
         if(!strncmp(str, flag->name, sep - str) && flag->name[sep - str] == 0)
-            break;
-    }
-    if (flag->name == NULL) {
-        return 2;
-    }
-    entry = cpuid_find_match(policy, flag->leaf, flag->subleaf);
-    resstr = entry->policy[flag->reg - 1];
-    num = strtoull(val, &endptr, 0);
-    flags[flag->length] = 0;
-    if (endptr != val) {
-        /* if this was a valid number, write the binary form into the string */
-        for (i = 0; i < flag->length; i++) {
-            flags[flag->length - 1 - i] = "01"[!!(num & (1 << i))];
-        }
-    } else {
-        switch(val[0]) {
-        case 'x': case 'k': case 's':
-            memset(flags, val[0], flag->length);
-            break;
-        default:
-            return 3;
-        }
-    }
-
-    if (resstr == NULL) {
-        resstr = strdup("xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx");
+            return cpuid_add(policy, flag, val);
     }
 
-    /* the family and model entry is potentially split up across
-     * two fields in Fn0000_0001_EAX, so handle them here separately.
-     */
-    if (!strncmp(str, "family", sep - str)) {
-        if (num < 16) {
-            memcpy(resstr + (32 - 4) - flag->bit, flags + 4, 4);
-            memcpy(resstr + (32 - 8) - 20, "00000000", 8);
-        } else {
-            num -= 15;
-            memcpy(resstr + (32 - 4) - flag->bit, "1111", 4);
-            for (i = 0; i < 7; i++) {
-                flags[7 - i] = "01"[num & 1];
-                num >>= 1;
-            }
-            memcpy(resstr + (32 - 8) - 20, flags, 8);
-        }
-    } else if (!strncmp(str, "model", sep - str)) {
-        memcpy(resstr + (32 - 4) - 16, flags, 4);
-        memcpy(resstr + (32 - 4) - flag->bit, flags + 4, 4);
-    } else {
-        memcpy(resstr + (32 - flag->length) - flag->bit, flags,
-               flag->length);
-    }
-    entry->policy[flag->reg - 1] = resstr;
-
-    return 0;
+    return 2;
 }
 
 /* parse a single list item from the legacy Python xend syntax, where
Signature · 1 line
2.40.0
#17 lore raw

Re: [PATCH 11/13] libxl: split logic to parse user provided CPUID features

Quoted 12 lines · Roger Pau Monne
tools/libs/light/libxl_cpuid.c +6 -0
@@ -88,6 +88,66 @@ static struct xc_xend_cpuid *cpuid_find_match(libxl_cpuid_policy *policy,
     return *list + i;
 }
 
+static int cpuid_add(libxl_cpuid_policy *policy, const struct cpuid_flags *flag,
+                     const char *val)
+{
+    struct xc_xend_cpuid *entry = cpuid_find_match(policy, flag->leaf,
+                                                   flag->subleaf);
+    unsigned long num;

tools/libs/light/libxl_cpuid.c static struct xc_xend_cpuid *cpuid_find_match(libxl_cpuid_policy *policy, · line 106

Quoted 10 lines
tools/libs/light/libxl_cpuid.c +10 -0
+    char flags[33], *resstr, *endptr;
+    unsigned int i;
+
+    resstr = entry->policy[flag->reg - 1];
+    num = strtoull(val, &endptr, 0);
+    flags[flag->length] = 0;
+    if (endptr != val) {
+        /* if this was a valid number, write the binary form into the string */
+        for (i = 0; i < flag->length; i++) {
+            flags[flag->length - 1 - i] = "01"[!!(num & (1 << i))];

I expect you've left this as is because you really only want to move code here? At the very least the UB should be eliminated imo, by using 1u in the shift. Even better might be "01"[(num >> i) & 1]. And of course using strtoull() when num is unsigned long is a little fishy as well ...

Jan

#18 lore raw

PATCH 12/13 libxl: use the cpuid feature names from cpufeatureset.h

In reply to Roger Pau Monne

The current implementation in libxl_cpuid_parse_config() requires keeping a list of cpuid feature bits that should be mostly in sync with the contents of cpufeatureset.h.

Avoid such duplication by using the automatically generated list of cpuid features in INIT_FEATURE_NAMES in order to map feature names to featureset bits, and then translate from featureset bits into cpuid leaf, subleaf, register tuple.

Note that the full contents of the previous cpuid translation table can't be removed. That's because some feature names allowed by libxl are not described in the featuresets, or because naming has diverged and the previous nomenclature is preserved for compatibility reasons.

Should result in no functional change observed by callers, albeit some new cpuid features will be available as a result of the change.

While there constify cpuid_flags name field.

---
Changes since v1:
- const unnamed structure cast.
- Declare struct feature_name outside the function.
- Use strcmp.
- Fix indentation.
- Add back missing feature name options.
- Return ERROR_NOMEM if allocation fails.
- Improve xl.cfg documentation about how to reference the features described in the public header.

---
 docs/man/xl.cfg.5.pod.in       |  24 +--
 tools/libs/light/libxl_cpuid.c | 267 ++++++++++++---------------------
 tools/xl/xl_parse.c            |   3 +
 3 files changed, 107 insertions(+), 187 deletions(-)
docs/man/xl.cfg.5.pod.in +8 -16
index 3979be2a590a..55161856f4c7 100644
@@ -2010,24 +2010,16 @@ proccount procpkg stepping
 
 =back
 
-List of keys taking a character:
+List of keys taking a character can be found in the public header file
+L<arch-x86/cpufeatureset.h|https://xenbits.xen.org/docs/unstable/hypercall/x86_64/include,public,arch-x86,cpufeatureset.h.html>
 
-=over 4
-
-3dnow 3dnowext 3dnowprefetch abm acpi adx aes altmovcr8 apic arat avx avx2
-avx512-4fmaps avx512-4vnniw avx512bw avx512cd avx512dq avx512er avx512f
-avx512ifma avx512pf avx512vbmi avx512vl bmi1 bmi2 clflushopt clfsh clwb cmov
-cmplegacy cmpxchg16 cmpxchg8 cmt cntxid dca de ds dscpl dtes64 erms est extapic
-f16c ffxsr fma fma4 fpu fsgsbase fxsr hle htt hypervisor ia64 ibs invpcid
-invtsc lahfsahf lm lwp mca mce misalignsse mmx mmxext monitor movbe mpx msr
-mtrr nodeid nx ospke osvw osxsave pae page1gb pat pbe pcid pclmulqdq pdcm
-perfctr_core perfctr_nb pge pku popcnt pse pse36 psn rdrand rdseed rdtscp rtm
-sha skinit smap smep smx ss sse sse2 sse3 sse4.1 sse4.2 sse4_1 sse4_2 sse4a
-ssse3 svm svm_decode svm_lbrv svm_npt svm_nrips svm_pausefilt svm_tscrate
-svm_vmcbclean syscall sysenter tbm tm tm2 topoext tsc tsc-deadline tsc_adjust
-umip vme vmx wdt x2apic xop xsave xtpr
+The feature names described in C<cpufeatureset.h> should be specified in all
+lowercase letters, and with underscores converted to hyphens.  For example in
+order to reference feature C<LAHF_LM> the string C<lahf-lm> should be used.
 
-=back
+Note that C<clflush> is described as an option that takes a value, and that
+takes precedence over the C<clflush> flag in C<cpufeatureset.h>.  The feature
+flag must be referenced as C<clfsh>.
 
 =back
 
tools/libs/light/libxl_cpuid.c +96 -171

347 lines not on this page · raw message

tools/xl/xl_parse.c +3 -0
index f036e56fc239..7bf587455d08 100644
@@ -2620,6 +2620,9 @@ skip_usbdev:
                 case 3:
                     errstr = "illegal CPUID value (must be: [0|1|x|k|s])";
                     break;
+                case ERROR_NOMEM:
+                    errstr = "out of memory";
+                    break;
                 default:
                     errstr = "unknown error";
                     break;
Signature · 1 line
2.40.0
#19 lore raw

PATCH 13/13 libxl: add support for parsing MSR features

In reply to Roger Pau Monne

Introduce support for handling MSR features in libxl_cpuid_parse_config(). The MSR policies are added to the libxl_cpuid_policy like the CPUID one, which gets passed to xc_cpuid_apply_policy().

This allows existing users of libxl to provide MSR related features as key=value pairs to libxl_cpuid_parse_config() without requiring the usage of a different API.

---
 tools/libs/light/libxl_cpuid.c | 57 +++++++++++++++++++++++++++++++++-
 1 file changed, 56 insertions(+), 1 deletion(-)
tools/libs/light/libxl_cpuid.c +56 -1
index cbbd5d31d63b..804dddb446c3 100644
@@ -149,6 +149,53 @@ static int cpuid_add(libxl_cpuid_policy *policy, const struct cpuid_flags *flag,
     return 0;
 }
 
+static struct xc_msr *msr_find_match(libxl_cpuid_policy *policy, uint32_t index)
+{
+    unsigned int i = 0;
+
+    if (policy->msr != NULL)
+        for (i = 0; policy->msr[i].index != XC_MSR_INPUT_UNUSED; i++)
+            if (policy->msr[i].index == index)
+                return &policy->msr[i];
+
+    policy->msr = realloc(policy->msr, sizeof(struct xc_msr) * (i + 2));
+    policy->msr[i].index = index;
+    memset(policy->msr[i].policy, 'x', ARRAY_SIZE(policy->msr[0].policy) - 1);
+    policy->msr[i].policy[ARRAY_SIZE(policy->msr[0].policy) - 1] = '\0';
+    policy->msr[i + 1].index = XC_MSR_INPUT_UNUSED;
+
+    return &policy->msr[i];
+}
+
+static int msr_add(libxl_cpuid_policy *policy, uint32_t index, unsigned int bit,
+                   const char *val)
+{
+    struct xc_msr *entry = msr_find_match(policy, index);
+
+    /* Only allow options taking a character for MSRs, no values allowed. */
+    if (strlen(val) != 1)
+        return 3;
+
+    switch (val[0]) {
+    case '0':
+    case '1':
+    case 'x':
+    case 'k':
+        entry->policy[63 - bit] = val[0];
+        break;
+
+    case 's':
+        /* Translate s -> k as xc_msr doesn't support the deprecated 's'. */
+        entry->policy[63 - bit] = 'k';
+        break;
+
+    default:
+        return 3;
+    }
+
+    return 0;
+}
+
 struct feature_name {
     const char *name;
     unsigned int bit;
@@ -328,7 +375,15 @@ int libxl_cpuid_parse_config(libxl_cpuid_policy_list *policy, const char* str)
     }
 
     case FEAT_MSR:
-        return 2;
+    {
+        unsigned int bit = feat->bit % 32;
+
+        if (feature_to_policy[feat->bit / 32].msr.reg == CPUID_REG_EDX)
+            bit += 32;
+
+        return msr_add(policy, feature_to_policy[feat->bit / 32].msr.index,
+                       bit, val);
+    }
     }
 
     return 2;
Signature · 1 line
2.40.0
#20 lore raw
In reply to Roger Pau Monne
Quoted 25 lines · Roger Pau Monne

Hello,

The following series adds support for handling guest MSR features as defined in arch-x86/cpufeatureset.h.

The end result is the user being able to use such features with the xl.cfg(5) cpuid option. This also involves adding support to all the underlying layers, so both libxl and libxc also get new functionality in order to properly parse those.

In order for such support to be as transparent as possible for existing users of libxl, both libxl_cpuid_policy_list and libxl_cpuid_policy are modified, so that the libxl_cpuid_policy_list type is no longer an array anymore, and libxl_cpuid_policy is converted into a structure with two fields to hold both a CPUID and MSR arrays. It's my thinking that current users of libxl had no business in poking at libxl_cpuid_policy_list, since the underlying type (struct xc_xend_cpuid) is opaque in that context. Also the format of the array (with a terminating empty element) is not documented in the public headers.

Some of the patches had been salvaged from a previous series of mine to introduce better cpu_policy management support in libxc and libxl, and hence contain some version notes about changes, or are already reviewed.

Quoted 8 lines

Thanks, Roger.

Roger Pau Monne (13):
libs/guest: simplify xc_cpuid_apply_policy()
libx86: introduce helper to fetch cpuid leaf libs/guest: allow fetching a specific CPUID leaf from a cpu policy
libx86: introduce helper to fetch msr entry
libs/guest: allow fetching a specific MSR entry from a cpu policy

I'm somewhat loath to introduce wrappers like this to structures which intentionally have O(1) lookup by index.

These only exist (AFAICT) to let libxl poke inside an opaque object, but I don't see them used.  Furthermore, I'm not sure that extending test-cpu-policy is much use - the existing {,de}serialise tests already exercise the logic, as you factored it out of said logic.

Quoted 1 line

libs/guest: replace usage of host featureset in xc_cpuid_apply_policy()

This looks fine, but it's fairly dependent on patch 1 which needs a rearrangement.

Quoted 1 line

libs/guest: rework xc_cpuid_xend_policy

Here, the xend is about the format of the argument, so is important to stay as part of the name.

Furthermore, this helper is deliberately not exported from xg_cpuid_x86.c such that xc_cpuid_apply_policy() is the single interface to use, and AFAICT that's still the case after the series too.

Quoted 6 lines

libs/guest: introduce support for setting guest MSRs
libxl: change the type of libxl_cpuid_policy_list
libxl: introduce MSR data in libxl_cpuid_policy
libxl: split logic to parse user provided CPUID features
libxl: use the cpuid feature names from cpufeatureset.h
libxl: add support for parsing MSR features

So by the end here, we're still using the cpuid="host:..." format, using enumerations from cpufeatureset.h, with a brand new MSR type, and a hand-coded mapping between a featureset number and a register?

I think this would be far more simple it it just used the featureset bitmap which already exists in xc_cpuid_apply_policy(), rather than borrowing the featureset names and coding around an interface that already exists.

I'm wary about the names themselves.  I already dislike that cpufeatureset is in public/ but this is exposing it more than previously.  We have previously elected to tweak naming in cpufeatureset, and there's currently a mess with feature naming where Intel and AMD have the same named bit in different positions.  IMO we need to retain this flexibility - perhaps we can do that by just extending libxl's alias lists if we decide we need to change the names in Xen.

~Andrew