Skip to contentSkip to conversation

libs/guest: new CPUID/MSR interface

PATCH Superseded Roger Pau Monne 4 participants 43 messages Apr 13, 2021 – Apr 28, 2021

v2 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v2 cover libs/guest: new CPUID/MSR interface

Hello,

The following series introduces a new CPUID/MSR interface for the xenguest library. Such interface handles both CPUID and MSRs using the same opaque object, and provides some helpers for the user to peek or modify such data without exposing the backing type. This is useful for future development as CPUID and MSRs are closely related, so it makes handling those much easier if they are inside the same object (ie: a change to a CPUID bit might expose or hide an MSR).

In this patch series libxl and other in tree users have been switched to use the new interface, so it shouldn't result in any functional change from a user point of view.

Note there are still some missing pieces likely. The way to modify CPUID data is not ideal, as it requires fetching a leaf and modifying it directly. We might want some kind of interface in order to set specific CPUID features more easily, but that's to be discussed, and would be done as a follow up series.

Thanks, Roger.

Roger Pau Monne (21):
libxl: don't ignore the return value from xc_cpuid_apply_policy libs/guest: rename xc_get_cpu_policy_size to xc_cpu_policy_get_size libs/guest: introduce xc_cpu_policy_t
libs/guest: introduce helper to fetch a system cpu policy libs/guest: introduce helper to fetch a domain cpu policy libs/guest: introduce helper to serialize a cpu policy
tools: switch existing users of xc_get_{system,domain}_cpu_policy libs/guest: introduce a helper to apply a cpu policy to a domain libs/guest: allow fetching a specific CPUID leaf from a cpu policy tests/cpu-policy: add sorted MSR test
libs/guest: allow fetching a specific MSR entry from a cpu policy libs/guest: allow updating a cpu policy CPUID data libs/guest: allow updating a cpu policy MSR data libs/guest: introduce helper to check cpu policy compatibility libs/guest: obtain a compatible cpu policy from two input ones libs/guest: make a cpu policy compatible with older Xen versions libs/guest: introduce helper set cpu topology in cpu policy libs/guest: rework xc_cpuid_xend_policy
libs/guest: apply a featureset into a cpu policy libs/{light,guest}: implement xc_cpuid_apply_policy in libxl libs/guest: (re)move xc_cpu_policy_apply_cpuid

 tools/include/libxl.h                    |   6 +-
 tools/include/xen-tools/libs.h           |   5 +
 tools/include/xenctrl.h                  | 102 ++--
 tools/libs/guest/Makefile                |   2 +-
 tools/libs/guest/xg_cpuid_x86.c          | 710 +++++++++++++++--------
 tools/libs/guest/xg_sr_common_x86.c      |  17 +-
 tools/libs/light/libxl_cpuid.c           | 230 +++++++-
 tools/libs/light/libxl_create.c          |   5 +-
 tools/libs/light/libxl_dom.c             |   2 +-
 tools/libs/light/libxl_internal.h        |  32 +-
 tools/libs/light/libxl_nocpuid.c         |   5 +-
 tools/misc/xen-cpuid.c                   |  23 +-
 tools/tests/cpu-policy/test-cpu-policy.c |  17 +
 13 files changed, 821 insertions(+), 335 deletions(-)
Signature · 1 line
2.30.1
#1 lore raw

PATCH v2 1/21 libxl: don't ignore the return value from xc_cpuid_apply_policy

Also change libxl__cpuid_legacy to propagate the error from xc_cpuid_apply_policy into callers.

---
Changes since 1:
- Return ERROR_FAIL on error.

---
 tools/libs/light/libxl_cpuid.c    | 15 +++++++++++----
 tools/libs/light/libxl_create.c   |  5 +++--
 tools/libs/light/libxl_dom.c      |  2 +-
 tools/libs/light/libxl_internal.h |  4 ++--
 tools/libs/light/libxl_nocpuid.c  |  5 +++--
 5 files changed, 20 insertions(+), 11 deletions(-)
tools/libs/light/libxl_cpuid.c +11 -4
index 289c59c7420..539fc4869e6 100644
@@ -419,11 +419,13 @@ int libxl_cpuid_parse_config_xend(libxl_cpuid_policy_list *cpuid,
     return 0;
 }
 
-void libxl__cpuid_legacy(libxl_ctx *ctx, uint32_t domid, bool restore,
-                         libxl_domain_build_info *info)
+int libxl__cpuid_legacy(libxl_ctx *ctx, uint32_t domid, bool restore,
+                        libxl_domain_build_info *info)
 {
+    GC_INIT(ctx);
     bool pae = true;
     bool itsc;
+    int rc;
 
     /*
      * Gross hack.  Using libxl_defbool_val() here causes libvirt to crash in
@@ -462,8 +464,13 @@ void libxl__cpuid_legacy(libxl_ctx *ctx, uint32_t domid, bool restore,
     itsc = (libxl_defbool_val(info->disable_migrate) ||
             info->tsc_mode == LIBXL_TSC_MODE_ALWAYS_EMULATE);
 
-    xc_cpuid_apply_policy(ctx->xch, domid, restore, NULL, 0,
-                          pae, itsc, nested_virt, info->cpuid);
+    rc = xc_cpuid_apply_policy(ctx->xch, domid, restore, NULL, 0,
+                               pae, itsc, nested_virt, info->cpuid);
+    if (rc)
+        LOGE(ERROR, "Failed to apply CPUID policy");
+
+    GC_FREE;
+    return rc ? ERROR_FAIL : 0;
 }
 
 static const char *input_names[2] = { "leaf", "subleaf" };
tools/libs/light/libxl_create.c +3 -2
index 0c64268f66d..2faa96d9c68 100644
@@ -1443,6 +1443,7 @@ int libxl__srm_callout_callback_static_data_done(unsigned int missing,
 
     libxl_domain_config *d_config = dcs->guest_config;
     libxl_domain_build_info *info = &d_config->b_info;
+    int rc = 0;
 
     /*
      * CPUID/MSR information is not present in pre Xen-4.14 streams.
@@ -1452,9 +1453,9 @@ int libxl__srm_callout_callback_static_data_done(unsigned int missing,
      * stream doesn't contain any CPUID data.
      */
     if (missing & XGR_SDD_MISSING_CPUID)
-        libxl__cpuid_legacy(ctx, dcs->guest_domid, true, info);
+        rc = libxl__cpuid_legacy(ctx, dcs->guest_domid, true, info);
 
-    return 0;
+    return rc;
 }
 
 void libxl__srm_callout_callback_restore_results(xen_pfn_t store_mfn,
tools/libs/light/libxl_dom.c +1 -1
index 842a51c86cb..e9f58ee4b2b 100644
@@ -384,7 +384,7 @@ int libxl__build_pre(libxl__gc *gc, uint32_t domid,
      * being migrated-in/restored have CPUID handled during the
      * static_data_done() callback. */
     if (!state->restore)
-        libxl__cpuid_legacy(ctx, domid, false, info);
+        rc = libxl__cpuid_legacy(ctx, domid, false, info);
 
     return rc;
 }
tools/libs/light/libxl_internal.h +2 -2
index c6a4a187f5b..44a2f3c8fe3 100644
@@ -2052,8 +2052,8 @@ typedef yajl_gen_status (*libxl__gen_json_callback)(yajl_gen hand, void *);
 _hidden char *libxl__object_to_json(libxl_ctx *ctx, const char *type,
                                     libxl__gen_json_callback gen, void *p);
 
-_hidden void libxl__cpuid_legacy(libxl_ctx *ctx, uint32_t domid, bool retore,
-                                 libxl_domain_build_info *info);
+_hidden int libxl__cpuid_legacy(libxl_ctx *ctx, uint32_t domid, bool retore,
+                                libxl_domain_build_info *info);
 
 /* Calls poll() again - useful to check whether a signaled condition
  * is still true.  Cannot fail.  Returns currently-true revents. */
tools/libs/light/libxl_nocpuid.c +3 -2
index f47336565b9..0630959e760 100644
@@ -34,9 +34,10 @@ int libxl_cpuid_parse_config_xend(libxl_cpuid_policy_list *cpuid,
     return 0;
 }
 
-void libxl__cpuid_legacy(libxl_ctx *ctx, uint32_t domid, bool restore,
-                         libxl_domain_build_info *info)
+int libxl__cpuid_legacy(libxl_ctx *ctx, uint32_t domid, bool restore,
+                        libxl_domain_build_info *info)
 {
+    return 0;
 }
 
 yajl_gen_status libxl_cpuid_policy_list_gen_json(yajl_gen hand,
Signature · 1 line
2.30.1
#2 lore raw

Re: [PATCH v2 01/21] libxl: don't ignore the return value from xc_cpuid_apply_policy

Quoted 37 lines · Roger Pau Monne

Also change libxl__cpuid_legacy to propagate the error from xc_cpuid_apply_policy into callers.

---
Changes since 1:
- Return ERROR_FAIL on error.

---
 tools/libs/light/libxl_cpuid.c    | 15 +++++++++++----
 tools/libs/light/libxl_create.c   |  5 +++--
 tools/libs/light/libxl_dom.c      |  2 +-
 tools/libs/light/libxl_internal.h |  4 ++--
 tools/libs/light/libxl_nocpuid.c  |  5 +++--
 5 files changed, 20 insertions(+), 11 deletions(-)
tools/libs/light/libxl_cpuid.c +4 -2
index 289c59c7420..539fc4869e6 100644
@@ -419,11 +419,13 @@ int libxl_cpuid_parse_config_xend(libxl_cpuid_policy_list *cpuid,
     return 0;
 }
 
-void libxl__cpuid_legacy(libxl_ctx *ctx, uint32_t domid, bool restore,
-                         libxl_domain_build_info *info)
+int libxl__cpuid_legacy(libxl_ctx *ctx, uint32_t domid, bool restore,
+                        libxl_domain_build_info *info)
 {
+    GC_INIT(ctx);
     bool pae = true;
     bool itsc;
+    int rc;
 
     /*
      * Gross hack.  Using libxl_defbool_val() here causes libvirt to crash in

tools/libs/light/libxl_cpuid.c void libxl__cpuid_legacy(libxl_ctx *ctx, uint32_t domid, bool restore, · line 470

Quoted 9 lines
tools/libs/light/libxl_cpuid.c +4 -2
@@ -462,8 +464,13 @@ void libxl__cpuid_legacy(libxl_ctx *ctx, uint32_t domid, bool restore,
     itsc = (libxl_defbool_val(info->disable_migrate) ||
             info->tsc_mode == LIBXL_TSC_MODE_ALWAYS_EMULATE);
 
-    xc_cpuid_apply_policy(ctx->xch, domid, restore, NULL, 0,
-                          pae, itsc, nested_virt, info->cpuid);
+    rc = xc_cpuid_apply_policy(ctx->xch, domid, restore, NULL, 0,
+                               pae, itsc, nested_virt, info->cpuid);
+    if (rc)
+        LOGE(ERROR, "Failed to apply CPUID policy");

Is logging errno is going to be accurate enough ? The xc_cpuid_apply_policy() seems to only set rc and never change errno directly. It would often return "-errno" or an error code. There's one instance where we have "rc = -EOPNOTSUPP" but errno doesn't seems to be change to the same value (when checking "featureset).

So maybe having "LOGEV(ERROR, -rc, ...)" would be better?

And it should be r instead of rc. The latter is for libxl error code, the former for system call/libxc.

tools/libs/light/libxl_cpuid.c void libxl__cpuid_legacy(libxl_ctx *ctx, uint32_t domid, bool restore, · line 473

Quoted 3 lines
tools/libs/light/libxl_cpuid.c +3 -0
+
+    GC_FREE;
+    return rc ? ERROR_FAIL : 0;

The rest looks good.

Thanks,

Signature · 1 line
Anthony PERARD
#3 lore raw

PATCH v2 2/21 libs/guest: rename xc_get_cpu_policy_size to xc_cpu_policy_get_size

In reply to Roger Pau Monne

Preparatory change to introduce a new set of xc_cpu_policy_* functions that will replace the current CPUID/MSR helpers.

No functional change intended.

---
 tools/include/xenctrl.h             | 2 +-
 tools/libs/guest/xg_cpuid_x86.c     | 6 +++---
 tools/libs/guest/xg_sr_common_x86.c | 2 +-
 tools/misc/xen-cpuid.c              | 2 +-
 4 files changed, 6 insertions(+), 6 deletions(-)
tools/include/xenctrl.h +1 -1
index 318920166c5..e91ff92b9b1 100644
@@ -2594,7 +2594,7 @@ int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
 int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
                           uint32_t *nr_features, uint32_t *featureset);
 
-int xc_get_cpu_policy_size(xc_interface *xch, uint32_t *nr_leaves,
+int xc_cpu_policy_get_size(xc_interface *xch, uint32_t *nr_leaves,
                            uint32_t *nr_msrs);
 int xc_get_system_cpu_policy(xc_interface *xch, uint32_t index,
                              uint32_t *nr_leaves, xen_cpuid_leaf_t *leaves,
tools/libs/guest/xg_cpuid_x86.c +3 -3
index 5ea69ad3d51..cc5cae95725 100644
@@ -109,7 +109,7 @@ const uint32_t *xc_get_static_cpu_featuremask(
     return masks[mask];
 }
 
-int xc_get_cpu_policy_size(xc_interface *xch, uint32_t *nr_leaves,
+int xc_cpu_policy_get_size(xc_interface *xch, uint32_t *nr_leaves,
                            uint32_t *nr_msrs)
 {
     struct xen_sysctl sysctl = {};
@@ -302,7 +302,7 @@ static int xc_cpuid_xend_policy(
         goto fail;
     }
 
-    rc = xc_get_cpu_policy_size(xch, &nr_leaves, &nr_msrs);
+    rc = xc_cpu_policy_get_size(xch, &nr_leaves, &nr_msrs);
     if ( rc )
     {
         PERROR("Failed to obtain policy info size");
@@ -448,7 +448,7 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
         goto out;
     }
 
-    rc = xc_get_cpu_policy_size(xch, &nr_leaves, &nr_msrs);
+    rc = xc_cpu_policy_get_size(xch, &nr_leaves, &nr_msrs);
     if ( rc )
     {
         PERROR("Failed to obtain policy info size");
tools/libs/guest/xg_sr_common_x86.c +1 -1
index 3168c5485fd..4982519e055 100644
@@ -50,7 +50,7 @@ int write_x86_cpu_policy_records(struct xc_sr_context *ctx)
     uint32_t nr_leaves = 0, nr_msrs = 0;
     int rc;
 
-    if ( xc_get_cpu_policy_size(xch, &nr_leaves, &nr_msrs) < 0 )
+    if ( xc_cpu_policy_get_size(xch, &nr_leaves, &nr_msrs) < 0 )
     {
         PERROR("Unable to get CPU Policy size");
         return -1;
tools/misc/xen-cpuid.c +1 -1
index 2d04162d8d8..52596c08c90 100644
@@ -462,7 +462,7 @@ int main(int argc, char **argv)
         if ( !xch )
             err(1, "xc_interface_open");
 
-        if ( xc_get_cpu_policy_size(xch, &max_leaves, &max_msrs) )
+        if ( xc_cpu_policy_get_size(xch, &max_leaves, &max_msrs) )
             err(1, "xc_get_cpu_policy_size(...)");
         if ( domid == -1 )
             printf("Xen reports there are maximum %u leaves and %u MSRs\n",
Signature · 1 line
2.30.1
#4 lore raw

PATCH v2 3/21 libs/guest: introduce xc_cpu_policy_t

In reply to Roger Pau Monne

Introduce an opaque type that is used to store the CPUID and MSRs policies of a domain. Such type uses the existing {cpuid,msr}_policy structures to store the data, but doesn't expose the type to the users of the xenguest library. There are also two arrays to allow for easier serialization without requiring an allocation each time.

Introduce an allocation (init) and freeing function (destroy) to manage the type.

Note the type is not yet used anywhere.

---
Changes since v1:
- Introduce two arrays to store serialized versions.

---
 tools/include/xenctrl.h         |  6 ++++++
 tools/libs/guest/xg_cpuid_x86.c | 18 ++++++++++++++++++
 2 files changed, 24 insertions(+)
tools/include/xenctrl.h +6 -0
index e91ff92b9b1..1aba814f01c 100644
@@ -2590,6 +2590,12 @@ int xc_psr_get_domain_data(xc_interface *xch, uint32_t domid,
 int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
                        xc_psr_feat_type type, xc_psr_hw_info *hw_info);
 
+typedef struct xc_cpu_policy *xc_cpu_policy_t;
+
+/* Create and free a xc_cpu_policy object. */
+xc_cpu_policy_t xc_cpu_policy_init(void);
+void xc_cpu_policy_destroy(xc_cpu_policy_t policy);
+
 int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
 int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
                           uint32_t *nr_features, uint32_t *featureset);
tools/libs/guest/xg_cpuid_x86.c +18 -0
index cc5cae95725..8e3a1a8cbf2 100644
@@ -39,6 +39,13 @@ enum {
 #define bitmaskof(idx)      (1u << ((idx) & 31))
 #define featureword_of(idx) ((idx) >> 5)
 
+struct xc_cpu_policy {
+    struct cpuid_policy cpuid;
+    struct msr_policy msr;
+    xen_cpuid_leaf_t leaves[CPUID_MAX_SERIALISED_LEAVES];
+    xen_msr_entry_t entries[MSR_MAX_SERIALISED_ENTRIES];
+};
+
 int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps)
 {
     DECLARE_SYSCTL;
@@ -660,3 +667,14 @@ out:
 
     return rc;
 }
+
+xc_cpu_policy_t xc_cpu_policy_init(void)
+{
+    return calloc(1, sizeof(struct xc_cpu_policy));
+}
+
+void xc_cpu_policy_destroy(xc_cpu_policy_t policy)
+{
+    if ( policy )
+        free(policy);
+}
Signature · 1 line
2.30.1
#5 lore raw

PATCH v2 4/21 libs/guest: introduce helper to fetch a system cpu policy

In reply to Roger Pau Monne

Such helper is based on the existing functions to fetch a CPUID and MSR policies, but uses the xc_cpu_policy_t type to return the data to the caller.

Note some helper functions are introduced, those are split from xc_cpu_policy_get_system because they will be used by other functions also.

No user of the interface introduced on the patch.

---
Changes since v1:
- Always return -1 on error from xc_cpu_policy_get_system.
- Only print detailed error messages if err_leaf or err_msr is != -1.
- Rename idx function parameter to policy_idx.

---
 tools/include/xenctrl.h         |  4 +++
 tools/libs/guest/xg_cpuid_x86.c | 54 +++++++++++++++++++++++++++++++++
 2 files changed, 58 insertions(+)
tools/include/xenctrl.h +4 -0
index 1aba814f01c..187df5c5d2d 100644
@@ -2596,6 +2596,10 @@ typedef struct xc_cpu_policy *xc_cpu_policy_t;
 xc_cpu_policy_t xc_cpu_policy_init(void);
 void xc_cpu_policy_destroy(xc_cpu_policy_t policy);
 
+/* Retrieve a system policy, or get/set a domains policy. */
+int xc_cpu_policy_get_system(xc_interface *xch, unsigned int policy_idx,
+                             xc_cpu_policy_t policy);
+
 int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
 int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
                           uint32_t *nr_features, uint32_t *featureset);
tools/libs/guest/xg_cpuid_x86.c +54 -0

57 lines not on this page · raw message

Signature · 1 line
2.30.1
Reviewed-by on v2 4/21
#6 lore raw

Re: [PATCH v2 04/21] libs/guest: introduce helper to fetch a system cpu policy

Quoted 11 lines · Roger Pau Monne

Such helper is based on the existing functions to fetch a CPUID and MSR policies, but uses the xc_cpu_policy_t type to return the data to the caller.

Note some helper functions are introduced, those are split from xc_cpu_policy_get_system because they will be used by other functions also.

No user of the interface introduced on the patch.

#7 lore raw

PATCH v2 5/21 libs/guest: introduce helper to fetch a domain cpu policy

In reply to Roger Pau Monne

Such helper is based on the existing functions to fetch a CPUID and MSR policies, but uses the xc_cpu_policy_t type to return the data to the caller.

No user of the interface introduced on the patch.

---
Changes since v1:
- Uniformly return -1 on error from xc_cpu_policy_get_domain.

---
 tools/include/xenctrl.h         |  2 ++
 tools/libs/guest/xg_cpuid_x86.c | 25 +++++++++++++++++++++++++
 2 files changed, 27 insertions(+)
tools/include/xenctrl.h +2 -0
index 187df5c5d2d..34d979d11da 100644
@@ -2599,6 +2599,8 @@ void xc_cpu_policy_destroy(xc_cpu_policy_t policy);
 /* Retrieve a system policy, or get/set a domains policy. */
 int xc_cpu_policy_get_system(xc_interface *xch, unsigned int policy_idx,
                              xc_cpu_policy_t policy);
+int xc_cpu_policy_get_domain(xc_interface *xch, uint32_t domid,
+                             xc_cpu_policy_t policy);
 
 int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
 int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
tools/libs/guest/xg_cpuid_x86.c +25 -0
index 78fbc7db9d3..1394e503f3d 100644
@@ -732,3 +732,28 @@ int xc_cpu_policy_get_system(xc_interface *xch, unsigned int policy_idx,
 
     return rc;
 }
+
+int xc_cpu_policy_get_domain(xc_interface *xch, uint32_t domid,
+                             xc_cpu_policy_t policy)
+{
+    unsigned int nr_leaves = ARRAY_SIZE(policy->leaves);
+    unsigned int nr_entries = ARRAY_SIZE(policy->entries);
+    int rc;
+
+    rc = xc_get_domain_cpu_policy(xch, domid, &nr_leaves, policy->leaves,
+                                  &nr_entries, policy->entries);
+    if ( rc )
+    {
+        PERROR("Failed to obtain domain %u policy", domid);
+        return rc;
+    }
+
+    rc = deserialize_policy(xch, policy, nr_leaves, nr_entries);
+    if ( rc )
+    {
+        errno = -rc;
+        rc = -1;
+    }
+
+    return rc;
+}
Signature · 1 line
2.30.1
#8 lore raw

PATCH v2 6/21 libs/guest: introduce helper to serialize a cpu policy

In reply to Roger Pau Monne

Such helper allow converting a cpu policy into an array of xen_cpuid_leaf_t and xen_msr_entry_t elements, which matches the current interface of the CPUID/MSR functions. This is required in order for the user to be able to parse the CPUID/MSR data.

No user of the interface introduced in this patch.

---
 tools/include/xenctrl.h         |  5 +++++
 tools/libs/guest/xg_cpuid_x86.c | 32 ++++++++++++++++++++++++++++++++
 2 files changed, 37 insertions(+)
tools/include/xenctrl.h +5 -0
index 34d979d11da..a4827b1ae6a 100644
@@ -2602,6 +2602,11 @@ int xc_cpu_policy_get_system(xc_interface *xch, unsigned int policy_idx,
 int xc_cpu_policy_get_domain(xc_interface *xch, uint32_t domid,
                              xc_cpu_policy_t policy);
 
+/* Manipulate a policy via architectural representations. */
+int xc_cpu_policy_serialise(xc_interface *xch, const xc_cpu_policy_t policy,
+                            xen_cpuid_leaf_t *leaves, uint32_t *nr_leaves,
+                            xen_msr_entry_t *msrs, uint32_t *nr_msrs);
+
 int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
 int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
                           uint32_t *nr_features, uint32_t *featureset);
tools/libs/guest/xg_cpuid_x86.c +32 -0
index 1394e503f3d..918591a128c 100644
@@ -757,3 +757,35 @@ int xc_cpu_policy_get_domain(xc_interface *xch, uint32_t domid,
 
     return rc;
 }
+
+int xc_cpu_policy_serialise(xc_interface *xch, const xc_cpu_policy_t p,
+                            xen_cpuid_leaf_t *leaves, uint32_t *nr_leaves,
+                            xen_msr_entry_t *msrs, uint32_t *nr_msrs)
+{
+    int rc;
+
+    if ( leaves )
+    {
+        rc = x86_cpuid_copy_to_buffer(&p->cpuid, leaves, nr_leaves);
+        if ( rc )
+        {
+            ERROR("Failed to serialize CPUID policy");
+            errno = -rc;
+            return -1;
+        }
+    }
+
+    if ( msrs )
+    {
+        rc = x86_msr_copy_to_buffer(&p->msr, msrs, nr_msrs);
+        if ( rc )
+        {
+            ERROR("Failed to serialize MSR policy");
+            errno = -rc;
+            return -1;
+        }
+    }
+
+    errno = 0;
+    return 0;
+}
Signature · 1 line
2.30.1
#9 lore raw

PATCH v2 7/21 tools: switch existing users of xc_get_{system,domain}_cpu_policy

In reply to Roger Pau Monne

With the introduction of xc_cpu_policy_get_{system,domain} and xc_cpu_policy_serialise the current users of xc_get_{system,domain}_cpu_policy can be switched to the new interface.

Note that xc_get_{system,domain}_cpu_policy is removed from the public interface and the functions are made static, since there are still internal consumers in xg_cpuid_x86.c

---
 tools/include/xenctrl.h             |  6 -----
 tools/libs/guest/xg_cpuid_x86.c     | 39 ++++++++++++++---------------
 tools/libs/guest/xg_sr_common_x86.c | 15 ++++++++---
 tools/misc/xen-cpuid.c              | 21 ++++++++++------
 4 files changed, 45 insertions(+), 36 deletions(-)
tools/include/xenctrl.h +0 -6
index a4827b1ae6a..e9a86d63bad 100644
@@ -2613,12 +2613,6 @@ int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
 
 int xc_cpu_policy_get_size(xc_interface *xch, uint32_t *nr_leaves,
                            uint32_t *nr_msrs);
-int xc_get_system_cpu_policy(xc_interface *xch, uint32_t index,
-                             uint32_t *nr_leaves, xen_cpuid_leaf_t *leaves,
-                             uint32_t *nr_msrs, xen_msr_entry_t *msrs);
-int xc_get_domain_cpu_policy(xc_interface *xch, uint32_t domid,
-                             uint32_t *nr_leaves, xen_cpuid_leaf_t *leaves,
-                             uint32_t *nr_msrs, xen_msr_entry_t *msrs);
 int xc_set_domain_cpu_policy(xc_interface *xch, uint32_t domid,
                              uint32_t nr_leaves, xen_cpuid_leaf_t *leaves,
                              uint32_t nr_msrs, xen_msr_entry_t *msrs,
tools/libs/guest/xg_cpuid_x86.c +19 -20
index 918591a128c..208a247bb6e 100644
@@ -135,9 +135,9 @@ int xc_cpu_policy_get_size(xc_interface *xch, uint32_t *nr_leaves,
     return ret;
 }
 
-int xc_get_system_cpu_policy(xc_interface *xch, uint32_t index,
-                             uint32_t *nr_leaves, xen_cpuid_leaf_t *leaves,
-                             uint32_t *nr_msrs, xen_msr_entry_t *msrs)
+static int get_system_cpu_policy(xc_interface *xch, uint32_t index,
+                                 uint32_t *nr_leaves, xen_cpuid_leaf_t *leaves,
+                                 uint32_t *nr_msrs, xen_msr_entry_t *msrs)
 {
     struct xen_sysctl sysctl = {};
     DECLARE_HYPERCALL_BOUNCE(leaves,
@@ -173,9 +173,9 @@ int xc_get_system_cpu_policy(xc_interface *xch, uint32_t index,
     return ret;
 }
 
-int xc_get_domain_cpu_policy(xc_interface *xch, uint32_t domid,
-                             uint32_t *nr_leaves, xen_cpuid_leaf_t *leaves,
-                             uint32_t *nr_msrs, xen_msr_entry_t *msrs)
+static int get_domain_cpu_policy(xc_interface *xch, uint32_t domid,
+                                 uint32_t *nr_leaves, xen_cpuid_leaf_t *leaves,
+                                 uint32_t *nr_msrs, xen_msr_entry_t *msrs)
 {
     DECLARE_DOMCTL;
     DECLARE_HYPERCALL_BOUNCE(leaves,
@@ -329,7 +329,7 @@ static int xc_cpuid_xend_policy(
     /* Get the domain's current policy. */
     nr_msrs = 0;
     nr_cur = nr_leaves;
-    rc = xc_get_domain_cpu_policy(xch, domid, &nr_cur, cur, &nr_msrs, NULL);
+    rc = get_domain_cpu_policy(xch, domid, &nr_cur, cur, &nr_msrs, NULL);
     if ( rc )
     {
         PERROR("Failed to obtain d%d current policy", domid);
@@ -340,10 +340,9 @@ static int xc_cpuid_xend_policy(
     /* Get the domain type's default policy. */
     nr_msrs = 0;
     nr_def = nr_leaves;
-    rc = xc_get_system_cpu_policy(xch,
-                                  di.hvm ? XEN_SYSCTL_cpu_policy_hvm_default
-                                         : XEN_SYSCTL_cpu_policy_pv_default,
-                                  &nr_def, def, &nr_msrs, NULL);
+    rc = get_system_cpu_policy(xch, di.hvm ? XEN_SYSCTL_cpu_policy_hvm_default
+                                           : XEN_SYSCTL_cpu_policy_pv_default,
+                               &nr_def, def, &nr_msrs, NULL);
     if ( rc )
     {
         PERROR("Failed to obtain %s def policy", di.hvm ? "hvm" : "pv");
@@ -354,8 +353,8 @@ static int xc_cpuid_xend_policy(
     /* Get the host policy. */
     nr_msrs = 0;
     nr_host = nr_leaves;
-    rc = xc_get_system_cpu_policy(xch, XEN_SYSCTL_cpu_policy_host,
-                                  &nr_host, host, &nr_msrs, NULL);
+    rc = get_system_cpu_policy(xch, XEN_SYSCTL_cpu_policy_host,
+                               &nr_host, host, &nr_msrs, NULL);
     if ( rc )
     {
         PERROR("Failed to obtain host policy");
@@ -486,9 +485,9 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
 
     /* Get the domain's default policy. */
     nr_msrs = 0;
-    rc = xc_get_system_cpu_policy(xch, di.hvm ? XEN_SYSCTL_cpu_policy_hvm_default
-                                              : XEN_SYSCTL_cpu_policy_pv_default,
-                                  &nr_leaves, leaves, &nr_msrs, NULL);
+    rc = get_system_cpu_policy(xch, di.hvm ? XEN_SYSCTL_cpu_policy_hvm_default
+                                           : XEN_SYSCTL_cpu_policy_pv_default,
+                               &nr_leaves, leaves, &nr_msrs, NULL);
     if ( rc )
     {
         PERROR("Failed to obtain %s default policy", di.hvm ? "hvm" : "pv");
@@ -715,8 +714,8 @@ int xc_cpu_policy_get_system(xc_interface *xch, unsigned int policy_idx,
     unsigned int nr_entries = ARRAY_SIZE(policy->entries);
     int rc;
 
-    rc = xc_get_system_cpu_policy(xch, policy_idx, &nr_leaves, policy->leaves,
-                                  &nr_entries, policy->entries);
+    rc = get_system_cpu_policy(xch, policy_idx, &nr_leaves, policy->leaves,
+                               &nr_entries, policy->entries);
     if ( rc )
     {
         PERROR("Failed to obtain %u policy", policy_idx);
@@ -740,8 +739,8 @@ int xc_cpu_policy_get_domain(xc_interface *xch, uint32_t domid,
     unsigned int nr_entries = ARRAY_SIZE(policy->entries);
     int rc;
 
-    rc = xc_get_domain_cpu_policy(xch, domid, &nr_leaves, policy->leaves,
-                                  &nr_entries, policy->entries);
+    rc = get_domain_cpu_policy(xch, domid, &nr_leaves, policy->leaves,
+                               &nr_entries, policy->entries);
     if ( rc )
     {
         PERROR("Failed to obtain domain %u policy", domid);
tools/libs/guest/xg_sr_common_x86.c +12 -3
index 4982519e055..15265e7a331 100644
@@ -48,6 +48,7 @@ int write_x86_cpu_policy_records(struct xc_sr_context *ctx)
     struct xc_sr_record cpuid = { .type = REC_TYPE_X86_CPUID_POLICY, };
     struct xc_sr_record msrs  = { .type = REC_TYPE_X86_MSR_POLICY, };
     uint32_t nr_leaves = 0, nr_msrs = 0;
+    xc_cpu_policy_t policy = NULL;
     int rc;
 
     if ( xc_cpu_policy_get_size(xch, &nr_leaves, &nr_msrs) < 0 )
@@ -58,20 +59,27 @@ int write_x86_cpu_policy_records(struct xc_sr_context *ctx)
 
     cpuid.data = malloc(nr_leaves * sizeof(xen_cpuid_leaf_t));
     msrs.data  = malloc(nr_msrs   * sizeof(xen_msr_entry_t));
-    if ( !cpuid.data || !msrs.data )
+    policy = xc_cpu_policy_init();
+    if ( !cpuid.data || !msrs.data || !policy )
     {
         ERROR("Cannot allocate memory for CPU Policy");
         rc = -1;
         goto out;
     }
 
-    if ( xc_get_domain_cpu_policy(xch, ctx->domid, &nr_leaves, cpuid.data,
-                                  &nr_msrs, msrs.data) )
+    if ( xc_cpu_policy_get_domain(xch, ctx->domid, policy) )
     {
         PERROR("Unable to get d%d CPU Policy", ctx->domid);
         rc = -1;
         goto out;
     }
+    if ( xc_cpu_policy_serialise(xch, policy, cpuid.data, &nr_leaves,
+                                 msrs.data, &nr_msrs) )
+    {
+        PERROR("Unable to serialize d%d CPU Policy", ctx->domid);
+        rc = -1;
+        goto out;
+    }
 
     cpuid.length = nr_leaves * sizeof(xen_cpuid_leaf_t);
     if ( cpuid.length )
@@ -94,6 +102,7 @@ int write_x86_cpu_policy_records(struct xc_sr_context *ctx)
  out:
     free(cpuid.data);
     free(msrs.data);
+    xc_cpu_policy_destroy(policy);
 
     return rc;
 }
tools/misc/xen-cpuid.c +14 -7
index 52596c08c90..8ac25581d68 100644
@@ -458,9 +458,12 @@ int main(int argc, char **argv)
         uint32_t i, max_leaves, max_msrs;
 
         xc_interface *xch = xc_interface_open(0, 0, 0);
+        xc_cpu_policy_t policy = xc_cpu_policy_init();
 
         if ( !xch )
             err(1, "xc_interface_open");
+        if ( !policy )
+            err(1, "xc_cpu_policy_init");
 
         if ( xc_cpu_policy_get_size(xch, &max_leaves, &max_msrs) )
             err(1, "xc_get_cpu_policy_size(...)");
@@ -481,10 +484,11 @@ int main(int argc, char **argv)
             uint32_t nr_leaves = max_leaves;
             uint32_t nr_msrs = max_msrs;
 
-            if ( xc_get_domain_cpu_policy(xch, domid, &nr_leaves, leaves,
-                                          &nr_msrs, msrs) )
-                err(1, "xc_get_domain_cpu_policy(, %d, %d,, %d,)",
-                    domid, nr_leaves, nr_msrs);
+            if ( xc_cpu_policy_get_domain(xch, domid, policy) )
+                err(1, "xc_cpu_policy_get_domain(, %d, )", domid);
+            if ( xc_cpu_policy_serialise(xch, policy, leaves, &nr_leaves,
+                                         msrs, &nr_msrs) )
+                err(1, "xc_cpu_policy_serialise");
 
             snprintf(name, sizeof(name), "Domain %d", domid);
             print_policy(name, leaves, nr_leaves, msrs, nr_msrs);
@@ -497,8 +501,7 @@ int main(int argc, char **argv)
                 uint32_t nr_leaves = max_leaves;
                 uint32_t nr_msrs = max_msrs;
 
-                if ( xc_get_system_cpu_policy(xch, i, &nr_leaves, leaves,
-                                              &nr_msrs, msrs) )
+                if ( xc_cpu_policy_get_system(xch, i, policy) )
                 {
                     if ( errno == EOPNOTSUPP )
                     {
@@ -507,14 +510,18 @@ int main(int argc, char **argv)
                         continue;
                     }
 
-                    err(1, "xc_get_system_cpu_policy(, %s,,)", sys_policies[i]);
+                    err(1, "xc_cpu_policy_get_system(, %s, )", sys_policies[i]);
                 }
+                if ( xc_cpu_policy_serialise(xch, policy, leaves, &nr_leaves,
+                                             msrs, &nr_msrs) )
+                    err(1, "xc_cpu_policy_serialise");
 
                 print_policy(sys_policies[i], leaves, nr_leaves,
                              msrs, nr_msrs);
             }
         }
 
+        xc_cpu_policy_destroy(policy);
         free(leaves);
         free(msrs);
         xc_interface_close(xch);
Signature · 1 line
2.30.1
#10 lore raw

PATCH v2 8/21 libs/guest: introduce a helper to apply a cpu policy to a domain

In reply to Roger Pau Monne

Such helper is very similar to the existing xc_set_domain_cpu_policy interface, but takes an opaque xc_cpu_policy_t instead of arrays of CPUID leaves and MSRs.

No callers of the interface introduced in this patch.

---
 tools/include/xenctrl.h         |  2 ++
 tools/libs/guest/xg_cpuid_x86.c | 29 +++++++++++++++++++++++++++++
 2 files changed, 31 insertions(+)
tools/include/xenctrl.h +2 -0
index e9a86d63bad..27cec1b93ff 100644
@@ -2601,6 +2601,8 @@ int xc_cpu_policy_get_system(xc_interface *xch, unsigned int policy_idx,
                              xc_cpu_policy_t policy);
 int xc_cpu_policy_get_domain(xc_interface *xch, uint32_t domid,
                              xc_cpu_policy_t policy);
+int xc_cpu_policy_set_domain(xc_interface *xch, uint32_t domid,
+                             const xc_cpu_policy_t policy);
 
 /* Manipulate a policy via architectural representations. */
 int xc_cpu_policy_serialise(xc_interface *xch, const xc_cpu_policy_t policy,
tools/libs/guest/xg_cpuid_x86.c +29 -0
index 208a247bb6e..8b48c51a8ee 100644
@@ -757,6 +757,35 @@ int xc_cpu_policy_get_domain(xc_interface *xch, uint32_t domid,
     return rc;
 }
 
+int xc_cpu_policy_set_domain(xc_interface *xch, uint32_t domid,
+                             const xc_cpu_policy_t policy)
+{
+    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
+    unsigned int nr_leaves = ARRAY_SIZE(policy->leaves);
+    unsigned int nr_entries = ARRAY_SIZE(policy->entries);
+    int rc;
+
+    rc = xc_cpu_policy_serialise(xch, policy, policy->leaves, &nr_leaves,
+                                 policy->entries, &nr_entries);
+    if ( rc )
+        return rc;
+
+    rc = xc_set_domain_cpu_policy(xch, domid, nr_leaves, policy->leaves,
+                                  nr_entries, policy->entries,
+                                  &err_leaf, &err_subleaf, &err_msr);
+    if ( rc )
+    {
+        ERROR("Failed to set domain %u policy (%d = %s)", domid, -rc,
+              strerror(-rc));
+        if ( err_leaf != -1 )
+            ERROR("CPUID leaf %u subleaf %u", err_leaf, err_subleaf);
+        if ( err_msr != -1 )
+            ERROR("MSR index %#x\n", err_msr);
+    }
+
+    return rc;
+}
+
 int xc_cpu_policy_serialise(xc_interface *xch, const xc_cpu_policy_t p,
                             xen_cpuid_leaf_t *leaves, uint32_t *nr_leaves,
                             xen_msr_entry_t *msrs, uint32_t *nr_msrs)
Signature · 1 line
2.30.1
#11 lore raw

PATCH v2 9/21 libs/guest: allow fetching a specific CPUID leaf from a cpu policy

In reply to Roger Pau Monne

Introduce an interface that returns a specific leaf/subleaf from a cpu policy in xen_cpuid_leaf_t format.

This is useful to callers can peek data from the opaque xc_cpu_policy_t type.

No caller of the interface introduced on this patch.

Note that callers of find_leaf need to be slightly adjusted to use the new helper parameters.

---
Changes since v1:
- Use find leaf.

---
 tools/include/xenctrl.h         |  3 +++
 tools/libs/guest/xg_cpuid_x86.c | 38 ++++++++++++++++++++++++++++-----
 2 files changed, 36 insertions(+), 5 deletions(-)
tools/include/xenctrl.h +3 -0
index 27cec1b93ff..cbca7209e34 100644
@@ -2608,6 +2608,9 @@ int xc_cpu_policy_set_domain(xc_interface *xch, uint32_t domid,
 int xc_cpu_policy_serialise(xc_interface *xch, const xc_cpu_policy_t policy,
                             xen_cpuid_leaf_t *leaves, uint32_t *nr_leaves,
                             xen_msr_entry_t *msrs, uint32_t *nr_msrs);
+int xc_cpu_policy_get_cpuid(xc_interface *xch, const xc_cpu_policy_t policy,
+                            uint32_t leaf, uint32_t subleaf,
+                            xen_cpuid_leaf_t *out);
 
 int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
 int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
tools/libs/guest/xg_cpuid_x86.c +33 -5
index 8b48c51a8ee..d146c5bbc99 100644
@@ -280,9 +280,9 @@ static int compare_leaves(const void *l, const void *r)
 
 static xen_cpuid_leaf_t *find_leaf(
     xen_cpuid_leaf_t *leaves, unsigned int nr_leaves,
-    const struct xc_xend_cpuid *xend)
+    unsigned int leaf, unsigned int subleaf)
 {
-    const xen_cpuid_leaf_t key = { xend->leaf, xend->subleaf };
+    const xen_cpuid_leaf_t key = { leaf, subleaf };
 
     return bsearch(&key, leaves, nr_leaves, sizeof(*leaves), compare_leaves);
 }
@@ -365,9 +365,12 @@ static int xc_cpuid_xend_policy(
     rc = -EINVAL;
     for ( ; xend->leaf != XEN_CPUID_INPUT_UNUSED; ++xend )
     {
-        xen_cpuid_leaf_t *cur_leaf = find_leaf(cur, nr_cur, xend);
-        const xen_cpuid_leaf_t *def_leaf = find_leaf(def, nr_def, xend);
-        const xen_cpuid_leaf_t *host_leaf = find_leaf(host, nr_host, xend);
+        xen_cpuid_leaf_t *cur_leaf = find_leaf(cur, nr_cur,
+                                               xend->leaf, xend->subleaf);
+        const xen_cpuid_leaf_t *def_leaf = find_leaf(def, nr_def,
+                                                     xend->leaf, xend->subleaf);
+        const xen_cpuid_leaf_t *host_leaf = find_leaf(host, nr_host, xend->leaf,
+                                                      xend->subleaf);
 
         if ( cur_leaf == NULL || def_leaf == NULL || host_leaf == NULL )
         {
@@ -817,3 +820,28 @@ int xc_cpu_policy_serialise(xc_interface *xch, const xc_cpu_policy_t p,
     errno = 0;
     return 0;
 }
+
+int xc_cpu_policy_get_cpuid(xc_interface *xch, const xc_cpu_policy_t policy,
+                            uint32_t leaf, uint32_t subleaf,
+                            xen_cpuid_leaf_t *out)
+{
+    unsigned int nr_leaves = ARRAY_SIZE(policy->leaves);
+    xen_cpuid_leaf_t *tmp;
+    int rc;
+
+    rc = xc_cpu_policy_serialise(xch, policy, policy->leaves, &nr_leaves,
+                                 NULL, 0);
+    if ( rc )
+        return rc;
+
+    tmp = find_leaf(policy->leaves, nr_leaves, leaf, subleaf);
+    if ( !tmp )
+    {
+        /* Unable to find a matching leaf. */
+        errno = ENOENT;
+        return -1;
+    }
+
+    *out = *tmp;
+    return 0;
+}
Signature · 1 line
2.30.1
#12 lore raw

PATCH v2 10/21 tests/cpu-policy: add sorted MSR test

In reply to Roger Pau Monne

Further changes will rely on MSR entries being sorted, so add a test to assert it.

---
Changes since v1:
- New in this version.

---
 tools/tests/cpu-policy/test-cpu-policy.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
tools/tests/cpu-policy/test-cpu-policy.c +17 -0
index 0fa209f1ea7..ed450a09970 100644
@@ -86,6 +86,16 @@ static bool leaves_are_sorted(const xen_cpuid_leaf_t *leaves, unsigned int nr)
     return true;
 }
 
+static bool msrs_are_sorted(const xen_msr_entry_t *entries, unsigned int nr)
+{
+    for ( unsigned int i = 1; i < nr; ++i )
+        /* MSR index went backwards => not sorted. */
+        if ( entries[i - 1].idx > entries[i].idx )
+            return false;
+
+    return true;
+}
+
 static void test_cpuid_current(void)
 {
     struct cpuid_policy p;
@@ -270,6 +280,13 @@ static void test_msr_serialise_success(void)
             goto test_done;
         }
 
+        if ( !msrs_are_sorted(msrs, nr) )
+        {
+            fail("  Test %s, MSR entries not sorted\n",
+                 t->name);
+            goto test_done;
+        }
+
     test_done:
         free(msrs);
     }
Signature · 1 line
2.30.1
#13 lore raw

PATCH v2 11/21 libs/guest: allow fetching a specific MSR entry from a cpu policy

In reply to Roger Pau Monne

Introduce an interface that returns a specific MSR entry from a cpu policy in xen_msr_entry_t format. Provide a helper to perform a binary search against an array of MSR entries.

This is useful to callers can peek data from the opaque xc_cpu_policy_t type.

No caller of the interface introduced on this patch.

---
Changes since v1:
- Introduce a helper to perform a binary search of the MSR entries array.

---
 tools/include/xenctrl.h         |  2 ++
 tools/libs/guest/xg_cpuid_x86.c | 42 +++++++++++++++++++++++++++++++++
 2 files changed, 44 insertions(+)
tools/include/xenctrl.h +2 -0
index cbca7209e34..605c632cf30 100644
@@ -2611,6 +2611,8 @@ int xc_cpu_policy_serialise(xc_interface *xch, const xc_cpu_policy_t policy,
 int xc_cpu_policy_get_cpuid(xc_interface *xch, const xc_cpu_policy_t policy,
                             uint32_t leaf, uint32_t subleaf,
                             xen_cpuid_leaf_t *out);
+int xc_cpu_policy_get_msr(xc_interface *xch, const xc_cpu_policy_t policy,
+                          uint32_t msr, xen_msr_entry_t *out);
 
 int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
 int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
tools/libs/guest/xg_cpuid_x86.c +42 -0

45 lines not on this page · raw message

Signature · 1 line
2.30.1
#14 lore raw

PATCH v2 12/21 libs/guest: allow updating a cpu policy CPUID data

In reply to Roger Pau Monne

Introduce a helper to update the CPUID policy using an array of xen_cpuid_leaf_t entries. Note the leaves present in the input xen_cpuid_leaf_t array will replace any existing leaves on the policy.

No user of the interface introduced on this patch.

---
Changes since v1:
- Don't use memcpy.
- Drop logic to update the leaf manually - x86_cpuid_copy_from_buffer already does it.
- Only print a failure message if err_leaf != -1.

---
 tools/include/xenctrl.h         |  3 +++
 tools/libs/guest/xg_cpuid_x86.c | 20 ++++++++++++++++++++
 2 files changed, 23 insertions(+)
tools/include/xenctrl.h +3 -0
index 605c632cf30..49f919f16a7 100644
@@ -2613,6 +2613,9 @@ int xc_cpu_policy_get_cpuid(xc_interface *xch, const xc_cpu_policy_t policy,
                             xen_cpuid_leaf_t *out);
 int xc_cpu_policy_get_msr(xc_interface *xch, const xc_cpu_policy_t policy,
                           uint32_t msr, xen_msr_entry_t *out);
+int xc_cpu_policy_update_cpuid(xc_interface *xch, xc_cpu_policy_t policy,
+                               const xen_cpuid_leaf_t *leaves,
+                               uint32_t nr);
 
 int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
 int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
tools/libs/guest/xg_cpuid_x86.c +20 -0
index a4307d50ddb..00595e14cc3 100644
@@ -887,3 +887,23 @@ int xc_cpu_policy_get_msr(xc_interface *xch, const xc_cpu_policy_t policy,
     *out = *tmp;
     return 0;
 }
+
+int xc_cpu_policy_update_cpuid(xc_interface *xch, xc_cpu_policy_t policy,
+                               const xen_cpuid_leaf_t *leaves,
+                               uint32_t nr)
+{
+    unsigned int err_leaf = -1, err_subleaf = -1;
+    int rc = x86_cpuid_copy_from_buffer(&policy->cpuid, leaves, nr,
+                                        &err_leaf, &err_subleaf);
+
+    if ( rc )
+    {
+        if ( err_leaf != -1 )
+            ERROR("Failed to update CPUID (err leaf %#x, subleaf %#x) (%d = %s)",
+                  err_leaf, err_subleaf, -rc, strerror(-rc));
+        errno = -rc;
+        rc = -1;
+    }
+
+    return rc;
+}
Signature · 1 line
2.30.1
#15 lore raw

PATCH v2 13/21 libs/guest: allow updating a cpu policy MSR data

In reply to Roger Pau Monne

Introduce a helper to update the MSR policy using an array of xen_msr_entry_t entries. Note the MSRs present in the input xen_msr_entry_t array will replace any existing entries on the policy.

No user of the interface introduced on this patch.

---
Changes since v1:
- Drop logic to update entries manually.
- Only print failure message if err_msr != -1.

---
 tools/include/xenctrl.h         |  2 ++
 tools/libs/guest/xg_cpuid_x86.c | 18 ++++++++++++++++++
 2 files changed, 20 insertions(+)
tools/include/xenctrl.h +2 -0
index 49f919f16a7..9a6d1b126d8 100644
@@ -2616,6 +2616,8 @@ int xc_cpu_policy_get_msr(xc_interface *xch, const xc_cpu_policy_t policy,
 int xc_cpu_policy_update_cpuid(xc_interface *xch, xc_cpu_policy_t policy,
                                const xen_cpuid_leaf_t *leaves,
                                uint32_t nr);
+int xc_cpu_policy_update_msrs(xc_interface *xch, xc_cpu_policy_t policy,
+                              const xen_msr_entry_t *msrs, uint32_t nr);
 
 int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
 int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
tools/libs/guest/xg_cpuid_x86.c +18 -0
index 00595e14cc3..26b09322dfa 100644
@@ -907,3 +907,21 @@ int xc_cpu_policy_update_cpuid(xc_interface *xch, xc_cpu_policy_t policy,
 
     return rc;
 }
+
+int xc_cpu_policy_update_msrs(xc_interface *xch, xc_cpu_policy_t policy,
+                              const xen_msr_entry_t *msrs, uint32_t nr)
+{
+    unsigned int err_msr = -1;
+    int rc = x86_msr_copy_from_buffer(&policy->msr, msrs, nr, &err_msr);
+
+    if ( rc )
+    {
+        if ( err_msr != -1 )
+            ERROR("Failed to deserialise MSRS (err index %#x) (%d = %s)",
+                  err_msr, -rc, strerror(-rc));
+        errno = -rc;
+        rc = -1;
+    }
+
+    return rc;
+}
Signature · 1 line
2.30.1
#16 lore raw

PATCH v2 14/21 libs/guest: introduce helper to check cpu policy compatibility

In reply to Roger Pau Monne

Such helpers is just a wrapper to the existing x86_cpu_policies_are_compatible function. This requires building policy.c from libx86 on user land also.

No user of the interface introduced.

---
Changes since v1:
- Initialize err.
- Explicitly name parameters as host and guest.

---
 tools/include/xenctrl.h         |  4 ++++
 tools/libs/guest/Makefile       |  2 +-
 tools/libs/guest/xg_cpuid_x86.c | 19 +++++++++++++++++++
 3 files changed, 24 insertions(+), 1 deletion(-)
tools/include/xenctrl.h +4 -0
index 9a6d1b126d8..5f699c09509 100644
@@ -2619,6 +2619,10 @@ int xc_cpu_policy_update_cpuid(xc_interface *xch, xc_cpu_policy_t policy,
 int xc_cpu_policy_update_msrs(xc_interface *xch, xc_cpu_policy_t policy,
                               const xen_msr_entry_t *msrs, uint32_t nr);
 
+/* Compatibility calculations. */
+bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host,
+                                 const xc_cpu_policy_t guest);
+
 int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
 int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
                           uint32_t *nr_features, uint32_t *featureset);
tools/libs/guest/Makefile +1 -1
index 604e1695d6a..6d2a1d5bbc0 100644
@@ -40,7 +40,7 @@ $(patsubst %.c,%.opic,$(ELF_SRCS-y)): CFLAGS += -Wno-pointer-sign
 ifeq ($(CONFIG_X86),y) # Add libx86 to the build
 vpath %.c ../../../xen/lib/x86
 
-SRCS-y                 += cpuid.c msr.c
+SRCS-y                 += cpuid.c msr.c policy.c
 endif
 
 # new domain builder
tools/libs/guest/xg_cpuid_x86.c +19 -0
index 26b09322dfa..bd2f31dd87f 100644
@@ -925,3 +925,22 @@ int xc_cpu_policy_update_msrs(xc_interface *xch, xc_cpu_policy_t policy,
 
     return rc;
 }
+
+bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host,
+                                 const xc_cpu_policy_t guest)
+{
+    struct cpu_policy_errors err = INIT_CPU_POLICY_ERRORS;
+    struct cpu_policy h = { &host->cpuid, &host->msr };
+    struct cpu_policy g = { &guest->cpuid, &guest->msr };
+    int rc = x86_cpu_policies_are_compatible(&h, &g, &err);
+
+    if ( !rc )
+        return true;
+
+    if ( err.leaf != -1 )
+        ERROR("Leaf %#x subleaf %#x is not compatible", err.leaf, err.subleaf);
+    if ( err.msr != -1 )
+        ERROR("MSR index %#x is not compatible", err.msr);
+
+    return false;
+}
Signature · 1 line
2.30.1
#17 lore raw

Re: [PATCH v2 14/21] libs/guest: introduce helper to check cpu policy compatibility

Quoted 12 lines · Roger Pau Monne
tools/libs/guest/xg_cpuid_x86.c +6 -0
@@ -925,3 +925,22 @@ int xc_cpu_policy_update_msrs(xc_interface *xch, xc_cpu_policy_t policy,
 
     return rc;
 }
+
+bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host,
+                                 const xc_cpu_policy_t guest)
+{
+    struct cpu_policy_errors err = INIT_CPU_POLICY_ERRORS;
+    struct cpu_policy h = { &host->cpuid, &host->msr };

tools/libs/guest/xg_cpuid_x86.c int xc_cpu_policy_update_msrs(xc_interface *xch, xc_cpu_policy_t policy, · line 943

Quoted 10 lines
tools/libs/guest/xg_cpuid_x86.c +10 -0
+    struct cpu_policy g = { &guest->cpuid, &guest->msr };
+    int rc = x86_cpu_policies_are_compatible(&h, &g, &err);
+
+    if ( !rc )
+        return true;
+
+    if ( err.leaf != -1 )
+        ERROR("Leaf %#x subleaf %#x is not compatible", err.leaf, err.subleaf);
+    if ( err.msr != -1 )
+        ERROR("MSR index %#x is not compatible", err.msr);

Personally I'm against making assumptions like these ones about what (in this case) INIT_CPU_POLICY_ERRORS actually expands to (i.e. three times -1). I can see how alternatives to this are quickly going to get ugly, so I'll leave it to others to judge.

Jan

#18 lore raw

Re: [PATCH v2 14/21] libs/guest: introduce helper to check cpu policy compatibility

Quoted 20 lines · Jan Beulich
Quoted 19 lines · Roger Pau Monne
tools/libs/guest/xg_cpuid_x86.c +13 -0
@@ -925,3 +925,22 @@ int xc_cpu_policy_update_msrs(xc_interface *xch, xc_cpu_policy_t policy,
 
     return rc;
 }
+
+bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host,
+                                 const xc_cpu_policy_t guest)
+{
+    struct cpu_policy_errors err = INIT_CPU_POLICY_ERRORS;
+    struct cpu_policy h = { &host->cpuid, &host->msr };
+    struct cpu_policy g = { &guest->cpuid, &guest->msr };
+    int rc = x86_cpu_policies_are_compatible(&h, &g, &err);
+
+    if ( !rc )
+        return true;
+
+    if ( err.leaf != -1 )
Quoted 8 lines

tools/libs/guest/xg_cpuid_x86.c int xc_cpu_policy_update_msrs(xc_interface *xch, xc_cpu_policy_t policy, · line 943

Quoted 3 lines
tools/libs/guest/xg_cpuid_x86.c +3 -0
+        ERROR("Leaf %#x subleaf %#x is not compatible", err.leaf, err.subleaf);
+    if ( err.msr != -1 )
+        ERROR("MSR index %#x is not compatible", err.msr);

Personally I'm against making assumptions like these ones about what (in this case) INIT_CPU_POLICY_ERRORS actually expands to (i.e. three times -1). I can see how alternatives to this are quickly going to get ugly, so I'll leave it to others to judge.

Would you like me to define a separate POLICY_ERROR? ie:

#define INIT_CPU_POLICY_ERROR -1

#define INIT_CPU_POLICY_ERRORS { INIT_CPU_POLICY_ERROR, \
                                 INIT_CPU_POLICY_ERROR, \
                                 INIT_CPU_POLICY_ERROR }

We already have a bunch of open coded -1 checks anyway, but might prevent new ones from appearing.

Thanks, Roger.

#19 lore raw

Re: [PATCH v2 14/21] libs/guest: introduce helper to check cpu policy compatibility

Quoted 30 lines · Roger Pau Monné
Quoted 20 lines · Jan Beulich
Quoted 19 lines · Roger Pau Monne
tools/libs/guest/xg_cpuid_x86.c +13 -0
@@ -925,3 +925,22 @@ int xc_cpu_policy_update_msrs(xc_interface *xch, xc_cpu_policy_t policy,
 
     return rc;
 }
+
+bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host,
+                                 const xc_cpu_policy_t guest)
+{
+    struct cpu_policy_errors err = INIT_CPU_POLICY_ERRORS;
+    struct cpu_policy h = { &host->cpuid, &host->msr };
+    struct cpu_policy g = { &guest->cpuid, &guest->msr };
+    int rc = x86_cpu_policies_are_compatible(&h, &g, &err);
+
+    if ( !rc )
+        return true;
+
+    if ( err.leaf != -1 )
Quoted 8 lines

tools/libs/guest/xg_cpuid_x86.c int xc_cpu_policy_update_msrs(xc_interface *xch, xc_cpu_policy_t policy, · line 943

Quoted 3 lines
tools/libs/guest/xg_cpuid_x86.c +3 -0
+        ERROR("Leaf %#x subleaf %#x is not compatible", err.leaf, err.subleaf);
+    if ( err.msr != -1 )
+        ERROR("MSR index %#x is not compatible", err.msr);

Personally I'm against making assumptions like these ones about what (in this case) INIT_CPU_POLICY_ERRORS actually expands to (i.e. three times -1). I can see how alternatives to this are quickly going to get ugly, so I'll leave it to others to judge.

Quoted 6 lines

Would you like me to define a separate POLICY_ERROR? ie:

#define INIT_CPU_POLICY_ERROR -1

#define INIT_CPU_POLICY_ERRORS { INIT_CPU_POLICY_ERROR, \
                                 INIT_CPU_POLICY_ERROR, \
                                 INIT_CPU_POLICY_ERROR }

I would prefer this; I'm not sure (nit: properly parenthesized) -1 is the value to use though, considering the fields are unsigned. I wonder what Andrew thinks, as he did introduce all of this.

Quoted 2 lines

We already have a bunch of open coded -1 checks anyway, but might prevent new ones from appearing.

Indeed.

Jan

#20 lore raw

PATCH v2 15/21 libs/guest: obtain a compatible cpu policy from two input ones

In reply to Roger Pau Monne

Introduce a helper to obtain a compatible cpu policy based on two input cpu policies. Currently this is done by and'ing all CPUID leaves and MSR entries, except for MSR_ARCH_CAPABILITIES which has the RSBA bit or'ed.

The _AC macro is pulled from libxl_internal.h into xen-tools/libs.h since it's required in order to use the msr-index.h header.

Note there's no need to place this helper in libx86, since the calculation of a compatible policy shouldn't be done from the hypervisor.

No callers of the interface introduced.

---
Changes since v1:
- Only AND the feature parts of cpuid.
- Use a binary search to find the matching leaves and msr entries.
- Remove default case from MSR level function.

---
 tools/include/xen-tools/libs.h    |   5 ++
 tools/include/xenctrl.h           |   4 +
 tools/libs/guest/xg_cpuid_x86.c   | 128 ++++++++++++++++++++++++++++++
 tools/libs/light/libxl_internal.h |   2 -
 4 files changed, 137 insertions(+), 2 deletions(-)
tools/include/xen-tools/libs.h +5 -0
index a16e0c38070..b9e89f9a711 100644
@@ -63,4 +63,9 @@
 #define ROUNDUP(_x,_w) (((unsigned long)(_x)+(1UL<<(_w))-1) & ~((1UL<<(_w))-1))
 #endif
 
+#ifndef _AC
+#define __AC(X,Y)   (X##Y)
+#define _AC(X,Y)    __AC(X,Y)
+#endif
+
 #endif	/* __XEN_TOOLS_LIBS__ */
tools/include/xenctrl.h +4 -0
index 5f699c09509..c41d794683c 100644
@@ -2622,6 +2622,10 @@ int xc_cpu_policy_update_msrs(xc_interface *xch, xc_cpu_policy_t policy,
 /* Compatibility calculations. */
 bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host,
                                  const xc_cpu_policy_t guest);
+int xc_cpu_policy_calc_compatible(xc_interface *xch,
+                                  const xc_cpu_policy_t p1,
+                                  const xc_cpu_policy_t p2,
+                                  xc_cpu_policy_t out);
 
 int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
 int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
tools/libs/guest/xg_cpuid_x86.c +128 -0

137 lines not on this page · raw message

tools/libs/light/libxl_internal.h +0 -2
index 44a2f3c8fe3..5709bcb93fa 100644
@@ -126,8 +126,6 @@
 #define PVSHIM_CMDLINE "pv-shim console=xen,pv"
 
 /* Size macros. */
-#define __AC(X,Y)   (X##Y)
-#define _AC(X,Y)    __AC(X,Y)
 #define MB(_mb)     (_AC(_mb, ULL) << 20)
 #define GB(_gb)     (_AC(_gb, ULL) << 30)
 
Signature · 1 line
2.30.1
#21 lore raw

Re: [PATCH v2 15/21] libs/guest: obtain a compatible cpu policy from two input ones

tools/libs/guest/xg_cpuid_x86.c bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host, · line 951

Quoted 8 lines · Roger Pau Monne
tools/libs/guest/xg_cpuid_x86.c +4 -0
@@ -944,3 +945,130 @@ bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host,
 
     return false;
 }
+
+static uint64_t level_msr(unsigned int index, uint64_t val1, uint64_t val2)
+{
+    uint64_t val = val1 & val2;;

For arbitrary MSRs this isn't going to do any good. If only very specific MSRs are assumed to make it here, I think this wants commenting on.

Also, nit: stray semicolon.

Quoted 5 lines
tools/libs/guest/xg_cpuid_x86.c +5 -0
+    switch ( index )
+    {
+    case MSR_ARCH_CAPABILITIES:
+        /*
+         * Set RSBA if present on any of the input values to notice the guest

tools/libs/guest/xg_cpuid_x86.c bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host, · line 967

Quoted 10 lines
tools/libs/guest/xg_cpuid_x86.c +10 -0
+         * might run on vulnerable hardware at some point.
+         */
+        val |= (val1 | val2) & ARCH_CAPS_RSBA;
+        break;
+    }
+
+    return val;
+}
+
+static bool level_leaf(xen_cpuid_leaf_t *l1, xen_cpuid_leaf_t *l2,

const (twice)?

Quoted 7 lines
tools/libs/guest/xg_cpuid_x86.c +7 -0
+                       xen_cpuid_leaf_t *out)
+{
+    *out = (xen_cpuid_leaf_t){ };
+
+    switch ( l1->leaf )
+    {
+    case 0x1:

tools/libs/guest/xg_cpuid_x86.c bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host, · line 984

Quoted 10 lines
tools/libs/guest/xg_cpuid_x86.c +10 -0
+    case 0x80000001:
+        out->c = l1->c & l2->c;
+        out->d = l1->d & l2->d;
+        return true;
+
+    case 0xd:
+        if ( l1->subleaf != 1 )
+            break;
+        out->a = l1->a & l2->a;
+        return true;

Could you explain your thinking behind this (a code comment would likely help)? You effectively discard everything except subleaf 1 by returning false in that case, don't you?

Quoted 4 lines
tools/libs/guest/xg_cpuid_x86.c +4 -0
+    case 0x7:
+        switch ( l1->subleaf )
+        {
+        case 0:

tools/libs/guest/xg_cpuid_x86.c bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host, · line 999

Quoted 10 lines
tools/libs/guest/xg_cpuid_x86.c +10 -0
+            out->b = l1->b & l2->b;
+            out->c = l1->c & l2->c;
+            out->d = l1->d & l2->d;
+            return true;
+
+        case 1:
+            out->a = l1->a & l2->a;
+            return true;
+        }
+        break;

Can we perhaps assume all subleaves here are going to hold flags, and hence and both sides together without regard to what subleaf we're actually dealing with (subleaf 1 remaining special as to EAX of course)? This would avoid having to remember to make yet another mechanical change when enabling a new subleaf.

tools/libs/guest/xg_cpuid_x86.c bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host, · line 1011

Quoted 11 lines
tools/libs/guest/xg_cpuid_x86.c +11 -0
+    case 0x80000007:
+        out->d = l1->d & l2->d;
+        return true;
+
+    case 0x80000008:
+        out->b = l1->b & l2->b;
+        return true;
+    }
+
+    return false;
+}

Considering your LFENCE-always-serializing patch, I assume whichever ends up going in last will take care of adding handling of that leaf here?

Jan

#22 lore raw

Re: [PATCH v2 15/21] libs/guest: obtain a compatible cpu policy from two input ones

Quoted 13 lines · Jan Beulich

tools/libs/guest/xg_cpuid_x86.c bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host, · line 951

Quoted 8 lines · Roger Pau Monne
tools/libs/guest/xg_cpuid_x86.c +4 -0
@@ -944,3 +945,130 @@ bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host,
 
     return false;
 }
+
+static uint64_t level_msr(unsigned int index, uint64_t val1, uint64_t val2)
+{
+    uint64_t val = val1 & val2;;

For arbitrary MSRs this isn't going to do any good. If only very specific MSRs are assumed to make it here, I think this wants commenting on.

I've added: "MSRs passed to level_msr are expected to be bitmaps of features"

Quoted 13 lines
Quoted 13 lines
tools/libs/guest/xg_cpuid_x86.c +13 -0
+                       xen_cpuid_leaf_t *out)
+{
+    *out = (xen_cpuid_leaf_t){ };
+
+    switch ( l1->leaf )
+    {
+    case 0x1:
+    case 0x80000001:
+        out->c = l1->c & l2->c;
+        out->d = l1->d & l2->d;
+        return true;
+
+    case 0xd:
Quoted 8 lines

tools/libs/guest/xg_cpuid_x86.c bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host, · line 984

Quoted 4 lines
tools/libs/guest/xg_cpuid_x86.c +4 -0
+        if ( l1->subleaf != 1 )
+            break;
+        out->a = l1->a & l2->a;
+        return true;

Could you explain your thinking behind this (a code comment would likely help)? You effectively discard everything except subleaf 1 by returning false in that case, don't you?

Yes, the intent is to only level the features bitfield found in subleaf 1.

I was planning for level_leaf so far in this series to deal with the feature leaves part of the featureset only. I guess you would also like to leverage other parts of the xstate leaf, like the max_size or the supported bits in xss_{low,high}?

Quoted 15 lines
Quoted 4 lines
tools/libs/guest/xg_cpuid_x86.c +4 -0
+    case 0x7:
+        switch ( l1->subleaf )
+        {
+        case 0:

tools/libs/guest/xg_cpuid_x86.c bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host, · line 999

Quoted 10 lines
tools/libs/guest/xg_cpuid_x86.c +10 -0
+            out->b = l1->b & l2->b;
+            out->c = l1->c & l2->c;
+            out->d = l1->d & l2->d;
+            return true;
+
+        case 1:
+            out->a = l1->a & l2->a;
+            return true;
+        }
+        break;
Quoted 3 lines

Can we perhaps assume all subleaves here are going to hold flags, and hence and both sides together without regard to what subleaf we're actually dealing with (subleaf 1 remaining special as to

I think you meant subleaf 0 EAX (the max subleaf value)?

subleaf 1 EAX is just a normal bitfield AFAIK.

Quoted 2 lines

EAX of course)? This would avoid having to remember to make yet another mechanical change when enabling a new subleaf.

Sure, seems like a fine approach since leaf 7 will only contain feature bitmaps.

Quoted 7 lines
Quoted 7 lines
tools/libs/guest/xg_cpuid_x86.c +7 -0
+    case 0x80000007:
+        out->d = l1->d & l2->d;
+        return true;
+
+    case 0x80000008:
+        out->b = l1->b & l2->b;
+        return true;
Quoted 8 lines

tools/libs/guest/xg_cpuid_x86.c bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host, · line 1011

Quoted 4 lines
tools/libs/guest/xg_cpuid_x86.c +4 -0
+    }
+
+    return false;
+}

Considering your LFENCE-always-serializing patch, I assume whichever ends up going in last will take care of adding handling of that leaf here?

Indeed.

Thanks, Roger.

#23 lore raw

Re: [PATCH v2 15/21] libs/guest: obtain a compatible cpu policy from two input ones

Quoted 15 lines · Roger Pau Monné
Quoted 13 lines · Jan Beulich

tools/libs/guest/xg_cpuid_x86.c bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host, · line 951

Quoted 8 lines · Roger Pau Monne
tools/libs/guest/xg_cpuid_x86.c +4 -0
@@ -944,3 +945,130 @@ bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host,
 
     return false;
 }
+
+static uint64_t level_msr(unsigned int index, uint64_t val1, uint64_t val2)
+{
+    uint64_t val = val1 & val2;;

For arbitrary MSRs this isn't going to do any good. If only very specific MSRs are assumed to make it here, I think this wants commenting on.

Quoted 2 lines

I've added: "MSRs passed to level_msr are expected to be bitmaps of features"

How does such a comment help? I.e. how does the caller tell which MSRs to pass here and which to deal with anyother way?

Quoted 22 lines
Quoted 13 lines
Quoted 13 lines
tools/libs/guest/xg_cpuid_x86.c +13 -0
+                       xen_cpuid_leaf_t *out)
+{
+    *out = (xen_cpuid_leaf_t){ };
+
+    switch ( l1->leaf )
+    {
+    case 0x1:
+    case 0x80000001:
+        out->c = l1->c & l2->c;
+        out->d = l1->d & l2->d;
+        return true;
+
+    case 0xd:
Quoted 8 lines

tools/libs/guest/xg_cpuid_x86.c bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host, · line 984

Quoted 4 lines
tools/libs/guest/xg_cpuid_x86.c +4 -0
+        if ( l1->subleaf != 1 )
+            break;
+        out->a = l1->a & l2->a;
+        return true;

Could you explain your thinking behind this (a code comment would likely help)? You effectively discard everything except subleaf 1 by returning false in that case, don't you?

Quoted 7 lines

Yes, the intent is to only level the features bitfield found in subleaf 1.

I was planning for level_leaf so far in this series to deal with the feature leaves part of the featureset only. I guess you would also like to leverage other parts of the xstate leaf, like the max_size or the supported bits in xss_{low,high}?

The latter is clearly one of the things to consider, yes (alongside the respective bits in sub-leaf 0 for XCR0). Sub-leaves > 1 may also need dealing with ECX. Yet then again some or all of this may need handling elsewhere, not the least because of the unusual handling of leaf 0xd in the hypervisor. What gets checked and/or adjusted where needs to be settled upon, and then the different parts of code would imo better cross-reference each other.

Quoted 15 lines
Quoted 15 lines
Quoted 14 lines
tools/libs/guest/xg_cpuid_x86.c +14 -0
+    case 0x7:
+        switch ( l1->subleaf )
+        {
+        case 0:
+            out->b = l1->b & l2->b;
+            out->c = l1->c & l2->c;
+            out->d = l1->d & l2->d;
+            return true;
+
+        case 1:
+            out->a = l1->a & l2->a;
+            return true;
+        }
+        break;
Quoted 7 lines
Quoted 3 lines

Can we perhaps assume all subleaves here are going to hold flags, and hence and both sides together without regard to what subleaf we're actually dealing with (subleaf 1 remaining special as to

I think you meant subleaf 0 EAX (the max subleaf value)?

subleaf 1 EAX is just a normal bitfield AFAIK.

Indeed I did.

Jan

Quoted 5 lines
Quoted 2 lines

EAX of course)? This would avoid having to remember to make yet another mechanical change when enabling a new subleaf.

Sure, seems like a fine approach since leaf 7 will only contain feature bitmaps.

#24 lore raw

Re: [PATCH v2 15/21] libs/guest: obtain a compatible cpu policy from two input ones

Quoted 16 lines · Jan Beulich
Quoted 15 lines · Roger Pau Monné
Quoted 13 lines · Jan Beulich

tools/libs/guest/xg_cpuid_x86.c bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host, · line 951

Quoted 8 lines · Roger Pau Monne
tools/libs/guest/xg_cpuid_x86.c +4 -0
@@ -944,3 +945,130 @@ bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host,
 
     return false;
 }
+
+static uint64_t level_msr(unsigned int index, uint64_t val1, uint64_t val2)
+{
+    uint64_t val = val1 & val2;;

For arbitrary MSRs this isn't going to do any good. If only very specific MSRs are assumed to make it here, I think this wants commenting on.

Quoted 5 lines
Quoted 2 lines

I've added: "MSRs passed to level_msr are expected to be bitmaps of features"

How does such a comment help? I.e. how does the caller tell which MSRs to pass here and which to deal with anyother way?

All MSRs should be passed to level_msr, but it's handling logic would need to be expanded to support MSRs that are not feature bitmaps.

It might be best to restore the previous switch and handle each MSR specifically?

From your reply to v1 I wrongly misunderstood that you initially wanted to handle all MSRs as bitmaps:

https://lore.kernel.org/xen-devel/[email protected]/

Quoted 30 lines
Quoted 22 lines
Quoted 13 lines
Quoted 13 lines
tools/libs/guest/xg_cpuid_x86.c +13 -0
+                       xen_cpuid_leaf_t *out)
+{
+    *out = (xen_cpuid_leaf_t){ };
+
+    switch ( l1->leaf )
+    {
+    case 0x1:
+    case 0x80000001:
+        out->c = l1->c & l2->c;
+        out->d = l1->d & l2->d;
+        return true;
+
+    case 0xd:
Quoted 8 lines

tools/libs/guest/xg_cpuid_x86.c bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host, · line 984

Quoted 4 lines
tools/libs/guest/xg_cpuid_x86.c +4 -0
+        if ( l1->subleaf != 1 )
+            break;
+        out->a = l1->a & l2->a;
+        return true;

Could you explain your thinking behind this (a code comment would likely help)? You effectively discard everything except subleaf 1 by returning false in that case, don't you?

Quoted 7 lines

Yes, the intent is to only level the features bitfield found in subleaf 1.

I was planning for level_leaf so far in this series to deal with the feature leaves part of the featureset only. I guess you would also like to leverage other parts of the xstate leaf, like the max_size or the supported bits in xss_{low,high}?

Quoted 7 lines

The latter is clearly one of the things to consider, yes (alongside the respective bits in sub-leaf 0 for XCR0). Sub-leaves > 1 may also need dealing with ECX. Yet then again some or all of this may need handling elsewhere, not the least because of the unusual handling of leaf 0xd in the hypervisor. What gets checked and/or adjusted where needs to be settled upon, and then the different parts of code would imo better cross-reference each other.

There's a comment in recalculate_xstate that mentions that Da1 leaf is the only piece of information preserved, and that everything else is derived from feature state. I don't think it makes sense to try to level anything apart from Da1 if it's going to be discarded by recalculate_xstate anyway?

I can add a comment here regarding why only Da1 is taken into account for leveling so far.

Thanks, Roger.

#25 lore raw

Re: [PATCH v2 15/21] libs/guest: obtain a compatible cpu policy from two input ones

Quoted 20 lines · Roger Pau Monné
Quoted 19 lines · Jan Beulich
Quoted 17 lines · Roger Pau Monné
Quoted 13 lines · Jan Beulich

tools/libs/guest/xg_cpuid_x86.c bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host, · line 951

Quoted 8 lines · Roger Pau Monne
tools/libs/guest/xg_cpuid_x86.c +4 -0
@@ -944,3 +945,130 @@ bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host,
 
     return false;
 }
+
+static uint64_t level_msr(unsigned int index, uint64_t val1, uint64_t val2)
+{
+    uint64_t val = val1 & val2;;

For arbitrary MSRs this isn't going to do any good. If only very specific MSRs are assumed to make it here, I think this wants commenting on.

I've added: "MSRs passed to level_msr are expected to be bitmaps of features"

Quoted 8 lines
Quoted 2 lines

How does such a comment help? I.e. how does the caller tell which MSRs to pass here and which to deal with anyother way?

All MSRs should be passed to level_msr, but it's handling logic would need to be expanded to support MSRs that are not feature bitmaps.

It might be best to restore the previous switch and handle each MSR specifically?

I think so, yes. We need to be very careful with what a possible default case does there, though.

Quoted 38 lines
Quoted 30 lines
Quoted 22 lines
Quoted 13 lines
Quoted 13 lines
tools/libs/guest/xg_cpuid_x86.c +13 -0
+                       xen_cpuid_leaf_t *out)
+{
+    *out = (xen_cpuid_leaf_t){ };
+
+    switch ( l1->leaf )
+    {
+    case 0x1:
+    case 0x80000001:
+        out->c = l1->c & l2->c;
+        out->d = l1->d & l2->d;
+        return true;
+
+    case 0xd:
Quoted 8 lines

tools/libs/guest/xg_cpuid_x86.c bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host, · line 984

Quoted 4 lines
tools/libs/guest/xg_cpuid_x86.c +4 -0
+        if ( l1->subleaf != 1 )
+            break;
+        out->a = l1->a & l2->a;
+        return true;

Could you explain your thinking behind this (a code comment would likely help)? You effectively discard everything except subleaf 1 by returning false in that case, don't you?

Quoted 7 lines

Yes, the intent is to only level the features bitfield found in subleaf 1.

I was planning for level_leaf so far in this series to deal with the feature leaves part of the featureset only. I guess you would also like to leverage other parts of the xstate leaf, like the max_size or the supported bits in xss_{low,high}?

Quoted 7 lines

The latter is clearly one of the things to consider, yes (alongside the respective bits in sub-leaf 0 for XCR0). Sub-leaves > 1 may also need dealing with ECX. Yet then again some or all of this may need handling elsewhere, not the least because of the unusual handling of leaf 0xd in the hypervisor. What gets checked and/or adjusted where needs to be settled upon, and then the different parts of code would imo better cross-reference each other.

Quoted 8 lines

There's a comment in recalculate_xstate that mentions that Da1 leaf is the only piece of information preserved, and that everything else is derived from feature state. I don't think it makes sense to try to level anything apart from Da1 if it's going to be discarded by recalculate_xstate anyway?

I can add a comment here regarding why only Da1 is taken into account for leveling so far.

Yes, this would help. Thanks.

Jan

#26 lore raw

Re: [PATCH v2 15/21] libs/guest: obtain a compatible cpu policy from two input ones

Quoted 24 lines · Jan Beulich
Quoted 23 lines · Roger Pau Monné
Quoted 21 lines · Jan Beulich
Quoted 15 lines · Roger Pau Monné
Quoted 13 lines · Jan Beulich

tools/libs/guest/xg_cpuid_x86.c bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host, · line 951

Quoted 8 lines · Roger Pau Monne
tools/libs/guest/xg_cpuid_x86.c +4 -0
@@ -944,3 +945,130 @@ bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host,
 
     return false;
 }
+
+static uint64_t level_msr(unsigned int index, uint64_t val1, uint64_t val2)
+{
+    uint64_t val = val1 & val2;;

For arbitrary MSRs this isn't going to do any good. If only very specific MSRs are assumed to make it here, I think this wants commenting on.

Quoted 2 lines

I've added: "MSRs passed to level_msr are expected to be bitmaps of features"

How does such a comment help? I.e. how does the caller tell which MSRs to pass here and which to deal with anyother way?

Quoted 8 lines
Quoted 5 lines

All MSRs should be passed to level_msr, but it's handling logic would need to be expanded to support MSRs that are not feature bitmaps.

It might be best to restore the previous switch and handle each MSR specifically?

I think so, yes. We need to be very careful with what a possible default case does there, though.

Maybe it would be better to handle level_msr in a way similar to
level_leaf: return true/false to notice whether the MSR should be added to the resulting compatible policy?

And then make the default case in level_msr just return false in order to prevent adding MSRs not properly leveled to the policy?

Thanks, Roger.

#27 lore raw

Re: [PATCH v2 15/21] libs/guest: obtain a compatible cpu policy from two input ones

Quoted 34 lines · Roger Pau Monné
Quoted 24 lines · Jan Beulich
Quoted 23 lines · Roger Pau Monné
Quoted 21 lines · Jan Beulich
Quoted 15 lines · Roger Pau Monné
Quoted 13 lines · Jan Beulich

Review bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host, · line 951

Quoted 8 lines · Roger Pau Monne
(quoted diff) +4 -0
@@ -944,3 +945,130 @@ bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host,
 
     return false;
 }
+
+static uint64_t level_msr(unsigned int index, uint64_t val1, uint64_t val2)
+{
+    uint64_t val = val1 & val2;;

For arbitrary MSRs this isn't going to do any good. If only very specific MSRs are assumed to make it here, I think this wants commenting on.

Quoted 2 lines

I've added: "MSRs passed to level_msr are expected to be bitmaps of features"

How does such a comment help? I.e. how does the caller tell which MSRs to pass here and which to deal with anyother way?

Quoted 8 lines
Quoted 5 lines

All MSRs should be passed to level_msr, but it's handling logic would need to be expanded to support MSRs that are not feature bitmaps.

It might be best to restore the previous switch and handle each MSR specifically?

I think so, yes. We need to be very careful with what a possible default case does there, though.

Quoted 6 lines

Maybe it would be better to handle level_msr in a way similar to
level_leaf: return true/false to notice whether the MSR should be added to the resulting compatible policy?

And then make the default case in level_msr just return false in order to prevent adding MSRs not properly leveled to the policy?

I'm afraid I'm not clear about the implications. What again is the (planned?) final effect of an MSR not getting added there?

Jan

#28 lore raw

Re: [PATCH v2 15/21] libs/guest: obtain a compatible cpu policy from two input ones

Quoted 39 lines · Jan Beulich
Quoted 38 lines · Roger Pau Monné
Quoted 24 lines · Jan Beulich
Quoted 23 lines · Roger Pau Monné
Quoted 21 lines · Jan Beulich
Quoted 15 lines · Roger Pau Monné
Quoted 13 lines · Jan Beulich

Review bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host, · line 951

Quoted 8 lines · Roger Pau Monne
(quoted diff) +4 -0
@@ -944,3 +945,130 @@ bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host,
 
     return false;
 }
+
+static uint64_t level_msr(unsigned int index, uint64_t val1, uint64_t val2)
+{
+    uint64_t val = val1 & val2;;

For arbitrary MSRs this isn't going to do any good. If only very specific MSRs are assumed to make it here, I think this wants commenting on.

Quoted 2 lines

I've added: "MSRs passed to level_msr are expected to be bitmaps of features"

How does such a comment help? I.e. how does the caller tell which MSRs to pass here and which to deal with anyother way?

Quoted 8 lines
Quoted 5 lines

All MSRs should be passed to level_msr, but it's handling logic would need to be expanded to support MSRs that are not feature bitmaps.

It might be best to restore the previous switch and handle each MSR specifically?

I think so, yes. We need to be very careful with what a possible default case does there, though.

Maybe it would be better to handle level_msr in a way similar to
level_leaf: return true/false to notice whether the MSR should be added to the resulting compatible policy?

Quoted 5 lines
Quoted 2 lines

And then make the default case in level_msr just return false in order to prevent adding MSRs not properly leveled to the policy?

I'm afraid I'm not clear about the implications. What again is the (planned?) final effect of an MSR not getting added there?

Adding the MSR with a 0 value will zero out any previous value on the 'out' policy, while not adding it would leave the previous value there given the current code in xc_cpu_policy_calc_compatible added by this patch.

I would expect callers of xc_cpu_policy_calc_compatible to pass a zeroed 'out' policy, so I think the end result should be the same.

Maybe I should also clear 'out' in xc_cpu_policy_calc_compatible, at which point both options will get the same exact result.

Thanks, Roger.

#29 lore raw

Re: [PATCH v2 15/21] libs/guest: obtain a compatible cpu policy from two input ones

Quoted 46 lines · Roger Pau Monné
Quoted 39 lines · Jan Beulich
Quoted 38 lines · Roger Pau Monné
Quoted 24 lines · Jan Beulich
Quoted 23 lines · Roger Pau Monné
Quoted 21 lines · Jan Beulich
Quoted 15 lines · Roger Pau Monné
Quoted 13 lines · Jan Beulich

Review bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host, · line 951

Quoted 8 lines · Roger Pau Monne
(quoted diff) +4 -0
@@ -944,3 +945,130 @@ bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host,
 
     return false;
 }
+
+static uint64_t level_msr(unsigned int index, uint64_t val1, uint64_t val2)
+{
+    uint64_t val = val1 & val2;;

For arbitrary MSRs this isn't going to do any good. If only very specific MSRs are assumed to make it here, I think this wants commenting on.

Quoted 2 lines

I've added: "MSRs passed to level_msr are expected to be bitmaps of features"

How does such a comment help? I.e. how does the caller tell which MSRs to pass here and which to deal with anyother way?

Quoted 8 lines
Quoted 5 lines

All MSRs should be passed to level_msr, but it's handling logic would need to be expanded to support MSRs that are not feature bitmaps.

It might be best to restore the previous switch and handle each MSR specifically?

I think so, yes. We need to be very careful with what a possible default case does there, though.

Maybe it would be better to handle level_msr in a way similar to
level_leaf: return true/false to notice whether the MSR should be added to the resulting compatible policy?

Quoted 5 lines
Quoted 2 lines

And then make the default case in level_msr just return false in order to prevent adding MSRs not properly leveled to the policy?

I'm afraid I'm not clear about the implications. What again is the (planned?) final effect of an MSR not getting added there?

Quoted 7 lines

Adding the MSR with a 0 value will zero out any previous value on the 'out' policy, while not adding it would leave the previous value there given the current code in xc_cpu_policy_calc_compatible added by this patch.

I would expect callers of xc_cpu_policy_calc_compatible to pass a zeroed 'out' policy, so I think the end result should be the same.

But we're not talking about actual MSR values here, as this is all about policy. So in the end we'll have to see how things need to be once we have the first non-feature-flag-like entries there. It feels as if simply zeroing can't be generally the right thing in such a case. It may e.g. be that min() is wanted instead.

Jan

#30 lore raw

Re: [PATCH v2 15/21] libs/guest: obtain a compatible cpu policy from two input ones

Quoted 52 lines · Jan Beulich
Quoted 51 lines · Roger Pau Monné
Quoted 39 lines · Jan Beulich
Quoted 38 lines · Roger Pau Monné
Quoted 24 lines · Jan Beulich
Quoted 23 lines · Roger Pau Monné
Quoted 21 lines · Jan Beulich
Quoted 15 lines · Roger Pau Monné
Quoted 13 lines · Jan Beulich

Review bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host, · line 951

Quoted 8 lines · Roger Pau Monne
(quoted diff) +4 -0
@@ -944,3 +945,130 @@ bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host,
 
     return false;
 }
+
+static uint64_t level_msr(unsigned int index, uint64_t val1, uint64_t val2)
+{
+    uint64_t val = val1 & val2;;

For arbitrary MSRs this isn't going to do any good. If only very specific MSRs are assumed to make it here, I think this wants commenting on.

Quoted 2 lines

I've added: "MSRs passed to level_msr are expected to be bitmaps of features"

How does such a comment help? I.e. how does the caller tell which MSRs to pass here and which to deal with anyother way?

Quoted 8 lines
Quoted 5 lines

All MSRs should be passed to level_msr, but it's handling logic would need to be expanded to support MSRs that are not feature bitmaps.

It might be best to restore the previous switch and handle each MSR specifically?

I think so, yes. We need to be very careful with what a possible default case does there, though.

Maybe it would be better to handle level_msr in a way similar to
level_leaf: return true/false to notice whether the MSR should be added to the resulting compatible policy?

Quoted 5 lines
Quoted 2 lines

And then make the default case in level_msr just return false in order to prevent adding MSRs not properly leveled to the policy?

I'm afraid I'm not clear about the implications. What again is the (planned?) final effect of an MSR not getting added there?

Adding the MSR with a 0 value will zero out any previous value on the 'out' policy, while not adding it would leave the previous value there given the current code in xc_cpu_policy_calc_compatible added by this patch.

Quoted 8 lines
Quoted 2 lines

I would expect callers of xc_cpu_policy_calc_compatible to pass a zeroed 'out' policy, so I think the end result should be the same.

But we're not talking about actual MSR values here, as this is all about policy. So in the end we'll have to see how things need to be once we have the first non-feature-flag-like entries there. It feels as if simply zeroing can't be generally the right thing in such a case. It may e.g. be that min() is wanted instead.

Maybe level_msr should return an error for MSRs not explicitly handled, that's propagated to the caller of xc_cpu_policy_calc_compatible.

That way addition of new MSRs are not likely to miss adding the required handling in level_msr?

Thanks, Roger.

#31 lore raw

Re: [PATCH v2 15/21] libs/guest: obtain a compatible cpu policy from two input ones

Quoted 62 lines · Roger Pau Monné
Quoted 52 lines · Jan Beulich
Quoted 51 lines · Roger Pau Monné
Quoted 39 lines · Jan Beulich
Quoted 38 lines · Roger Pau Monné
Quoted 24 lines · Jan Beulich
Quoted 23 lines · Roger Pau Monné
Quoted 21 lines · Jan Beulich
Quoted 15 lines · Roger Pau Monné
Quoted 13 lines · Jan Beulich

Review bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host, · line 951

Quoted 8 lines · Roger Pau Monne
(quoted diff) +4 -0
@@ -944,3 +945,130 @@ bool xc_cpu_policy_is_compatible(xc_interface *xch, const xc_cpu_policy_t host,
 
     return false;
 }
+
+static uint64_t level_msr(unsigned int index, uint64_t val1, uint64_t val2)
+{
+    uint64_t val = val1 & val2;;

For arbitrary MSRs this isn't going to do any good. If only very specific MSRs are assumed to make it here, I think this wants commenting on.

Quoted 2 lines

I've added: "MSRs passed to level_msr are expected to be bitmaps of features"

How does such a comment help? I.e. how does the caller tell which MSRs to pass here and which to deal with anyother way?

Quoted 8 lines
Quoted 5 lines

All MSRs should be passed to level_msr, but it's handling logic would need to be expanded to support MSRs that are not feature bitmaps.

It might be best to restore the previous switch and handle each MSR specifically?

I think so, yes. We need to be very careful with what a possible default case does there, though.

Maybe it would be better to handle level_msr in a way similar to
level_leaf: return true/false to notice whether the MSR should be added to the resulting compatible policy?

Quoted 5 lines
Quoted 2 lines

And then make the default case in level_msr just return false in order to prevent adding MSRs not properly leveled to the policy?

I'm afraid I'm not clear about the implications. What again is the (planned?) final effect of an MSR not getting added there?

Adding the MSR with a 0 value will zero out any previous value on the 'out' policy, while not adding it would leave the previous value there given the current code in xc_cpu_policy_calc_compatible added by this patch.

Quoted 8 lines
Quoted 2 lines

I would expect callers of xc_cpu_policy_calc_compatible to pass a zeroed 'out' policy, so I think the end result should be the same.

But we're not talking about actual MSR values here, as this is all about policy. So in the end we'll have to see how things need to be once we have the first non-feature-flag-like entries there. It feels as if simply zeroing can't be generally the right thing in such a case. It may e.g. be that min() is wanted instead.

Quoted 6 lines

Maybe level_msr should return an error for MSRs not explicitly handled, that's propagated to the caller of xc_cpu_policy_calc_compatible.

That way addition of new MSRs are not likely to miss adding the required handling in level_msr?

Perhaps, yes.

Jan

#32 lore raw

Re: [PATCH v2 15/21] libs/guest: obtain a compatible cpu policy from two input ones

In reply to Roger Pau Monne
Quoted 5 lines · Roger Pau Monne

Introduce a helper to obtain a compatible cpu policy based on two input cpu policies. Currently this is done by and'ing all CPUID leaves and MSR entries, except for MSR_ARCH_CAPABILITIES which has the RSBA bit or'ed.

I thought canonical source for compatibility was to be put into the hypervisor, thus all this functionality would be in the hypervisor. Am I misremembering?

Wei.

#33 lore raw

Re: [PATCH v2 15/21] libs/guest: obtain a compatible cpu policy from two input ones

Quoted 10 lines · Wei Liu
Quoted 5 lines · Roger Pau Monne

Introduce a helper to obtain a compatible cpu policy based on two input cpu policies. Currently this is done by and'ing all CPUID leaves and MSR entries, except for MSR_ARCH_CAPABILITIES which has the RSBA bit or'ed.

I thought canonical source for compatibility was to be put into the hypervisor, thus all this functionality would be in the hypervisor. Am I misremembering?

Andrew said something similar on v1, but I'm not able to figure how this would be used by the hypervisor.

It's my understating that the toolstack will attempt to generate a CPU policy and forward it to the hypervisor, which will either accept or reject it based on the capabilities of the system. I'm not sure I see why we would need to give the hypervisor two policies in order to generate a resulting compatible one - it should all be done by the toolstack AFAICT.

If there's a use case for this being in the hypervisor I'm happy to add it there, but so far I haven't been able to come up with one myself, and hence I don't see the need to make the code available.

Thanks, Roger.

#34 lore raw

Re: [PATCH v2 15/21] libs/guest: obtain a compatible cpu policy from two input ones

Quoted 22 lines · Roger Pau Monné
Quoted 10 lines · Wei Liu
Quoted 5 lines · Roger Pau Monne

Introduce a helper to obtain a compatible cpu policy based on two input cpu policies. Currently this is done by and'ing all CPUID leaves and MSR entries, except for MSR_ARCH_CAPABILITIES which has the RSBA bit or'ed.

I thought canonical source for compatibility was to be put into the hypervisor, thus all this functionality would be in the hypervisor. Am I misremembering?

Andrew said something similar on v1, but I'm not able to figure how this would be used by the hypervisor.

It's my understating that the toolstack will attempt to generate a CPU policy and forward it to the hypervisor, which will either accept or reject it based on the capabilities of the system. I'm not sure I see why we would need to give the hypervisor two policies in order to generate a resulting compatible one - it should all be done by the toolstack AFAICT.

Quoted 3 lines

If there's a use case for this being in the hypervisor I'm happy to add it there, but so far I haven't been able to come up with one myself, and hence I don't see the need to make the code available.

I have no opinion here. Just wanted to get on the same page.

Thanks for the explanation.

Wei.

Quoted 2 lines

Thanks, Roger.

#35 lore raw

PATCH v2 16/21 libs/guest: make a cpu policy compatible with older Xen versions

In reply to Roger Pau Monne

Older Xen versions used to expose some CPUID bits which are no longer exposed by default. In order to keep a compatible behavior with guests migrated from versions of Xen that don't encode the CPUID data on the migration stream introduce a function that sets the same bits as older Xen versions.

This is pulled out from xc_cpuid_apply_policy which already has this logic present.

No functional change intended.

---
Changes since v1:
- Move comments and explicitly mention pre-4.14 Xen.

---
 tools/include/xenctrl.h         |  4 +++
 tools/libs/guest/xg_cpuid_x86.c | 46 ++++++++++++++++++++++++++++-----
 2 files changed, 43 insertions(+), 7 deletions(-)
tools/include/xenctrl.h +4 -0
index c41d794683c..89a73fd6823 100644
@@ -2627,6 +2627,10 @@ int xc_cpu_policy_calc_compatible(xc_interface *xch,
                                   const xc_cpu_policy_t p2,
                                   xc_cpu_policy_t out);
 
+/* Make a policy compatible with pre-4.14 Xen versions. */
+int xc_cpu_policy_make_compatible(xc_interface *xch, xc_cpu_policy_t policy,
+                                  bool hvm);
+
 int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
 int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
                           uint32_t *nr_features, uint32_t *featureset);
tools/libs/guest/xg_cpuid_x86.c +39 -7

65 lines not on this page · raw message

Signature · 1 line
2.30.1
#36 lore raw

PATCH v2 17/21 libs/guest: introduce helper set cpu topology in cpu policy

In reply to Roger Pau Monne

This logic is pulled out from xc_cpuid_apply_policy and placed into a separate helper. Note the legacy part of the introduced function, as long term Xen will require a proper topology setter function capable of expressing a more diverse set of topologies.

No functional change intended.

---
- s/xc_cpu_policy_topology/xc_cpu_policy_legacy_topology/

---
 tools/include/xenctrl.h         |   4 +
 tools/libs/guest/xg_cpuid_x86.c | 182 +++++++++++++++++---------------
 2 files changed, 103 insertions(+), 83 deletions(-)
tools/include/xenctrl.h +4 -0
index 89a73fd6823..ec184bccd3f 100644
@@ -2631,6 +2631,10 @@ int xc_cpu_policy_calc_compatible(xc_interface *xch,
 int xc_cpu_policy_make_compatible(xc_interface *xch, xc_cpu_policy_t policy,
                                   bool hvm);
 
+/* Setup the legacy policy topology. */
+int xc_cpu_policy_legacy_topology(xc_interface *xch, xc_cpu_policy_t policy,
+                                  bool hvm);
+
 int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
 int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
                           uint32_t *nr_features, uint32_t *featureset);
tools/libs/guest/xg_cpuid_x86.c +99 -83

207 lines not on this page · raw message

Signature · 1 line
2.30.1
#37 lore raw

PATCH v2 18/21 libs/guest: rework xc_cpuid_xend_policy

In reply to Roger Pau Monne

Rename xc_cpuid_xend_policy to xc_cpu_policy_apply_cpuid and make it public. Modify the function internally to use the new xc_cpu_policy_* set of functions. Also don't apply the passed policy to a domain directly, and instead modify the provided xc_cpu_policy_t. The caller will be responsible of applying the modified cpu policy to the domain.

Note that further patches will end up removing this function, as the callers should have the necessary helpers to modify an xc_cpu_policy_t themselves.

No functional change intended.

---
 tools/include/xenctrl.h         |   4 +
 tools/libs/guest/xg_cpuid_x86.c | 182 ++++++++++++++------------------
 2 files changed, 84 insertions(+), 102 deletions(-)
tools/include/xenctrl.h +4 -0
index ec184bccd3f..ce0785f7654 100644
@@ -2635,6 +2635,10 @@ int xc_cpu_policy_make_compatible(xc_interface *xch, xc_cpu_policy_t policy,
 int xc_cpu_policy_legacy_topology(xc_interface *xch, xc_cpu_policy_t policy,
                                   bool hvm);
 
+/* Apply an xc_xend_cpuid object to the policy. */
+int xc_cpu_policy_apply_cpuid(xc_interface *xch, xc_cpu_policy_t policy,
+                              const struct xc_xend_cpuid *cpuid, bool hvm);
+
 int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
 int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
                           uint32_t *nr_features, uint32_t *featureset);
tools/libs/guest/xg_cpuid_x86.c +80 -102

251 lines not on this page · raw message

Signature · 1 line
2.30.1
#38 lore raw

PATCH v2 19/21 libs/guest: apply a featureset into a cpu policy

In reply to Roger Pau Monne

Pull out the code from xc_cpuid_apply_policy that applies a featureset to a cpu policy and place it on it's own standalone function that's part of the public interface.

No functional change intended.

---
 tools/include/xenctrl.h         |  5 ++
 tools/libs/guest/xg_cpuid_x86.c | 95 ++++++++++++++++++++-------------
 2 files changed, 62 insertions(+), 38 deletions(-)
tools/include/xenctrl.h +5 -0
index ce0785f7654..3fef954d1d1 100644
@@ -2639,6 +2639,11 @@ int xc_cpu_policy_legacy_topology(xc_interface *xch, xc_cpu_policy_t policy,
 int xc_cpu_policy_apply_cpuid(xc_interface *xch, xc_cpu_policy_t policy,
                               const struct xc_xend_cpuid *cpuid, bool hvm);
 
+/* Apply a featureset to the policy. */
+int xc_cpu_policy_apply_featureset(xc_interface *xch, xc_cpu_policy_t policy,
+                                   const uint32_t *featureset,
+                                   unsigned int nr_features);
+
 int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
 int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
                           uint32_t *nr_features, uint32_t *featureset);
tools/libs/guest/xg_cpuid_x86.c +57 -38

106 lines not on this page · raw message

Signature · 1 line
2.30.1
#39 lore raw

PATCH v2 20/21 libs/{light,guest}: implement xc_cpuid_apply_policy in libxl

In reply to Roger Pau Monne

With the addition of the xc_cpu_policy_* now libxl can have better control over the cpu policy, this allows removing the xc_cpuid_apply_policy function and instead coding the required bits by libxl in libxl__cpuid_legacy directly.

Remove xc_cpuid_apply_policy.

---
 tools/include/xenctrl.h         |  18 -----
 tools/libs/guest/xg_cpuid_x86.c | 126 --------------------------------
 tools/libs/light/libxl_cpuid.c  |  87 +++++++++++++++++++++-
 3 files changed, 83 insertions(+), 148 deletions(-)
tools/include/xenctrl.h +0 -18
index 3fef954d1d1..c6ee1142e8e 100644
@@ -1890,24 +1890,6 @@ struct xc_xend_cpuid {
     char *policy[4];
 };
 
-/*
- * Make adjustments to the CPUID settings for a domain.
- *
- * This path is used in two cases.  First, for fresh boots of the domain, and
- * secondly for migrate-in/restore of pre-4.14 guests (where CPUID data was
- * missing from the stream).  The @restore parameter distinguishes these
- * cases, and the generated policy must be compatible with a 4.13.
- *
- * Either pass a full new @featureset (and @nr_features), or adjust individual
- * features (@pae, @itsc, @nested_virt).
- *
- * Then (optionally) apply legacy XEND overrides (@xend) to the result.
- */
-int xc_cpuid_apply_policy(xc_interface *xch,
-                          uint32_t domid, bool restore,
-                          const uint32_t *featureset,
-                          unsigned int nr_features, bool pae, bool itsc,
-                          bool nested_virt, const struct xc_xend_cpuid *xend);
 int xc_mca_op(xc_interface *xch, struct xen_mc *mc);
 int xc_mca_op_inject_v2(xc_interface *xch, unsigned int flags,
                         xc_cpumap_t cpumap, unsigned int nr_cpus);
tools/libs/guest/xg_cpuid_x86.c +0 -126

132 lines not on this page · raw message

tools/libs/light/libxl_cpuid.c +83 -4

115 lines not on this page · raw message

Signature · 1 line
2.30.1
#40 lore raw

Re: [PATCH v2 20/21] libs/{light,guest}: implement xc_cpuid_apply_policy in libxl

Quoted 6 lines · Roger Pau Monne
tools/libs/light/libxl_cpuid.c +0 -0
index 539fc4869e6..cadc8b2a05e 100644
@@ -436,6 +438,42 @@ int libxl__cpuid_legacy(libxl_ctx *ctx, uint32_t domid, bool restore,
      */

tools/libs/light/libxl_cpuid.c int libxl__cpuid_legacy(libxl_ctx *ctx, uint32_t domid, bool restore, · line 448

Quoted 10 lines
tools/libs/light/libxl_cpuid.c +8 -0
     bool nested_virt = info->nested_hvm.val > 0;
 
+    policy = xc_cpu_policy_init();
+    if (!policy) {
+        LOGE(ERROR, "Failed to init CPU policy");
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+    rc = xc_cpu_policy_get_domain(ctx->xch, domid, policy);

libxc return values should be stored in 'r'.

tools/libs/light/libxl_cpuid.c int libxl__cpuid_legacy(libxl_ctx *ctx, uint32_t domid, bool restore, · line 450

Quoted 2 lines
tools/libs/light/libxl_cpuid.c +2 -0
+    if (rc) {
+        LOGE(ERROR, "Failed to fetch domain %u CPU policy", domid);

It's probably better to use LOGED(ERROR, domid, ...) to log the domid. libvirt might be able to use that domid information, but I'm not sure about that. At least, using LOG*D macros would make logging the domid consistent with the rest of libxl.

Also use the LOG*D variant for other LOG*, to record the domid every time.

tools/libs/light/libxl_cpuid.c int libxl__cpuid_legacy(libxl_ctx *ctx, uint32_t domid, bool restore, · line 456

Quoted 6 lines
tools/libs/light/libxl_cpuid.c +6 -0
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+    /*
+     * Account for feature which have been disabled by default since Xen 4.13,

Why the 's' from "features" has been removed? Also, if it must be removed then I'm pretty sure that we should also change "have been" to "has been".

Thanks,

Signature · 1 line
Anthony PERARD
#41 lore raw

PATCH v2 21/21 libs/guest: (re)move xc_cpu_policy_apply_cpuid

In reply to Roger Pau Monne

Move the logic from xc_cpu_policy_apply_cpuid into libxl, now that the xc_cpu_policy_* helpers allow modifying a cpu policy. By moving such parsing into libxl directly we can get rid of xc_xend_cpuid, as libxl will now implement it's own private type for storing CPUID information, which currently matches xc_xend_cpuid.

Note the function logic is moved as-is, but requires adapting to the libxl coding style.

No functional change intended.

---
 tools/include/libxl.h             |   6 +-
 tools/include/xenctrl.h           |  30 -------
 tools/libs/guest/xg_cpuid_x86.c   | 125 ---------------------------
 tools/libs/light/libxl_cpuid.c    | 136 +++++++++++++++++++++++++++++-
 tools/libs/light/libxl_internal.h |  26 ++++++
 5 files changed, 162 insertions(+), 161 deletions(-)
tools/include/libxl.h +3 -3
index ae7fe27c1f2..150b7ba85ac 100644
@@ -1375,10 +1375,10 @@ void libxl_bitmap_init(libxl_bitmap *map);
 void libxl_bitmap_dispose(libxl_bitmap *map);
 
 /*
- * libxl_cpuid_policy is opaque in the libxl ABI.  Users of both libxl and
- * libxc may not make assumptions about xc_xend_cpuid.
+ * libxl_cpuid_policy is opaque in the libxl ABI. Users of libxl may not make
+ * assumptions about libxl__cpuid_policy.
  */
-typedef struct xc_xend_cpuid libxl_cpuid_policy;
+typedef struct libxl__cpuid_policy libxl_cpuid_policy;
 typedef libxl_cpuid_policy * libxl_cpuid_policy_list;
 void libxl_cpuid_dispose(libxl_cpuid_policy_list *cpuid_list);
 int libxl_cpuid_policy_list_length(const libxl_cpuid_policy_list *l);
tools/include/xenctrl.h +0 -30
index c6ee1142e8e..044d05321a3 100644
@@ -1864,32 +1864,6 @@ int xc_domain_debug_control(xc_interface *xch,
 
 #if defined(__i386__) || defined(__x86_64__)
 
-/*
- * CPUID policy data, expressed in the legacy XEND format.
- *
- * Policy is an array of strings, 32 chars long:
- *   policy[0] = eax
- *   policy[1] = ebx
- *   policy[2] = ecx
- *   policy[3] = edx
- *
- * The format of the string is the following:
- *   '1' -> force to 1
- *   '0' -> force to 0
- *   'x' -> we don't care (use default)
- *   'k' -> pass through host value
- *   's' -> legacy alias for 'k'
- */
-struct xc_xend_cpuid {
-    union {
-        struct {
-            uint32_t leaf, subleaf;
-        };
-        uint32_t input[2];
-    };
-    char *policy[4];
-};
-
 int xc_mca_op(xc_interface *xch, struct xen_mc *mc);
 int xc_mca_op_inject_v2(xc_interface *xch, unsigned int flags,
                         xc_cpumap_t cpumap, unsigned int nr_cpus);
@@ -2617,10 +2591,6 @@ int xc_cpu_policy_make_compatible(xc_interface *xch, xc_cpu_policy_t policy,
 int xc_cpu_policy_legacy_topology(xc_interface *xch, xc_cpu_policy_t policy,
                                   bool hvm);
 
-/* Apply an xc_xend_cpuid object to the policy. */
-int xc_cpu_policy_apply_cpuid(xc_interface *xch, xc_cpu_policy_t policy,
-                              const struct xc_xend_cpuid *cpuid, bool hvm);
-
 /* Apply a featureset to the policy. */
 int xc_cpu_policy_apply_featureset(xc_interface *xch, xc_cpu_policy_t policy,
                                    const uint32_t *featureset,
tools/libs/guest/xg_cpuid_x86.c +0 -125

131 lines not on this page · raw message

tools/libs/light/libxl_cpuid.c +133 -3

160 lines not on this page · raw message

tools/libs/light/libxl_internal.h +26 -0
index 5709bcb93fa..d130c073ac4 100644
@@ -2050,6 +2050,32 @@ typedef yajl_gen_status (*libxl__gen_json_callback)(yajl_gen hand, void *);
 _hidden char *libxl__object_to_json(libxl_ctx *ctx, const char *type,
                                     libxl__gen_json_callback gen, void *p);
 
+/*
+ * CPUID policy data, expressed in the internal libxl format.
+ *
+ * Policy is an array of strings, 32 chars long:
+ *   policy[0] = eax
+ *   policy[1] = ebx
+ *   policy[2] = ecx
+ *   policy[3] = edx
+ *
+ * The format of the string is the following:
+ *   '1' -> force to 1
+ *   '0' -> force to 0
+ *   'x' -> we don't care (use default)
+ *   'k' -> pass through host value
+ *   's' -> legacy alias for 'k'
+ */
+struct libxl__cpuid_policy {
+    union {
+        struct {
+            uint32_t leaf, subleaf;
+        };
+        uint32_t input[2];
+    };
+    char *policy[4];
+};
+
 _hidden int libxl__cpuid_legacy(libxl_ctx *ctx, uint32_t domid, bool retore,
                                 libxl_domain_build_info *info);
 
Signature · 1 line
2.30.1
#42 lore raw

Re: [PATCH v2 21/21] libs/guest: (re)move xc_cpu_policy_apply_cpuid

tools/libs/light/libxl_cpuid.c int libxl_cpuid_parse_config_xend(libxl_cpuid_policy_list *cpuid, · line 422

Quoted 9 lines · Roger Pau Monne
tools/libs/light/libxl_cpuid.c +1 -0
index cadc8b2a05e..6be2d773d1d 100644
@@ -419,6 +419,136 @@ int libxl_cpuid_parse_config_xend(libxl_cpuid_policy_list *cpuid,
     return 0;
 }
 
+static int apply_cpuid(libxl_ctx *ctx, xc_cpu_policy_t policy,

I'm pretty sure we want libxl__gc here instead of ctx. ctx will then by available via CTX. (and the GC_* macro will not need to be called)

Beside that, there is also the need to store libxc return values in 'r' instead 'rc', and maybe use LOG*D macros to log the domid on error messages.

Thanks,

Signature · 1 line
Anthony PERARD