Skip to contentSkip to conversation

Enable legacy LBR support for guest

PATCH In review Yang Weijiang 2 participants 5 messages Jan 22, 2022 – Feb 10, 2022 KVM

lore
Share & tools
lore
#0 lore raw

v5 cover Enable legacy LBR support for guest

KVM legacy LBR patches have been merged in kernel 5.12, this patchset is to expose the feature to guest from the perf capability MSR. Qemu can add LBR format in cpu option to achieve it, e.g., -cpu host,lbr-fmt=0x5, the format should match host value in IA32_PERF_CAPABILITIES.

Note, KVM legacy LBR solution accelerates guest perf performace by LBR MSR passthrough so it requires guest cpu model matches that of host's, i.e., only -cpu host is supported.

Change in v5:
1. This patchset is rebased on tip : 6621441db5
2. No functional change since v4.

Yang Weijiang (2):
qdev-properties: Add a new macro with bitmask check for uint64_t
property
target/i386: Add lbr-fmt vPMU option to support guest LBR

 hw/core/qdev-properties.c    | 19 +++++++++++++++++
 include/hw/qdev-properties.h | 12 +++++++++++
 target/i386/cpu.c            | 40 ++++++++++++++++++++++++++++++++++++
 target/i386/cpu.h            | 10 +++++++++
 4 files changed, 81 insertions(+)
Signature · 1 line
2.27.0
#1 lore raw

PATCH v5 1/2 qdev-properties: Add a new macro with bitmask check for uint64_t property

The DEFINE_PROP_UINT64_CHECKMASK maro applies certain mask check agaist user-supplied property value, reject the value if it violates the bitmask.

---
 hw/core/qdev-properties.c    | 19 +++++++++++++++++++
 include/hw/qdev-properties.h | 12 ++++++++++++
 2 files changed, 31 insertions(+)
hw/core/qdev-properties.c +19 -0
index c34aac6ebc..27566e5ef7 100644
@@ -428,6 +428,25 @@ const PropertyInfo qdev_prop_int64 = {
     .set_default_value = qdev_propinfo_set_default_value_int,
 };
 
+static void set_uint64_checkmask(Object *obj, Visitor *v, const char *name,
+                      void *opaque, Error **errp)
+{
+    Property *prop = opaque;
+    uint64_t *ptr = object_field_prop_ptr(obj, prop);
+
+    visit_type_uint64(v, name, ptr, errp);
+    if (*ptr & ~prop->bitmask) {
+        error_setg(errp, "Property value for '%s' violates bitmask '0x%lx'",
+                   name, prop->bitmask);
+    }
+}
+
+const PropertyInfo qdev_prop_uint64_checkmask = {
+    .name  = "uint64",
+    .get   = get_uint64,
+    .set   = set_uint64_checkmask,
+};
+
 /* --- string --- */
 
 static void release_string(Object *obj, const char *name, void *opaque)
include/hw/qdev-properties.h +12 -0
index f7925f67d0..e1df08876c 100644
@@ -17,6 +17,7 @@ struct Property {
     const PropertyInfo *info;
     ptrdiff_t    offset;
     uint8_t      bitnr;
+    uint64_t     bitmask;
     bool         set_default;
     union {
         int64_t i;
@@ -54,6 +55,7 @@ extern const PropertyInfo qdev_prop_uint16;
 extern const PropertyInfo qdev_prop_uint32;
 extern const PropertyInfo qdev_prop_int32;
 extern const PropertyInfo qdev_prop_uint64;
+extern const PropertyInfo qdev_prop_uint64_checkmask;
 extern const PropertyInfo qdev_prop_int64;
 extern const PropertyInfo qdev_prop_size;
 extern const PropertyInfo qdev_prop_string;
@@ -103,6 +105,16 @@ extern const PropertyInfo qdev_prop_link;
                 .set_default = true,                         \
                 .defval.u    = (bool)_defval)
 
+/**
+ * The DEFINE_PROP_UINT64_CHECKMASK macro checks a user-supplied value
+ * against corresponding bitmask, rejects the value if it violates.
+ * The default value is set in instance_init().
+ */
+#define DEFINE_PROP_UINT64_CHECKMASK(_name, _state, _field, _bitmask)   \
+    DEFINE_PROP(_name, _state, _field, qdev_prop_uint64_checkmask, uint64_t, \
+                .bitmask    = (_bitmask),                     \
+                .set_default = false)
+
 #define PROP_ARRAY_LEN_PREFIX "len-"
 
 /**
Signature · 1 line
2.27.0
#2 lore raw

PATCH v5 2/2 target/i386: Add lbr-fmt vPMU option to support guest LBR

In reply to Yang Weijiang

The Last Branch Recording (LBR) is a performance monitor unit (PMU) feature on Intel processors which records a running trace of the most recent branches taken by the processor in the LBR stack. This option indicates the LBR format to enable for guest perf.

The LBR feature is enabled if below conditions are met:
1) KVM is enabled and the PMU is enabled.
2) msr-based-feature IA32_PERF_CAPABILITIES is supporterd on KVM.
3) Supported returned value for lbr_fmt from above msr is non-zero.
4) Guest vcpu model does support FEAT_1_ECX.CPUID_EXT_PDCM.
5) User-provided lbr-fmt value doesn't violate its bitmask (0x3f).
6) Target guest LBR format matches that of host.

---
 target/i386/cpu.c | 40 ++++++++++++++++++++++++++++++++++++++++
 target/i386/cpu.h | 10 ++++++++++
 2 files changed, 50 insertions(+)
target/i386/cpu.c +40 -0
index aa9e636800..55eb519214 100644
@@ -6280,6 +6280,7 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
     CPUX86State *env = &cpu->env;
     Error *local_err = NULL;
     static bool ht_warned;
+    uint64_t requested_lbr_fmt;
 
     if (cpu->apic_id == UNASSIGNED_APIC_ID) {
         error_setg(errp, "apic-id property was not initialized properly");
@@ -6297,6 +6298,42 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
         goto out;
     }
 
+    /*
+     * Override env->features[FEAT_PERF_CAPABILITIES].LBR_FMT
+     * with user-provided setting.
+     */
+    if (cpu->lbr_fmt != ~PERF_CAP_LBR_FMT) {
+        if ((cpu->lbr_fmt & PERF_CAP_LBR_FMT) != cpu->lbr_fmt) {
+            error_setg(errp, "invalid lbr-fmt");
+            return;
+        }
+        env->features[FEAT_PERF_CAPABILITIES] &= ~PERF_CAP_LBR_FMT;
+        env->features[FEAT_PERF_CAPABILITIES] |= cpu->lbr_fmt;
+    }
+
+    /*
+     * vPMU LBR is supported when 1) KVM is enabled 2) Option pmu=on and
+     * 3)vPMU LBR format matches that of host setting.
+     */
+    requested_lbr_fmt =
+        env->features[FEAT_PERF_CAPABILITIES] & PERF_CAP_LBR_FMT;
+    if (requested_lbr_fmt && kvm_enabled()) {
+        uint64_t host_perf_cap =
+            x86_cpu_get_supported_feature_word(FEAT_PERF_CAPABILITIES, false);
+        uint64_t host_lbr_fmt = host_perf_cap & PERF_CAP_LBR_FMT;
+
+        if (!cpu->enable_pmu) {
+            error_setg(errp, "vPMU: LBR is unsupported without pmu=on");
+            return;
+        }
+        if (requested_lbr_fmt != host_lbr_fmt) {
+            error_setg(errp, "vPMU: the lbr-fmt value (0x%lx) mismatches "
+                        "the host supported value (0x%lx).",
+                        requested_lbr_fmt, host_lbr_fmt);
+            return;
+        }
+    }
+
     x86_cpu_filter_features(cpu, cpu->check_cpuid || cpu->enforce_cpuid);
 
     if (cpu->enforce_cpuid && x86_cpu_have_filtered_features(cpu)) {
@@ -6649,6 +6686,8 @@ static void x86_cpu_initfn(Object *obj)
     object_property_add_alias(obj, "sse4_2", obj, "sse4.2");
 
     object_property_add_alias(obj, "hv-apicv", obj, "hv-avic");
+    cpu->lbr_fmt = ~PERF_CAP_LBR_FMT;
+    object_property_add_alias(obj, "lbr_fmt", obj, "lbr-fmt");
 
     if (xcc->model) {
         x86_cpu_load_model(cpu, xcc->model);
@@ -6804,6 +6843,7 @@ static Property x86_cpu_properties[] = {
 #endif
     DEFINE_PROP_INT32("node-id", X86CPU, node_id, CPU_UNSET_NUMA_NODE_ID),
     DEFINE_PROP_BOOL("pmu", X86CPU, enable_pmu, false),
+    DEFINE_PROP_UINT64_CHECKMASK("lbr-fmt", X86CPU, lbr_fmt, PERF_CAP_LBR_FMT),
 
     DEFINE_PROP_UINT32("hv-spinlocks", X86CPU, hyperv_spinlock_attempts,
                        HYPERV_SPINLOCK_NEVER_NOTIFY),
target/i386/cpu.h +10 -0
index 9911d7c871..ee8974abc9 100644
@@ -383,6 +383,7 @@ typedef enum X86Seg {
 #define ARCH_CAP_TSX_CTRL_MSR		(1<<7)
 
 #define MSR_IA32_PERF_CAPABILITIES      0x345
+#define PERF_CAP_LBR_FMT                0x3f
 
 #define MSR_IA32_TSX_CTRL		0x122
 #define MSR_IA32_TSCDEADLINE            0x6e0
@@ -1775,6 +1776,15 @@ struct X86CPU {
      */
     bool enable_pmu;
 
+    /*
+     * Enable LBR_FMT bits of IA32_PERF_CAPABILITIES MSR.
+     * This can't be initialized with a default because it doesn't have
+     * stable ABI support yet. It is only allowed to pass all LBR_FMT bits
+     * returned by kvm_arch_get_supported_msr_feature()(which depends on both
+     * host CPU and kernel capabilities) to the guest.
+     */
+    uint64_t lbr_fmt;
+
     /* LMCE support can be enabled/disabled via cpu option 'lmce=on/off'. It is
      * disabled by default to avoid breaking migration between QEMU with
      * different LMCE configurations.
Signature · 1 line
2.27.0
#3 lore raw
In reply to Yang Weijiang

Hi Weijiang,

Quoted 3 lines · Yang Weijiang

KVM legacy LBR patches have been merged in kernel 5.12, this patchset is to expose the feature to guest from the perf capability MSR. Qemu can add LBR format in cpu option to achieve it, e.g., -cpu host,lbr-fmt=0x5,

Some older Intel CPUs may have lbr-fmt=LBR_FORMAT_32 (which is 0), would you help verify that KVM is supported on these platforms ? If so, how do we enable guest LBR form the QEMU side, w/ -cpu host,lbr-fmt=0x0 ?

Quoted 4 lines

the format should match host value in IA32_PERF_CAPABILITIES.

Note, KVM legacy LBR solution accelerates guest perf performace by LBR MSR passthrough so it requires guest cpu model matches that of host's, i.e.,

Would you help add live migration support across host/guest CPU models when hosts at both ends have the same number of LBR entries and the same lbr-fmt ?

Thanks,
Like Xu

Quoted 5 lines

only -cpu host is supported.

Change in v5:
1. This patchset is rebased on tip : 6621441db5
2. No functional change since v4.

#4 lore raw
Quoted 10 lines · Like Xu

Hi Weijiang,

Quoted 3 lines · Yang Weijiang

KVM legacy LBR patches have been merged in kernel 5.12, this patchset is to expose the feature to guest from the perf capability MSR. Qemu can add LBR format in cpu option to achieve it, e.g., -cpu host,lbr-fmt=0x5,

Some older Intel CPUs may have lbr-fmt=LBR_FORMAT_32 (which is 0), would you help verify that KVM is supported on these platforms ? If so, how do we enable guest LBR form the QEMU side, w/ -cpu host,lbr-fmt=0x0 ?

Hi, Like, do you know which cpu model or platform so that I can have a test on?

Quoted 8 lines
Quoted 5 lines

the format should match host value in IA32_PERF_CAPABILITIES.

Note, KVM legacy LBR solution accelerates guest perf performace by LBR MSR passthrough so it requires guest cpu model matches that of host's, i.e.,

Would you help add live migration support across host/guest CPU models when hosts at both ends have the same number of LBR entries and the same lbr-fmt ?

Yes, I'm working on this part for Arch LBR, then enable it for legacy LBR as well.

Quoted 9 lines

Thanks,
Like Xu

Quoted 5 lines

only -cpu host is supported.

Change in v5:
    1. This patchset is rebased on tip : 6621441db5
    2. No functional change since v4.