Skip to contentSkip to conversation

arm64: Support perf event modifiers :G and :H

PATCH Superseded Andrew Murray 3 participants 10 messages Dec 10, 2018 – Dec 11, 2018

Subsystems (4)

v6 of 13 Superseded Latest is v13 →

lore
Share & tools
lore
#0 lore raw

v6 cover arm64: Support perf event modifiers :G and :H

This patchset provides support for perf event modifiers :G and :H which allows for filtering of PMU events between host and guests when used with KVM.

As the underlying hardware cannot distinguish between guest and host context, the performance counters must be stopped and started upon entry/exit to the guest. This is performed at EL2 in a way that minimizes overhead and improves accuracy of recording events that only occur in the requested context.

This has been tested with VHE and non-VHE kernels with a KVM guest.

Changes from v5:

- Tweak logic in use of kvm_set_pmu_events

Changes from v4:

- Prevent unnecessary write_sysreg calls by improving __pmu_switch_to_xxx logic.

Changes from v3:

- Remove confusing _only suffix from bitfields in kvm_cpu_context
- Remove unnecessary condition when clearing event bits in disable
- Simplify API of KVM accessors
- Prevent unnecessary setting of pmcnten when guest/host events are the same.

Changes from v2:

- Ensured that exclude_kernel works for guest
- Removed unnecessary exclusion of EL2 with exclude_host on !VHE
- Renamed kvm_clr_set_host_pmu_events to reflect args order
- Added additional information to isb patch

Changes from v1:

- Removed unnecessary exclusion of EL1 with exclude_guest on VHE
- Removed unnecessary isb from existing perf_event.c driver
- Folded perf_event.c patches together
- Added additional information to last patch commit message

Andrew Murray (4):
arm64: arm_pmu: remove unnecessary isb instruction
arm64: KVM: add accessors to track guest/host only counters
arm64: arm_pmu: Add support for exclude_host/exclude_guest attributes
arm64: KVM: Enable support for :G/:H perf event modifiers

 arch/arm64/include/asm/kvm_host.h | 24 ++++++++++++++++++
 arch/arm64/kernel/perf_event.c    | 52 +++++++++++++++++++++++++++++++++------
 arch/arm64/kvm/hyp/switch.c       | 38 ++++++++++++++++++++++++++++
 3 files changed, 106 insertions(+), 8 deletions(-)
Signature · 1 line
2.7.4
#1 lore raw

PATCH v6 1/4 arm64: arm_pmu: remove unnecessary isb instruction

The armv8pmu_enable_event_counter function issues an isb instruction after enabling a pair of counters - this doesn't provide any value and is inconsistent with the armv8pmu_disable_event_counter.

In any case armv8pmu_enable_event_counter is always called with the PMU stopped. Starting the PMU with armv8pmu_start results in an isb instruction being issued prior to writing to PMCR_EL0.

Let's remove the unnecessary isb instruction.

---
 arch/arm64/kernel/perf_event.c | 1 -
 1 file changed, 1 deletion(-)
arch/arm64/kernel/perf_event.c +0 -1
index 8e38d52..de564ae 100644
@@ -652,7 +652,6 @@ static inline void armv8pmu_enable_event_counter(struct perf_event *event)
 	armv8pmu_enable_counter(idx);
 	if (armv8pmu_event_is_chained(event))
 		armv8pmu_enable_counter(idx - 1);
-	isb();
 }
 
 static inline int armv8pmu_disable_counter(int idx)
Signature · 1 line
2.7.4
#2 lore raw

PATCH v6 2/4 arm64: KVM: add accessors to track guest/host only counters

In reply to Andrew Murray

In order to effeciently enable/disable guest/host only perf counters at guest entry/exit we add bitfields to kvm_cpu_context for guest and host events as well as accessors for updating them.

---
 arch/arm64/include/asm/kvm_host.h | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
arch/arm64/include/asm/kvm_host.h +24 -0
index 1550192..800c87b 100644
@@ -203,6 +203,8 @@ struct kvm_cpu_context {
 	};
 
 	struct kvm_vcpu *__hyp_running_vcpu;
+	u32 events_host;
+	u32 events_guest;
 };
 
 typedef struct kvm_cpu_context kvm_cpu_context_t;
@@ -467,11 +469,33 @@ void kvm_arch_vcpu_load_fp(struct kvm_vcpu *vcpu);
 void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu);
 void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu);
 
+#define KVM_PMU_EVENTS_HOST	1
+#define KVM_PMU_EVENTS_GUEST	2
+
 #ifdef CONFIG_KVM /* Avoid conflicts with core headers if CONFIG_KVM=n */
 static inline int kvm_arch_vcpu_run_pid_change(struct kvm_vcpu *vcpu)
 {
 	return kvm_arch_vcpu_run_map_fp(vcpu);
 }
+static inline void kvm_set_pmu_events(u32 set, int flags)
+{
+	kvm_cpu_context_t *ctx = this_cpu_ptr(&kvm_host_cpu_state);
+
+	if (flags & KVM_PMU_EVENTS_HOST)
+		ctx->events_host |= set;
+	if (flags & KVM_PMU_EVENTS_GUEST)
+		ctx->events_guest |= set;
+}
+static inline void kvm_clr_pmu_events(u32 clr)
+{
+	kvm_cpu_context_t *ctx = this_cpu_ptr(&kvm_host_cpu_state);
+
+	ctx->events_host &= ~clr;
+	ctx->events_guest &= ~clr;
+}
+#else
+static inline void kvm_set_pmu_events(u32 set, int flags) {}
+static inline void kvm_clr_pmu_events(u32 clr) {}
 #endif
 
 static inline void kvm_arm_vhe_guest_enter(void)
Signature · 1 line
2.7.4
#3 lore raw

Re: [PATCH v6 2/4] arm64: KVM: add accessors to track guest/host only counters

Quoted 14 lines · Andrew Murray

In order to effeciently enable/disable guest/host only perf counters at guest entry/exit we add bitfields to kvm_cpu_context for guest and host events as well as accessors for updating them.

---
 arch/arm64/include/asm/kvm_host.h | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
arch/arm64/include/asm/kvm_host.h +0 -0
index 1550192..800c87b 100644

arch/arm64/include/asm/kvm_host.h struct kvm_cpu_context { · line 207

Quoted 5 lines
arch/arm64/include/asm/kvm_host.h +2 -0
@@ -203,6 +203,8 @@ struct kvm_cpu_context {
 	};
 
 	struct kvm_vcpu *__hyp_running_vcpu;
+	u32 events_host;
+	u32 events_guest;

This is confusing to me.

These values appear only to be used for the host instance, which makes me wonder why we add them to kvm_cpu_context, which is also used for the guest state? Should we not instead consider moving them to their own data structure and add a per-cpu data structure or something more fancy like having a new data structure, kvm_percpu_host_data, which contains the kvm_cpu_context and the events flags?

I don't know much about perf, but doesn't this design also imply that you can only set these modifiers at a per-cpu level, and not attach the modifiers to a task/vcpu or vm ? Is that by design?

Thanks,

Christoffer

Quoted 40 lines
arch/arm64/include/asm/kvm_host.h +22 -0
 };
 
 typedef struct kvm_cpu_context kvm_cpu_context_t;
@@ -467,11 +469,33 @@ void kvm_arch_vcpu_load_fp(struct kvm_vcpu *vcpu);
 void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu);
 void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu);
 
+#define KVM_PMU_EVENTS_HOST	1
+#define KVM_PMU_EVENTS_GUEST	2
+
 #ifdef CONFIG_KVM /* Avoid conflicts with core headers if CONFIG_KVM=n */
 static inline int kvm_arch_vcpu_run_pid_change(struct kvm_vcpu *vcpu)
 {
 	return kvm_arch_vcpu_run_map_fp(vcpu);
 }
+static inline void kvm_set_pmu_events(u32 set, int flags)
+{
+	kvm_cpu_context_t *ctx = this_cpu_ptr(&kvm_host_cpu_state);
+
+	if (flags & KVM_PMU_EVENTS_HOST)
+		ctx->events_host |= set;
+	if (flags & KVM_PMU_EVENTS_GUEST)
+		ctx->events_guest |= set;
+}
+static inline void kvm_clr_pmu_events(u32 clr)
+{
+	kvm_cpu_context_t *ctx = this_cpu_ptr(&kvm_host_cpu_state);
+
+	ctx->events_host &= ~clr;
+	ctx->events_guest &= ~clr;
+}
+#else
+static inline void kvm_set_pmu_events(u32 set, int flags) {}
+static inline void kvm_clr_pmu_events(u32 clr) {}
 #endif
 
 static inline void kvm_arm_vhe_guest_enter(void)
Signature · 1 line
2.7.4
#4 lore raw

Re: [PATCH v6 2/4] arm64: KVM: add accessors to track guest/host only counters

Quoted 23 lines · Christoffer Dall
Quoted 14 lines · Andrew Murray

In order to effeciently enable/disable guest/host only perf counters at guest entry/exit we add bitfields to kvm_cpu_context for guest and host events as well as accessors for updating them.

---
 arch/arm64/include/asm/kvm_host.h | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
arch/arm64/include/asm/kvm_host.h +0 -0
index 1550192..800c87b 100644

arch/arm64/include/asm/kvm_host.h struct kvm_cpu_context { · line 207

Quoted 5 lines
arch/arm64/include/asm/kvm_host.h +2 -0
@@ -203,6 +203,8 @@ struct kvm_cpu_context {
 	};
 
 	struct kvm_vcpu *__hyp_running_vcpu;
+	u32 events_host;
+	u32 events_guest;

This is confusing to me.

Quoted 3 lines

These values appear only to be used for the host instance, which makes me wonder why we add them to kvm_cpu_context, which is also used for the guest state? Should we not instead consider moving them to their own

Indeed they are only used for the host instance (i.e. not arch.ctxt). I hadn't realised the structure was used in other contexts. So it isn't optimal to have additional fields that aren't always used.

Quoted 3 lines

data structure and add a per-cpu data structure or something more fancy like having a new data structure, kvm_percpu_host_data, which contains the kvm_cpu_context and the events flags?

Using a percpu for the guest/host events was an approach I took prior to sharing on the list - an additional hypervisor mapping is needed (such that the percpu can be accessed from the hyp switch code) and I assume there to be a very marginal additional amount of work resulting from it switching between host/guest.

I also considered using the unused ctxt.sys_regs[PMCNTENSET_EL0] register though this feels like a hack and would probably involve a system register read in the switch code to read the current PMU state.

I attempted the fancy approach (see below) - the struct and variable naming isn't ideal (virt/arm/arm.c defines kvm_cpu_context_t and of course is shared with arm32). Also I updated asm-offsets.c to reflect the now nested struct (for use with get_vcpu_ptr) - it's not strictly necessary but adds robustness.

What are your thoughts on the best approach?

Quoted 4 lines

I don't know much about perf, but doesn't this design also imply that you can only set these modifiers at a per-cpu level, and not attach the modifiers to a task/vcpu or vm ? Is that by design?

You can set these modifiers on a task and the perf core will take care of disabling the perf_event when the task is scheduled in/out.

Here it is..

arch/arm64/include/asm/kvm_host.h +14 -6
index 800c87b..1f4a78a 100644
@@ -203,11 +203,19 @@ struct kvm_cpu_context {
        };
 
        struct kvm_vcpu *__hyp_running_vcpu;
+};
+
+struct kvm_pmu_events {
        u32 events_host;
        u32 events_guest;
 };
 
-typedef struct kvm_cpu_context kvm_cpu_context_t;
+struct kvm_host_data {
+       struct kvm_cpu_context __kvm_cpu_state;
+       struct kvm_pmu_events pmu_events;
+};
+
+typedef struct kvm_host_data kvm_cpu_context_t;
 
 struct kvm_vcpu_arch {
        struct kvm_cpu_context ctxt;
@@ -243,7 +251,7 @@ struct kvm_vcpu_arch {
        struct kvm_guest_debug_arch external_debug_state;
 
        /* Pointer to host CPU context */
-       kvm_cpu_context_t *host_cpu_context;
+       struct kvm_cpu_context *host_cpu_context;
 
        struct thread_info *host_thread_info;   /* hyp VA */
        struct user_fpsimd_state *host_fpsimd_state;    /* hyp VA */
@@ -482,16 +490,16 @@ static inline void kvm_set_pmu_events(u32 set, int flags)
        kvm_cpu_context_t *ctx = this_cpu_ptr(&kvm_host_cpu_state);
 
        if (flags & KVM_PMU_EVENTS_HOST)
-               ctx->events_host |= set;
+               ctx->pmu_events.events_host |= set;
        if (flags & KVM_PMU_EVENTS_GUEST)
-               ctx->events_guest |= set;
+               ctx->pmu_events.events_guest |= set;
 }
 static inline void kvm_clr_pmu_events(u32 clr)
 {
        kvm_cpu_context_t *ctx = this_cpu_ptr(&kvm_host_cpu_state);
 
-       ctx->events_host &= ~clr;
-       ctx->events_guest &= ~clr;
+       ctx->pmu_events.events_host &= ~clr;
+       ctx->pmu_events.events_guest &= ~clr;
 }
 #else
 static inline void kvm_set_pmu_events(u32 set, int flags) {}
arch/arm64/kernel/asm-offsets.c +2 -1
index 323aeb5..da34022 100644
@@ -142,7 +142,8 @@ int main(void)
   DEFINE(CPU_FP_REGS,          offsetof(struct kvm_regs, fp_regs));
   DEFINE(VCPU_FPEXC32_EL2,     offsetof(struct kvm_vcpu, arch.ctxt.sys_regs[FPEXC32_EL2]));
   DEFINE(VCPU_HOST_CONTEXT,    offsetof(struct kvm_vcpu, arch.host_cpu_context));
-  DEFINE(HOST_CONTEXT_VCPU,    offsetof(struct kvm_cpu_context, __hyp_running_vcpu));
+  DEFINE(HOST_CONTEXT_VCPU,    offsetof(struct kvm_cpu_context, __hyp_running_vcpu)
+                               + offsetof(struct kvm_host_data, __kvm_cpu_state));
 #endif
 #ifdef CONFIG_CPU_PM
   DEFINE(CPU_SUSPEND_SZ,       sizeof(struct cpu_suspend_ctx));
arch/arm64/kvm/hyp/switch.c +13 -4
index e505cad..5f0d571 100644
@@ -375,8 +375,14 @@ static bool __hyp_text __hyp_switch_fpsimd(struct kvm_vcpu *vcpu)
 
 static bool __hyp_text __pmu_switch_to_guest(struct kvm_cpu_context *host_ctxt)
 {
-       u32 clr = host_ctxt->events_host & ~host_ctxt->events_guest;
-       u32 set = host_ctxt->events_guest & ~host_ctxt->events_host;
+       struct kvm_host_data *host;
+       struct kvm_pmu_events *events;
+
+       host = container_of(host_ctxt, struct kvm_host_data, __kvm_cpu_state);
+       pmu = &host_data->pmu_events;
+
+       u32 clr = pmu->events_host & ~pmu->events_guest;
+       u32 set = pmu->events_guest & ~pmu->events_host;
 
        if (clr)
                write_sysreg(clr, pmcntenclr_el0);
@@ -389,8 +395,11 @@ static bool __hyp_text __pmu_switch_to_guest(struct kvm_cpu_context *host_ctxt)
 
 static void __hyp_text __pmu_switch_to_host(struct kvm_cpu_context *host_ctxt)
 {
-       u32 clr = host_ctxt->events_guest & ~host_ctxt->events_host;
-       u32 set = host_ctxt->events_host & ~host_ctxt->events_guest;
+       struct kvm_host_data *host = container_of(host_ctxt, struct kvm_host_data, __kvm_cpu_state);
+       struct kvm_pmu_events *pmu = &host->pmu_events;
+
+       u32 clr = pmu->events_guest & ~pmu->events_host;
+       u32 set = pmu->events_host & ~pmu->events_guest;
 
        if (clr)
                write_sysreg(clr, pmcntenclr_el0);
virt/kvm/arm/arm.c +1 -1
index 150c8a6..6ba7de1 100644
@@ -374,7 +374,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
        }
 
        vcpu->cpu = cpu;
-       vcpu->arch.host_cpu_context = this_cpu_ptr(&kvm_host_cpu_state);
+       vcpu->arch.host_cpu_context = &(this_cpu_ptr(&kvm_host_cpu_state)->__kvm_cpu_state);
 
        kvm_arm_set_running_vcpu(vcpu);
        kvm_vgic_load(vcpu);

Thanks,

Andrew Murray

Quoted 46 lines

Thanks,

Christoffer

Quoted 40 lines
arch/arm64/include/asm/kvm_host.h +22 -0
 };
 
 typedef struct kvm_cpu_context kvm_cpu_context_t;
@@ -467,11 +469,33 @@ void kvm_arch_vcpu_load_fp(struct kvm_vcpu *vcpu);
 void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu);
 void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu);
 
+#define KVM_PMU_EVENTS_HOST	1
+#define KVM_PMU_EVENTS_GUEST	2
+
 #ifdef CONFIG_KVM /* Avoid conflicts with core headers if CONFIG_KVM=n */
 static inline int kvm_arch_vcpu_run_pid_change(struct kvm_vcpu *vcpu)
 {
 	return kvm_arch_vcpu_run_map_fp(vcpu);
 }
+static inline void kvm_set_pmu_events(u32 set, int flags)
+{
+	kvm_cpu_context_t *ctx = this_cpu_ptr(&kvm_host_cpu_state);
+
+	if (flags & KVM_PMU_EVENTS_HOST)
+		ctx->events_host |= set;
+	if (flags & KVM_PMU_EVENTS_GUEST)
+		ctx->events_guest |= set;
+}
+static inline void kvm_clr_pmu_events(u32 clr)
+{
+	kvm_cpu_context_t *ctx = this_cpu_ptr(&kvm_host_cpu_state);
+
+	ctx->events_host &= ~clr;
+	ctx->events_guest &= ~clr;
+}
+#else
+static inline void kvm_set_pmu_events(u32 set, int flags) {}
+static inline void kvm_clr_pmu_events(u32 clr) {}
 #endif
 
 static inline void kvm_arm_vhe_guest_enter(void)
Signature · 1 line
2.7.4
#5 lore raw

Re: [PATCH v6 2/4] arm64: KVM: add accessors to track guest/host only counters

Quoted 43 lines · Andrew Murray
Quoted 23 lines · Christoffer Dall
Quoted 14 lines · Andrew Murray

In order to effeciently enable/disable guest/host only perf counters at guest entry/exit we add bitfields to kvm_cpu_context for guest and host events as well as accessors for updating them.

---
 arch/arm64/include/asm/kvm_host.h | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
arch/arm64/include/asm/kvm_host.h +0 -0
index 1550192..800c87b 100644

arch/arm64/include/asm/kvm_host.h struct kvm_cpu_context { · line 207

Quoted 5 lines
arch/arm64/include/asm/kvm_host.h +2 -0
@@ -203,6 +203,8 @@ struct kvm_cpu_context {
 	};
 
 	struct kvm_vcpu *__hyp_running_vcpu;
+	u32 events_host;
+	u32 events_guest;

This is confusing to me.

Quoted 3 lines

These values appear only to be used for the host instance, which makes me wonder why we add them to kvm_cpu_context, which is also used for the guest state? Should we not instead consider moving them to their own

Indeed they are only used for the host instance (i.e. not arch.ctxt). I hadn't realised the structure was used in other contexts. So it isn't optimal to have additional fields that aren't always used.

Quoted 3 lines

data structure and add a per-cpu data structure or something more fancy like having a new data structure, kvm_percpu_host_data, which contains the kvm_cpu_context and the events flags?

Using a percpu for the guest/host events was an approach I took prior to sharing on the list - an additional hypervisor mapping is needed (such that the percpu can be accessed from the hyp switch code) and I assume there to be a very marginal additional amount of work resulting from it switching between host/guest.

Quoted 3 lines

I also considered using the unused ctxt.sys_regs[PMCNTENSET_EL0] register though this feels like a hack and would probably involve a system register read in the switch code to read the current PMU state.

Yeah, that doesn't sound overly appealing.

Quoted 8 lines

I attempted the fancy approach (see below) - the struct and variable naming isn't ideal (virt/arm/arm.c defines kvm_cpu_context_t and of course is shared with arm32). Also I updated asm-offsets.c to reflect the now nested struct (for use with get_vcpu_ptr) - it's not strictly necessary but adds robustness.

What are your thoughts on the best approach?

The naming and typedef is the biggest issue with the patch below, but that can be easily fixed.

The fact that you keep the context pointer on the vcpu structure makes this much less painful that it could have been, so I think this is acceptable.

I can also live with mapping the additional data structure if necessary.

Quoted 8 lines
Quoted 5 lines

I don't know much about perf, but doesn't this design also imply that you can only set these modifiers at a per-cpu level, and not attach the modifiers to a task/vcpu or vm ? Is that by design?

You can set these modifiers on a task and the perf core will take care of disabling the perf_event when the task is scheduled in/out.

I now remember how this works, thanks for the nudge in the right direction.

Quoted 15 lines
arch/arm64/include/asm/kvm_host.h +3 -0
index 800c87b..1f4a78a 100644
@@ -203,11 +203,19 @@ struct kvm_cpu_context {
        };
 
        struct kvm_vcpu *__hyp_running_vcpu;
+};
+
+struct kvm_pmu_events {
        u32 events_host;
        u32 events_guest;

arch/arm64/include/asm/kvm_host.h struct kvm_cpu_context { · line 219

Quoted 10 lines
arch/arm64/include/asm/kvm_host.h +6 -1
 };
 
-typedef struct kvm_cpu_context kvm_cpu_context_t;
+struct kvm_host_data {
+       struct kvm_cpu_context __kvm_cpu_state;
+       struct kvm_pmu_events pmu_events;
+};
+
+typedef struct kvm_host_data kvm_cpu_context_t;
 

This look really strange. I think we can just keep the old typedef, and if you like you can introduce a kvm_host_data_t...

Quoted 12 lines
arch/arm64/include/asm/kvm_host.h +1 -1
 struct kvm_vcpu_arch {
        struct kvm_cpu_context ctxt;
@@ -243,7 +251,7 @@ struct kvm_vcpu_arch {
        struct kvm_guest_debug_arch external_debug_state;
 
        /* Pointer to host CPU context */
-       kvm_cpu_context_t *host_cpu_context;
+       struct kvm_cpu_context *host_cpu_context;
 
        struct thread_info *host_thread_info;   /* hyp VA */
        struct user_fpsimd_state *host_fpsimd_state;    /* hyp VA */

arch/arm64/include/asm/kvm_host.h static inline void kvm_set_pmu_events(u32 set, int flags) · line 495

Quoted 8 lines
arch/arm64/include/asm/kvm_host.h +2 -2
@@ -482,16 +490,16 @@ static inline void kvm_set_pmu_events(u32 set, int flags)
        kvm_cpu_context_t *ctx = this_cpu_ptr(&kvm_host_cpu_state);
 
        if (flags & KVM_PMU_EVENTS_HOST)
-               ctx->events_host |= set;
+               ctx->pmu_events.events_host |= set;
        if (flags & KVM_PMU_EVENTS_GUEST)
-               ctx->events_guest |= set;
+               ctx->pmu_events.events_guest |= set;

...which would make it easier to understand what is going on here.

Quoted 67 lines
arch/arm64/include/asm/kvm_host.h +2 -2
 }
 static inline void kvm_clr_pmu_events(u32 clr)
 {
        kvm_cpu_context_t *ctx = this_cpu_ptr(&kvm_host_cpu_state);
 
-       ctx->events_host &= ~clr;
-       ctx->events_guest &= ~clr;
+       ctx->pmu_events.events_host &= ~clr;
+       ctx->pmu_events.events_guest &= ~clr;
 }
 #else
 static inline void kvm_set_pmu_events(u32 set, int flags) {}
arch/arm64/kernel/asm-offsets.c +2 -1
index 323aeb5..da34022 100644
@@ -142,7 +142,8 @@ int main(void)
   DEFINE(CPU_FP_REGS,          offsetof(struct kvm_regs, fp_regs));
   DEFINE(VCPU_FPEXC32_EL2,     offsetof(struct kvm_vcpu, arch.ctxt.sys_regs[FPEXC32_EL2]));
   DEFINE(VCPU_HOST_CONTEXT,    offsetof(struct kvm_vcpu, arch.host_cpu_context));
-  DEFINE(HOST_CONTEXT_VCPU,    offsetof(struct kvm_cpu_context, __hyp_running_vcpu));
+  DEFINE(HOST_CONTEXT_VCPU,    offsetof(struct kvm_cpu_context, __hyp_running_vcpu)
+                               + offsetof(struct kvm_host_data, __kvm_cpu_state));
 #endif
 #ifdef CONFIG_CPU_PM
   DEFINE(CPU_SUSPEND_SZ,       sizeof(struct cpu_suspend_ctx));
arch/arm64/kvm/hyp/switch.c +13 -4
index e505cad..5f0d571 100644
@@ -375,8 +375,14 @@ static bool __hyp_text __hyp_switch_fpsimd(struct kvm_vcpu *vcpu)
 
 static bool __hyp_text __pmu_switch_to_guest(struct kvm_cpu_context *host_ctxt)
 {
-       u32 clr = host_ctxt->events_host & ~host_ctxt->events_guest;
-       u32 set = host_ctxt->events_guest & ~host_ctxt->events_host;
+       struct kvm_host_data *host;
+       struct kvm_pmu_events *events;
+
+       host = container_of(host_ctxt, struct kvm_host_data, __kvm_cpu_state);
+       pmu = &host_data->pmu_events;
+
+       u32 clr = pmu->events_host & ~pmu->events_guest;
+       u32 set = pmu->events_guest & ~pmu->events_host;
 
        if (clr)
                write_sysreg(clr, pmcntenclr_el0);
@@ -389,8 +395,11 @@ static bool __hyp_text __pmu_switch_to_guest(struct kvm_cpu_context *host_ctxt)
 
 static void __hyp_text __pmu_switch_to_host(struct kvm_cpu_context *host_ctxt)
 {
-       u32 clr = host_ctxt->events_guest & ~host_ctxt->events_host;
-       u32 set = host_ctxt->events_host & ~host_ctxt->events_guest;
+       struct kvm_host_data *host = container_of(host_ctxt, struct kvm_host_data, __kvm_cpu_state);
+       struct kvm_pmu_events *pmu = &host->pmu_events;
+
+       u32 clr = pmu->events_guest & ~pmu->events_host;
+       u32 set = pmu->events_host & ~pmu->events_guest;
 
        if (clr)
                write_sysreg(clr, pmcntenclr_el0);
virt/kvm/arm/arm.c +0 -0
index 150c8a6..6ba7de1 100644

virt/kvm/arm/arm.c void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) · line 380

Quoted 8 lines
virt/kvm/arm/arm.c +1 -1
@@ -374,7 +374,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
        }
 
        vcpu->cpu = cpu;
-       vcpu->arch.host_cpu_context = this_cpu_ptr(&kvm_host_cpu_state);
+       vcpu->arch.host_cpu_context = &(this_cpu_ptr(&kvm_host_cpu_state)->__kvm_cpu_state);
 
        kvm_arm_set_running_vcpu(vcpu);
        kvm_vgic_load(vcpu);

Thanks,

Christoffer

#6 lore raw

PATCH v6 3/4 arm64: arm_pmu: Add support for exclude_host/exclude_guest attributes

In reply to Andrew Murray

Add support for the :G and :H attributes in perf by handling the exclude_host/exclude_guest event attributes.

We notify KVM of counters that we wish to be enabled or disabled on guest entry/exit and thus defer from starting or stopping :G events as per the events exclude_host attribute.

With both VHE and non-VHE we switch the counters between host/guest at EL2. We are able to eliminate counters counting host events on the boundaries of guest entry/exit when using :G by filtering out EL2 for exclude_host. However when using :H unless exclude_hv is set on non-VHE then there is a small blackout window at the guest entry/exit where host events are not captured.

---
 arch/arm64/kernel/perf_event.c | 51 ++++++++++++++++++++++++++++++++++++------
 1 file changed, 44 insertions(+), 7 deletions(-)
arch/arm64/kernel/perf_event.c +44 -7
index de564ae..4a3c73d 100644
@@ -26,6 +26,7 @@
 
 #include <linux/acpi.h>
 #include <linux/clocksource.h>
+#include <linux/kvm_host.h>
 #include <linux/of.h>
 #include <linux/perf/arm_pmu.h>
 #include <linux/platform_device.h>
@@ -647,11 +648,26 @@ static inline int armv8pmu_enable_counter(int idx)
 
 static inline void armv8pmu_enable_event_counter(struct perf_event *event)
 {
+	struct perf_event_attr *attr = &event->attr;
 	int idx = event->hw.idx;
+	int flags = 0;
+	u32 counter_bits = BIT(ARMV8_IDX_TO_COUNTER(idx));
 
-	armv8pmu_enable_counter(idx);
 	if (armv8pmu_event_is_chained(event))
-		armv8pmu_enable_counter(idx - 1);
+		counter_bits |= BIT(ARMV8_IDX_TO_COUNTER(idx - 1));
+
+	if (!attr->exclude_host)
+		flags |= KVM_PMU_EVENTS_HOST;
+	if (!attr->exclude_guest)
+		flags |= KVM_PMU_EVENTS_GUEST;
+
+	kvm_set_pmu_events(counter_bits, flags);
+
+	if (!attr->exclude_host) {
+		armv8pmu_enable_counter(idx);
+		if (armv8pmu_event_is_chained(event))
+			armv8pmu_enable_counter(idx - 1);
+	}
 }
 
 static inline int armv8pmu_disable_counter(int idx)
@@ -664,11 +680,20 @@ static inline int armv8pmu_disable_counter(int idx)
 static inline void armv8pmu_disable_event_counter(struct perf_event *event)
 {
 	struct hw_perf_event *hwc = &event->hw;
+	struct perf_event_attr *attr = &event->attr;
 	int idx = hwc->idx;
+	u32 counter_bits = BIT(ARMV8_IDX_TO_COUNTER(idx));
 
 	if (armv8pmu_event_is_chained(event))
-		armv8pmu_disable_counter(idx - 1);
-	armv8pmu_disable_counter(idx);
+		counter_bits |= BIT(ARMV8_IDX_TO_COUNTER(idx - 1));
+
+	kvm_clr_pmu_events(counter_bits);
+
+	if (!attr->exclude_host) {
+		if (armv8pmu_event_is_chained(event))
+			armv8pmu_disable_counter(idx - 1);
+		armv8pmu_disable_counter(idx);
+	}
 }
 
 static inline int armv8pmu_enable_intens(int idx)
@@ -943,16 +968,25 @@ static int armv8pmu_set_event_filter(struct hw_perf_event *event,
 	 * Therefore we ignore exclude_hv in this configuration, since
 	 * there's no hypervisor to sample anyway. This is consistent
 	 * with other architectures (x86 and Power).
+	 *
+	 * To eliminate counting host events on the boundaries of
+	 * guest entry/exit we ensure EL2 is not included in hyp mode
+	 * with !exclude_host.
 	 */
 	if (is_kernel_in_hyp_mode()) {
-		if (!attr->exclude_kernel)
+		if (!attr->exclude_kernel && !attr->exclude_host)
 			config_base |= ARMV8_PMU_INCLUDE_EL2;
 	} else {
-		if (attr->exclude_kernel)
-			config_base |= ARMV8_PMU_EXCLUDE_EL1;
 		if (!attr->exclude_hv)
 			config_base |= ARMV8_PMU_INCLUDE_EL2;
 	}
+
+	/*
+	 * Filter out !VHE kernels and guest kernels
+	 */
+	if (attr->exclude_kernel)
+		config_base |= ARMV8_PMU_EXCLUDE_EL1;
+
 	if (attr->exclude_user)
 		config_base |= ARMV8_PMU_EXCLUDE_EL0;
 
@@ -976,6 +1010,9 @@ static void armv8pmu_reset(void *info)
 		armv8pmu_disable_intens(idx);
 	}
 
+	/* Clear the counters we flip at guest entry/exit */
+	kvm_clr_pmu_events(U32_MAX);
+
 	/*
 	 * Initialize & Reset PMNC. Request overflow interrupt for
 	 * 64 bit cycle counter but cheat in armv8pmu_write_counter().
Signature · 1 line
2.7.4
#7 lore raw

PATCH v6 4/4 arm64: KVM: Enable support for :G/:H perf event modifiers

In reply to Andrew Murray

Enable/disable event counters as appropriate when entering and exiting the guest to enable support for guest or host only event counting.

For both VHE and non-VHE we switch the counters between host/guest at EL2. EL2 is filtered out by the PMU when we are using the :G modifier.

The PMU may be on when we change which counters are enabled however we avoid adding an isb as we instead rely on existing context synchronisation events: the isb in kvm_arm_vhe_guest_exit for VHE and the eret from the hvc in kvm_call_hyp.

---
 arch/arm64/kvm/hyp/switch.c | 38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)
arch/arm64/kvm/hyp/switch.c +38 -0
index d496ef5..e505cad 100644
@@ -373,6 +373,32 @@ static bool __hyp_text __hyp_switch_fpsimd(struct kvm_vcpu *vcpu)
 	return true;
 }
 
+static bool __hyp_text __pmu_switch_to_guest(struct kvm_cpu_context *host_ctxt)
+{
+	u32 clr = host_ctxt->events_host & ~host_ctxt->events_guest;
+	u32 set = host_ctxt->events_guest & ~host_ctxt->events_host;
+
+	if (clr)
+		write_sysreg(clr, pmcntenclr_el0);
+
+	if (set)
+		write_sysreg(set, pmcntenset_el0);
+
+	return (clr || set);
+}
+
+static void __hyp_text __pmu_switch_to_host(struct kvm_cpu_context *host_ctxt)
+{
+	u32 clr = host_ctxt->events_guest & ~host_ctxt->events_host;
+	u32 set = host_ctxt->events_host & ~host_ctxt->events_guest;
+
+	if (clr)
+		write_sysreg(clr, pmcntenclr_el0);
+
+	if (set)
+		write_sysreg(set, pmcntenset_el0);
+}
+
 /*
  * Return true when we were able to fixup the guest exit and should return to
  * the guest, false when we should restore the host state and return to the
@@ -488,12 +514,15 @@ int kvm_vcpu_run_vhe(struct kvm_vcpu *vcpu)
 {
 	struct kvm_cpu_context *host_ctxt;
 	struct kvm_cpu_context *guest_ctxt;
+	bool pmu_switch_needed;
 	u64 exit_code;
 
 	host_ctxt = vcpu->arch.host_cpu_context;
 	host_ctxt->__hyp_running_vcpu = vcpu;
 	guest_ctxt = &vcpu->arch.ctxt;
 
+	pmu_switch_needed = __pmu_switch_to_guest(host_ctxt);
+
 	sysreg_save_host_state_vhe(host_ctxt);
 
 	__activate_traps(vcpu);
@@ -524,6 +553,9 @@ int kvm_vcpu_run_vhe(struct kvm_vcpu *vcpu)
 
 	__debug_switch_to_host(vcpu);
 
+	if (pmu_switch_needed)
+		__pmu_switch_to_host(host_ctxt);
+
 	return exit_code;
 }
 
@@ -532,6 +564,7 @@ int __hyp_text __kvm_vcpu_run_nvhe(struct kvm_vcpu *vcpu)
 {
 	struct kvm_cpu_context *host_ctxt;
 	struct kvm_cpu_context *guest_ctxt;
+	bool pmu_switch_needed;
 	u64 exit_code;
 
 	vcpu = kern_hyp_va(vcpu);
@@ -540,6 +573,8 @@ int __hyp_text __kvm_vcpu_run_nvhe(struct kvm_vcpu *vcpu)
 	host_ctxt->__hyp_running_vcpu = vcpu;
 	guest_ctxt = &vcpu->arch.ctxt;
 
+	pmu_switch_needed = __pmu_switch_to_guest(host_ctxt);
+
 	__sysreg_save_state_nvhe(host_ctxt);
 
 	__activate_traps(vcpu);
@@ -586,6 +621,9 @@ int __hyp_text __kvm_vcpu_run_nvhe(struct kvm_vcpu *vcpu)
 	 */
 	__debug_switch_to_host(vcpu);
 
+	if (pmu_switch_needed)
+		__pmu_switch_to_host(host_ctxt);
+
 	return exit_code;
 }
 
Signature · 1 line
2.7.4
#8 lore raw
In reply to Andrew Murray

Hi Andrew,

Quoted 10 lines · Andrew Murray

This patchset provides support for perf event modifiers :G and :H which allows for filtering of PMU events between host and guests when used with KVM.

As the underlying hardware cannot distinguish between guest and host context, the performance counters must be stopped and started upon entry/exit to the guest. This is performed at EL2 in a way that minimizes overhead and improves accuracy of recording events that only occur in the requested context.

Quoted 5 lines

This has been tested with VHE and non-VHE kernels with a KVM guest.

Changes from v5:

- Tweak logic in use of kvm_set_pmu_events

This looks good to me, but I'll need Acks from the KVM folks on patches 2 and 4 before I can take it (and Christoffer has some questions on patch 2 as well).

Also, please be aware that I'm planning to close the arm64 tree later this week so we don't have to deal with late issues going into the holidays. Last Christmas was ruined by specdown and meltre, so I think many of us are due a proper break this year :)

Cheers,

Will

#9 lore raw
Quoted 19 lines · Will Deacon

Hi Andrew,

Quoted 10 lines · Andrew Murray

This patchset provides support for perf event modifiers :G and :H which allows for filtering of PMU events between host and guests when used with KVM.

As the underlying hardware cannot distinguish between guest and host context, the performance counters must be stopped and started upon entry/exit to the guest. This is performed at EL2 in a way that minimizes overhead and improves accuracy of recording events that only occur in the requested context.

Quoted 5 lines

This has been tested with VHE and non-VHE kernels with a KVM guest.

Changes from v5:

- Tweak logic in use of kvm_set_pmu_events

Quoted 8 lines

This looks good to me, but I'll need Acks from the KVM folks on patches 2 and 4 before I can take it (and Christoffer has some questions on patch 2 as well).

Also, please be aware that I'm planning to close the arm64 tree later this week so we don't have to deal with late issues going into the holidays. Last Christmas was ruined by specdown and meltre, so I think many of us are due a proper break this year :)

Thanks for the heads up and sounding completely reasonable to me.

Thanks,

Andrew Murray

Quoted 4 lines

Cheers,

Will