arm/kvm: Enable support for KVM_ARM_VCPU_PMU_V3_FILTER
PATCH v3 1/1 arm/kvm: Enable support for KVM_ARM_VCPU_PMU_V3_FILTER
The KVM_ARM_VCPU_PMU_V3_FILTER provide the ability to let the VMM decide which PMU events are provided to the guest. Add a new option pmu-filter as -accel sub-option to set the PMU Event Filtering. Without the filter, the KVM will expose all events from the host to guest by default.
The pmu-filter has such format:
pmu-filter="{A,D}:start-end[;{A,D}:start-end...]"
The A means "allow" and D means "deny", start is the first event of the range and the end is the last one. The first registered range defines the global policy(global ALLOW if the first @action is DENY, global DENY if the first @action is ALLOW). The start and end only support hex format now. For example:
pmu-filter="A:0x11-0x11;A:0x23-0x3a;D:0x30-0x30"
Since the first action is allow, we have a global deny policy. It will allow event 0x11 (The cycle counter), events 0x23 to 0x3a is also allowed except the event 0x30 is denied, and all the other events are disallowed.
Here is an real example shows how to use the PMU Event Filtering, when we launch a guest by use kvm, add such command line:
# qemu-system-aarch64 \ -accel kvm,pmu-filter="D:0x11-0x11"
Since the first action is deny, we have a global allow policy. This disables the filtering of the cycle counter (event 0x11 being CPU_CYCLES).
And then in guest, use the perf to count the cycle:
# perf stat sleep 1
Performance counter stats for 'sleep 1':
1.22 msec task-clock # 0.001 CPUs utilized
1 context-switches # 820.695 /sec
0 cpu-migrations # 0.000 /sec
55 page-faults # 45.138 K/sec
<not supported> cycles
1128954 instructions
227031 branches # 186.323 M/sec
8686 branch-misses # 3.83% of all branches 1.002492480 seconds time elapsed
0.001752000 seconds user
0.000000000 seconds sysAs we can see, the cycle counter has been disabled in the guest, but other pmu events are still work.
---v2->v3:
- Improve commits message, use kernel doc wording, add more explaination on
filter example, fix some typo error. [Eric]
- Add g_free() in kvm_arch_set_pmu_filter() to prevent memory leak. [Eric]
- Add more precise error message report. [Eric]
- In options doc, add pmu-filter rely on KVM_ARM_VCPU_PMU_V3_FILTER support in
KVM. [Eric]
v1->v2:
- Add more description for allow and deny meaning in
commit message. [Sebastian]
- Small improvement. [Sebastian]
v2: https://lore.kernel.org/all/[email protected]/
v1: https://lore.kernel.org/all/[email protected]/
--- include/sysemu/kvm_int.h | 1 + qemu-options.hx | 21 +++++++++++++ target/arm/kvm.c | 23 ++++++++++++++ target/arm/kvm64.c | 68 ++++++++++++++++++++++++++++++++++++++++ 4 files changed, 113 insertions(+)
include/sysemu/kvm_int.h +1 -0
@@ -120,6 +120,7 @@ struct KVMStateuint32_t xen_caps;uint16_t xen_gnttab_max_frames;uint16_t xen_evtchn_max_pirq; +char *kvm_pmu_filter; }; void kvm_memory_listener_register(KVMState *s, KVMMemoryListener *kml,
qemu-options.hx +21 -0
@@ -187,6 +187,7 @@ DEF("accel", HAS_ARG, QEMU_OPTION_accel, " tb-size=n (TCG translation block cache size)\n" " dirty-ring-size=n (KVM dirty ring GFN count, default 0)\n" " eager-split-size=n (KVM Eager Page Split chunk size, default 0, disabled. ARM only)\n" + " pmu-filter={A,D}:start-end[;...] (KVM PMU Event Filter, default no filter. ARM only)\n" " notify-vmexit=run|internal-error|disable,notify-window=n (enable notify VM exit and set notify window, x86 only)\n" " thread=single|multi (enable multi-threaded TCG)\n", QEMU_ARCH_ALL) SRST
@@ -259,6 +260,26 @@ SRST impact on the memory. By default, this feature is disabled (eager-split-size=0). + ``pmu-filter={A,D}:start-end[;...]`` + KVM implements pmu event filtering to prevent a guest from being able to + sample certain events. It depends on the KVM_ARM_VCPU_PMU_V3_FILTER attr + supported in KVM. It has the following format: + + pmu-filter="{A,D}:start-end[;{A,D}:start-end...]" + + The A means "allow" and D means "deny", start is the first event of the + range and the end is the last one. The first registered range defines + the global policy(global ALLOW if the first @action is DENY, global DENY + if the first @action is ALLOW). The start and end only support hex + format now. For example: + + pmu-filter="A:0x11-0x11;A:0x23-0x3a;D:0x30-0x30" + + Since the first action is allow, we have a global deny policy. It + will allow event 0x11 (The cycle counter), events 0x23 to 0x3a is + also allowed except the event 0x30 is denied, and all the other events + are disallowed. + ``notify-vmexit=run|internal-error|disable,notify-window=n`` Enables or disables notify VM exit support on x86 host and specify the corresponding notify window to trigger the VM exit if enabled.
target/arm/kvm.c +23 -0
@@ -1108,6 +1108,22 @@ static void kvm_arch_set_eager_split_size(Object *obj, Visitor *v, s->kvm_eager_split_size = value; } +static char *kvm_arch_get_pmu_filter (Object *obj,Error **errp) +{ +KVMState *s =KVM_STATE (obj); + +return g_strdup (s->kvm_pmu_filter); +} + +static void kvm_arch_set_pmu_filter (Object *obj,const char *pmu_filter, +Error **errp) +{ +KVMState *s =KVM_STATE (obj); + +g_free (s->kvm_pmu_filter); + s->kvm_pmu_filter =g_strdup (pmu_filter); +} +void kvm_arch_accel_class_init (ObjectClass *oc) { object_class_property_add(oc, "eager-split-size", "size",
@@ -1116,4 +1132,11 @@ void kvm_arch_accel_class_init(ObjectClass *oc)object_class_property_set_description (oc,"eager-split-size" ,"Eager Page Split chunk size for hugepages. (default: 0, disabled)" ); + +object_class_property_add_str (oc,"pmu-filter" , + kvm_arch_get_pmu_filter, + kvm_arch_set_pmu_filter); + +object_class_property_set_description (oc,"pmu-filter" , +"PMU Event Filtering description for guest pmu. (default: NULL, disabled)" ); }
target/arm/kvm64.c +68 -0
86 lines not on this page · raw message
Signature · 1 line
2.40.1
Quoted 14 lines · Shaoqin Huang
target/arm/kvm64.c +14
-0+static void kvm_arm_pmu_filter_init (CPUState *cs) +{ +static bool pmu_filter_init = false; +struct kvm_pmu_event_filter filter; +struct kvm_device_attr attr = { + .group =KVM_ARM_VCPU_PMU_V3_CTRL , + .attr =KVM_ARM_VCPU_PMU_V3_FILTER , + .addr = (uint64_t )&filter, + }; +KVMState *kvm_state = cs->kvm_state; +char *tmp; +char *str, act; + +if (!kvm_state->kvm_pmu_filter)
target/arm/kvm64.c static bool kvm_arm_set_device_attr(CPUState *cs, struct kvm_device_attr *attr, · line 158
Quoted 10 lines
target/arm/kvm64.c +10
-0+return ; + +if (kvm_vcpu_ioctl (cs,KVM_HAS_DEVICE_ATTR , attr)) { +error_report ("The kernel doesn't support the pmu event filter!\n" ); +abort (); + } + +/* The filter only needs to be initialized for 1 vcpu. */ +if (!pmu_filter_init) + pmu_filter_init = true;
Imho this is missing an else to bail out. Or the shorter version
if (pmu_filter_init) return; pmu_filter_init = true;
which could also move above the other tests.
Sebastian
Quoted 29 lines · Sebastian Ott
Quoted 18 lines · Shaoqin Huang
target/arm/kvm64.c +18
-0+static void kvm_arm_pmu_filter_init (CPUState *cs) +{ + static bool pmu_filter_init = false; + struct kvm_pmu_event_filter filter; + struct kvm_device_attr attr = { + .group = KVM_ARM_VCPU_PMU_V3_CTRL, + .attr = KVM_ARM_VCPU_PMU_V3_FILTER, + .addr = (uint64_t)&filter, + }; + KVMState *kvm_state = cs->kvm_state; + char *tmp; + char *str, act; + + if (!kvm_state->kvm_pmu_filter) + return; + + if (kvm_vcpu_ioctl(cs, KVM_HAS_DEVICE_ATTR, attr)) { + error_report("The kernel doesn't support the pmu eventQuoted 7 lines
filter!\n");
+abort();
+ }
+
+ /* The filter only needs to be initialized for 1 vcpu. */ + if (!pmu_filter_init)
+pmu_filter_init= true;Imho this is missing an else to bail out. Or the shorter version
Quoted 5 lines
if (
pmu_filter_init)
return;pmu_filter_init = true;
Yes. This is what I want to do. Thanks for fixing it.
Quoted 4 lines
which could also move above the other tests.
Sebastian
Signature · 1 line
Shaoqin
Hi Shaoqin,
Quoted 96 lines · Shaoqin Huang
The
KVM_ARM_VCPU_PMU_V3_FILTERprovide the ability to let the VMM decide which PMU events are provided to the guest. Add a new optionpmu-filteras -accel sub-option to set the PMU Event Filtering. Without the filter, the KVM will expose all events from the host to guest by default.The
pmu-filterhas such format:pmu-filter="{A,D}:start-end[;{A,D}:start-end...]"The A means "allow" and D means "deny", start is the first event of the range and the end is the last one. The first registered range defines the global
policy(global ALLOW if the first @action is DENY, global DENY if the first @action is ALLOW). The start and end only support hex format now. For example:pmu-filter="A:0x11-0x11;A:0x23-0x3a;D:0x30-0x30"Since the first action is allow, we have a global deny policy. It will allow event
0x11(The cycle counter), events0x23to0x3ais also allowed except the event0x30is denied, and all the other events are disallowed.Here is an real example shows how to use the PMU Event Filtering, when we launch a guest by use kvm, add such command line:
# qemu-system-aarch64 \ -accel kvm,pmu-filter="D:0x11-0x11"Since the first action is deny, we have a global allow policy. This disables the filtering of the cycle counter (event
0x11beingCPU_CYCLES).And then in guest, use the perf to count the cycle:
# perf stat sleep 1Performance counter stats for 'sleep 1':
1.22 msec task-clock # 0.001 CPUs utilized
1 context-switches # 820.695 /sec
0 cpu-migrations # 0.000 /sec
55 page-faults # 45.138 K/sec
<not supported> cycles
1128954 instructions
227031 branches # 186.323 M/sec
8686 branch-misses # 3.83% of all branches1.002492480 seconds time elapsed
0.001752000 seconds user
0.000000000 seconds sysAs we can see, the cycle counter has been disabled in the guest, but other pmu events are still work.
Signed-off-by Shaoqin Huang <[email protected]>---
v2->v3:
- Improve commits message, use kernel doc wording, add more explaination on
filter example, fix some typo error. [Eric]
- Addg_free()inkvm_arch_set_pmu_filter()to prevent memory leak. [Eric]
- Add more precise error message report. [Eric]
- In options doc, add pmu-filter rely onKVM_ARM_VCPU_PMU_V3_FILTERsupport in
KVM. [Eric]
v1->v2:
- Add more description for allow and deny meaning in
commit message. [Sebastian]
- Small improvement. [Sebastian]v2: https://lore.kernel.org/all/[email protected]/
v1: https://lore.kernel.org/all/[email protected]/--- include/sysemu/kvm_int.h | 1 + qemu-options.hx | 21 +++++++++++++ target/arm/kvm.c | 23 ++++++++++++++ target/arm/kvm64.c | 68 ++++++++++++++++++++++++++++++++++++++++ 4 files changed, 113 insertions(+)include/sysemu/kvm_int.h +1
-0@@ -120,6 +120,7 @@ struct KVMStateuint32_t xen_caps;uint16_t xen_gnttab_max_frames;uint16_t xen_evtchn_max_pirq; +char *kvm_pmu_filter; }; void kvm_memory_listener_register(KVMState *s, KVMMemoryListener *kml,qemu-options.hx +0
-0
qemu-options.hx DEF("accel", HAS_ARG, QEMU_OPTION_accel, · line 190
Quoted 4 lines
qemu-options.hx +1
-0@@ -187,6 +187,7 @@ DEF("accel", HAS_ARG, QEMU_OPTION_accel, " tb-size=n (TCG translation block cache size)\n" " dirty-ring-size=n (KVM dirty ring GFN count, default 0)\n" " eager-split-size=n (KVM Eager Page Split chunk size, default 0, disabled. ARM only)\n" + " pmu-filter={A,D}:start-end[;...] (KVM PMU Event Filter, default no filter. ARM only)\n"
^^^^^^^
Potential alignment issue, or the email isn't shown for me correctly. Besides, why not follow the pattern in the commit log, which is nicer than what's of being:
pmu-filter={A,D}:start-end[;...]
to
pmu-filter="{A,D}:start-end[;{A,D}:start-end...]
qemu-options.hx SRST · line 264
Quoted 9 lines
qemu-options.hx +2
-0" notify-vmexit=run|internal-error|disable,notify-window=n (enable notify VM exit and set notify window, x86 only)\n" " thread=single|multi (enable multi-threaded TCG)\n", QEMU_ARCH_ALL) SRST@@ -259,6 +260,26 @@ SRST impact on the memory. By default, this feature is disabled (eager-split-size=0). + ``pmu-filter={A,D}:start-end[;...]`` + KVM implements pmu event filtering to prevent a guest from being able to
^^^^ ^^^^^^^^^^^^^^^^^^^
Alignment "PMU Event Filtering" to be consistent
qemu-options.hx SRST · line 265
Quoted 1 line
qemu-options.hx +1
-0+ sample certain events. It depends on the KVM_ARM_VCPU_PMU_V3_FILTER attr
^^^^
attributeQuoted 48 lines
qemu-options.hx +17
-0+ supported in KVM. It has the following format: + + pmu-filter="{A,D}:start-end[;{A,D}:start-end...]" + + The A means "allow" and D means "deny", start is the first event of the + range and the end is the last one. The first registered range defines + the global policy(global ALLOW if the first @action is DENY, global DENY + if the first @action is ALLOW). The start and end only support hex + format now. For example: + + pmu-filter="A:0x11-0x11;A:0x23-0x3a;D:0x30-0x30" + + Since the first action is allow, we have a global deny policy. It + will allow event 0x11 (The cycle counter), events 0x23 to 0x3a is + also allowed except the event 0x30 is denied, and all the other events + are disallowed. + ``notify-vmexit=run|internal-error|disable,notify-window=n`` Enables or disables notify VM exit support on x86 host and specify the corresponding notify window to trigger the VM exit if enabled.target/arm/kvm.c +16
-0@@ -1108,6 +1108,22 @@ static void kvm_arch_set_eager_split_size(Object *obj, Visitor *v, s->kvm_eager_split_size = value; } +static char *kvm_arch_get_pmu_filter (Object *obj,Error **errp) +{ +KVMState *s =KVM_STATE (obj); + +return g_strdup (s->kvm_pmu_filter); +} + +static void kvm_arch_set_pmu_filter (Object *obj,const char *pmu_filter, +Error **errp) +{ +KVMState *s =KVM_STATE (obj); + +g_free (s->kvm_pmu_filter); + s->kvm_pmu_filter =g_strdup (pmu_filter); +} +void kvm_arch_accel_class_init (ObjectClass *oc) { object_class_property_add(oc, "eager-split-size", "size",
target/arm/kvm.c void kvm_arch_accel_class_init(ObjectClass *oc) · line 1141
Quoted 10 lines
target/arm/kvm.c +7
-0@@ -1116,4 +1132,11 @@ void kvm_arch_accel_class_init(ObjectClass *oc)object_class_property_set_description (oc,"eager-split-size" ,"Eager Page Split chunk size for hugepages. (default: 0, disabled)" ); + +object_class_property_add_str (oc,"pmu-filter" , + kvm_arch_get_pmu_filter, + kvm_arch_set_pmu_filter); + +object_class_property_set_description (oc,"pmu-filter" , +"PMU Event Filtering description for guest pmu. (default: NULL, disabled)" );
^^^
PMUQuoted 17 lines
target/arm/kvm.c +0
-0}target/arm/kvm64.c +1
-0@@ -10,6 +10,7 @@*/ #include "qemu/osdep.h" +#include <asm-arm64/kvm.h> #include <sys/ioctl.h> #include <sys/ptrace.h> @@ -131,6 +132,70 @@ static bool kvm_arm_set_device_attr(CPUState *cs, struct kvm_device_attr *attr,return true; }
target/arm/kvm64.c static bool kvm_arm_set_device_attr(CPUState *cs, struct kvm_device_attr *attr, · line 144
Quoted 10 lines
target/arm/kvm64.c +10
-0+static void kvm_arm_pmu_filter_init (CPUState *cs) +{ +static bool pmu_filter_init = false; +struct kvm_pmu_event_filter filter; +struct kvm_device_attr attr = { + .group =KVM_ARM_VCPU_PMU_V3_CTRL , + .attr =KVM_ARM_VCPU_PMU_V3_FILTER , + .addr = (uint64_t )&filter, + }; +KVMState *kvm_state = cs->kvm_state;
I would move @kvm_state to the beginning of the function since it's the container to everything else.
target/arm/kvm64.c static bool kvm_arm_set_device_attr(CPUState *cs, struct kvm_device_attr *attr, · line 155
Quoted 11 lines
target/arm/kvm64.c +11
-0+char *tmp; +char *str, act; + +if (!kvm_state->kvm_pmu_filter) +return ; + +if (kvm_vcpu_ioctl (cs,KVM_HAS_DEVICE_ATTR , attr)) { +error_report ("The kernel doesn't support the pmu event filter!\n" ); +abort (); + } +
s/attr/&attr ?
The connection between vCPU and attribute query was set up in Linux v4.10 by commit f577f6c2a6a ("arm64: KVM: Introduce per-vcpu kvm device controls"), and the capability depends on KVM_CAP_VCPU_ATTRIBUTES. I think KVM_CAP_VCPU_ATTRIBUTES needs to be checked prior to kvm_vcpu_ioctl(cs, KVM_HAS_DEVICE_ATTR, ...)
Besides, the PMU Event Filtering was introduced to Linux v4.10. It means the user can crash qemu when "pmu-filter" is provided on Linux v4.9. So the correct behavior would be warning and ignore "pmu-filter" since it's an add-on and best-effort feature.
target/arm/kvm64.c static bool kvm_arm_set_device_attr(CPUState *cs, struct kvm_device_attr *attr, · line 159
Quoted 4 lines
target/arm/kvm64.c +4
-0+/* The filter only needs to be initialized for 1 vcpu. */ +if (!pmu_filter_init) + pmu_filter_init = true; +
{ } has been missed. QEMU needs it even for the block with single line of code.
Quoted 29 lines
target/arm/kvm64.c +29
-0+ tmp =g_strdup (kvm_state->kvm_pmu_filter); + +for (str =strtok (tmp,";" ); str !=NULL ; str =strtok (NULL ,";" )) { +unsigned short start =0 , end =0 ; + +sscanf (str,"%c:%hx-%hx" , &act, &start, &end); +if ((act !='A' && act !='D' ) || (!start && !end)) { +error_report ("skipping invalid filter %s\n" , str); +continue ; + } + + filter = (struct kvm_pmu_event_filter ) { + .base_event = start, + .nevents = end - start +1 , + .action = act =='A' ?KVM_PMU_EVENT_ALLOW : +KVM_PMU_EVENT_DENY , + }; + +if (!kvm_arm_set_device_attr (cs, &attr,"PMU Event Filter" )) { +if (errno ==EINVAL ) +error_report ("Invalid filter range [0x%x-0x%x]. " +"ARMv8.0 support 10 bits event space, " +"ARMv8.1 support 16 bits event space" , + start, end); +else if (errno ==ENODEV ) +error_report ("GIC not initialized" ); +else if (errno ==ENXIO ) +error_report ("PMUv3 not properly configured or in-kernel irqchip " +"not configured." );
target/arm/kvm64.c static bool kvm_arm_set_device_attr(CPUState *cs, struct kvm_device_attr *attr, · line 198
Quoted 10 lines
target/arm/kvm64.c +10
-0+else if (errno ==EBUSY ) +error_report ("PMUv3 already initialized or a VCPU has already run" ); + +abort (); + } + } + +g_free (tmp); +} +
{ } has been missed.
g_strsplit() may be good fit to parse "pmu-filter". cpu-target.c::parse_cpu_option() is the example for its usage.
As I explained above, it wouldn't a "abort()" since "pmu-filter" is an add-on and best-effort attempt. We probably just warn done by warn_report() instead of raising error if the PMU Event Filter fails to be set.
target/arm/kvm64.c void kvm_arm_pmu_init(CPUState *cs) · line 214
Quoted 13 lines
target/arm/kvm64.c +3
-0void kvm_arm_pmu_init (CPUState *cs) {struct kvm_device_attr attr = {@@ -141,6 +206,9 @@ void kvm_arm_pmu_init(CPUState *cs)if (!ARM_CPU (cs)->has_pmu) {return ; } + +kvm_arm_pmu_filter_init (cs); +if (!kvm_arm_set_device_attr (cs, &attr,"PMU" )) {error_report ("failed to init PMU" );abort ();
Thanks,
Gavin
Hi Gavin,
Quoted 109 lines · Gavin Shan
Hi Shaoqin,
Quoted 95 lines · Shaoqin Huang
The
KVM_ARM_VCPU_PMU_V3_FILTERprovide the ability to let the VMM decide which PMU events are provided to the guest. Add a new optionpmu-filteras -accel sub-option to set the PMU Event Filtering. Without the filter, the KVM will expose all events from the host to guest by default.The
pmu-filterhas such format:pmu-filter="{A,D}:start-end[;{A,D}:start-end...]"
The A means "allow" and D means "deny", start is the first event of the range and the end is the last one. The first registered range defines the global
policy(global ALLOW if the first @action is DENY, global DENY if the first @action is ALLOW). The start and end only support hex format now. For example:pmu-filter="A:0x11-0x11;A:0x23-0x3a;D:0x30-0x30"
Since the first action is allow, we have a global deny policy. It will allow event
0x11(The cycle counter), events0x23to0x3ais also allowed except the event0x30is denied, and all the other events are disallowed.Here is an real example shows how to use the PMU Event Filtering, when we launch a guest by use kvm, add such command line:
# qemu-system-aarch64 \ -accel kvm,pmu-filter="D:0x11-0x11"Since the first action is deny, we have a global allow policy. This disables the filtering of the cycle counter (event
0x11beingCPU_CYCLES).And then in guest, use the perf to count the cycle:
# perf stat sleep 1Performance counter stats for 'sleep 1':
1.22 msec task-clock # 0.001 CPUs utilized 1 context-switches # 820.695 /sec 0 cpu-migrations # 0.000 /sec 55 page-faults # 45.138 K/sec <not supported> cycles
1128954 instructions
227031 branches # 186.323 M/sec 8686 branch-misses # 3.83% of all branches1.002492480 seconds time elapsed
0.001752000 seconds user
0.000000000 seconds sysAs we can see, the cycle counter has been disabled in the guest, but other pmu events are still work.
Signed-off-by Shaoqin Huang <[email protected]>---
v2->v3:
- Improve commits message, use kernel doc wording, add more explaination on filter example, fix some typo error. [Eric]
- Addg_free()inkvm_arch_set_pmu_filter()to prevent memory leak. [Eric]
- Add more precise error message report. [Eric]
- In options doc, add pmu-filter rely onKVM_ARM_VCPU_PMU_V3_FILTERsupport in KVM. [Eric]
v1->v2:
- Add more description for allow and deny meaning in commit message. [Sebastian]
- Small improvement. [Sebastian]v2: https://lore.kernel.org/all/[email protected]/
v1: https://lore.kernel.org/all/[email protected]/
---
include/sysemu/kvm_int.h| 1 +
qemu-options.hx | 21 +++++++++++++
target/arm/kvm.c | 23 ++++++++++++++ target/arm/kvm64.c | 68 ++++++++++++++++++++++++++++++++++++++++4 files changed, 113 insertions(+)include/sysemu/kvm_int.h +1
-0@@ -120,6 +120,7 @@ struct KVMState uint32_t xen_caps; uint16_t xen_gnttab_max_frames; uint16_t xen_evtchn_max_pirq; + char *kvm_pmu_filter; }; void kvm_memory_listener_register(KVMState *s, KVMMemoryListener *kml,qemu-options.hx +0
-0@@ -187,6 +187,7 @@ DEF("accel", HAS_ARG, QEMU_OPTION_accel,Quoted 5 lines
" tb-size=n (TCG translation block cache size)\n"
" dirty-ring-size=n (KVM dirty ring GFN count, default 0)\n"
" eager-split-size=n (KVM Eager Page Split chunk size, default 0, disabled. ARM only)\n" + " pmu-filter={A,D}:start-end[;...] (KVM PMU Event
Filter, default no filter. ARM only)\n"^^^^^^^
Potential alignment issue, or the email isn't shown for me correctly. Besides, why not follow the pattern in the commit log, which is nicer than what's of being:
Quoted 6 lines
pmu-filter={A,D}:start-end[;...]
to
pmu-filter="{A,D}:start-end[;{A,D}:start-end...]
Ok. I can replace it with the better format.
Quoted 12 lines
Quoted 9 lines
" notify-vmexit=run|internal-error|disable,notify-window=n (enable notify VM exit and set notify window, x86 only)\n"
" thread=single|multi (enable multi-threaded TCG)\n",QEMU_ARCH_ALL) SRSTqemu-options.hx +0
-0@@ -259,6 +260,26 @@ SRSTimpact on the memory. By default, this feature is disabled (eager-split-size=0).
+ ``pmu-filter={A,D}:start-end[;...]``
+ KVM implements pmu event filtering to prevent a guest from being able to^^^^ ^^^^^^^^^^^^^^^^^^^ Alignment "PMU Event Filtering" to be consistent
Thanks for pointing it out. It should be an alignment issue. I will fix it.
Quoted 60 lines
Quoted 2 lines
+ sample certain events. It depends on the
KVM_ARM_VCPU_PMU_V3_FILTERattr^^^^ attribute
Quoted 56 lines
+ supported in KVM. It has the following format:
qemu-options.hx +4
-0+ + pmu-filter="{A,D}:start-end[;{A,D}:start-end...]" + + The A means "allow" and D means "deny", start is the first eventof the
+ range and the end is the last one. The first registered range defines
+ the global policy(global ALLOW if the first @action is DENY, global DENY
+ if the first @action is ALLOW). The start and end only support hex + format now. For example:qemu-options.hx +6
-0+ + pmu-filter="A:0x11-0x11;A:0x23-0x3a;D:0x30-0x30" + + Since the first action is allow, we have a global deny policy. It + will allow event 0x11 (The cycle counter), events 0x23 to 0x3a is + also allowed except the event 0x30 is denied, and all the otherevents
+ are disallowed.
+
``notify-vmexit=run|internal-error|disable,notify-window=n`` Enables or disables notify VM exit support on x86 host and specify the corresponding notify window to trigger the VM exit if enabled.target/arm/kvm.c +18
-0@@ -1108,6 +1108,22 @@ static voidkvm_arch_set_eager_split_size (Object *obj, Visitor *v, s->kvm_eager_split_size = value; } +static char *kvm_arch_get_pmu_filter(Object *obj, Error **errp) +{ + KVMState *s = KVM_STATE(obj); + + return g_strdup(s->kvm_pmu_filter); +} + +static void kvm_arch_set_pmu_filter (Object *obj,const char *pmu_filter, + Error **errp) +{ + KVMState *s = KVM_STATE(obj); + + g_free(s->kvm_pmu_filter); + s->kvm_pmu_filter = g_strdup(pmu_filter); +} + void kvm_arch_accel_class_init(ObjectClass *oc) { object_class_property_add(oc, "eager-split-size", "size",@@ -1116,4 +1132,11 @@ void kvm_arch_accel_class_init(ObjectClass *oc) object_class_property_set_description(oc, "eager-split-size", "Eager Page Split chunk size for hugepages. (default: 0, disabled)"); + + object_class_property_add_str(oc, "pmu-filter",
Quoted 8 lines
target/arm/kvm.c void kvm_arch_accel_class_init(ObjectClass *oc)
Quoted 6 lines
target/arm/kvm.c +5
-0+ kvm_arch_get_pmu_filter, + kvm_arch_set_pmu_filter); + + object_class_property_set_description(oc, "pmu-filter", + "PMU Event Filtering description for guest pmu. (default: NULL, disabled)");^^^ PMU
Got it.
Quoted 20 lines
Quoted 20 lines
target/arm/kvm.c +0
-0}target/arm/kvm64.c +6
-0@@ -10,6 +10,7 @@*/ #include "qemu/osdep.h" +#include <asm-arm64/kvm.h> #include <sys/ioctl.h> #include <sys/ptrace.h>@@ -131,6 +132,70 @@ static bool kvm_arm_set_device_attr(CPUState *cs, struct kvm_device_attr *attr, return true; } +static void kvm_arm_pmu_filter_init (CPUState *cs) +{ + static bool pmu_filter_init = false; + struct kvm_pmu_event_filter filter; + struct kvm_device_attr attr = {
Quoted 9 lines
target/arm/kvm64.c static bool kvm_arm_set_device_attr(CPUState *cs, struct kvm_device_attr *attr, · line 144
Quoted 5 lines
target/arm/kvm64.c +5
-0+ .group = KVM_ARM_VCPU_PMU_V3_CTRL, + .attr = KVM_ARM_VCPU_PMU_V3_FILTER, + .addr = (uint64_t)&filter, + }; + KVMState *kvm_state = cs->kvm_state;I would move @kvm_state to the beginning of the function since it's the container to everything else.
Ok. I can move it to the first.
Quoted 6 lines
Quoted 6 lines
target/arm/kvm64.c +6
-0+ char *tmp; + char *str, act; + + if (!kvm_state->kvm_pmu_filter) + return; +
Quoted 9 lines
target/arm/kvm64.c static bool kvm_arm_set_device_attr(CPUState *cs, struct kvm_device_attr *attr, · line 155
Quoted 6 lines
target/arm/kvm64.c +5
-0+ if (kvm_vcpu_ioctl(cs, KVM_HAS_DEVICE_ATTR, attr)) { + error_report("The kernel doesn't support the pmu event filter!\n"); + abort(); + } +s/attr/&attr ?
It should be &attr.
Quoted 5 lines
The connection between vCPU and attribute query was set up in Linux v4.10 by commit
f577f6c2a6a("arm64: KVM: Introduce per-vcpu kvm device controls"), and the capability depends onKVM_CAP_VCPU_ATTRIBUTES. I thinkKVM_CAP_VCPU_ATTRIBUTESneeds to be checked prior tokvm_vcpu_ioctl(cs, KVM_HAS_DEVICE_ATTR, ...)
I searched the KVM_CAP_VCPU_ATTRIBUTES, no where check the capability. And the current usage of KVM_HAS_DEVICE_ATTR never check the KVM_CAP_VCPU_ATTRIBUTES. I guess we don't need it.
Quoted 4 lines
Besides, the PMU Event Filtering was introduced to Linux v4.10. It means the user can crash qemu when "pmu-filter" is provided on Linux v4.9. So the correct behavior would be warning and ignore "pmu-filter" since it's an add-on and best-effort feature.
Thanks for pointing this out, I think it's reasonable. I can replace all error_report with wran_report and replace delete all abort().
Quoted 9 lines
target/arm/kvm64.c static bool kvm_arm_set_device_attr(CPUState *cs, struct kvm_device_attr *attr, · line 159
Quoted 6 lines
target/arm/kvm64.c +4
-0+ /* The filter only needs to be initialized for 1 vcpu. */ + if (!pmu_filter_init) + pmu_filter_init = true; +{ } has been missed. QEMU needs it even for the block with single line of code.
Ok. I will add it.
Quoted 44 lines
Quoted 33 lines
target/arm/kvm64.c +28
-0+ tmp = g_strdup(kvm_state->kvm_pmu_filter); + + for (str = strtok(tmp, ";"); str != NULL; str = strtok(NULL, ";")) { + unsigned short start = 0, end = 0; + + sscanf(str, "%c:%hx-%hx", &act, &start, &end); + if ((act != 'A' && act != 'D') || (!start && !end)) { + error_report("skipping invalid filter %s\n", str); + continue; + } + + filter = (struct kvm_pmu_event_filter) { + .base_event = start, + .nevents = end - start + 1, + .action = act == 'A' ? KVM_PMU_EVENT_ALLOW : + KVM_PMU_EVENT_DENY, + }; + + if (!kvm_arm_set_device_attr(cs, &attr, "PMU Event Filter")) { + if (errno == EINVAL) + error_report("Invalid filter range [0x%x-0x%x]. " + "ARMv8.0 support 10 bits event space, " + "ARMv8.1 support 16 bits event space", + start, end); + else if (errno == ENODEV) + error_report("GIC not initialized"); + else if (errno == ENXIO) + error_report("PMUv3 not properly configured orin-kernel irqchip "
+ "not configured.");
+ else if (errno == EBUSY)
+error_report("PMUv3 already initialized or a VCPU has already run");target/arm/kvm64.c static bool kvm_arm_set_device_attr(CPUState *cs, struct kvm_device_attr *attr, · line 198
Quoted 8 lines
target/arm/kvm64.c +8
-0+ + abort(); + } + } + + g_free(tmp); +} +{ } has been missed.
Quoted 3 lines
g_strsplit()may be good fit to parse "pmu-filter". cpu-target.c::parse_cpu_option() is the example for its usage.
Ok. I can follow the implementation. It actually much simpler.
Quoted 4 lines
As I explained above, it wouldn't a "
abort()" since "pmu-filter" is an add-on and best-effort attempt. We probably just warn done bywarn_report()instead of raising error if the PMU Event Filter fails to be set.
Will do that.
Thanks,
Shaoqin
Quoted 17 lines
target/arm/kvm64.c void kvm_arm_pmu_init(CPUState *cs) · line 214
Quoted 13 lines
void
kvm_arm_pmu_init(CPUState *cs)
{
structkvm_device_attrattr = {target/arm/kvm64.c +3
-0@@ -141,6 +206,9 @@ void kvm_arm_pmu_init(CPUState *cs) if (!ARM_CPU(cs)->has_pmu) { return; } + + kvm_arm_pmu_filter_init(cs); + if (!kvm_arm_set_device_attr(cs, &attr, "PMU")) { error_report("failed to init PMU"); abort();Thanks,
Gavin
Signature · 1 line
Shaoqin