Add vPMU vPMU support under TCG mode
cover Add vPMU vPMU support under TCG mode
QEMU has implemented cycle count support for guest VM under TCG mode. But this feature is not complete. In fact using perf inside a 64-bit Linux guest VM (under TCG) can cause the following kernel panic because some PMU registers are not implemented.
[ 329.445970] [<fffffe000009e600>] armv8pmu_enable_event+0x58/0x8c [ 329.446713] [<fffffe0000621e74>] armpmu_start+0x4c/0x74
This patchset solves the problem by adding support for missing vPMU registers. Basic perf test can work now under TCG by applying this patchset.
[root@localhost ~]# perf stat ls
Performance counter stats for 'ls':
226.740256 task-clock (msec) # 0.312 CPUs utilized
76 context-switches # 0.335 K/sec
0 cpu-migrations # 0.000 K/sec
64 page-faults # 0.282 K/sec
186,031,410 cycles # 0.820 GHz (36.40%)
<not supported> stalled-cycles-frontend
<not supported> stalled-cycles-backend
<not counted> instructions (0.00%)
<not supported> branches
<not counted> branch-misses (0.00%)Thanks,
-Wei
Wei Huang (4):
target-arm: Add support for PMU register PMSELR_EL0
target-arm: Add support for AArch64 PMU register PMXEVTYPER_EL0
target-arm: Add support for PMU register PMINTENSET_EL1
target-arm: Hook up TCG vPMU with CPU pmu option
target/arm/cpu.c | 2 +- target/arm/cpu.h | 1 + target/arm/helper.c | 41 +++++++++++++++++++++++++++++++++++------ 3 files changed, 37 insertions(+), 7 deletions(-)
Signature · 1 line
1.8.3.1
PATCH 1/4 target-arm: Add support for PMU register PMSELR_EL0
This patch adds support for AArch64 register PMSELR_EL0. The existing PMSELR definition is revised accordingly.
--- target/arm/cpu.h | 1 + target/arm/helper.c | 24 +++++++++++++++++++----- 2 files changed, 20 insertions(+), 5 deletions(-)
target/arm/cpu.h +1 -0
@@ -307,6 +307,7 @@ typedef struct CPUARMState {uint32_t c9_pmovsr;/* perf monitor overflow status */ uint32_t c9_pmxevtyper;/* perf monitor event type */ uint32_t c9_pmuserenr;/* perf monitor user enable */ +uint32_t c9_pmselr;/* perf monitor counter selection register */ uint32_t c9_pminten;/* perf monitor interrupt enables */ union { /* Memory attribute redirection */ struct {
target/arm/helper.c +19 -5
@@ -975,6 +975,15 @@ static uint64_t pmccntr_read(CPUARMState *env, const ARMCPRegInfo *ri)return total_ticks - env->cp15.c15_ccnt; } +static void pmselr_write (CPUARMState *env,const ARMCPRegInfo *ri, +uint64_t value) +{ +/* only cycle counter selection is supported */ +if (value ==0x1f ) { + env->cp15.c9_pmselr = value; + } +} +static void pmccntr_write (CPUARMState *env,const ARMCPRegInfo *ri,uint64_t value) {
@@ -1194,12 +1203,17 @@ static const ARMCPRegInfo v7_cp_reginfo[] = {/* Unimplemented so WI. */ { .name = "PMSWINC", .cp = 15, .crn = 9, .crm = 12, .opc1 = 0, .opc2 = 4, .access = PL0_W, .accessfn = pmreg_access, .type = ARM_CP_NOP }, - /* Since we don't implement any events, writing to PMSELR is UNPREDICTABLE. - * We choose to RAZ/WI. - */ { .name = "PMSELR", .cp = 15, .crn = 9, .crm = 12, .opc1 = 0, .opc2 = 5, - .access = PL0_RW, .type = ARM_CP_CONST, .resetvalue = 0, - .accessfn = pmreg_access }, + .access = PL0_RW, .type = ARM_CP_ALIAS, + .fieldoffset = offsetof(CPUARMState, cp15.c9_pmselr), + .accessfn = pmreg_access, .writefn = pmselr_write, + .raw_writefn = raw_write}, + { .name = "PMSELR_EL0", .state = ARM_CP_STATE_AA64, + .opc0 = 3, .opc1 = 3, .crn = 9, .crm = 12, .opc2 = 5, + .access = PL0_RW, .accessfn = pmreg_access, + .type = ARM_CP_IO, + .fieldoffset = offsetof(CPUARMState, cp15.c9_pmselr), + .writefn = pmselr_write, .raw_writefn = raw_write, }, #ifndef CONFIG_USER_ONLY { .name = "PMCCNTR", .cp = 15, .crn = 9, .crm = 13, .opc1 = 0, .opc2 = 0, .access = PL0_RW, .resetvalue = 0, .type = ARM_CP_IO,
Signature · 1 line
1.8.3.1
Re: [Qemu-arm] [PATCH V1 1/4] target-arm: Add support for PMU register PMSELR_EL0
Quoted 26 lines · Wei Huang
This patch adds support for AArch64 register
PMSELR_EL0. The existing PMSELR definition is revised accordingly.Signed-off-by Wei Huang <[email protected]>--- target/arm/cpu.h | 1 + target/arm/helper.c | 24 +++++++++++++++++++----- 2 files changed, 20 insertions(+), 5 deletions(-)target/arm/cpu.h +1
-0@@ -307,6 +307,7 @@ typedef struct CPUARMState {uint32_t c9_pmovsr;/* perf monitor overflow status */ uint32_t c9_pmxevtyper;/* perf monitor event type */ uint32_t c9_pmuserenr;/* perf monitor user enable */ +uint32_t c9_pmselr;/* perf monitor counter selection register */ uint32_t c9_pminten;/* perf monitor interrupt enables */ union { /* Memory attribute redirection */ struct {target/arm/helper.c +0
-0
target/arm/helper.c static uint64_t pmccntr_read(CPUARMState *env, const ARMCPRegInfo *ri) · line 984
Quoted 10 lines
target/arm/helper.c +7
-0@@ -975,6 +975,15 @@ static uint64_t pmccntr_read(CPUARMState *env, const ARMCPRegInfo *ri)return total_ticks - env->cp15.c15_ccnt; } +static void pmselr_write (CPUARMState *env,const ARMCPRegInfo *ri, +uint64_t value) +{ +/* only cycle counter selection is supported */ +if (value ==0x1f ) { + env->cp15.c9_pmselr = value; + }
This looks like it's trying to add PMUv2 (and ARMv8) semantics for PMSELR, but it doesn't quite get them right.
In the old PMUv1 (which is what we currently implement), 0x1f was reserved and thus effectively writing anything to PMSELR in a system with no implemented counters was UNPREDICTABLE.
Quoted 1 line
From PMUv2 (and including v8)
0x1fis supported for selecting
the cycle counter and must cause:
* PMXEVTYPER reads and writes to access PMCCFILTR
* PMXEVCNTR access to be CONSTRAINED UNPREDICTABLE (we already get this right because 'RAZ/WI' is one of the permitted choices)
The SEL field is 5 bits and the rest is RAZ/WI so we should mask value with 0x1f first.
Writing something to PMSELR which isn't 0x1f but is greater than or equal to the number of implemented counters (so for QEMU, anything but 0x1f) is permitted, but makes PMXEVTYPER/PMXEVCNTR accesses CONSTRAINED UNPREDICTABLE. So you need to allow pmselr to read back as the non-0x1f value, and have the handling of "is this 0x1f or not" happen in the read/write functions for PMXEVTYPER, I think.
(Also I think it's nice when we're making an UNPREDICTABLE or CONSTRAINED UNPREDICTABLE choice of behaviour to note it in a comment.)
Quoted 14 lines
target/arm/helper.c +2
-4+} +static void pmccntr_write (CPUARMState *env,const ARMCPRegInfo *ri,uint64_t value) {@@ -1194,12 +1203,17 @@ static const ARMCPRegInfo v7_cp_reginfo[] = {/* Unimplemented so WI. */ { .name = "PMSWINC", .cp = 15, .crn = 9, .crm = 12, .opc1 = 0, .opc2 = 4, .access = PL0_W, .accessfn = pmreg_access, .type = ARM_CP_NOP }, - /* Since we don't implement any events, writing to PMSELR is UNPREDICTABLE. - * We choose to RAZ/WI. - */ { .name = "PMSELR", .cp = 15, .crn = 9, .crm = 12, .opc1 = 0, .opc2 = 5, - .access = PL0_RW, .type = ARM_CP_CONST, .resetvalue = 0,
target/arm/helper.c static const ARMCPRegInfo v7_cp_reginfo[] = { · line 1215
Quoted 10 lines
target/arm/helper.c +9
-1- .accessfn = pmreg_access }, + .access = PL0_RW, .type = ARM_CP_ALIAS, + .fieldoffset = offsetof(CPUARMState, cp15.c9_pmselr), + .accessfn = pmreg_access, .writefn = pmselr_write, + .raw_writefn = raw_write}, + { .name = "PMSELR_EL0", .state = ARM_CP_STATE_AA64, + .opc0 = 3, .opc1 = 3, .crn = 9, .crm = 12, .opc2 = 5, + .access = PL0_RW, .accessfn = pmreg_access, + .type = ARM_CP_IO, + .fieldoffset = offsetof(CPUARMState, cp15.c9_pmselr),
For a 64-bit register the field in the CPU struct has to be a uint64_t, so you need to change it from the uint32_t it is at the moment. The AArch32 regdef struct then needs to change to use offsetoflow32() where it currently has offsetof().
Quoted 6 lines
target/arm/helper.c +1
-1+ .writefn = pmselr_write, .raw_writefn = raw_write, },#ifndef CONFIG_USER_ONLY { .name = "PMCCNTR", .cp = 15, .crn = 9, .crm = 13, .opc1 = 0, .opc2 = 0, .access = PL0_RW, .resetvalue = 0, .type = ARM_CP_IO, --1.8.3.1
thanks
-- PMM
PATCH 2/4 target-arm: Add support for AArch64 PMU register PMXEVTYPER_EL0
In reply to Wei HuangIn order to support Linux perf, which uses PMXEVTYPER register, this patch adds access support for PMXEVTYPER_EL0.
--- target/arm/helper.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
target/arm/helper.c +8 -1
@@ -1233,10 +1233,17 @@ static const ARMCPRegInfo v7_cp_reginfo[] = { .fieldoffset = offsetof(CPUARMState, cp15.pmccfiltr_el0), .resetvalue = 0, }, { .name = "PMXEVTYPER", .cp = 15, .crn = 9, .crm = 13, .opc1 = 0, .opc2 = 1, - .access = PL0_RW, + .access = PL0_RW, .type = ARM_CP_ALIAS, .fieldoffset = offsetof(CPUARMState, cp15.c9_pmxevtyper), .accessfn = pmreg_access, .writefn = pmxevtyper_write, .raw_writefn = raw_write }, + { .name = "PMXEVTYPER_EL0", .state = ARM_CP_STATE_AA64, + .opc0 = 3, .opc1 = 3, .crn = 9, .crm = 13, .opc2 = 1, + .access = PL0_RW, .accessfn = pmreg_access, + .type = ARM_CP_IO, + .fieldoffset = offsetof(CPUARMState, cp15.c9_pmxevtyper), + .writefn = pmxevtyper_write, .raw_writefn = raw_write, + .resetvalue = 0x0 }, /* Unimplemented, RAZ/WI. */ { .name = "PMXEVCNTR", .cp = 15, .crn = 9, .crm = 13, .opc1 = 0, .opc2 = 2, .access = PL0_RW, .type = ARM_CP_CONST, .resetvalue = 0,
Signature · 1 line
1.8.3.1
Re: [Qemu-arm] [PATCH V1 2/4] target-arm: Add support for AArch64 PMU register PMXEVTYPER_EL0
Quoted 15 lines · Wei Huang
In order to support Linux perf, which uses PMXEVTYPER register, this patch adds access support for
PMXEVTYPER_EL0.Signed-off-by Wei Huang <[email protected]>--- target/arm/helper.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)target/arm/helper.c +0
-0@@ -1233,10 +1233,17 @@ static const ARMCPRegInfo v7_cp_reginfo[] = { .fieldoffset = offsetof(CPUARMState, cp15.pmccfiltr_el0), .resetvalue = 0, },
target/arm/helper.c static const ARMCPRegInfo v7_cp_reginfo[] = { · line 1243
Quoted 10 lines
target/arm/helper.c +5
-1{ .name = "PMXEVTYPER", .cp = 15, .crn = 9, .crm = 13, .opc1 = 0, .opc2 = 1, - .access = PL0_RW, + .access = PL0_RW, .type = ARM_CP_ALIAS, .fieldoffset = offsetof(CPUARMState, cp15.c9_pmxevtyper), .accessfn = pmreg_access, .writefn = pmxevtyper_write, .raw_writefn = raw_write }, + { .name = "PMXEVTYPER_EL0", .state = ARM_CP_STATE_AA64, + .opc0 = 3, .opc1 = 3, .crn = 9, .crm = 13, .opc2 = 1, + .access = PL0_RW, .accessfn = pmreg_access, + .type = ARM_CP_IO,
Why is this marked as ARM_CP_IO ?
target/arm/helper.c static const ARMCPRegInfo v7_cp_reginfo[] = { · line 1246
Quoted 3 lines
target/arm/helper.c +3
-0+ .fieldoffset = offsetof(CPUARMState, cp15.c9_pmxevtyper), + .writefn = pmxevtyper_write, .raw_writefn = raw_write, + .resetvalue = 0x0 },
As with patch 1, the c9_pmxevtyper field needs to become a uint64_t so you can use it in a 64-bit regdef.
(Also the semantics of PMXEVTYPER need to change, see comments on patch 1.)
target/arm/helper.c static const ARMCPRegInfo v7_cp_reginfo[] = { · line 1249
Quoted 3 lines
target/arm/helper.c +0
-0/* Unimplemented, RAZ/WI. */ { .name = "PMXEVCNTR", .cp = 15, .crn = 9, .crm = 13, .opc1 = 0, .opc2 = 2, .access = PL0_RW, .type = ARM_CP_CONST, .resetvalue = 0,
thanks
-- PMM
PATCH 3/4 target-arm: Add support for PMU register PMINTENSET_EL1
In reply to Wei HuangThis patch adds access support for PMINTENSET_EL1.
--- target/arm/helper.c | 8 ++++++++ 1 file changed, 8 insertions(+)
target/arm/helper.c +8 -0
@@ -1261,9 +1261,17 @@ static const ARMCPRegInfo v7_cp_reginfo[] = { .writefn = pmuserenr_write, .raw_writefn = raw_write }, { .name = "PMINTENSET", .cp = 15, .crn = 9, .crm = 14, .opc1 = 0, .opc2 = 1, .access = PL1_RW, .accessfn = access_tpm, + .type = ARM_CP_ALIAS, .fieldoffset = offsetof(CPUARMState, cp15.c9_pminten), .resetvalue = 0, .writefn = pmintenset_write, .raw_writefn = raw_write }, + { .name = "PMINTENSET_EL1", .state = ARM_CP_STATE_AA64, + .opc0 = 3, .opc1 = 0, .crn = 9, .crm = 14, .opc2 = 1, + .access = PL1_RW, .accessfn = access_tpm, + .type = ARM_CP_IO, + .fieldoffset = offsetof(CPUARMState, cp15.c9_pminten), + .writefn = pmintenset_write, .raw_writefn = raw_write, + .resetvalue = 0x0 }, { .name = "PMINTENCLR", .cp = 15, .crn = 9, .crm = 14, .opc1 = 0, .opc2 = 2, .access = PL1_RW, .accessfn = access_tpm, .type = ARM_CP_ALIAS, .fieldoffset = offsetof(CPUARMState, cp15.c9_pminten),
Signature · 1 line
1.8.3.1
Re: [Qemu-devel] [PATCH V1 3/4] target-arm: Add support for PMU register PMINTENSET_EL1
Quoted 20 lines · Wei Huang
This patch adds access support for
PMINTENSET_EL1.Signed-off-by Wei Huang <[email protected]>--- target/arm/helper.c | 8 ++++++++ 1 file changed, 8 insertions(+)target/arm/helper.c +2
-0@@ -1261,9 +1261,17 @@ static const ARMCPRegInfo v7_cp_reginfo[] = { .writefn = pmuserenr_write, .raw_writefn = raw_write }, { .name = "PMINTENSET", .cp = 15, .crn = 9, .crm = 14, .opc1 = 0, .opc2 = 1, .access = PL1_RW, .accessfn = access_tpm, + .type = ARM_CP_ALIAS, .fieldoffset = offsetof(CPUARMState, cp15.c9_pminten), .resetvalue = 0, .writefn = pmintenset_write, .raw_writefn = raw_write }, + { .name = "PMINTENSET_EL1", .state = ARM_CP_STATE_AA64,
target/arm/helper.c static const ARMCPRegInfo v7_cp_reginfo[] = {
Quoted 10 lines
target/arm/helper.c +6
-1+ .opc0 = 3, .opc1 = 0, .crn = 9, .crm = 14, .opc2 = 1, + .access = PL1_RW, .accessfn = access_tpm, + .type = ARM_CP_IO, + .fieldoffset = offsetof(CPUARMState, cp15.c9_pminten), + .writefn = pmintenset_write, .raw_writefn = raw_write, + .resetvalue = 0x0 }, { .name = "PMINTENCLR", .cp = 15, .crn = 9, .crm = 14, .opc1 = 0, .opc2 = 2, .access = PL1_RW, .accessfn = access_tpm, .type = ARM_CP_ALIAS, .fieldoffset = offsetof(CPUARMState, cp15.c9_pminten), --
c9_pminten needs to become 64 bits.
We should add PMINTENCLR_EL1 too -- it's a bit weird to provide the SET function but not its parter CLR operation.
thanks
-- PMM
PATCH 4/4 target-arm: Hook up TCG vPMU with CPU pmu option
In reply to Wei HuangRemove the checking of kvm_enabled(). With this, .pmu option can also control vPMU under TCG mode.
--- target/arm/cpu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
target/arm/cpu.c +1 -1
@@ -692,7 +692,7 @@ static void arm_cpu_realizefn(DeviceState *dev, Error **errp) cpu->id_aa64pfr0 &= ~0xf000 ; } -if (!cpu->has_pmu || !kvm_enabled ()) { +if (!cpu->has_pmu) { cpu->has_pmu = false;unset_feature (env,ARM_FEATURE_PMU ); }
Signature · 1 line
1.8.3.1
Re: [Qemu-arm] [PATCH V1 4/4] target-arm: Hook up TCG vPMU with CPU pmu option
Quoted 13 lines · Wei Huang
Remove the checking of
kvm_enabled(). With this, .pmu option can also control vPMU under TCG mode.Signed-off-by Wei Huang <[email protected]>--- target/arm/cpu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)target/arm/cpu.c +0
-0
target/arm/cpu.c static void arm_cpu_realizefn(DeviceState *dev, Error **errp) · line 698
Quoted 8 lines
target/arm/cpu.c +1
-1@@ -692,7 +692,7 @@ static void arm_cpu_realizefn(DeviceState *dev, Error **errp) cpu->id_aa64pfr0 &= ~0xf000 ; } -if (!cpu->has_pmu || !kvm_enabled ()) { +if (!cpu->has_pmu) { cpu->has_pmu = false;unset_feature (env,ARM_FEATURE_PMU ); }
I notice that the ARM_FEATURE_PMU bit doesn't seem to control whether we actually provide the TCG PMU registers, so this patch is OK but doesn't actually change the behaviour of the emulated cpu AFAICT.
Don't we now need to remove the hack where we mask out the PMUVer field of ID_AA64DFR1_EL1 ?
thanks
-- PMM