arm: pmu: Fixes for bare metal
v2 cover arm: pmu: Fixes for bare metal
There are some tests that fail when running on bare metal (including a passthrough prototype). There are three issues with the tests. The first one is that there are some missing isb()'s between enabling event counting and the actual counting. This wasn't an issue on KVM as trapping on registers served as context synchronization events. The second issue is that some tests assume that registers reset to 0. And finally, the third issue is that overflowing the low counter of a chained event sets the overflow flag in PMVOS and some tests fail by checking for it not being set.
Addressed all comments from the previous version:
- added some isb()'s as suggested by Alexandru.
- fixed a couple of confusing comments (Alexandru).
- check for overflow in the low counter in the interrupt_overflow test (Marc).
Thanks!
Ricardo
Ricardo Koller (3):
arm: pmu: Add missing isb()'s after sys register writing
arm: pmu: Reset the pmu registers before starting some tests
arm: pmu: Check for overflow in the low counter in chained counters
tests
arm/pmu.c | 55 ++++++++++++++++++++++++++++++++++++++----------------- 1 file changed, 38 insertions(+), 17 deletions(-)
Signature · 6 lines
2.37.1.455.g008518b4e5-goog _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v2 1/3 arm: pmu: Add missing isb()'s after sys register writing
There are various pmu tests that require an isb() between enabling counting and the actual counting. This can lead to count registers reporting less events than expected; the actual enabling happens after some events have happened. For example, some missing isb()'s in the pmu-sw-incr test lead to the following errors on bare-metal:
INFO: pmu: pmu-sw-incr: SW_INCR counter #0 has value 4294967280
PASS: pmu: pmu-sw-incr: PWSYNC does not increment if PMCR.E is unset
FAIL: pmu: pmu-sw-incr: counter #1 after + 100 SW_INCR
FAIL: pmu: pmu-sw-incr: counter #0 after + 100 SW_INCR
INFO: pmu: pmu-sw-incr: counter values after 100 SW_INCR #0=82 #1=98
PASS: pmu: pmu-sw-incr: overflow on counter #0 after 100 SW_INCR
SUMMARY: 4 tests, 2 unexpected failures
Add the missing isb()'s on all failing tests, plus some others that seem required:
- after clearing the overflow signal in the IRQ handler to avoid spurious interrupts.
- after direct writes to PMSWINC_EL0 for software to read the correct value for PMEVNCTR0_EL0 (from ARM DDI 0487H.a, page D13-5237).
--- arm/pmu.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
arm/pmu.c +16 -0
@@ -307,6 +307,7 @@ static void irq_handler(struct pt_regs *regs) } }write_sysreg (ALL_SET , pmovsclr_el0); +isb (); }else { pmu_stats.unexpected = true; }
@@ -534,10 +535,12 @@ static void test_sw_incr(void)write_sysreg_s (0x3 ,PMCNTENSET_EL0 );write_regn_el0 (pmevcntr,0 ,PRE_OVERFLOW ); +isb ();for (i =0 ; i <100 ; i++)write_sysreg (0x1 , pmswinc_el0); +isb ();report_info ("SW_INCR counter #0 has value %ld" ,read_regn_el0 (pmevcntr,0 ));report (read_regn_el0 (pmevcntr,0 ) ==PRE_OVERFLOW ,"PWSYNC does not increment if PMCR.E is unset" );
@@ -547,10 +550,12 @@ static void test_sw_incr(void)write_regn_el0 (pmevcntr,0 ,PRE_OVERFLOW );write_sysreg_s (0x3 ,PMCNTENSET_EL0 );set_pmcr (pmu.pmcr_ro |PMU_PMCR_E ); +isb ();for (i =0 ; i <100 ; i++)write_sysreg (0x3 , pmswinc_el0); +isb ();report (read_regn_el0 (pmevcntr,0 ) ==84 ,"counter #1 after + 100 SW_INCR" );report (read_regn_el0 (pmevcntr,1 ) ==100 ,"counter #0 after + 100 SW_INCR" );
@@ -618,9 +623,12 @@ static void test_chained_sw_incr(void)write_regn_el0 (pmevcntr,0 ,PRE_OVERFLOW );set_pmcr (pmu.pmcr_ro |PMU_PMCR_E ); +isb (); +for (i =0 ; i <100 ; i++)write_sysreg (0x1 , pmswinc_el0); +isb ();report (!read_sysreg (pmovsclr_el0) && (read_regn_el0 (pmevcntr,1 ) ==1 ),"no overflow and chain counter incremented after 100 SW_INCR/CHAIN" ); report_info("overflow=0x%lx, #0=%ld #1=%ld", read_sysreg(pmovsclr_el0),
@@ -634,9 +642,12 @@ static void test_chained_sw_incr(void)write_regn_el0 (pmevcntr,1 ,ALL_SET );write_sysreg_s (0x3 ,PMCNTENSET_EL0 );set_pmcr (pmu.pmcr_ro |PMU_PMCR_E ); +isb (); +for (i =0 ; i <100 ; i++)write_sysreg (0x1 , pmswinc_el0); +isb (); report((read_sysreg(pmovsclr_el0) == 0x2) && (read_regn_el0(pmevcntr, 1) == 0) && (read_regn_el0(pmevcntr, 0) == 84),
@@ -821,10 +832,13 @@ static void test_overflow_interrupt(void)report (expect_interrupts (0 ),"no overflow interrupt after preset" );set_pmcr (pmu.pmcr_ro |PMU_PMCR_E ); +isb (); +for (i =0 ; i <100 ; i++)write_sysreg (0x2 , pmswinc_el0);set_pmcr (pmu.pmcr_ro); +isb ();report (expect_interrupts (0 ),"no overflow interrupt after counting" );/* enable interrupts */
@@ -879,6 +893,7 @@ static bool check_cycles_increase(void)set_pmccfiltr (0 );/* count cycles in EL0, EL1, but not EL2 */ set_pmcr (get_pmcr () |PMU_PMCR_LC |PMU_PMCR_C |PMU_PMCR_E ); +isb ();for (int i =0 ; i <NR_SAMPLES ; i++) {uint64_t a, b;
@@ -894,6 +909,7 @@ static bool check_cycles_increase(void) }set_pmcr (get_pmcr () & ~PMU_PMCR_E ); +isb ();return success; }
Signature · 1 line
2.37.1.455.g008518b4e5-goog
Re: [kvm-unit-tests PATCH v2 1/3] arm: pmu: Add missing isb()'s after sys register writing
Hi,
Quoted 14 lines · Ricardo Koller
There are various pmu tests that require an
isb()between enabling counting and the actual counting. This can lead to count registers reporting less events than expected; the actual enabling happens after some events have happened. For example, some missing isb()'s in the pmu-sw-incr test lead to the following errors on bare-metal:INFO: pmu: pmu-sw-incr:
SW_INCRcounter #0 has value 4294967280
PASS: pmu: pmu-sw-incr: PWSYNC does not increment if PMCR.E is unset
FAIL: pmu: pmu-sw-incr: counter #1 after + 100SW_INCR
FAIL: pmu: pmu-sw-incr: counter #0 after + 100SW_INCR
INFO: pmu: pmu-sw-incr: counter values after 100SW_INCR#0=82 #1=98
PASS: pmu: pmu-sw-incr: overflow on counter #0 after 100SW_INCR
SUMMARY: 4 tests, 2 unexpected failures
Quoted 4 lines
Add the missing isb()'s on all failing tests, plus some others that seem required:
- after clearing the overflow signal in the IRQ handler to avoid spurious interrupts.
Nitpick, but it doesn't avoid (eliminates) spurious interrupts, it makes them less likely.
Quoted 14 lines
- after direct writes to
PMSWINC_EL0for software to read the correct value forPMEVNCTR0_EL0(from ARM DDI 0487H.a, page D13-5237).Signed-off-by Ricardo Koller <[email protected]>--- arm/pmu.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)arm/pmu.c +0
-0
arm/pmu.c static void test_overflow_interrupt(void) · line 841
Quoted 10 lines
arm/pmu.c +3
-0@@ -821,10 +832,13 @@ static void test_overflow_interrupt(void)report (expect_interrupts (0 ),"no overflow interrupt after preset" );set_pmcr (pmu.pmcr_ro |PMU_PMCR_E ); +isb (); +for (i =0 ; i <100 ; i++)write_sysreg (0x2 , pmswinc_el0);set_pmcr (pmu.pmcr_ro); +isb ();
A context synchronization event affects system register writes that come before the context synchronization event in program order, but if there are multiple system register writes, it doesn't perform them in program order (if that makes sense).
So it might happen that the CPU decides to perform the write to PMCR_EL1 which disables the PMU *before* the writes to PMSWINC_EL0. Which means that even if PMINTENSET_EL1 allows the PMU to assert interrupts when it shouldn't (thus causing the test to fail), those interrupt won't be asserted by the PMU because the PMU is disabled and the test would pass.
You need an ISB after the PMSWINC_EL0 writes and before disabling the PMU.
Thanks,
Alex
arm/pmu.c static void test_overflow_interrupt(void) · line 842
Quoted 1 line
arm/pmu.c +0
-0report (expect_interrupts (0 ),"no overflow interrupt after counting" );
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [kvm-unit-tests PATCH v2 1/3] arm: pmu: Add missing isb()'s after sys register writing
Quoted 47 lines · Alexandru Elisei
Hi,
Quoted 14 lines · Ricardo Koller
There are various pmu tests that require an
isb()between enabling counting and the actual counting. This can lead to count registers reporting less events than expected; the actual enabling happens after some events have happened. For example, some missing isb()'s in the pmu-sw-incr test lead to the following errors on bare-metal:INFO: pmu: pmu-sw-incr:
SW_INCRcounter #0 has value 4294967280
PASS: pmu: pmu-sw-incr: PWSYNC does not increment if PMCR.E is unset
FAIL: pmu: pmu-sw-incr: counter #1 after + 100SW_INCR
FAIL: pmu: pmu-sw-incr: counter #0 after + 100SW_INCR
INFO: pmu: pmu-sw-incr: counter values after 100SW_INCR#0=82 #1=98
PASS: pmu: pmu-sw-incr: overflow on counter #0 after 100SW_INCR
SUMMARY: 4 tests, 2 unexpected failuresQuoted 4 lines
Add the missing isb()'s on all failing tests, plus some others that seem required:
- after clearing the overflow signal in the IRQ handler to avoid spurious interrupts.Nitpick, but it doesn't avoid (eliminates) spurious interrupts, it makes them less likely.
Quoted 22 lines
- after direct writes to
PMSWINC_EL0for software to read the correct value forPMEVNCTR0_EL0(from ARM DDI 0487H.a, page D13-5237).Signed-off-by Ricardo Koller <[email protected]>--- arm/pmu.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)arm/pmu.c +2
-0@@ -821,10 +832,13 @@ static void test_overflow_interrupt(void)report (expect_interrupts (0 ),"no overflow interrupt after preset" );set_pmcr (pmu.pmcr_ro |PMU_PMCR_E ); +isb (); +for (i =0 ; i <100 ; i++)write_sysreg (0x2 , pmswinc_el0);
Quoted 7 lines
arm/pmu.c static void test_overflow_interrupt(void) · line 841
Quoted 2 lines
arm/pmu.c +1
-0set_pmcr (pmu.pmcr_ro); +isb ();A context synchronization event affects system register writes that come before the context synchronization event in program order, but if there are multiple system register writes, it doesn't perform them in program order (if that makes sense).
Good point, didn't think of that case. Added the missing isb() in v3.
Thanks,
Ricardo
Quoted 13 lines
So it might happen that the CPU decides to perform the write to
PMCR_EL1which disables the PMU *before* the writes toPMSWINC_EL0. Which means that even ifPMINTENSET_EL1allows the PMU to assert interrupts when it shouldn't (thus causing the test to fail), those interrupt won't be asserted by the PMU because the PMU is disabled and the test would pass.You need an ISB after the
PMSWINC_EL0writes and before disabling the PMU.Thanks,
AlexQuoted 1 line
arm/pmu.c +0
-0report (expect_interrupts (0 ),"no overflow interrupt after counting" );
PATCH v2 2/3 arm: pmu: Reset the pmu registers before starting some tests
In reply to Ricardo KollerSome registers like the PMOVS reset to an architecturally UNKNOWN value. Most tests expect them to be reset (mostly zeroed) using pmu_reset(). Add a pmu_reset() on all the tests that need one.
As a bonus, fix a couple of comments related to the register state before a sub-test.
--- arm/pmu.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
arm/pmu.c +4 -2
@@ -826,7 +826,7 @@ static void test_overflow_interrupt(void)write_regn_el0 (pmevcntr,1 ,PRE_OVERFLOW );isb (); -/* interrupts are disabled */ +/* interrupts are disabled (PMINTENSET_EL1 == 0) */ mem_access_loop (addr,200 , pmu.pmcr_ro |PMU_PMCR_E );report (expect_interrupts (0 ),"no overflow interrupt after preset" );
@@ -841,7 +841,7 @@ static void test_overflow_interrupt(void)isb ();report (expect_interrupts (0 ),"no overflow interrupt after counting" ); -/* enable interrupts */ +/* enable interrupts (PMINTENSET_EL1 <= ALL_SET) */ pmu_reset_stats ();
@@ -889,6 +889,7 @@ static bool check_cycles_increase(void)bool success = true;/* init before event access, this test only cares about cycle count */ +pmu_reset ();set_pmcntenset (1 <<PMU_CYCLE_IDX );set_pmccfiltr (0 );/* count cycles in EL0, EL1, but not EL2 */
@@ -943,6 +944,7 @@ static bool check_cpi(int cpi)uint32_t pmcr =get_pmcr () |PMU_PMCR_LC |PMU_PMCR_C |PMU_PMCR_E ;/* init before event access, this test only cares about cycle count */ +pmu_reset ();set_pmcntenset (1 <<PMU_CYCLE_IDX );set_pmccfiltr (0 );/* count cycles in EL0, EL1, but not EL2 */
Signature · 1 line
2.37.1.455.g008518b4e5-goog
Re: [kvm-unit-tests PATCH v2 2/3] arm: pmu: Reset the pmu registers before starting some tests
Hi Ricardo,
Quoted 8 lines · Ricardo Koller
Some registers like the PMOVS reset to an architecturally UNKNOWN value. Most tests expect them to be reset (mostly zeroed) using
pmu_reset(). Add apmu_reset()on all the tests that need one.As a bonus, fix a couple of comments related to the register state before a sub-test.
Signed-off-by Ricardo Koller <[email protected]>
Eric
Quoted 35 lines
--- arm/pmu.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)arm/pmu.c +3
-2@@ -826,7 +826,7 @@ static void test_overflow_interrupt(void)write_regn_el0 (pmevcntr,1 ,PRE_OVERFLOW );isb (); -/* interrupts are disabled */ +/* interrupts are disabled (PMINTENSET_EL1 == 0) */ mem_access_loop (addr,200 , pmu.pmcr_ro |PMU_PMCR_E );report (expect_interrupts (0 ),"no overflow interrupt after preset" );@@ -841,7 +841,7 @@ static void test_overflow_interrupt(void)isb ();report (expect_interrupts (0 ),"no overflow interrupt after counting" ); -/* enable interrupts */ +/* enable interrupts (PMINTENSET_EL1 <= ALL_SET) */ pmu_reset_stats ();@@ -889,6 +889,7 @@ static bool check_cycles_increase(void)bool success = true;/* init before event access, this test only cares about cycle count */ +pmu_reset ();set_pmcntenset (1 <<PMU_CYCLE_IDX );set_pmccfiltr (0 );/* count cycles in EL0, EL1, but not EL2 */
arm/pmu.c static bool check_cpi(int cpi) · line 950
Quoted 7 lines
arm/pmu.c +1
-0@@ -943,6 +944,7 @@ static bool check_cpi(int cpi)uint32_t pmcr =get_pmcr () |PMU_PMCR_LC |PMU_PMCR_C |PMU_PMCR_E ;/* init before event access, this test only cares about cycle count */ +pmu_reset ();set_pmcntenset (1 <<PMU_CYCLE_IDX );set_pmccfiltr (0 );/* count cycles in EL0, EL1, but not EL2 */
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v2 3/3 arm: pmu: Check for overflow in the low counter in chained counters tests
In reply to Ricardo KollerA chained event overflowing on the low counter can set the overflow flag in PMOVS. KVM does not set it, but real HW and the fast-model seem to. Moreover, the AArch64.IncrementEventCounter() pseudocode in the ARM ARM (DDI 0487H.a, J1.1.1 "aarch64/debug") also sets the PMOVS bit on overflow.
The pmu chain tests fail on bare metal when checking the overflow flag of the low counter _not_ being set on overflow. Fix by checking for overflow. Note that this test fails in KVM without the respective fix.
--- arm/pmu.c | 33 ++++++++++++++++++--------------- 1 file changed, 18 insertions(+), 15 deletions(-)
arm/pmu.c +18 -15
@@ -583,7 +583,7 @@ static void test_chained_counters(void)precise_instrs_loop (22 , pmu.pmcr_ro |PMU_PMCR_E );report (read_regn_el0 (pmevcntr,1 ) ==1 ,"CHAIN counter #1 incremented" ); -report (!read_sysreg (pmovsclr_el0),"no overflow recorded for chained incr #1" ); +report (read_sysreg (pmovsclr_el0) ==0x1 ,"overflow recorded for chained incr #1" );/* test 64b overflow */
@@ -595,7 +595,7 @@ static void test_chained_counters(void)precise_instrs_loop (22 , pmu.pmcr_ro |PMU_PMCR_E );report_info ("overflow reg = 0x%lx" ,read_sysreg (pmovsclr_el0));report (read_regn_el0 (pmevcntr,1 ) ==2 ,"CHAIN counter #1 set to 2" ); -report (!read_sysreg (pmovsclr_el0),"no overflow recorded for chained incr #2" ); +report (read_sysreg (pmovsclr_el0) ==0x1 ,"overflow recorded for chained incr #2" );write_regn_el0 (pmevcntr,0 ,PRE_OVERFLOW );write_regn_el0 (pmevcntr,1 ,ALL_SET );
@@ -603,7 +603,7 @@ static void test_chained_counters(void)precise_instrs_loop (22 , pmu.pmcr_ro |PMU_PMCR_E );report_info ("overflow reg = 0x%lx" ,read_sysreg (pmovsclr_el0));report (!read_regn_el0 (pmevcntr,1 ),"CHAIN counter #1 wrapped" ); -report (read_sysreg (pmovsclr_el0) ==0x2 ,"overflow on chain counter" ); +report (read_sysreg (pmovsclr_el0) ==0x3 ,"overflow on even and odd counters" ); }static void test_chained_sw_incr (void )
@@ -629,8 +629,9 @@ static void test_chained_sw_incr(void)write_sysreg (0x1 , pmswinc_el0);isb (); -report (!read_sysreg (pmovsclr_el0) && (read_regn_el0 (pmevcntr,1 ) ==1 ), -"no overflow and chain counter incremented after 100 SW_INCR/CHAIN" ); +report ((read_sysreg (pmovsclr_el0) ==0x1 ) && + (read_regn_el0 (pmevcntr,1 ) ==1 ), +"overflow and chain counter incremented after 100 SW_INCR/CHAIN" );report_info ("overflow=0x%lx, #0=%ld #1=%ld" ,read_sysreg (pmovsclr_el0),read_regn_el0 (pmevcntr,0 ),read_regn_el0 (pmevcntr,1 ));
@@ -648,10 +649,10 @@ static void test_chained_sw_incr(void)write_sysreg (0x1 , pmswinc_el0);isb (); -report ((read_sysreg (pmovsclr_el0) ==0x2 ) && +report ((read_sysreg (pmovsclr_el0) ==0x3 ) && (read_regn_el0 (pmevcntr,1 ) ==0 ) && (read_regn_el0 (pmevcntr,0 ) ==84 ), -"overflow on chain counter and expected values after 100 SW_INCR/CHAIN" ); +"overflow on even and odd counters, and expected values after 100 SW_INCR/CHAIN" );report_info ("overflow=0x%lx, #0=%ld #1=%ld" ,read_sysreg (pmovsclr_el0),read_regn_el0 (pmevcntr,0 ),read_regn_el0 (pmevcntr,1 )); }
@@ -731,8 +732,9 @@ static void test_chain_promotion(void)report_info ("MEM_ACCESS counter #0 has value 0x%lx" ,read_regn_el0 (pmevcntr,0 )); -report ((read_regn_el0 (pmevcntr,1 ) ==1 ) && !read_sysreg (pmovsclr_el0), -"CHAIN counter enabled: CHAIN counter was incremented and no overflow" ); +report ((read_regn_el0 (pmevcntr,1 ) ==1 ) && + (read_sysreg (pmovsclr_el0) ==0x1 ), +"CHAIN counter enabled: CHAIN counter was incremented and overflow" );report_info ("CHAIN counter #1 = 0x%lx, overflow=0x%lx" ,read_regn_el0 (pmevcntr,1 ),read_sysreg (pmovsclr_el0));
@@ -759,8 +761,9 @@ static void test_chain_promotion(void)report_info ("MEM_ACCESS counter #0 has value 0x%lx" ,read_regn_el0 (pmevcntr,0 )); -report ((read_regn_el0 (pmevcntr,1 ) ==1 ) && !read_sysreg (pmovsclr_el0), -"32b->64b: CHAIN counter incremented and no overflow" ); +report ((read_regn_el0 (pmevcntr,1 ) ==1 ) && + (read_sysreg (pmovsclr_el0) ==0x1 ), +"32b->64b: CHAIN counter incremented and overflow" );report_info ("CHAIN counter #1 = 0x%lx, overflow=0x%lx" ,read_regn_el0 (pmevcntr,1 ),read_sysreg (pmovsclr_el0));
@@ -867,8 +870,8 @@ static void test_overflow_interrupt(void)write_regn_el0 (pmevcntr,0 ,PRE_OVERFLOW );isb ();mem_access_loop (addr,200 , pmu.pmcr_ro |PMU_PMCR_E ); -report (expect_interrupts (0 ), -"no overflow interrupt expected on 32b boundary" ); +report (expect_interrupts (0x1 ), +"expect overflow interrupt on 32b boundary" );/* overflow on odd counter */ pmu_reset_stats ();
@@ -876,8 +879,8 @@ static void test_overflow_interrupt(void)write_regn_el0 (pmevcntr,1 ,ALL_SET );isb ();mem_access_loop (addr,400 , pmu.pmcr_ro |PMU_PMCR_E ); -report (expect_interrupts (0x2 ), - "expect overflow interrupt on odd counter"); +report (expect_interrupts (0x3 ), + "expect overflow interrupt on even and odd counter"); }#endif
Signature · 1 line
2.37.1.455.g008518b4e5-goog
Re: [kvm-unit-tests PATCH v2 3/3] arm: pmu: Check for overflow in the low counter in chained counters tests
Quoted 59 lines · Ricardo Koller
A chained event overflowing on the low counter can set the overflow flag in PMOVS. KVM does not set it, but real HW and the fast-model seem to. Moreover, the
AArch64.IncrementEventCounter()pseudocode in the ARM ARM (DDI 0487H.a, J1.1.1 "aarch64/debug") also sets the PMOVS bit on overflow.The pmu chain tests fail on bare metal when checking the overflow flag of the low counter _not_ being set on overflow. Fix by checking for overflow. Note that this test fails in KVM without the respective fix.
Signed-off-by Ricardo Koller <[email protected]>--- arm/pmu.c | 33 ++++++++++++++++++--------------- 1 file changed, 18 insertions(+), 15 deletions(-)arm/pmu.c +6
-5@@ -583,7 +583,7 @@ static void test_chained_counters(void)precise_instrs_loop (22 , pmu.pmcr_ro |PMU_PMCR_E );report (read_regn_el0 (pmevcntr,1 ) ==1 ,"CHAIN counter #1 incremented" ); -report (!read_sysreg (pmovsclr_el0),"no overflow recorded for chained incr #1" ); +report (read_sysreg (pmovsclr_el0) ==0x1 ,"overflow recorded for chained incr #1" );/* test 64b overflow */ @@ -595,7 +595,7 @@ static void test_chained_counters(void)precise_instrs_loop (22 , pmu.pmcr_ro |PMU_PMCR_E );report_info ("overflow reg = 0x%lx" ,read_sysreg (pmovsclr_el0));report (read_regn_el0 (pmevcntr,1 ) ==2 ,"CHAIN counter #1 set to 2" ); -report (!read_sysreg (pmovsclr_el0),"no overflow recorded for chained incr #2" ); +report (read_sysreg (pmovsclr_el0) ==0x1 ,"overflow recorded for chained incr #2" );write_regn_el0 (pmevcntr,0 ,PRE_OVERFLOW );write_regn_el0 (pmevcntr,1 ,ALL_SET );@@ -603,7 +603,7 @@ static void test_chained_counters(void)precise_instrs_loop (22 , pmu.pmcr_ro |PMU_PMCR_E );report_info ("overflow reg = 0x%lx" ,read_sysreg (pmovsclr_el0));report (!read_regn_el0 (pmevcntr,1 ),"CHAIN counter #1 wrapped" ); -report (read_sysreg (pmovsclr_el0) ==0x2 ,"overflow on chain counter" ); +report (read_sysreg (pmovsclr_el0) ==0x3 ,"overflow on even and odd counters" ); }static void test_chained_sw_incr (void )@@ -629,8 +629,9 @@ static void test_chained_sw_incr(void)write_sysreg (0x1 , pmswinc_el0);isb (); -report (!read_sysreg (pmovsclr_el0) && (read_regn_el0 (pmevcntr,1 ) ==1 ), -"no overflow and chain counter incremented after 100 SW_INCR/CHAIN" ); +report ((read_sysreg (pmovsclr_el0) ==0x1 ) && + (read_regn_el0 (pmevcntr,1 ) ==1 ), +"overflow and chain counter incremented after 100 SW_INCR/CHAIN" );report_info ("overflow=0x%lx, #0=%ld #1=%ld" ,read_sysreg (pmovsclr_el0),read_regn_el0 (pmevcntr,0 ),read_regn_el0 (pmevcntr,1 ));
arm/pmu.c static void test_chained_sw_incr(void) · line 655
Quoted 9 lines
arm/pmu.c +2
-2@@ -648,10 +649,10 @@ static void test_chained_sw_incr(void)write_sysreg (0x1 , pmswinc_el0);isb (); -report ((read_sysreg (pmovsclr_el0) ==0x2 ) && +report ((read_sysreg (pmovsclr_el0) ==0x3 ) && (read_regn_el0 (pmevcntr,1 ) ==0 ) && (read_regn_el0 (pmevcntr,0 ) ==84 ), -"overflow on chain counter and expected values after 100 SW_INCR/CHAIN" ); +"overflow on even and odd counters, and expected values after 100 SW_INCR/CHAIN" );
Besides the extra space, this doesn't read well (to me).
Thanks,
drew
Re: [kvm-unit-tests PATCH v2 3/3] arm: pmu: Check for overflow in the low counter in chained counters tests
Quoted 62 lines · Andrew Jones
Quoted 61 lines · Ricardo Koller
A chained event overflowing on the low counter can set the overflow flag in PMOVS. KVM does not set it, but real HW and the fast-model seem to. Moreover, the
AArch64.IncrementEventCounter()pseudocode in the ARM ARM (DDI 0487H.a, J1.1.1 "aarch64/debug") also sets the PMOVS bit on overflow.The pmu chain tests fail on bare metal when checking the overflow flag of the low counter _not_ being set on overflow. Fix by checking for overflow. Note that this test fails in KVM without the respective fix.
Signed-off-by Ricardo Koller <[email protected]>--- arm/pmu.c | 33 ++++++++++++++++++--------------- 1 file changed, 18 insertions(+), 15 deletions(-)arm/pmu.c +6
-5@@ -583,7 +583,7 @@ static void test_chained_counters(void)precise_instrs_loop (22 , pmu.pmcr_ro |PMU_PMCR_E );report (read_regn_el0 (pmevcntr,1 ) ==1 ,"CHAIN counter #1 incremented" ); -report (!read_sysreg (pmovsclr_el0),"no overflow recorded for chained incr #1" ); +report (read_sysreg (pmovsclr_el0) ==0x1 ,"overflow recorded for chained incr #1" );/* test 64b overflow */ @@ -595,7 +595,7 @@ static void test_chained_counters(void)precise_instrs_loop (22 , pmu.pmcr_ro |PMU_PMCR_E );report_info ("overflow reg = 0x%lx" ,read_sysreg (pmovsclr_el0));report (read_regn_el0 (pmevcntr,1 ) ==2 ,"CHAIN counter #1 set to 2" ); -report (!read_sysreg (pmovsclr_el0),"no overflow recorded for chained incr #2" ); +report (read_sysreg (pmovsclr_el0) ==0x1 ,"overflow recorded for chained incr #2" );write_regn_el0 (pmevcntr,0 ,PRE_OVERFLOW );write_regn_el0 (pmevcntr,1 ,ALL_SET );@@ -603,7 +603,7 @@ static void test_chained_counters(void)precise_instrs_loop (22 , pmu.pmcr_ro |PMU_PMCR_E );report_info ("overflow reg = 0x%lx" ,read_sysreg (pmovsclr_el0));report (!read_regn_el0 (pmevcntr,1 ),"CHAIN counter #1 wrapped" ); -report (read_sysreg (pmovsclr_el0) ==0x2 ,"overflow on chain counter" ); +report (read_sysreg (pmovsclr_el0) ==0x3 ,"overflow on even and odd counters" ); }static void test_chained_sw_incr (void )@@ -629,8 +629,9 @@ static void test_chained_sw_incr(void)write_sysreg (0x1 , pmswinc_el0);isb (); -report (!read_sysreg (pmovsclr_el0) && (read_regn_el0 (pmevcntr,1 ) ==1 ), -"no overflow and chain counter incremented after 100 SW_INCR/CHAIN" ); +report ((read_sysreg (pmovsclr_el0) ==0x1 ) && + (read_regn_el0 (pmevcntr,1 ) ==1 ), +"overflow and chain counter incremented after 100 SW_INCR/CHAIN" );report_info ("overflow=0x%lx, #0=%ld #1=%ld" ,read_sysreg (pmovsclr_el0),read_regn_el0 (pmevcntr,0 ),read_regn_el0 (pmevcntr,1 ));@@ -648,10 +649,10 @@ static void test_chained_sw_incr(void)write_sysreg (0x1 , pmswinc_el0);
Quoted 9 lines
arm/pmu.c static void test_chained_sw_incr(void) · line 655
Quoted 7 lines
arm/pmu.c +2
-2isb (); -report ((read_sysreg (pmovsclr_el0) ==0x2 ) && +report ((read_sysreg (pmovsclr_el0) ==0x3 ) && (read_regn_el0 (pmevcntr,1 ) ==0 ) && (read_regn_el0 (pmevcntr,0 ) ==84 ), -"overflow on chain counter and expected values after 100 SW_INCR/CHAIN" ); +"overflow on even and odd counters, and expected values after 100 SW_INCR/CHAIN" );Besides the extra space, this doesn't read well (to me).
Replaced the sentence with something simpler and hopefully nicer (in v3).
Thanks,
Ricardo
Quoted 3 lines
Thanks,
drew
Re: [kvm-unit-tests PATCH v2 3/3] arm: pmu: Check for overflow in the low counter in chained counters tests
In reply to Ricardo KollerHi Ricardo,
Quoted 107 lines · Ricardo Koller
A chained event overflowing on the low counter can set the overflow flag in PMOVS. KVM does not set it, but real HW and the fast-model seem to. Moreover, the
AArch64.IncrementEventCounter()pseudocode in the ARM ARM (DDI 0487H.a, J1.1.1 "aarch64/debug") also sets the PMOVS bit on overflow.The pmu chain tests fail on bare metal when checking the overflow flag of the low counter _not_ being set on overflow. Fix by checking for overflow. Note that this test fails in KVM without the respective fix.
Signed-off-by Ricardo Koller <[email protected]>--- arm/pmu.c | 33 ++++++++++++++++++--------------- 1 file changed, 18 insertions(+), 15 deletions(-)arm/pmu.c +16
-13@@ -583,7 +583,7 @@ static void test_chained_counters(void)precise_instrs_loop (22 , pmu.pmcr_ro |PMU_PMCR_E );report (read_regn_el0 (pmevcntr,1 ) ==1 ,"CHAIN counter #1 incremented" ); -report (!read_sysreg (pmovsclr_el0),"no overflow recorded for chained incr #1" ); +report (read_sysreg (pmovsclr_el0) ==0x1 ,"overflow recorded for chained incr #1" );/* test 64b overflow */ @@ -595,7 +595,7 @@ static void test_chained_counters(void)precise_instrs_loop (22 , pmu.pmcr_ro |PMU_PMCR_E );report_info ("overflow reg = 0x%lx" ,read_sysreg (pmovsclr_el0));report (read_regn_el0 (pmevcntr,1 ) ==2 ,"CHAIN counter #1 set to 2" ); -report (!read_sysreg (pmovsclr_el0),"no overflow recorded for chained incr #2" ); +report (read_sysreg (pmovsclr_el0) ==0x1 ,"overflow recorded for chained incr #2" );write_regn_el0 (pmevcntr,0 ,PRE_OVERFLOW );write_regn_el0 (pmevcntr,1 ,ALL_SET );@@ -603,7 +603,7 @@ static void test_chained_counters(void)precise_instrs_loop (22 , pmu.pmcr_ro |PMU_PMCR_E );report_info ("overflow reg = 0x%lx" ,read_sysreg (pmovsclr_el0));report (!read_regn_el0 (pmevcntr,1 ),"CHAIN counter #1 wrapped" ); -report (read_sysreg (pmovsclr_el0) ==0x2 ,"overflow on chain counter" ); +report (read_sysreg (pmovsclr_el0) ==0x3 ,"overflow on even and odd counters" ); }static void test_chained_sw_incr (void )@@ -629,8 +629,9 @@ static void test_chained_sw_incr(void)write_sysreg (0x1 , pmswinc_el0);isb (); -report (!read_sysreg (pmovsclr_el0) && (read_regn_el0 (pmevcntr,1 ) ==1 ), -"no overflow and chain counter incremented after 100 SW_INCR/CHAIN" ); +report ((read_sysreg (pmovsclr_el0) ==0x1 ) && + (read_regn_el0 (pmevcntr,1 ) ==1 ), +"overflow and chain counter incremented after 100 SW_INCR/CHAIN" );report_info ("overflow=0x%lx, #0=%ld #1=%ld" ,read_sysreg (pmovsclr_el0),read_regn_el0 (pmevcntr,0 ),read_regn_el0 (pmevcntr,1 ));@@ -648,10 +649,10 @@ static void test_chained_sw_incr(void)write_sysreg (0x1 , pmswinc_el0);isb (); -report ((read_sysreg (pmovsclr_el0) ==0x2 ) && +report ((read_sysreg (pmovsclr_el0) ==0x3 ) && (read_regn_el0 (pmevcntr,1 ) ==0 ) && (read_regn_el0 (pmevcntr,0 ) ==84 ), -"overflow on chain counter and expected values after 100 SW_INCR/CHAIN" ); +"overflow on even and odd counters, and expected values after 100 SW_INCR/CHAIN" );report_info ("overflow=0x%lx, #0=%ld #1=%ld" ,read_sysreg (pmovsclr_el0),read_regn_el0 (pmevcntr,0 ),read_regn_el0 (pmevcntr,1 )); }@@ -731,8 +732,9 @@ static void test_chain_promotion(void)report_info ("MEM_ACCESS counter #0 has value 0x%lx" ,read_regn_el0 (pmevcntr,0 )); -report ((read_regn_el0 (pmevcntr,1 ) ==1 ) && !read_sysreg (pmovsclr_el0), -"CHAIN counter enabled: CHAIN counter was incremented and no overflow" ); +report ((read_regn_el0 (pmevcntr,1 ) ==1 ) && + (read_sysreg (pmovsclr_el0) ==0x1 ), +"CHAIN counter enabled: CHAIN counter was incremented and overflow" );report_info ("CHAIN counter #1 = 0x%lx, overflow=0x%lx" ,read_regn_el0 (pmevcntr,1 ),read_sysreg (pmovsclr_el0));@@ -759,8 +761,9 @@ static void test_chain_promotion(void)report_info ("MEM_ACCESS counter #0 has value 0x%lx" ,read_regn_el0 (pmevcntr,0 )); -report ((read_regn_el0 (pmevcntr,1 ) ==1 ) && !read_sysreg (pmovsclr_el0), -"32b->64b: CHAIN counter incremented and no overflow" ); +report ((read_regn_el0 (pmevcntr,1 ) ==1 ) && + (read_sysreg (pmovsclr_el0) ==0x1 ), +"32b->64b: CHAIN counter incremented and overflow" );report_info ("CHAIN counter #1 = 0x%lx, overflow=0x%lx" ,read_regn_el0 (pmevcntr,1 ),read_sysreg (pmovsclr_el0));@@ -867,8 +870,8 @@ static void test_overflow_interrupt(void)write_regn_el0 (pmevcntr,0 ,PRE_OVERFLOW );isb ();mem_access_loop (addr,200 , pmu.pmcr_ro |PMU_PMCR_E ); -report (expect_interrupts (0 ), -"no overflow interrupt expected on 32b boundary" ); +report (expect_interrupts (0x1 ), +"expect overflow interrupt on 32b boundary" );/* overflow on odd counter */ pmu_reset_stats ();
arm/pmu.c static void test_overflow_interrupt(void) · line 886
Quoted 10 lines
arm/pmu.c +2
-2@@ -876,8 +879,8 @@ static void test_overflow_interrupt(void)write_regn_el0 (pmevcntr,1 ,ALL_SET );isb ();mem_access_loop (addr,400 , pmu.pmcr_ro |PMU_PMCR_E ); -report (expect_interrupts (0x2 ), - "expect overflow interrupt on odd counter"); +report (expect_interrupts (0x3 ), + "expect overflow interrupt on even and odd counter"); }#endif
Besides Drew's comment and assuming Marc's fix in sync,
I definitively missed that spec detail when originally writing the test, thank you for fixing.
Eric
Marc Zyngier
maintainerRe: [kvm-unit-tests PATCH v2 3/3] arm: pmu: Check for overflow in the low counter in chained counters tests
Quoted 3 lines · Eric Auger
I definitively missed that spec detail when originally writing the test, thank you for fixing.
To be fair, it is only in a very recent version of the ARM ARM that this detail came to light, and we collectively misunderstood the spec for a few years, resulting in a over-complicated KVM infrastructure (the new emulation code is so much simpler).
BTW, I think I forgot to Cc you and Andrew on the KVM rework at [1], sorry about that.
Thanks,
M.
[1] https://lore.kernel.org/kvm/[email protected]
Signature · 1 line
Without deviation from the norm, progress is not possible.