Miscellaneous PMU fixes
cover Miscellaneous PMU fixes
This series contain several small fixes in PMU related code. The first patches is a sanity check for counter mappings in DT while rest of the patches updates mhpmevent correctly during init and counter config matching.
Atish Patra (3):
lib: sbi: Map only the counters enabled in hardware
lib: sbi: Disable interrupt during config matching
lib: sbi: Disable interrupt and inhibit counting in M-mode during init
include/sbi/riscv_encoding.h | 3 ++- lib/sbi/sbi_hart.c | 18 ++++++++++++++++++ lib/sbi/sbi_pmu.c | 16 ++++++++++++---- 3 files changed, 32 insertions(+), 5 deletions(-)
--
2.30.2
PATCH 1/3 lib: sbi: Map only the counters enabled in hardware
The counter mapping in DT may be incorrect if all the counters specified in the mapping are actually not physically present in the hardware. OpenSBI should only keep a mapping of counters enabled in hardware and defined in DT. This assume that all the programmable hpmcounters are consecutive as it doesn't make sense to build a system with sparse hpmcounters.
--- lib/sbi/sbi_pmu.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
lib/sbi/sbi_pmu.c +5 -1
@@ -181,6 +181,9 @@ static int pmu_add_hw_event_map(u32 eidx_start, u32 eidx_end, u32 cmap,int i =0 ;bool is_overlap;struct sbi_pmu_hw_event *event = &hw_event_map[num_hw_events]; +struct sbi_scratch *scratch =sbi_scratch_thishart_ptr (); +int hw_ctr_avail =sbi_hart_mhpm_count (scratch); +uint32_t ctr_avail_mask = ((uint32_t )(~0 ) >> (32 - (hw_ctr_avail +3 )));/* The first two counters are reserved by priv spec */ if (eidx_start >SBI_PMU_HW_INSTRUCTIONS && (cmap &SBI_PMU_FIXED_CTR_MASK ))
@@ -208,7 +211,8 @@ static int pmu_add_hw_event_map(u32 eidx_start, u32 eidx_end, u32 cmap, } event->select_mask = select_mask; - event->counters = cmap; +/* Map the only the counters that are available in the hardware */ + event->counters = cmap & ctr_avail_mask; event->select = select; num_hw_events++;
Signature · 1 line
2.30.2
PATCH 1/3 lib: sbi: Map only the counters enabled in hardware
Quoted 9 lines · Atish Patra
The counter mapping in DT may be incorrect if all the counters specified in the mapping are actually not physically present in the hardware. OpenSBI should only keep a mapping of counters enabled in hardware and defined in DT. This assume that all the programmable hpmcounters are consecutive as it doesn't make sense to build a system with sparse hpmcounters.
Signed-off-by Atish Patra <[email protected]>
Looks good to me.
Regards,
Anup
Quoted 31 lines
--- lib/sbi/sbi_pmu.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)lib/sbi/sbi_pmu.c +5
-2@@ -181,6 +181,9 @@ static int pmu_add_hw_event_map(u32 eidx_start, u32 eidx_end, u32 cmap,int i =0 ;bool is_overlap;struct sbi_pmu_hw_event *event = &hw_event_map[num_hw_events]; +struct sbi_scratch *scratch =sbi_scratch_thishart_ptr (); +int hw_ctr_avail =sbi_hart_mhpm_count (scratch); +uint32_t ctr_avail_mask = ((uint32_t )(~0 ) >> (32 - (hw_ctr_avail +3 )));/* The first two counters are reserved by priv spec */ if (eidx_start >SBI_PMU_HW_INSTRUCTIONS && (cmap &SBI_PMU_FIXED_CTR_MASK ))@@ -208,7 +211,8 @@ static int pmu_add_hw_event_map(u32 eidx_start, u32 eidx_end, u32 cmap, } event->select_mask = select_mask; - event->counters = cmap; +/* Map the only the counters that are available in the hardware */ + event->counters = cmap & ctr_avail_mask; event->select = select; num_hw_events++; --2.30.2
Anup Patel
AppliedPATCH 1/3 lib: sbi: Map only the counters enabled in hardware
Quoted 10 lines · Anup Patel
Quoted 8 lines · Atish Patra
The counter mapping in DT may be incorrect if all the counters specified in the mapping are actually not physically present in the hardware. OpenSBI should only keep a mapping of counters enabled in hardware and defined in DT. This assume that all the programmable hpmcounters are consecutive as it doesn't make sense to build a system with sparse hpmcounters.
Quoted 5 lines
Quoted 1 line
Signed-off-by Atish Patra <[email protected]>Looks good to me.
Reviewed-by Anup Patel <[email protected]>
Applied this patch to the riscv/openbis repo.
Regards,
Anup
Quoted 35 lines
Regards,
AnupQuoted 31 lines
--- lib/sbi/sbi_pmu.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)lib/sbi/sbi_pmu.c +5
-2@@ -181,6 +181,9 @@ static int pmu_add_hw_event_map(u32 eidx_start, u32 eidx_end, u32 cmap,int i =0 ;bool is_overlap;struct sbi_pmu_hw_event *event = &hw_event_map[num_hw_events]; +struct sbi_scratch *scratch =sbi_scratch_thishart_ptr (); +int hw_ctr_avail =sbi_hart_mhpm_count (scratch); +uint32_t ctr_avail_mask = ((uint32_t )(~0 ) >> (32 - (hw_ctr_avail +3 )));/* The first two counters are reserved by priv spec */ if (eidx_start >SBI_PMU_HW_INSTRUCTIONS && (cmap &SBI_PMU_FIXED_CTR_MASK ))@@ -208,7 +211,8 @@ static int pmu_add_hw_event_map(u32 eidx_start, u32 eidx_end, u32 cmap, } event->select_mask = select_mask; - event->counters = cmap; +/* Map the only the counters that are available in the hardware */ + event->counters = cmap & ctr_avail_mask; event->select = select; num_hw_events++; --2.30.2
PATCH 2/3 lib: sbi: Disable interrupt during config matching
In reply to Atish PatraPMU overflow interrupt should be disabled durinig initial configuration of counters. They should be enabled while starting counters.
--- include/sbi/riscv_encoding.h | 3 ++- lib/sbi/sbi_pmu.c | 10 +++++++--- 2 files changed, 9 insertions(+), 4 deletions(-)
include/sbi/riscv_encoding.h +2 -1
@@ -181,13 +181,14 @@#define MHPMEVENT_VSINH (_UL(1) << 59)#define MHPMEVENT_VUINH (_UL(1) << 58)#else -#define MHPMEVENTH_OF (_UL(1) << 31) +#define MHPMEVENTH_OF (_ULL(1) << 31)#define MHPMEVENTH_MINH (_ULL(1) << 30)#define MHPMEVENTH_SINH (_ULL(1) << 29)#define MHPMEVENTH_UINH (_ULL(1) << 28)#define MHPMEVENTH_VSINH (_ULL(1) << 27)#define MHPMEVENTH_VUINH (_ULL(1) << 26) +#define MHPMEVENT_OF (MHPMEVENTH_OF << 32)#define MHPMEVENT_MINH (MHPMEVENTH_MINH << 32)#define MHPMEVENT_SINH (MHPMEVENTH_SINH << 32)#define MHPMEVENT_UINH (MHPMEVENTH_UINH << 32)
lib/sbi/sbi_pmu.c +7 -3
@@ -256,7 +256,7 @@ static int pmu_ctr_enable_irq_hw(int ctr_idx)#if __riscv_xlen ==32 mhpmevent_csr =CSR_MHPMEVENT3H + ctr_idx -3 ; - of_mask = ~MHPMEVENTH_OF ; + of_mask = (uint32_t )~MHPMEVENTH_OF ;#else mhpmevent_csr =CSR_MHPMEVENT3 + ctr_idx -3 ; of_mask = ~MHPMEVENT_OF ;
@@ -470,9 +470,13 @@ static int pmu_update_hw_mhpmevent(struct sbi_pmu_hw_event *hw_evt, int ctr_idx,if (!mhpmevent_val || ctr_idx <3 || ctr_idx >=SBI_PMU_HW_CTR_MAX )return SBI_EFAIL ; -/* Always clear the OVF bit and inhibit countin of events in M-mode */ +/** +* Always set the OVF bit(disable interrupts) and inhibit counting of +* events in M-mode. The OVF bit should be enabled during the start call. +*/ if (sbi_hart_has_feature (scratch,SBI_HART_HAS_SSCOFPMF )) - mhpmevent_val = (mhpmevent_val & ~MHPMEVENT_SSCOF_MASK ) |MHPMEVENT_MINH ; + mhpmevent_val = (mhpmevent_val & ~MHPMEVENT_SSCOF_MASK ) | +MHPMEVENT_MINH |MHPMEVENT_OF ;/* Update the inhibit flags based on inhibit flags received from supervisor */ pmu_update_inhibit_flags (flags, &mhpmevent_val);
Signature · 1 line
2.30.2
PATCH 2/3 lib: sbi: Disable interrupt during config matching
Hello Atish!
Quoted 63 lines · On Fri, 7 Jan 2022 10:54:08 -0800 Atish Patra <[email protected]> wrote:
PMU overflow interrupt should be disabled durinig initial configuration of counters. They should be enabled while starting counters.
Fixes 730f01bb41a6 ("lib: sbi: Support sscofpmf extension inOpenSBI")
Signed-off-by Atish Patra <[email protected]>--- include/sbi/riscv_encoding.h | 3 ++- lib/sbi/sbi_pmu.c | 10 +++++++--- 2 files changed, 9 insertions(+), 4 deletions(-)(quoted diff) +0
-0
b/include/sbi/riscv_encoding.hindex8a29f8e05c9a..ac8590d72e4f100644include/sbi/riscv_encoding.h +2
-1@@ -181,13 +181,14 @@#define MHPMEVENT_VSINH (_UL(1) << 59)#define MHPMEVENT_VUINH (_UL(1) << 58)#else -#define MHPMEVENTH_OF (_UL(1) << 31) +#define MHPMEVENTH_OF (_ULL(1) << 31)#define MHPMEVENTH_MINH (_ULL(1) << 30)#define MHPMEVENTH_SINH (_ULL(1) << 29)#define MHPMEVENTH_UINH (_ULL(1) << 28)#define MHPMEVENTH_VSINH (_ULL(1) << 27)#define MHPMEVENTH_VUINH (_ULL(1) << 26) +#define MHPMEVENT_OF (MHPMEVENTH_OF << 32)#define MHPMEVENT_MINH (MHPMEVENTH_MINH << 32)#define MHPMEVENT_SINH (MHPMEVENTH_SINH << 32)#define MHPMEVENT_UINH (MHPMEVENTH_UINH << 32)lib/sbi/sbi_pmu.c +1
-1@@ -256,7 +256,7 @@ static int pmu_ctr_enable_irq_hw(int ctr_idx)#if __riscv_xlen ==32 mhpmevent_csr =CSR_MHPMEVENT3H + ctr_idx -3 ; - of_mask = ~MHPMEVENTH_OF ; + of_mask = (uint32_t )~MHPMEVENTH_OF ;#else mhpmevent_csr =CSR_MHPMEVENT3 + ctr_idx -3 ; of_mask = ~MHPMEVENT_OF ;@@ -470,9 +470,13 @@ static int pmu_update_hw_mhpmevent(struct
sbi_pmu_hw_event*hw_evt, intctr_idx, if (!mhpmevent_val ||ctr_idx< 3 ||ctr_idx>=SBI_PMU_HW_CTR_MAX) returnSBI_EFAIL;lib/sbi/sbi_pmu.c +0
-1-/* Always clear the OVF bit and inhibit countin of events in M-mode */
lib/sbi/sbi_pmu.c +2
-0+/** +* Always set the OVF bit(disable interrupts) and inhibit counting of
lib/sbi/sbi_pmu.c +1
-0+* events in M-mode. The OVF bit should be enabled during the start call.
lib/sbi/sbi_pmu.c +1
-1+*/ if (sbi_hart_has_feature (scratch,SBI_HART_HAS_SSCOFPMF )) - mhpmevent_val = (mhpmevent_val &~MHPMEVENT_SSCOF_MASK) |
MHPMEVENT_MINH;lib/sbi/sbi_pmu.c +1
-0+ mhpmevent_val = (mhpmevent_val &~MHPMEVENT_SSCOF_MASK) |
lib/sbi/sbi_pmu.c static int pmu_update_hw_mhpmevent(struct sbi_pmu_hw_event *hw_evt, int ctr_idx, · line 479
Quoted 1 line
lib/sbi/sbi_pmu.c +1
-0+MHPMEVENT_MINH |MHPMEVENT_OF ;
Is there some strong reason to inhibit M-Mode counting here or it is just a workaround until mode filtering support will come up ?
Quoted 3 lines
/* Update the inhibit flags based on inhibit flags received
from supervisor */pmu_update_inhibit_flags(flags, &mhpmevent_val);
PATCH 2/3 lib: sbi: Disable interrupt during config matching
Quoted 65 lines · Nikita Shubin
Hello Atish!
Quoted 59 lines · On Fri, 7 Jan 2022 10:54:08 -0800 Atish Patra <[email protected]> wrote:
PMU overflow interrupt should be disabled durinig initial configuration of counters. They should be enabled while starting counters.
Fixes 730f01bb41a6 ("lib: sbi: Support sscofpmf extension inOpenSBI")
Signed-off-by Atish Patra <[email protected]>--- include/sbi/riscv_encoding.h | 3 ++- lib/sbi/sbi_pmu.c | 10 +++++++--- 2 files changed, 9 insertions(+), 4 deletions(-)(quoted diff) +0
-0
b/include/sbi/riscv_encoding.hindex8a29f8e05c9a..ac8590d72e4f100644include/sbi/riscv_encoding.h +2
-1@@ -181,13 +181,14 @@#define MHPMEVENT_VSINH (_UL(1) << 59)#define MHPMEVENT_VUINH (_UL(1) << 58)#else -#define MHPMEVENTH_OF (_UL(1) << 31) +#define MHPMEVENTH_OF (_ULL(1) << 31)#define MHPMEVENTH_MINH (_ULL(1) << 30)#define MHPMEVENTH_SINH (_ULL(1) << 29)#define MHPMEVENTH_UINH (_ULL(1) << 28)#define MHPMEVENTH_VSINH (_ULL(1) << 27)#define MHPMEVENTH_VUINH (_ULL(1) << 26) +#define MHPMEVENT_OF (MHPMEVENTH_OF << 32)#define MHPMEVENT_MINH (MHPMEVENTH_MINH << 32)#define MHPMEVENT_SINH (MHPMEVENTH_SINH << 32)#define MHPMEVENT_UINH (MHPMEVENTH_UINH << 32)lib/sbi/sbi_pmu.c +1
-1@@ -256,7 +256,7 @@ static int pmu_ctr_enable_irq_hw(int ctr_idx)#if __riscv_xlen ==32 mhpmevent_csr =CSR_MHPMEVENT3H + ctr_idx -3 ; - of_mask = ~MHPMEVENTH_OF ; + of_mask = (uint32_t )~MHPMEVENTH_OF ;#else mhpmevent_csr =CSR_MHPMEVENT3 + ctr_idx -3 ; of_mask = ~MHPMEVENT_OF ;@@ -470,9 +470,13 @@ static int pmu_update_hw_mhpmevent(struct
sbi_pmu_hw_event*hw_evt, intctr_idx, if (!mhpmevent_val ||ctr_idx< 3 ||ctr_idx>=SBI_PMU_HW_CTR_MAX) returnSBI_EFAIL;- /* Always clear the OVF bit and inhibit countin of events in
M-mode */lib/sbi/sbi_pmu.c +2
-0+/** +* Always set the OVF bit(disable interrupts) and inhibit counting of
+ * events in M-mode. The OVF bit should be enabled during
the start call.
+ */
if (sbi_hart_has_feature(scratch, SBI_HART_HAS_SSCOFPMF))
Quoted 9 lines
Quoted 5 lines
-
mhpmevent_val= (mhpmevent_val&
~MHPMEVENT_SSCOF_MASK) |MHPMEVENT_MINH;
+mhpmevent_val= (mhpmevent_val&
~MHPMEVENT_SSCOF_MASK) |
+MHPMEVENT_MINH|MHPMEVENT_OF;Is there some strong reason to inhibit M-Mode counting here or it is just a workaround until mode filtering support will come up ?
As a security reason, any perf analysis shouldn't allow inspecting the M-mode for hardware counters.
Privilege mode filtering is already supported in Qemu. You can set exclude user/kernel
from perf command line for Qemu specific events (dTLB-load-misses, dTLB-store-misses, iTLB-load-misses)
Quoted 9 lines
Quoted 3 lines
/* Update the inhibit flags based on inhibit flags receivedfrom supervisor */
pmu_update_inhibit_flags(flags, &mhpmevent_val);--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi
Signature · 2 lines
Regards, Atish
PATCH 2/3 lib: sbi: Disable interrupt during config matching
Hello Atish!
Quoted 8 lines · On Sun, 9 Jan 2022 17:19:10 -0800 Atish Patra <[email protected]> wrote:
Quoted 8 lines
Quoted 7 lines
lib/sbi/sbi_pmu.c +1
-1+*/ if (sbi_hart_has_feature (scratch,SBI_HART_HAS_SSCOFPMF )) - mhpmevent_val = (mhpmevent_val &~MHPMEVENT_SSCOF_MASK) |
MHPMEVENT_MINH;
+mhpmevent_val= (mhpmevent_val&
~MHPMEVENT_SSCOF_MASK) |
+MHPMEVENT_MINH|MHPMEVENT_OF;
Quoted 7 lines
Quoted 3 lines
Is there some strong reason to inhibit M-Mode counting here or it is just a workaround until mode filtering support will come up ?
As a security reason, any perf analysis shouldn't allow inspecting the M-mode for hardware counters.
It looks a bit strange to me just to disable something that we have and it is working.
May be it's worth considering putting control of such a behavior in FW_OPTIONS or a compile option ?
I see strong benefits in being able to profile M-Mode also.
Quoted 5 lines
Privilege mode filtering is already supported in Qemu. You can set exclude user/kernel
from perf command line for Qemu specific events (dTLB-load-misses, dTLB-store-misses, iTLB-load-misses)
I see now - didn't study your v5 series carefully enough.
Quoted 13 lines
Quoted 10 lines
Quoted 4 lines
/* Update the inhibit flags based on inhibit flags receivedfrom supervisor */
pmu_update_inhibit_flags(flags, &mhpmevent_val);--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi
PATCH 2/3 lib: sbi: Disable interrupt during config matching
Quoted 18 lines · Nikita Shubin
Hello Atish!
Quoted 12 lines · On Sun, 9 Jan 2022 17:19:10 -0800 Atish Patra <[email protected]> wrote:
Quoted 11 lines
Quoted 7 lines
lib/sbi/sbi_pmu.c +1
-1+*/ if (sbi_hart_has_feature (scratch,SBI_HART_HAS_SSCOFPMF )) - mhpmevent_val = (mhpmevent_val &~MHPMEVENT_SSCOF_MASK) |
MHPMEVENT_MINH;
+mhpmevent_val= (mhpmevent_val&
~MHPMEVENT_SSCOF_MASK) |
+MHPMEVENT_MINH|MHPMEVENT_OF;Is there some strong reason to inhibit M-Mode counting here or it is just a workaround until mode filtering support will come up ?
Quoted 7 lines
Quoted 3 lines
As a security reason, any perf analysis shouldn't allow inspecting the M-mode for hardware counters.
It looks a bit strange to me just to disable something that we have and it is working.
We have privilege mode filtering working at S/U level because current perf has options for that. However, profiling the highest privilege mode(M-mode in RISC-V) is not an option for current
perf tool.
Quoted 5 lines
May be it's worth considering putting control of such a behavior in
FW_OPTIONSor a compile option ?I see strong benefits in being able to profile M-Mode also.
Yes but with an user aware of doing that. That's why firmware events are defined in the first place.
If we just enable the M-mode profiling by default without user explicitly requesting for it, the profiling data
may be inaccurate as well apart from the security concerns.
I was thinking about enabling M-mode profiling via the following route:
1. Add a firmware option to perf tool. At first iteration, it will be a RISC-V specific option.
2. If the user explicitly requests for firmware (M-mode in this case), MINH bit should be cleared.
Otherwise, it should be enabled.
We should add this feature only after the kernel feature is accepted. I am open to any other suggestions as well.
Quoted 22 lines
Quoted 5 lines
Privilege mode filtering is already supported in Qemu. You can set exclude user/kernel
from perf command line for Qemu specific events (dTLB-load-misses, dTLB-store-misses, iTLB-load-misses)I see now - didn't study your v5 series carefully enough.
Quoted 13 lines
Quoted 10 lines
Quoted 4 lines
/* Update the inhibit flags based on inhibit flags receivedfrom supervisor */
pmu_update_inhibit_flags(flags, &mhpmevent_val);--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi
Signature · 2 lines
Regards, Atish
PATCH 2/3 lib: sbi: Disable interrupt during config matching
Quoted 26 lines · On Mon, 10 Jan 2022 12:25:40 -0800 Atish Patra <[email protected]> wrote:
Quoted 24 lines · Nikita Shubin
Hello Atish!
Quoted 17 lines · On Sun, 9 Jan 2022 17:19:10 -0800 Atish Patra <[email protected]> wrote:
Quoted 10 lines
Quoted 9 lines
+ */
if (sbi_hart_has_feature(scratch,SBI_HART_HAS_SSCOFPMF))
-mhpmevent_val= (mhpmevent_val&
~MHPMEVENT_SSCOF_MASK) |MHPMEVENT_MINH;
+mhpmevent_val= (mhpmevent_val&
~MHPMEVENT_SSCOF_MASK) |
+MHPMEVENT_MINH|MHPMEVENT_OF;Quoted 3 lines
Is there some strong reason to inhibit M-Mode counting here or it is just a workaround until mode filtering support will come up ?
As a security reason, any perf analysis shouldn't allow inspecting the M-mode for hardware counters.
Quoted 8 lines
Quoted 3 lines
It looks a bit strange to me just to disable something that we have and it is working.
We have privilege mode filtering working at S/U level because current perf has options for that. However, profiling the highest privilege mode(M-mode in RISC-V) is not an option for current
perf tool.
I've already done a patch for this with new exclude_machine option, didn't sent it yet cause, it is totally useless without your pmu series merged.
Quoted 4 lines
Quoted 4 lines
May be it's worth considering putting control of such a behavior in
FW_OPTIONSor a compile option ?
Quoted 8 lines
Quoted 2 lines
I see strong benefits in being able to profile M-Mode also.
Yes but with an user aware of doing that. That's why firmware events are defined in the first place.
If we just enable the M-mode profiling by default without user explicitly requesting for it, the profiling data
may be inaccurate as well apart from the security concerns.
So inhibiting by default is really some kind of workaround.
Quoted 4 lines
I was thinking about enabling M-mode profiling via the following route:
Quoted 8 lines
1. Add a firmware option to perf tool. At first iteration, it will be a RISC-V specific option.
2. If the user explicitly requests for firmware (M-mode in this case), MINH bit should be cleared.
Otherwise, it should be enabled.We should add this feature only after the kernel feature is accepted. I am open to any other suggestions as well.
I thought an opposite way first OpenSBI and PMU v5 accepted then perf tools :).
I have something like this already, through i didn't find a good way to make it architecture specific.
Okay - i'll fire an RFC then to collect comments, option is not RISC-V specific in my version, but harmless for machines without M-Mode.
Than we can work a way to make it secure and available.
Quoted 25 lines
Quoted 23 lines
Quoted 6 lines
Privilege mode filtering is already supported in Qemu. You can set exclude user/kernel
from perf command line for Qemu specific events (dTLB-load-misses, dTLB-store-misses, iTLB-load-misses)I see now - didn't study your v5 series carefully enough.
Quoted 13 lines
Quoted 10 lines
Quoted 4 lines
/* Update the inhibit flags based on inhibit flagsreceived from supervisor */
pmu_update_inhibit_flags(flags, &mhpmevent_val);--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi
PATCH 2/3 lib: sbi: Disable interrupt during config matching
Quoted 68 lines · Nikita Shubin
Quoted 26 lines · On Mon, 10 Jan 2022 12:25:40 -0800 Atish Patra <[email protected]> wrote:
Quoted 24 lines · Nikita Shubin
Hello Atish!
Quoted 17 lines · On Sun, 9 Jan 2022 17:19:10 -0800 Atish Patra <[email protected]> wrote:
Quoted 10 lines
Quoted 9 lines
+ */
if (sbi_hart_has_feature(scratch,SBI_HART_HAS_SSCOFPMF))
-mhpmevent_val= (mhpmevent_val&
~MHPMEVENT_SSCOF_MASK) |MHPMEVENT_MINH;
+mhpmevent_val= (mhpmevent_val&
~MHPMEVENT_SSCOF_MASK) |
+MHPMEVENT_MINH|MHPMEVENT_OF;Quoted 3 lines
Is there some strong reason to inhibit M-Mode counting here or it is just a workaround until mode filtering support will come up ?
As a security reason, any perf analysis shouldn't allow inspecting the M-mode for hardware counters.
Quoted 8 lines
Quoted 3 lines
It looks a bit strange to me just to disable something that we have and it is working.
We have privilege mode filtering working at S/U level because current perf has options for that. However, profiling the highest privilege mode(M-mode in RISC-V) is not an option for current
perf tool.I've already done a patch for this with new
exclude_machineoption, didn't sent it yet cause, it is totally useless without your pmu series merged.Quoted 4 lines
Quoted 4 lines
May be it's worth considering putting control of such a behavior in
FW_OPTIONSor a compile option ?Quoted 8 lines
Quoted 2 lines
I see strong benefits in being able to profile M-Mode also.
Yes but with an user aware of doing that. That's why firmware events are defined in the first place.
If we just enable the M-mode profiling by default without user explicitly requesting for it, the profiling data
may be inaccurate as well apart from the security concerns.So inhibiting by default is really some kind of workaround.
Quoted 10 lines
I was thinking about enabling M-mode profiling via the following route:
1. Add a firmware option to perf tool. At first iteration, it will be a RISC-V specific option.
2. If the user explicitly requests for firmware (M-mode in this case), MINH bit should be cleared.
Otherwise, it should be enabled.
Quoted 6 lines
Quoted 2 lines
We should add this feature only after the kernel feature is accepted. I am open to any other suggestions as well.
I thought an opposite way first OpenSBI and PMU v5 accepted then perf tools :).
PMU series must be merged before any changes to perf tools can be merged. I am hoping the PMU series can make it in 5.18.
Regarding the OpenSBI changes, they are very small changes (few lines at best). That's why, I would at least like to see a broader agreement on the firmware option in the community.
We can patch OpenSBI after that.
Quoted 3 lines
I have something like this already, through i didn't find a good way to make it architecture specific.
Documenting is enough :).
Quoted 3 lines
Okay - i'll fire an RFC then to collect comments, option is not RISC-V specific in my version, but harmless for machines without M-Mode.
Thanks for the quick patch. I have left some comments.
Quoted 28 lines
Than we can work a way to make it secure and available.
Quoted 25 lines
Quoted 23 lines
Quoted 6 lines
Privilege mode filtering is already supported in Qemu. You can set exclude user/kernel
from perf command line for Qemu specific events (dTLB-load-misses, dTLB-store-misses, iTLB-load-misses)I see now - didn't study your v5 series carefully enough.
Quoted 13 lines
Quoted 10 lines
Quoted 4 lines
/* Update the inhibit flags based on inhibit flagsreceived from supervisor */
pmu_update_inhibit_flags(flags, &mhpmevent_val);--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi
Signature · 2 lines
Regards, Atish
PATCH 2/3 lib: sbi: Disable interrupt during config matching
Quoted 77 lines · On Tue, 11 Jan 2022 11:09:49 -0800 Atish Patra <[email protected]> wrote:
Quoted 68 lines · Nikita Shubin
Quoted 26 lines · On Mon, 10 Jan 2022 12:25:40 -0800 Atish Patra <[email protected]> wrote:
Quoted 24 lines · Nikita Shubin
Hello Atish!
Quoted 17 lines · On Sun, 9 Jan 2022 17:19:10 -0800 Atish Patra <[email protected]> wrote:
Quoted 10 lines
Quoted 9 lines
+ */
if (sbi_hart_has_feature(scratch,SBI_HART_HAS_SSCOFPMF))
-mhpmevent_val= (mhpmevent_val&
~MHPMEVENT_SSCOF_MASK) |MHPMEVENT_MINH;
+mhpmevent_val= (mhpmevent_val&
~MHPMEVENT_SSCOF_MASK) |
+MHPMEVENT_MINH|MHPMEVENT_OF;Quoted 3 lines
Is there some strong reason to inhibit M-Mode counting here or it is just a workaround until mode filtering support will come up ?
As a security reason, any perf analysis shouldn't allow inspecting the M-mode for hardware counters.
Quoted 8 lines
Quoted 3 lines
It looks a bit strange to me just to disable something that we have and it is working.
We have privilege mode filtering working at S/U level because current perf has options for that. However, profiling the highest privilege mode(M-mode in RISC-V) is not an option for current perf tool.
I've already done a patch for this with new
exclude_machineoption, didn't sent it yet cause, it is totally useless without your pmu series merged.Quoted 4 lines
Quoted 4 lines
May be it's worth considering putting control of such a behavior in
FW_OPTIONSor a compile option ?Quoted 8 lines
Quoted 2 lines
I see strong benefits in being able to profile M-Mode also.
Yes but with an user aware of doing that. That's why firmware events are defined in the first place.
If we just enable the M-mode profiling by default without user explicitly requesting for it, the profiling data
may be inaccurate as well apart from the security concerns.So inhibiting by default is really some kind of workaround.
Quoted 10 lines
I was thinking about enabling M-mode profiling via the following route:
1. Add a firmware option to perf tool. At first iteration, it will be a RISC-V specific option.
2. If the user explicitly requests for firmware (M-mode in this case), MINH bit should be cleared.
Otherwise, it should be enabled.Quoted 6 lines
Quoted 2 lines
We should add this feature only after the kernel feature is accepted. I am open to any other suggestions as well.
I thought an opposite way first OpenSBI and PMU v5 accepted then perf tools :).
Quoted 8 lines
PMU series must be merged before any changes to perf tools can be merged. I am hoping the PMU series can make it in 5.18.
Regarding the OpenSBI changes, they are very small changes (few lines at best). That's why, I would at least like to see a broader agreement on the firmware option in the community.
We can patch OpenSBI after that.
That might be the best way to go, as it seems RISC-V mail-lists are overcrowded and patches are not recieving enough attention.
Quoted 4 lines
Quoted 3 lines
I have something like this already, through i didn't find a good way to make it architecture specific.
Quoted 8 lines
Documenting is enough :).
Quoted 3 lines
Okay - i'll fire an RFC then to collect comments, option is not RISC-V specific in my version, but harmless for machines without M-Mode.
Thanks for the quick patch. I have left some comments.
Thank you for your comments and your HPM patches, i am already using then on our piece of RTL.
Quoted 30 lines
Quoted 28 lines
Than we can work a way to make it secure and available.
Quoted 25 lines
Quoted 23 lines
Quoted 6 lines
Privilege mode filtering is already supported in Qemu. You can set exclude user/kernel
from perf command line for Qemu specific events (dTLB-load-misses, dTLB-store-misses, iTLB-load-misses)I see now - didn't study your v5 series carefully enough.
Quoted 13 lines
Quoted 10 lines
Quoted 4 lines
/* Update the inhibit flags based on inhibit flagsreceived from supervisor */
pmu_update_inhibit_flags(flags, &mhpmevent_val);--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi
PATCH 2/3 lib: sbi: Disable interrupt during config matching
In reply to Atish PatraQuoted 7 lines · Atish Patra
PMU overflow interrupt should be disabled durinig initial configuration of counters. They should be enabled while starting counters.
Fixes 730f01bb41a6 ("lib: sbi: Support sscofpmf extension in OpenSBI")Signed-off-by Atish Patra <[email protected]>
Overall, this patch looks good but I agree that for debugging purposes we need an option in FW_OPTIONS which can allow monitoring events in M-mode (i.e. MINH=0). The actual behaviour of the option is a separate topic/patch.
Regards,
Anup
Quoted 57 lines
--- include/sbi/riscv_encoding.h | 3 ++- lib/sbi/sbi_pmu.c | 10 +++++++--- 2 files changed, 9 insertions(+), 4 deletions(-)include/sbi/riscv_encoding.h +2
-1@@ -181,13 +181,14 @@#define MHPMEVENT_VSINH (_UL(1) << 59)#define MHPMEVENT_VUINH (_UL(1) << 58)#else -#define MHPMEVENTH_OF (_UL(1) << 31) +#define MHPMEVENTH_OF (_ULL(1) << 31)#define MHPMEVENTH_MINH (_ULL(1) << 30)#define MHPMEVENTH_SINH (_ULL(1) << 29)#define MHPMEVENTH_UINH (_ULL(1) << 28)#define MHPMEVENTH_VSINH (_ULL(1) << 27)#define MHPMEVENTH_VUINH (_ULL(1) << 26) +#define MHPMEVENT_OF (MHPMEVENTH_OF << 32)#define MHPMEVENT_MINH (MHPMEVENTH_MINH << 32)#define MHPMEVENT_SINH (MHPMEVENTH_SINH << 32)#define MHPMEVENT_UINH (MHPMEVENTH_UINH << 32)lib/sbi/sbi_pmu.c +7
-4@@ -256,7 +256,7 @@ static int pmu_ctr_enable_irq_hw(int ctr_idx)#if __riscv_xlen ==32 mhpmevent_csr =CSR_MHPMEVENT3H + ctr_idx -3 ; - of_mask = ~MHPMEVENTH_OF ; + of_mask = (uint32_t )~MHPMEVENTH_OF ;#else mhpmevent_csr =CSR_MHPMEVENT3 + ctr_idx -3 ; of_mask = ~MHPMEVENT_OF ;@@ -470,9 +470,13 @@ static int pmu_update_hw_mhpmevent(struct sbi_pmu_hw_event *hw_evt, int ctr_idx,if (!mhpmevent_val || ctr_idx <3 || ctr_idx >=SBI_PMU_HW_CTR_MAX )return SBI_EFAIL ; -/* Always clear the OVF bit and inhibit countin of events in M-mode */ +/** +* Always set the OVF bit(disable interrupts) and inhibit counting of +* events in M-mode. The OVF bit should be enabled during the start call. +*/ if (sbi_hart_has_feature (scratch,SBI_HART_HAS_SSCOFPMF )) - mhpmevent_val = (mhpmevent_val & ~MHPMEVENT_SSCOF_MASK ) |MHPMEVENT_MINH ; + mhpmevent_val = (mhpmevent_val & ~MHPMEVENT_SSCOF_MASK ) | +MHPMEVENT_MINH |MHPMEVENT_OF ;/* Update the inhibit flags based on inhibit flags received from supervisor */ pmu_update_inhibit_flags (flags, &mhpmevent_val); --2.30.2
Anup Patel
AppliedPATCH 2/3 lib: sbi: Disable interrupt during config matching
Quoted 8 lines · Anup Patel
Quoted 6 lines · Atish Patra
PMU overflow interrupt should be disabled durinig initial configuration of counters. They should be enabled while starting counters.
Fixes 730f01bb41a6 ("lib: sbi: Support sscofpmf extension in OpenSBI")
Quoted 8 lines
Quoted 1 line
Signed-off-by Atish Patra <[email protected]>Overall, this patch looks good but I agree that for debugging purposes we need an option in
FW_OPTIONSwhich can allow monitoring events in M-mode (i.e. MINH=0). The actual behaviour of the option is a separate topic/patch.Reviewed-by Anup Patel <[email protected]>
Applied this patch to the riscv/openbis repo.
Regards,
Anup
Quoted 61 lines
Regards,
AnupQuoted 57 lines
--- include/sbi/riscv_encoding.h | 3 ++- lib/sbi/sbi_pmu.c | 10 +++++++--- 2 files changed, 9 insertions(+), 4 deletions(-)include/sbi/riscv_encoding.h +2
-1@@ -181,13 +181,14 @@#define MHPMEVENT_VSINH (_UL(1) << 59)#define MHPMEVENT_VUINH (_UL(1) << 58)#else -#define MHPMEVENTH_OF (_UL(1) << 31) +#define MHPMEVENTH_OF (_ULL(1) << 31)#define MHPMEVENTH_MINH (_ULL(1) << 30)#define MHPMEVENTH_SINH (_ULL(1) << 29)#define MHPMEVENTH_UINH (_ULL(1) << 28)#define MHPMEVENTH_VSINH (_ULL(1) << 27)#define MHPMEVENTH_VUINH (_ULL(1) << 26) +#define MHPMEVENT_OF (MHPMEVENTH_OF << 32)#define MHPMEVENT_MINH (MHPMEVENTH_MINH << 32)#define MHPMEVENT_SINH (MHPMEVENTH_SINH << 32)#define MHPMEVENT_UINH (MHPMEVENTH_UINH << 32)lib/sbi/sbi_pmu.c +7
-4@@ -256,7 +256,7 @@ static int pmu_ctr_enable_irq_hw(int ctr_idx)#if __riscv_xlen ==32 mhpmevent_csr =CSR_MHPMEVENT3H + ctr_idx -3 ; - of_mask = ~MHPMEVENTH_OF ; + of_mask = (uint32_t )~MHPMEVENTH_OF ;#else mhpmevent_csr =CSR_MHPMEVENT3 + ctr_idx -3 ; of_mask = ~MHPMEVENT_OF ;@@ -470,9 +470,13 @@ static int pmu_update_hw_mhpmevent(struct sbi_pmu_hw_event *hw_evt, int ctr_idx,if (!mhpmevent_val || ctr_idx <3 || ctr_idx >=SBI_PMU_HW_CTR_MAX )return SBI_EFAIL ; -/* Always clear the OVF bit and inhibit countin of events in M-mode */ +/** +* Always set the OVF bit(disable interrupts) and inhibit counting of +* events in M-mode. The OVF bit should be enabled during the start call. +*/ if (sbi_hart_has_feature (scratch,SBI_HART_HAS_SSCOFPMF )) - mhpmevent_val = (mhpmevent_val & ~MHPMEVENT_SSCOF_MASK ) |MHPMEVENT_MINH ; + mhpmevent_val = (mhpmevent_val & ~MHPMEVENT_SSCOF_MASK ) | +MHPMEVENT_MINH |MHPMEVENT_OF ;/* Update the inhibit flags based on inhibit flags received from supervisor */ pmu_update_inhibit_flags (flags, &mhpmevent_val); --2.30.2
PATCH 3/3 lib: sbi: Disable interrupt and inhibit counting in M-mode during init
In reply to Atish PatraCurrently, the mhpmevent CSRs are untouched during hart init during cold/warm boot. Ideally, we should clear out all the bits except overflow and MINH bit. That is required to disable overflow interrupt and inhbit counting in M-mode to avoid any spurious interrupts before perf start.
--- lib/sbi/sbi_hart.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
lib/sbi/sbi_hart.c +18 -0
@@ -40,6 +40,9 @@ static unsigned long hart_features_offset;static void mstatus_init (struct sbi_scratch *scratch) {unsigned long mstatus_val =0 ; +int cidx; +unsigned int num_mhpm =sbi_hart_mhpm_count (scratch); +uint64_t mhpmevent_init_val =0 ;/* Enable FPU */ if (misa_extension ('D' ) ||misa_extension ('F' ))
@@ -68,6 +71,21 @@ static void mstatus_init(struct sbi_scratch *scratch)if (sbi_hart_has_feature (scratch,SBI_HART_HAS_MCOUNTINHIBIT ))csr_write (CSR_MCOUNTINHIBIT ,0xFFFFFFF8 ); +/** +* The mhpmeventn[h] CSR should be initialized with interrupt disabled +* and inhibited running in M-mode during init. +* To keep it simple, only contiguous mhpmcounters are supported as a +* platform with discontiguous mhpmcounters may not make much sense. +*/ + mhpmevent_init_val |= (MHPMEVENT_OF |MHPMEVENT_MINH ); +for (cidx =0 ; cidx < num_mhpm; cidx++) { +#if __riscv_xlen ==32 +csr_write_num (CSR_MHPMEVENT3 + cidx, mhpmevent_init_val &0xFFFFFFFF ); +csr_write_num (CSR_MHPMEVENT3H + cidx, mhpmevent_init_val >>BITS_PER_LONG ); +#else +csr_write_num (CSR_MHPMEVENT3 + cidx, mhpmevent_init_val); +#endif + }/* Disable all interrupts */ csr_write (CSR_MIE ,0 );
Signature · 1 line
2.30.2
PATCH 3/3 lib: sbi: Disable interrupt and inhibit counting in M-mode during init
Quoted 5 lines · Atish Patra
Currently, the mhpmevent CSRs are untouched during hart init during cold/warm boot. Ideally, we should clear out all the bits except overflow and MINH bit. That is required to disable overflow interrupt and inhbit counting in M-mode to avoid any spurious
s/inhbit/inhibit/
Quoted 3 lines
interrupts before perf start.
Signed-off-by Atish Patra <[email protected]>
Apart from a minor comment above, it looks good to me.
Regards,
Anup
Quoted 43 lines
--- lib/sbi/sbi_hart.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)lib/sbi/sbi_hart.c +18
-1@@ -40,6 +40,9 @@ static unsigned long hart_features_offset;static void mstatus_init (struct sbi_scratch *scratch) {unsigned long mstatus_val =0 ; +int cidx; +unsigned int num_mhpm =sbi_hart_mhpm_count (scratch); +uint64_t mhpmevent_init_val =0 ;/* Enable FPU */ if (misa_extension ('D' ) ||misa_extension ('F' ))@@ -68,6 +71,21 @@ static void mstatus_init(struct sbi_scratch *scratch)if (sbi_hart_has_feature (scratch,SBI_HART_HAS_MCOUNTINHIBIT ))csr_write (CSR_MCOUNTINHIBIT ,0xFFFFFFF8 ); +/** +* The mhpmeventn[h] CSR should be initialized with interrupt disabled +* and inhibited running in M-mode during init. +* To keep it simple, only contiguous mhpmcounters are supported as a +* platform with discontiguous mhpmcounters may not make much sense. +*/ + mhpmevent_init_val |= (MHPMEVENT_OF |MHPMEVENT_MINH ); +for (cidx =0 ; cidx < num_mhpm; cidx++) { +#if __riscv_xlen ==32 +csr_write_num (CSR_MHPMEVENT3 + cidx, mhpmevent_init_val &0xFFFFFFFF ); +csr_write_num (CSR_MHPMEVENT3H + cidx, mhpmevent_init_val >>BITS_PER_LONG ); +#else +csr_write_num (CSR_MHPMEVENT3 + cidx, mhpmevent_init_val); +#endif + }/* Disable all interrupts */ csr_write (CSR_MIE ,0 ); --2.30.2
Anup Patel
AppliedPATCH 3/3 lib: sbi: Disable interrupt and inhibit counting in M-mode during init
Quoted 10 lines · Anup Patel
Quoted 5 lines · Atish Patra
Currently, the mhpmevent CSRs are untouched during hart init during cold/warm boot. Ideally, we should clear out all the bits except overflow and MINH bit. That is required to disable overflow interrupt and inhbit counting in M-mode to avoid any spurious
s/inhbit/inhibit/
Quoted 7 lines
Quoted 3 lines
interrupts before perf start.
Signed-off-by Atish Patra <[email protected]>Apart from a minor comment above, it looks good to me.
Reviewed-by Anup Patel <[email protected]>
I have taken care of the minor comment while merging this patch.
Applied this patch to the riscv/openbis repo.
Regards,
Anup
Quoted 47 lines
Regards,
AnupQuoted 43 lines
--- lib/sbi/sbi_hart.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)lib/sbi/sbi_hart.c +18
-1@@ -40,6 +40,9 @@ static unsigned long hart_features_offset;static void mstatus_init (struct sbi_scratch *scratch) {unsigned long mstatus_val =0 ; +int cidx; +unsigned int num_mhpm =sbi_hart_mhpm_count (scratch); +uint64_t mhpmevent_init_val =0 ;/* Enable FPU */ if (misa_extension ('D' ) ||misa_extension ('F' ))@@ -68,6 +71,21 @@ static void mstatus_init(struct sbi_scratch *scratch)if (sbi_hart_has_feature (scratch,SBI_HART_HAS_MCOUNTINHIBIT ))csr_write (CSR_MCOUNTINHIBIT ,0xFFFFFFF8 ); +/** +* The mhpmeventn[h] CSR should be initialized with interrupt disabled +* and inhibited running in M-mode during init. +* To keep it simple, only contiguous mhpmcounters are supported as a +* platform with discontiguous mhpmcounters may not make much sense. +*/ + mhpmevent_init_val |= (MHPMEVENT_OF |MHPMEVENT_MINH ); +for (cidx =0 ; cidx < num_mhpm; cidx++) { +#if __riscv_xlen ==32 +csr_write_num (CSR_MHPMEVENT3 + cidx, mhpmevent_init_val &0xFFFFFFFF ); +csr_write_num (CSR_MHPMEVENT3H + cidx, mhpmevent_init_val >>BITS_PER_LONG ); +#else +csr_write_num (CSR_MHPMEVENT3 + cidx, mhpmevent_init_val); +#endif + }/* Disable all interrupts */ csr_write (CSR_MIE ,0 ); --2.30.2