OpenSBI PMU improvements
cover OpenSBI PMU improvements
This series aims to improve SBI PMU implementation in following ways:
1) Optimize memory usage by reducing global data to track per-HART firmware counters (almost 80KB saved)
2) Allow platform to implement custom firmware counters and quirks related to hardware counters (This will be useful for T-Head)
3) Few other fixes for better SBI spec compliance
These patches can also be found in sbi_pmu_imp_v1 branch at:
https://github.com/avpatel/opensbi.git
Anup Patel (7):
lib: sbi_pmu: Remove "event_idx" member from struct sbi_pmu_fw_event
lib: sbi_pmu: Replace sbi_pmu_ctr_read() with sbi_pmu_ctr_fw_read()
lib: sbi_pmu: Firmware counters are always 64 bits wide
lib: sbi_pmu: Simplify FW counters to reduce memory usage
lib: sbi_pmu: Add custom PMU device operations
lib: sbi: Print platform PMU device at boot-time
include: sbi: Reduce includes in sbi_pmu.h
include/sbi/sbi_pmu.h | 67 ++++++++++-- lib/sbi/sbi_ecall_pmu.c | 3 +- lib/sbi/sbi_init.c | 4 + lib/sbi/sbi_pmu.c | 218 +++++++++++++++++++++++----------------- lib/sbi/sbi_trap.c | 1 + 5 files changed, 190 insertions(+), 103 deletions(-)
Signature · 1 line
2.34.1
PATCH 1/7 lib: sbi_pmu: Remove "event_idx" member from struct sbi_pmu_fw_event
The "event_idx" member of struct sbi_pmu_fw_event is not used anywhere so let us remove it.
--- lib/sbi/sbi_pmu.c | 3 --- 1 file changed, 3 deletions(-)
lib/sbi/sbi_pmu.c +0 -3
@@ -35,9 +35,6 @@ struct sbi_pmu_hw_event {/** Representation of a firmware event */ struct sbi_pmu_fw_event { -/* Event associated with the particular counter */ -uint32_t event_idx; -/* Current value of the counter */ unsigned long curr_count;
Signature · 1 line
2.34.1
PATCH 2/7 lib: sbi_pmu: Replace sbi_pmu_ctr_read() with sbi_pmu_ctr_fw_read()
In reply to Anup PatelThe "read a firmware counter" SBI call should only work for firmware counters so let us replace sbi_pmu_ctr_read() with sbi_pmu_ctr_fw_read() which works only on firmware counters.
--- include/sbi/sbi_pmu.h | 2 +- lib/sbi/sbi_ecall_pmu.c | 3 ++- lib/sbi/sbi_pmu.c | 45 +++++++---------------------------------- 3 files changed, 10 insertions(+), 40 deletions(-)
include/sbi/sbi_pmu.h +1 -1
@@ -53,7 +53,7 @@ int sbi_pmu_add_hw_event_counter_map(u32 eidx_start, u32 eidx_end, u32 cmap);int sbi_pmu_add_raw_event_counter_map (uint64_t select,uint64_t select_mask,u32 cmap); -int sbi_pmu_ctr_read (uint32_t cidx,unsigned long *cval); +int sbi_pmu_ctr_fw_read (uint32_t cidx,uint64_t *cval);int sbi_pmu_ctr_stop (unsigned long cidx_base,unsigned long cidx_mask,unsigned long flag);
lib/sbi/sbi_ecall_pmu.c +2 -1
@@ -51,7 +51,8 @@ static int sbi_ecall_pmu_handler(unsigned long extid, unsigned long funcid,break ;case SBI_EXT_PMU_COUNTER_FW_READ : - ret =sbi_pmu_ctr_read (regs->a0, out_val); + ret =sbi_pmu_ctr_fw_read (regs->a0, &temp); + *out_val = temp;break ;case SBI_EXT_PMU_COUNTER_START :
lib/sbi/sbi_pmu.c +7 -38
57 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH 3/7 lib: sbi_pmu: Firmware counters are always 64 bits wide
In reply to Anup PatelAs-per SBI specification, all firmware counters are always 64 bits wide so let us update the SBI PMU implementation to reflect this fact.
--- lib/sbi/sbi_pmu.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
lib/sbi/sbi_pmu.c +3 -3
@@ -36,7 +36,7 @@ struct sbi_pmu_hw_event {/** Representation of a firmware event */ struct sbi_pmu_fw_event {/* Current value of the counter */ -unsigned long curr_count; +uint64_t curr_count;/* A flag indicating pmu event monitoring is started */ bool bStarted;
@@ -719,8 +719,8 @@ int sbi_pmu_ctr_get_info(uint32_t cidx, unsigned long *ctr_info) }else {/* it's a firmware counter */ cinfo.type =SBI_PMU_CTR_TYPE_FW ; -/* Firmware counters are XLEN bits wide */ - cinfo.width =BITS_PER_LONG -1 ; +/* Firmware counters are always 64 bits wide */ + cinfo.width =63 ; } *ctr_info = cinfo.value;
Signature · 1 line
2.34.1
PATCH 4/7 lib: sbi_pmu: Simplify FW counters to reduce memory usage
In reply to Anup PatelCurrently, we have 32 elements (i.e. SBI_PMU_FW_EVENT_MAX) array of "struct sbi_pmu_fw_event" for each of 128 possible HARTs (i.e. SBI_HARTMASK_MAX_BITS).
To reduce memory usage of OpenSBI, we update FW counter implementation as follows:
1) Remove SBI_PMU_FW_EVENT_MAX
2) Remove "struct sbi_pmu_fw_event"
3) Create per-HART bitmap of XLEN bits to track FW counters which are started on each HART
4) Create per-HART uint64_t array to track values of FW counters on each HART.
--- include/sbi/sbi_pmu.h | 3 -- lib/sbi/sbi_pmu.c | 78 ++++++++++++++++++++----------------------- 2 files changed, 36 insertions(+), 45 deletions(-)
include/sbi/sbi_pmu.h +0 -3
@@ -19,9 +19,6 @@/* Maximum number of hardware events that can mapped by OpenSBI */ #define SBI_PMU_HW_EVENT_MAX 256 -/* Maximum number of firmware events that can mapped by OpenSBI */ -#define SBI_PMU_FW_EVENT_MAX 32 -/* Counter related macros */ #define SBI_PMU_FW_CTR_MAX 16#define SBI_PMU_HW_CTR_MAX 32
lib/sbi/sbi_pmu.c +36 -42
161 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH 5/7 lib: sbi_pmu: Add custom PMU device operations
In reply to Anup PatelWe extend SBI PMU implementation to allow custom PMU device operations which a platform can use for platform specific quirks.
The custom PMU device operations added by this patch include:
1) Operations to allow a platform implement custom firmware events. These custom firmware events can be SBI vendor extension related events or platform specific per-HART events are not possible to count through HPM CSRs.
2) Operations to allow a platform implement custom way for enabling (or disabling) an overflow interrupt (e.g. T-Head C9xx).
--- include/sbi/sbi_pmu.h | 57 ++++++++++++++++++++++ lib/sbi/sbi_pmu.c | 108 ++++++++++++++++++++++++++++++++++-------- 2 files changed, 144 insertions(+), 21 deletions(-)
include/sbi/sbi_pmu.h +57 -0
@@ -25,6 +25,63 @@#define SBI_PMU_CTR_MAX (SBI_PMU_HW_CTR_MAX + SBI_PMU_FW_CTR_MAX)#define SBI_PMU_FIXED_CTR_MASK 0x07 +struct sbi_pmu_device { +/** Name of the PMU platform device */ +char name[32 ]; + +/** +* Validate event code of custom firmware event +* Note: SBI_PMU_FW_MAX <= event_idx_code +*/ +int (*fw_event_validate_code)(uint32_t event_idx_code); + +/** +* Match custom firmware counter with custom firmware event +* Note: 0 <= counter_index < SBI_PMU_FW_CTR_MAX +*/ +bool (*fw_counter_match_code)(uint32_t counter_index, +uint32_t event_idx_code); + +/** +* Read value of custom firmware counter +* Note: 0 <= counter_index < SBI_PMU_FW_CTR_MAX +*/ +uint64_t (*fw_counter_read_value)(uint32_t counter_index); + +/** +* Start custom firmware counter +* Note: SBI_PMU_FW_MAX <= event_idx_code +* Note: 0 <= counter_index < SBI_PMU_FW_CTR_MAX +*/ +int (*fw_counter_start)(uint32_t counter_index, +uint32_t event_idx_code, +uint64_t init_val,bool init_val_update); + +/** +* Stop custom firmware counter +* Note: 0 <= counter_index < SBI_PMU_FW_CTR_MAX +*/ +int (*fw_counter_stop)(uint32_t counter_index); + +/** +* Custom enable irq for hardware counter +* Note: 0 <= counter_index < SBI_PMU_HW_CTR_MAX +*/ +void (*hw_counter_enable_irq)(uint32_t counter_index); + +/** +* Custom disable irq for hardware counter +* Note: 0 <= counter_index < SBI_PMU_HW_CTR_MAX +*/ +void (*hw_counter_disable_irq)(uint32_t counter_index); +}; + +/** Get the PMU platform device */ +const struct sbi_pmu_device *sbi_pmu_get_device (void ); + +/** Set the PMU platform device */ +void sbi_pmu_set_device (const struct sbi_pmu_device *dev); +/** Initialize PMU */ int sbi_pmu_init (struct sbi_scratch *scratch,bool cold_boot);
lib/sbi/sbi_pmu.c +87 -21
206 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH 6/7 lib: sbi: Print platform PMU device at boot-time
In reply to Anup PatelLet us print the platform PMU device name at the boot-time so that users know whether the underlying platform has custom per-HART PMU operations.
--- lib/sbi/sbi_init.c | 4 ++++ 1 file changed, 4 insertions(+)
lib/sbi/sbi_init.c +4 -0
@@ -63,6 +63,7 @@ static void sbi_boot_print_banner(struct sbi_scratch *scratch)static void sbi_boot_print_general (struct sbi_scratch *scratch) {char str[128 ]; +const struct sbi_pmu_device *pdev;const struct sbi_hsm_device *hdev;const struct sbi_ipi_device *idev;const struct sbi_timer_device *tdev;
@@ -93,6 +94,9 @@ static void sbi_boot_print_general(struct sbi_scratch *scratch) hdev =sbi_hsm_get_device ();sbi_printf ("Platform HSM Device : %s\n" , (hdev) ? hdev->name :"---" ); + pdev =sbi_pmu_get_device (); +sbi_printf ("Platform PMU Device : %s\n" , + (pdev) ? pdev->name :"---" ); srdev =sbi_system_reset_get_device (SBI_SRST_RESET_TYPE_COLD_REBOOT ,0 );sbi_printf ("Platform Reboot Device : %s\n" , (srdev) ? srdev->name :"---" );
Signature · 1 line
2.34.1
PATCH 7/7 include: sbi: Reduce includes in sbi_pmu.h
In reply to Anup PatelThe sbi_pmu.h should only include minimal required headers whereas sbi_pmu.c should include all required headers.
--- include/sbi/sbi_pmu.h | 5 ++--- lib/sbi/sbi_pmu.c | 2 ++ lib/sbi/sbi_trap.c | 1 + 3 files changed, 5 insertions(+), 3 deletions(-)
include/sbi/sbi_pmu.h +2 -3
@@ -11,9 +11,8 @@#define __SBI_PMU_H__#include <sbi/sbi_types.h> -#include <sbi/sbi_hartmask.h> -#include <sbi/sbi_scratch.h> -#include <sbi/sbi_ecall_interface.h> + +struct sbi_scratch ;/* Event related macros */ /* Maximum number of hardware events that can mapped by OpenSBI */
lib/sbi/sbi_pmu.c +2 -0
@@ -10,7 +10,9 @@#include <sbi/riscv_asm.h> #include <sbi/sbi_bitops.h> #include <sbi/sbi_console.h> +#include <sbi/sbi_ecall_interface.h> #include <sbi/sbi_hart.h> +#include <sbi/sbi_hartmask.h> #include <sbi/sbi_platform.h> #include <sbi/sbi_pmu.h> #include <sbi/sbi_scratch.h>
lib/sbi/sbi_trap.c +1 -0
@@ -9,6 +9,7 @@#include <sbi/riscv_asm.h> #include <sbi/riscv_encoding.h> +#include <sbi/sbi_bitops.h> #include <sbi/sbi_console.h> #include <sbi/sbi_ecall.h> #include <sbi/sbi_error.h>
Signature · 1 line
2.34.1
PATCH 5/7 lib: sbi_pmu: Add custom PMU device operations
In reply to Anup PatelQuoted 11 lines · Anup Patel
We extend SBI PMU implementation to allow custom PMU device operations which a platform can use for platform specific quirks.
The custom PMU device operations added by this patch include:
1) Operations to allow a platform implement custom firmware events. These custom firmware events can be SBI vendor extension related events or platform specific per-HART events are not possible to count through HPM CSRs.
2) Operations to allow a platform implement custom way for enabling (or disabling) an overflow interrupt (e.g. T-Head C9xx).
Quoted 6 lines
Signed-off-by Anup Patel <[email protected]>--- include/sbi/sbi_pmu.h | 57 ++++++++++++++++++++++ lib/sbi/sbi_pmu.c | 108 ++++++++++++++++++++++++++++++++++-------- 2 files changed, 144 insertions(+), 21 deletions(-)
PATCH 6/7 lib: sbi: Print platform PMU device at boot-time
In reply to a message not in the archiveQuoted 29 lines · Atish Patra
Quoted 25 lines · Anup Patel
Let us print the platform PMU device name at the boot-time so that users know whether the underlying platform has custom per-HART PMU operations.
Signed-off-by Anup Patel <[email protected]>Reviewed-by Andrew Jones <[email protected]>--- lib/sbi/sbi_init.c | 4 ++++ 1 file changed, 4 insertions(+)lib/sbi/sbi_init.c +1
-0@@ -63,6 +63,7 @@ static void sbi_boot_print_banner(struct sbi_scratch *scratch)static void sbi_boot_print_general (struct sbi_scratch *scratch) {char str[128 ]; +const struct sbi_pmu_device *pdev;const struct sbi_hsm_device *hdev;const struct sbi_ipi_device *idev;const struct sbi_timer_device *tdev;@@ -93,6 +94,9 @@ static void sbi_boot_print_general(struct sbi_scratch *scratch) hdev =sbi_hsm_get_device (); sbi_printf("Platform HSM Device : %s\n",
Quoted 8 lines
lib/sbi/sbi_init.c static void sbi_boot_print_general(struct sbi_scratch *scratch) · line 99
Quoted 4 lines
lib/sbi/sbi_init.c +3
-0(hdev) ? hdev->name : "---"); + pdev =sbi_pmu_get_device (); +sbi_printf ("Platform PMU Device : %s\n" , + (pdev) ? pdev->name :"---" );Just a nit pick thought ;)
Should we only print the entire line if pdev is present ?
In general, I agree with your suggestion but for consistency let us keep it as-is. You can send a separate patch print device for various things only if xdev != NULL.
Quoted 9 lines
Quoted 8 lines
lib/sbi/sbi_init.c +0
-1srdev =sbi_system_reset_get_device (SBI_SRST_RESET_TYPE_COLD_REBOOT ,0 );sbi_printf ("Platform Reboot Device : %s\n" , (srdev) ? srdev->name :"---" ); --2.34.1
Quoted 7 lines
Otherwise, LGTM.
Reviewed-by Atish Patra <[email protected]>--
Regards,
Atish
Regards,
Anup
Anup Patel
Appliedcover OpenSBI PMU improvements
In reply to Anup PatelQuoted 11 lines · Anup Patel
This series aims to improve SBI PMU implementation in following ways:
1) Optimize memory usage by reducing global data to track per-HART firmware counters (almost 80KB saved)
2) Allow platform to implement custom firmware counters and quirks related to hardware counters (This will be useful for T-Head)
3) Few other fixes for better SBI spec complianceThese patches can also be found in
sbi_pmu_imp_v1branch at:
https://github.com/avpatel/opensbi.git
Quoted 8 lines
Anup Patel (7):
lib:sbi_pmu: Remove "event_idx" member from structsbi_pmu_fw_event
lib:sbi_pmu: Replacesbi_pmu_ctr_read()withsbi_pmu_ctr_fw_read()
lib:sbi_pmu: Firmware counters are always 64 bits wide
lib:sbi_pmu: Simplify FW counters to reduce memory usage
lib:sbi_pmu: Add custom PMU device operations
lib: sbi: Print platform PMU device at boot-time
include: sbi: Reduce includes insbi_pmu.h
Applied this series to the riscv/opensbi repo.
Andrew (Drew) had already reviewed most of the patches before it landed on OpenSBI mailing list so I have added his Reviewed-by.
Thanks,
Anup
Quoted 11 lines
include/sbi/sbi_pmu.h | 67 ++++++++++-- lib/sbi/sbi_ecall_pmu.c | 3 +- lib/sbi/sbi_init.c | 4 + lib/sbi/sbi_pmu.c | 218 +++++++++++++++++++++++----------------- lib/sbi/sbi_trap.c | 1 + 5 files changed, 190 insertions(+), 103 deletions(-)--
2.34.1