Skip to contentSkip to conversation

Add support for T-HEAD C9xx PMU extensions

PATCH Superseded Heiko Stuebner 3 participants 11 messages Sep 8, 2022 – Sep 25, 2022 Arm/Allwinner sunXi SoC support

v3 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v3 cover Add support for T-HEAD C9xx PMU extensions

The T-HEAD C9XX cores implement functionality very similar to SSCOFPMF. Instead of implementing a separate interface into SBI as done in v1, we can add the C9XX tidbits with quite minimal overhead and leaverage the existing SBI pmu interface including giving access to the firmware counters.

The current only hickup is the detection override in sbi_hart, where I still need to find out why the MHPMCOUNTERs are not writeable at _that_ point of the boot, but with this series on top of openSBI 1.1 and the matching kernel perf patch I get reasonable results for example for:

perf stat -e cycles -e instructions -e L1-icache-load-misses -e L1-icache-loads ls

Performance counter stats for 'ls':

          17119496      cycles
           2867765      instructions              #    0.17  insn per cycle
             55929      L1-icache-load-misses     #    1.61% of all L1-icache accesses
           3467346      L1-icache-loads
       0.030156216 seconds time elapsed

       0.000000000 seconds user
       0.023496000 seconds sys

changes in v3:
- follow Atish's advice and implement an abstraction to not pollute the core pmu with cpu-specific variants

changes in v2:
- don't implement a separate interface but instead modify the sbi-pmu to allow the c9xx to use standard pmu interface

Heiko Stuebner (5):
lib: sbi: do platform-specific extension population earlier
lib: sbi_pmu: move pmu irq information into pmu itself
lib: sbi_pmu: add ability to override methods on non-standard
platforms
platform: generic: add extensions_init handler and platform-override
platform: generic: allwinner: add support for c9xx pmu

 include/sbi/sbi_pmu.h                        |  16 +++
 lib/sbi/sbi_hart.c                           |  21 ++-
 lib/sbi/sbi_pmu.c                            |  31 +++++
 platform/generic/allwinner/sun20i-d1.c       |  76 +++++++++++
 platform/generic/include/platform_override.h |   1 +
 platform/generic/include/thead_c9xx.h        | 127 +++++++++++++++++++
 platform/generic/platform.c                  |   9 ++
 7 files changed, 274 insertions(+), 7 deletions(-)
 create mode 100644 platform/generic/include/thead_c9xx.h
Signature · 1 line
2.35.1
#1 lore raw

PATCH v3 1/5 lib: sbi: do platform-specific extension population earlier

Some of the more specific detections in hart_detect_features() might need to check platform-specific extensions so might need the platform-extensions being populated earlier.

So move the call to sbi_platform_extensions_init() to an earlier place.

---
 lib/sbi/sbi_hart.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
lib/sbi/sbi_hart.c +5 -5
index 1294868..9540e5c 100644
@@ -553,6 +553,11 @@ static int hart_detect_features(struct sbi_scratch *scratch)
 	hfeatures->pmp_count = 0;
 	hfeatures->mhpm_count = 0;
 
+	/* Let platform populate extensions */
+	rc = sbi_platform_extensions_init(sbi_platform_thishart_ptr());
+	if (rc)
+		return rc;
+
 #define __check_csr(__csr, __rdonly, __wrval, __field, __skip)	\
 	oldval = csr_read_allowed(__csr, (ulong)&trap);			\
 	if (!trap.cause) {						\
@@ -681,11 +686,6 @@ __mhpm_skip:
 					SBI_HART_EXT_SMSTATEEN, true);
 	}
 
-	/* Let platform populate extensions */
-	rc = sbi_platform_extensions_init(sbi_platform_thishart_ptr());
-	if (rc)
-		return rc;
-
 	/* Mark hart feature detection done */
 	hfeatures->detected = true;
 
Signature · 1 line
2.35.1
#2 lore raw

PATCH v3 2/5 lib: sbi_pmu: move pmu irq information into pmu itself

In reply to Heiko Stuebner

Don't spread checking for pmu extensions through the code but instead introduce a sbi-pmu function that other code can call to get the correct information about the existence of the pmu interrupt.

---
 include/sbi/sbi_pmu.h |  3 +++
 lib/sbi/sbi_hart.c    |  4 ++--
 lib/sbi/sbi_pmu.c     | 10 ++++++++++
 3 files changed, 15 insertions(+), 2 deletions(-)
include/sbi/sbi_pmu.h +3 -0
index f5e3dbd..1c9a997 100644
@@ -34,6 +34,9 @@ int sbi_pmu_init(struct sbi_scratch *scratch, bool cold_boot);
 /** Reset PMU during hart exit */
 void sbi_pmu_exit(struct sbi_scratch *scratch);
 
+/** Return the pmu irq bit depending on extension existence */
+int sbi_pmu_irq_bit(void);
+
 /**
  * Add the hardware event to counter mapping information. This should be called
  * from the platform code to update the mapping table.
lib/sbi/sbi_hart.c +2 -2
index 9540e5c..45fbcde 100644
@@ -19,6 +19,7 @@
 #include <sbi/sbi_hart.h>
 #include <sbi/sbi_math.h>
 #include <sbi/sbi_platform.h>
+#include <sbi/sbi_pmu.h>
 #include <sbi/sbi_string.h>
 #include <sbi/sbi_trap.h>
 
@@ -208,8 +209,7 @@ static int delegate_traps(struct sbi_scratch *scratch)
 
 	/* Send M-mode interrupts and most exceptions to S-mode */
 	interrupts = MIP_SSIP | MIP_STIP | MIP_SEIP;
-	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SSCOFPMF))
-		interrupts |= MIP_LCOFIP;
+	interrupts |= sbi_pmu_irq_bit();
 
 	exceptions = (1U << CAUSE_MISALIGNED_FETCH) | (1U << CAUSE_BREAKPOINT) |
 		     (1U << CAUSE_USER_ECALL);
lib/sbi/sbi_pmu.c +10 -0
index 5a294a2..efbfbb6 100644
@@ -367,6 +367,16 @@ skip_inhibit_update:
 	return 0;
 }
 
+int sbi_pmu_irq_bit(void)
+{
+	struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
+
+	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SSCOFPMF))
+		return MIP_LCOFIP;
+
+	return 0;
+}
+
 static int pmu_ctr_start_fw(uint32_t cidx, uint32_t fw_evt_code,
 			    uint64_t ival, bool ival_update)
 {
Signature · 1 line
2.35.1
#3 lore raw

PATCH v3 3/5 lib: sbi_pmu: add ability to override methods on non-standard platforms

In reply to Heiko Stuebner

Some platforms may not implement the PMU in a standard way, but instead deviate on some functionality like overflow handling.

Implement an abstraction that those platforms can hook into. This way they can still use the standard pmu interface between kernel and firmware and profit from things like firmware counters, without needing to create their own.

---
 include/sbi/sbi_pmu.h | 13 +++++++++++++
 lib/sbi/sbi_hart.c    |  7 +++++++
 lib/sbi/sbi_pmu.c     | 21 +++++++++++++++++++++
 3 files changed, 41 insertions(+)
include/sbi/sbi_pmu.h +13 -0
index 1c9a997..7c8cfe7 100644
@@ -28,6 +28,16 @@
 #define SBI_PMU_CTR_MAX	   (SBI_PMU_HW_CTR_MAX + SBI_PMU_FW_CTR_MAX)
 #define SBI_PMU_FIXED_CTR_MASK 0x07
 
+struct sbi_pmu_platform_ops {
+	int (*ctr_enable_irq_hw)(int ctr_idx);
+	int (*ctr_enable_ovf)(int ctr_idx);
+	int (*irq_bit)(void);
+	void (*counter_data)(unsigned int *count, unsigned int *bits);
+};
+
+/** Set platform-specific override ops */
+void sbi_pmu_set_platform_ops(struct sbi_pmu_platform_ops *ops);
+
 /** Initialize PMU */
 int sbi_pmu_init(struct sbi_scratch *scratch, bool cold_boot);
 
@@ -37,6 +47,9 @@ void sbi_pmu_exit(struct sbi_scratch *scratch);
 /** Return the pmu irq bit depending on extension existence */
 int sbi_pmu_irq_bit(void);
 
+/** Allow non-standard platforms to override probed counter information */
+void sbi_pmu_override_counter_data(unsigned int *count, unsigned int *bits);
+
 /**
  * Add the hardware event to counter mapping information. This should be called
  * from the platform code to update the mapping table.
lib/sbi/sbi_hart.c +7 -0
index 45fbcde..6506a19 100644
@@ -632,6 +632,13 @@ __mhpm_skip:
 #undef __check_csr_2
 #undef __check_csr
 
+	/**
+	 * Allow non-standard implementations to override the detected
+	 * values for number of counters and bits.
+	 */
+	sbi_pmu_override_counter_data(&hfeatures->mhpm_count,
+				      &hfeatures->mhpm_bits);
+
 	/* Detect if hart supports Priv v1.10 */
 	val = csr_read_allowed(CSR_MCOUNTEREN, (unsigned long)&trap);
 	if (!trap.cause)
lib/sbi/sbi_pmu.c +21 -0
index efbfbb6..08fb242 100644
@@ -77,6 +77,8 @@ static uint32_t num_hw_ctrs;
 /* Maximum number of counters available */
 static uint32_t total_ctrs;
 
+static struct sbi_pmu_platform_ops *pmu_platform_ops;
+
 /* Helper macros to retrieve event idx and code type */
 #define get_cidx_type(x) ((x & SBI_PMU_EVENT_IDX_TYPE_MASK) >> 16)
 #define get_cidx_code(x) (x & SBI_PMU_EVENT_IDX_CODE_MASK)
@@ -358,6 +360,9 @@ static int pmu_ctr_start_hw(uint32_t cidx, uint64_t ival, bool ival_update)
 
 	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SSCOFPMF))
 		pmu_ctr_enable_irq_hw(cidx);
+	else if (pmu_platform_ops && pmu_platform_ops->ctr_enable_irq_hw)
+		pmu_platform_ops->ctr_enable_irq_hw(cidx);
+
 	csr_write(CSR_MCOUNTINHIBIT, mctr_inhbt);
 
 skip_inhibit_update:
@@ -373,6 +378,8 @@ int sbi_pmu_irq_bit(void)
 
 	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SSCOFPMF))
 		return MIP_LCOFIP;
+	else if (pmu_platform_ops && pmu_platform_ops->irq_bit)
+		return pmu_platform_ops->irq_bit();
 
 	return 0;
 }
@@ -534,6 +541,8 @@ static int pmu_update_hw_mhpmevent(struct sbi_pmu_hw_event *hw_evt, int ctr_idx,
 	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SSCOFPMF))
 		mhpmevent_val = (mhpmevent_val & ~MHPMEVENT_SSCOF_MASK) |
 				 MHPMEVENT_MINH | MHPMEVENT_OF;
+	else if (pmu_platform_ops && pmu_platform_ops->ctr_enable_ovf)
+		pmu_platform_ops->ctr_enable_ovf(ctr_idx);
 
 	/* Update the inhibit flags based on inhibit flags received from supervisor */
 	pmu_update_inhibit_flags(flags, &mhpmevent_val);
@@ -822,3 +831,15 @@ int sbi_pmu_init(struct sbi_scratch *scratch, bool cold_boot)
 
 	return 0;
 }
+
+void sbi_pmu_override_counter_data(unsigned int *count,
+				   unsigned int *bits)
+{
+	if (pmu_platform_ops && pmu_platform_ops->counter_data)
+		pmu_platform_ops->counter_data(count, bits);
+}
+
+void sbi_pmu_set_platform_ops(struct sbi_pmu_platform_ops *ops)
+{
+	pmu_platform_ops = ops;
+}
Signature · 1 line
2.35.1
#4 lore raw

PATCH v3 3/5 lib: sbi_pmu: add ability to override methods on non-standard platforms

Quoted 98 lines · Heiko Stuebner

Some platforms may not implement the PMU in a standard way, but instead deviate on some functionality like overflow handling.

Implement an abstraction that those platforms can hook into. This way they can still use the standard pmu interface between kernel and firmware and profit from things like firmware counters, without needing to create their own.

---
 include/sbi/sbi_pmu.h | 13 +++++++++++++
 lib/sbi/sbi_hart.c    |  7 +++++++
 lib/sbi/sbi_pmu.c     | 21 +++++++++++++++++++++
 3 files changed, 41 insertions(+)
include/sbi/sbi_pmu.h +13 -0
index 1c9a997..7c8cfe7 100644
@@ -28,6 +28,16 @@
 #define SBI_PMU_CTR_MAX	   (SBI_PMU_HW_CTR_MAX + SBI_PMU_FW_CTR_MAX)
 #define SBI_PMU_FIXED_CTR_MASK 0x07
 
+struct sbi_pmu_platform_ops {
+	int (*ctr_enable_irq_hw)(int ctr_idx);
+	int (*ctr_enable_ovf)(int ctr_idx);
+	int (*irq_bit)(void);
+	void (*counter_data)(unsigned int *count, unsigned int *bits);
+};
+
+/** Set platform-specific override ops */
+void sbi_pmu_set_platform_ops(struct sbi_pmu_platform_ops *ops);
+
 /** Initialize PMU */
 int sbi_pmu_init(struct sbi_scratch *scratch, bool cold_boot);
 
@@ -37,6 +47,9 @@ void sbi_pmu_exit(struct sbi_scratch *scratch);
 /** Return the pmu irq bit depending on extension existence */
 int sbi_pmu_irq_bit(void);
 
+/** Allow non-standard platforms to override probed counter information */
+void sbi_pmu_override_counter_data(unsigned int *count, unsigned int *bits);
+
 /**
  * Add the hardware event to counter mapping information. This should be called
  * from the platform code to update the mapping table.
lib/sbi/sbi_hart.c +7 -0
index 45fbcde..6506a19 100644
@@ -632,6 +632,13 @@ __mhpm_skip:
 #undef __check_csr_2
 #undef __check_csr
 
+	/**
+	 * Allow non-standard implementations to override the detected
+	 * values for number of counters and bits.
+	 */
+	sbi_pmu_override_counter_data(&hfeatures->mhpm_count,
+				      &hfeatures->mhpm_bits);
+
 	/* Detect if hart supports Priv v1.10 */
 	val = csr_read_allowed(CSR_MCOUNTEREN, (unsigned long)&trap);
 	if (!trap.cause)
lib/sbi/sbi_pmu.c +7 -0
index efbfbb6..08fb242 100644
@@ -77,6 +77,8 @@ static uint32_t num_hw_ctrs;
 /* Maximum number of counters available */
 static uint32_t total_ctrs;
 
+static struct sbi_pmu_platform_ops *pmu_platform_ops;
+
 /* Helper macros to retrieve event idx and code type */
 #define get_cidx_type(x) ((x & SBI_PMU_EVENT_IDX_TYPE_MASK) >> 16)
 #define get_cidx_code(x) (x & SBI_PMU_EVENT_IDX_CODE_MASK)
@@ -358,6 +360,9 @@ static int pmu_ctr_start_hw(uint32_t cidx, uint64_t ival, bool ival_update)
 
 	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SSCOFPMF))
 		pmu_ctr_enable_irq_hw(cidx);
+	else if (pmu_platform_ops && pmu_platform_ops->ctr_enable_irq_hw)
+		pmu_platform_ops->ctr_enable_irq_hw(cidx);
+
 	csr_write(CSR_MCOUNTINHIBIT, mctr_inhbt);
 
 skip_inhibit_update:
@@ -373,6 +378,8 @@ int sbi_pmu_irq_bit(void)
 
 	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SSCOFPMF))
 		return MIP_LCOFIP;
+	else if (pmu_platform_ops && pmu_platform_ops->irq_bit)
+		return pmu_platform_ops->irq_bit();
 
 	return 0;
 }

lib/sbi/sbi_pmu.c static int pmu_update_hw_mhpmevent(struct sbi_pmu_hw_event *hw_evt, int ctr_idx, · line 546

Quoted 6 lines
lib/sbi/sbi_pmu.c +2 -0
@@ -534,6 +541,8 @@ static int pmu_update_hw_mhpmevent(struct sbi_pmu_hw_event *hw_evt, int ctr_idx,
 	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SSCOFPMF))
 		mhpmevent_val = (mhpmevent_val & ~MHPMEVENT_SSCOF_MASK) |
 				 MHPMEVENT_MINH | MHPMEVENT_OF;
+	else if (pmu_platform_ops && pmu_platform_ops->ctr_enable_ovf)
+		pmu_platform_ops->ctr_enable_ovf(ctr_idx);
 

Based on these 'if sscofpmf else pmu_platform' conditionals it looks like the pmu platform framework is actually specifically for a sscofpmf quirk. How about introducing a general quirk / pmu quirk framework and then adding a sscofpmf quirk. Then, where needed we'd have something like

  if (pmu_quirks[SSCOFPMF])
     sscofpmf_ops = (struct sscofpmf_quirk_ops *)pmu_quirks[SSCOFPMF];

  if sbi_hart_has_extension(scratch, SBI_HART_EXT_SSCOFPMF)
     ...
  else if (sscofpmf_ops && sscofpmf_ops->func_needed_here)
     sscofpmf_ops->func_needed_here(...);

Thanks,
drew

Quoted 26 lines
lib/sbi/sbi_pmu.c +12 -0
 	/* Update the inhibit flags based on inhibit flags received from supervisor */
 	pmu_update_inhibit_flags(flags, &mhpmevent_val);
@@ -822,3 +831,15 @@ int sbi_pmu_init(struct sbi_scratch *scratch, bool cold_boot)
 
 	return 0;
 }
+
+void sbi_pmu_override_counter_data(unsigned int *count,
+				   unsigned int *bits)
+{
+	if (pmu_platform_ops && pmu_platform_ops->counter_data)
+		pmu_platform_ops->counter_data(count, bits);
+}
+
+void sbi_pmu_set_platform_ops(struct sbi_pmu_platform_ops *ops)
+{
+	pmu_platform_ops = ops;
+}
Signature · 7 lines
2.35.1


-- 
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi
#5 lore raw

PATCH v3 3/5 lib: sbi_pmu: add ability to override methods on non-standard platforms

Hi Drew,

Am Donnerstag, 8. September 2022, 18:02:11 CEST schrieb Andrew Jones:

Quoted 111 lines · Heiko Stuebner
Quoted 98 lines

Some platforms may not implement the PMU in a standard way, but instead deviate on some functionality like overflow handling.

Implement an abstraction that those platforms can hook into. This way they can still use the standard pmu interface between kernel and firmware and profit from things like firmware counters, without needing to create their own.

---
 include/sbi/sbi_pmu.h | 13 +++++++++++++
 lib/sbi/sbi_hart.c    |  7 +++++++
 lib/sbi/sbi_pmu.c     | 21 +++++++++++++++++++++
 3 files changed, 41 insertions(+)
include/sbi/sbi_pmu.h +13 -0
index 1c9a997..7c8cfe7 100644
@@ -28,6 +28,16 @@
 #define SBI_PMU_CTR_MAX	   (SBI_PMU_HW_CTR_MAX + SBI_PMU_FW_CTR_MAX)
 #define SBI_PMU_FIXED_CTR_MASK 0x07
 
+struct sbi_pmu_platform_ops {
+	int (*ctr_enable_irq_hw)(int ctr_idx);
+	int (*ctr_enable_ovf)(int ctr_idx);
+	int (*irq_bit)(void);
+	void (*counter_data)(unsigned int *count, unsigned int *bits);
+};
+
+/** Set platform-specific override ops */
+void sbi_pmu_set_platform_ops(struct sbi_pmu_platform_ops *ops);
+
 /** Initialize PMU */
 int sbi_pmu_init(struct sbi_scratch *scratch, bool cold_boot);
 
@@ -37,6 +47,9 @@ void sbi_pmu_exit(struct sbi_scratch *scratch);
 /** Return the pmu irq bit depending on extension existence */
 int sbi_pmu_irq_bit(void);
 
+/** Allow non-standard platforms to override probed counter information */
+void sbi_pmu_override_counter_data(unsigned int *count, unsigned int *bits);
+
 /**
  * Add the hardware event to counter mapping information. This should be called
  * from the platform code to update the mapping table.
lib/sbi/sbi_hart.c +7 -0
index 45fbcde..6506a19 100644
@@ -632,6 +632,13 @@ __mhpm_skip:
 #undef __check_csr_2
 #undef __check_csr
 
+	/**
+	 * Allow non-standard implementations to override the detected
+	 * values for number of counters and bits.
+	 */
+	sbi_pmu_override_counter_data(&hfeatures->mhpm_count,
+				      &hfeatures->mhpm_bits);
+
 	/* Detect if hart supports Priv v1.10 */
 	val = csr_read_allowed(CSR_MCOUNTEREN, (unsigned long)&trap);
 	if (!trap.cause)
lib/sbi/sbi_pmu.c +7 -0
index efbfbb6..08fb242 100644
@@ -77,6 +77,8 @@ static uint32_t num_hw_ctrs;
 /* Maximum number of counters available */
 static uint32_t total_ctrs;
 
+static struct sbi_pmu_platform_ops *pmu_platform_ops;
+
 /* Helper macros to retrieve event idx and code type */
 #define get_cidx_type(x) ((x & SBI_PMU_EVENT_IDX_TYPE_MASK) >> 16)
 #define get_cidx_code(x) (x & SBI_PMU_EVENT_IDX_CODE_MASK)
@@ -358,6 +360,9 @@ static int pmu_ctr_start_hw(uint32_t cidx, uint64_t ival, bool ival_update)
 
 	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SSCOFPMF))
 		pmu_ctr_enable_irq_hw(cidx);
+	else if (pmu_platform_ops && pmu_platform_ops->ctr_enable_irq_hw)
+		pmu_platform_ops->ctr_enable_irq_hw(cidx);
+
 	csr_write(CSR_MCOUNTINHIBIT, mctr_inhbt);
 
 skip_inhibit_update:
@@ -373,6 +378,8 @@ int sbi_pmu_irq_bit(void)
 
 	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SSCOFPMF))
 		return MIP_LCOFIP;
+	else if (pmu_platform_ops && pmu_platform_ops->irq_bit)
+		return pmu_platform_ops->irq_bit();
 
 	return 0;
 }

lib/sbi/sbi_pmu.c static int pmu_update_hw_mhpmevent(struct sbi_pmu_hw_event *hw_evt, int ctr_idx, · line 546

Quoted 6 lines
lib/sbi/sbi_pmu.c +2 -0
@@ -534,6 +541,8 @@ static int pmu_update_hw_mhpmevent(struct sbi_pmu_hw_event *hw_evt, int ctr_idx,
 	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SSCOFPMF))
 		mhpmevent_val = (mhpmevent_val & ~MHPMEVENT_SSCOF_MASK) |
 				 MHPMEVENT_MINH | MHPMEVENT_OF;
+	else if (pmu_platform_ops && pmu_platform_ops->ctr_enable_ovf)
+		pmu_platform_ops->ctr_enable_ovf(ctr_idx);
 

Based on these 'if sscofpmf else pmu_platform' conditionals it looks like the pmu platform framework is actually specifically for a sscofpmf quirk. How about introducing a general quirk / pmu quirk framework and then adding a sscofpmf quirk. Then, where needed we'd have something like

Quoted 7 lines
  if (pmu_quirks[SSCOFPMF])
     sscofpmf_ops = (struct sscofpmf_quirk_ops *)pmu_quirks[SSCOFPMF];

  if sbi_hart_has_extension(scratch, SBI_HART_EXT_SSCOFPMF)
     ...
  else if (sscofpmf_ops && sscofpmf_ops->func_needed_here)
     sscofpmf_ops->func_needed_here(...);

as Anup wrote in his mail, the sbi-pmu meanwhile already got that struct sbi_pmu_device as abstraction from somewhere else.

Which in turn handles separately from the sscofpmf extension (similar to what I did here originally).

So for the next version I'll only use that with some extensions :-)

Heiko

Quoted 19 lines
Quoted 18 lines
lib/sbi/sbi_pmu.c +12 -0
 	/* Update the inhibit flags based on inhibit flags received from supervisor */
 	pmu_update_inhibit_flags(flags, &mhpmevent_val);
@@ -822,3 +831,15 @@ int sbi_pmu_init(struct sbi_scratch *scratch, bool cold_boot)
 
 	return 0;
 }
+
+void sbi_pmu_override_counter_data(unsigned int *count,
+				   unsigned int *bits)
+{
+	if (pmu_platform_ops && pmu_platform_ops->counter_data)
+		pmu_platform_ops->counter_data(count, bits);
+}
+
+void sbi_pmu_set_platform_ops(struct sbi_pmu_platform_ops *ops)
+{
+	pmu_platform_ops = ops;
+}
#6 lore raw

PATCH v3 4/5 platform: generic: add extensions_init handler and platform-override

In reply to Heiko Stuebner

Init of non-standard extensions is a platform-specific thing, so allow generic platforms to do this via a platform-override.

---
 platform/generic/include/platform_override.h | 1 +
 platform/generic/platform.c                  | 9 +++++++++
 2 files changed, 10 insertions(+)
platform/generic/include/platform_override.h +1 -0
index e55da25..06a26dd 100644
@@ -22,6 +22,7 @@ struct platform_override {
 	void (*early_exit)(const struct fdt_match *match);
 	void (*final_exit)(const struct fdt_match *match);
 	int (*fdt_fixup)(void *fdt, const struct fdt_match *match);
+	int (*extensions_init)(const struct fdt_match *match);
 	int (*vendor_ext_check)(long extid, const struct fdt_match *match);
 	int (*vendor_ext_provider)(long extid, long funcid,
 				   const struct sbi_trap_regs *regs,
platform/generic/platform.c +9 -0
index cc3620f..de406e3 100644
@@ -203,6 +203,14 @@ static void generic_final_exit(void)
 		generic_plat->final_exit(generic_plat_match);
 }
 
+static int generic_extensions_init(void)
+{
+	if (generic_plat && generic_plat->extensions_init)
+		return generic_plat->extensions_init(generic_plat_match);
+
+	return 0;
+}
+
 static int generic_domains_init(void)
 {
 	return fdt_domains_populate(fdt_get_address());
@@ -248,6 +256,7 @@ const struct sbi_platform_operations platform_ops = {
 	.final_init		= generic_final_init,
 	.early_exit		= generic_early_exit,
 	.final_exit		= generic_final_exit,
+	.extensions_init	= generic_extensions_init,
 	.domains_init		= generic_domains_init,
 	.console_init		= fdt_serial_init,
 	.irqchip_init		= fdt_irqchip_init,
Signature · 1 line
2.35.1
#7 lore raw

PATCH v3 4/5 platform: generic: add extensions_init handler and platform-override

Quoted 14 lines · Heiko Stuebner

Init of non-standard extensions is a platform-specific thing, so allow generic platforms to do this via a platform-override.

---
 platform/generic/include/platform_override.h | 1 +
 platform/generic/platform.c                  | 9 +++++++++
 2 files changed, 10 insertions(+)
platform/generic/include/platform_override.h +0 -0
index e55da25..06a26dd 100644

platform/generic/include/platform_override.h struct platform_override { · line 25

Quoted 4 lines
platform/generic/include/platform_override.h +1 -0
@@ -22,6 +22,7 @@ struct platform_override {
 	void (*early_exit)(const struct fdt_match *match);
 	void (*final_exit)(const struct fdt_match *match);
 	int (*fdt_fixup)(void *fdt, const struct fdt_match *match);
+	int (*extensions_init)(const struct fdt_match *match);

If we go with the quirk concept, then maybe this should be renamed to quirks_init() and it'd be where pmu_quirks[] would get initialized by the platform.

Thanks,
drew

Quoted 38 lines
platform/generic/include/platform_override.h +0 -0
 	int (*vendor_ext_check)(long extid, const struct fdt_match *match);
 	int (*vendor_ext_provider)(long extid, long funcid,
 				   const struct sbi_trap_regs *regs,
platform/generic/platform.c +9 -0
index cc3620f..de406e3 100644
@@ -203,6 +203,14 @@ static void generic_final_exit(void)
 		generic_plat->final_exit(generic_plat_match);
 }
 
+static int generic_extensions_init(void)
+{
+	if (generic_plat && generic_plat->extensions_init)
+		return generic_plat->extensions_init(generic_plat_match);
+
+	return 0;
+}
+
 static int generic_domains_init(void)
 {
 	return fdt_domains_populate(fdt_get_address());
@@ -248,6 +256,7 @@ const struct sbi_platform_operations platform_ops = {
 	.final_init		= generic_final_init,
 	.early_exit		= generic_early_exit,
 	.final_exit		= generic_final_exit,
+	.extensions_init	= generic_extensions_init,
 	.domains_init		= generic_domains_init,
 	.console_init		= fdt_serial_init,
 	.irqchip_init		= fdt_irqchip_init,
Signature · 7 lines
2.35.1


-- 
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi
#8 lore raw

PATCH v3 5/5 platform: generic: allwinner: add support for c9xx pmu

In reply to Heiko Stuebner

With the T-HEAD C9XX cores being designed before or during ratification of the SSCOFPMF extension, they implement a PMU extension that behaves very similar but not equal to it by providing overflow interrupts though in a slightly different registers format.

The sun20i-d1 is using this core. So implement the necessary overrides to allow its pmu to be used via the standard sbi-pmu extension.

For now it's also the only soc using this core, so keep the additional code in the d1-space for now.

---
 platform/generic/allwinner/sun20i-d1.c |  76 +++++++++++++++
 platform/generic/include/thead_c9xx.h  | 127 +++++++++++++++++++++++++
 2 files changed, 203 insertions(+)
 create mode 100644 platform/generic/include/thead_c9xx.h
platform/generic/allwinner/sun20i-d1.c +76 -0
index 5b2656c..55d58eb 100644
@@ -5,11 +5,13 @@
  */
 
 #include <platform_override.h>
+#include <thead_c9xx.h>
 #include <sbi/riscv_io.h>
 #include <sbi/sbi_bitops.h>
 #include <sbi/sbi_ecall_interface.h>
 #include <sbi/sbi_error.h>
 #include <sbi/sbi_hsm.h>
+#include <sbi/sbi_pmu.h>
 #include <sbi_utils/fdt/fdt_helper.h>
 #include <sbi_utils/irqchip/fdt_irqchip_plic.h>
 
@@ -199,6 +201,79 @@ static int sun20i_d1_final_init(bool cold_boot, const struct fdt_match *match)
 	return 0;
 }
 
+static int thead_c9xx_pmu_ctr_enable_irq_hw(int ctr_idx)
+{
+	unsigned long val;
+	unsigned long mip_val;
+
+	if (ctr_idx >= SBI_PMU_HW_CTR_MAX)
+		return SBI_EFAIL;
+
+	mip_val = csr_read(CSR_MIP);
+	/**
+	 * Clear out the OF bit so that next interrupt can be enabled.
+	 * This should be done only when the corresponding overflow interrupt
+	 * bit is cleared. That indicates that software has already handled the
+	 * previous interrupts or the hardware yet to set an overflow interrupt.
+	 * Otherwise, there will be race conditions where we may clear the bit
+	 * the software is yet to handle the interrupt.
+	 */
+	if (!(mip_val & THEAD_C9XX_MIP_MOIP)) {
+		val = csr_read(THEAD_C9XX_CSR_MCOUNTEROF);
+		val &= ~(1 << ctr_idx);
+		csr_write(THEAD_C9XX_CSR_MCOUNTEROF, val);
+	}
+
+	/**
+	 * SSCOFPMF uses the OF bit for enabling/disabling the interrupt,
+	 * while the C9XX has designated enable bits.
+	 * So enable per-counter interrupt on C9xx here.
+	 */
+	val = csr_read(THEAD_C9XX_CSR_MCOUNTERINTEN);
+	val |= (1 << ctr_idx);
+	csr_write(THEAD_C9XX_CSR_MCOUNTERINTEN, val);
+
+	return 0;
+}
+
+static int thead_c9xx_pmu_ctr_enable_ovf(int ctr_idx)
+{
+	unsigned long val;
+
+	val = csr_read(THEAD_C9XX_CSR_MCOUNTERINTEN);
+	val &= ~(1 << ctr_idx);
+	csr_write(THEAD_C9XX_CSR_MCOUNTERINTEN, val);
+
+	return 0;
+}
+
+static int thead_c9xx_pmu_irq_bit(void)
+{
+	return THEAD_C9XX_MIP_MOIP;
+}
+
+static void thead_c9xx_pmu_counter_data(unsigned int *count,
+					unsigned int *bits)
+{
+	/* auto-detection doesn't work on t-head c9xx cores */
+	*count = 29;
+	*bits = 64;
+}
+
+struct sbi_pmu_platform_ops thead_c9xx_pmu_platform_ops = {
+	.ctr_enable_irq_hw = thead_c9xx_pmu_ctr_enable_irq_hw,
+	.ctr_enable_ovf = thead_c9xx_pmu_ctr_enable_ovf,
+	.irq_bit = thead_c9xx_pmu_irq_bit,
+	.counter_data = thead_c9xx_pmu_counter_data,
+};
+
+static int sun20i_d1_extensions_init(const struct fdt_match *match)
+{
+	sbi_pmu_set_platform_ops(&thead_c9xx_pmu_platform_ops);
+
+	return 0;
+}
+
 static const struct fdt_match sun20i_d1_match[] = {
 	{ .compatible = "allwinner,sun20i-d1" },
 	{ },
@@ -207,4 +282,5 @@ static const struct fdt_match sun20i_d1_match[] = {
 const struct platform_override sun20i_d1 = {
 	.match_table	= sun20i_d1_match,
 	.final_init	= sun20i_d1_final_init,
+	.extensions_init = sun20i_d1_extensions_init,
 };
platform/generic/include/thead_c9xx.h new file +127 -0

127 lines not on this page · raw message

Signature · 1 line
2.35.1
#9 lore raw

v3 cover Add support for T-HEAD C9xx PMU extensions

In reply to Heiko Stuebner

Hi Heiko,

Quoted 36 lines · Heiko Stuebner

The T-HEAD C9XX cores implement functionality very similar to SSCOFPMF. Instead of implementing a separate interface into SBI as done in v1, we can add the C9XX tidbits with quite minimal overhead and leaverage the existing SBI pmu interface including giving access to the firmware counters.

The current only hickup is the detection override in sbi_hart, where I still need to find out why the MHPMCOUNTERs are not writeable at _that_ point of the boot, but with this series on top of openSBI 1.1 and the matching kernel perf patch I get reasonable results for example for:

perf stat -e cycles -e instructions -e L1-icache-load-misses -e L1-icache-loads ls

Performance counter stats for 'ls':

          17119496      cycles
           2867765      instructions              #    0.17  insn per cycle
             55929      L1-icache-load-misses     #    1.61% of all L1-icache accesses
           3467346      L1-icache-loads
       0.030156216 seconds time elapsed

       0.000000000 seconds user
       0.023496000 seconds sys

changes in v3:
- follow Atish's advice and implement an abstraction to not pollute the core pmu with cpu-specific variants

changes in v2:
- don't implement a separate interface but instead modify the sbi-pmu to allow the c9xx to use standard pmu interface

Quoted 7 lines

Heiko Stuebner (5):
lib: sbi: do platform-specific extension population earlier
lib: sbi_pmu: move pmu irq information into pmu itself
lib: sbi_pmu: add ability to override methods on non-standard
platforms
platform: generic: add extensions_init handler and platform-override
platform: generic: allwinner: add support for c9xx pmu

Please rebase this upon the latest OpenSBI. We already have "struct sbi_pmu_device" which platform can register and you can extend it in this series.

Regards,
Anup

Quoted 19 lines
 include/sbi/sbi_pmu.h                        |  16 +++
 lib/sbi/sbi_hart.c                           |  21 ++-
 lib/sbi/sbi_pmu.c                            |  31 +++++
 platform/generic/allwinner/sun20i-d1.c       |  76 +++++++++++
 platform/generic/include/platform_override.h |   1 +
 platform/generic/include/thead_c9xx.h        | 127 +++++++++++++++++++
 platform/generic/platform.c                  |   9 ++
 7 files changed, 274 insertions(+), 7 deletions(-)
 create mode 100644 platform/generic/include/thead_c9xx.h

--
2.35.1

--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

#10 lore raw

v3 cover Add support for T-HEAD C9xx PMU extensions

Am Donnerstag, 8. September 2022, 18:07:59 CEST schrieb Anup Patel:

Quoted 47 lines

Hi Heiko,

Quoted 36 lines · Heiko Stuebner

The T-HEAD C9XX cores implement functionality very similar to SSCOFPMF. Instead of implementing a separate interface into SBI as done in v1, we can add the C9XX tidbits with quite minimal overhead and leaverage the existing SBI pmu interface including giving access to the firmware counters.

The current only hickup is the detection override in sbi_hart, where I still need to find out why the MHPMCOUNTERs are not writeable at _that_ point of the boot, but with this series on top of openSBI 1.1 and the matching kernel perf patch I get reasonable results for example for:

perf stat -e cycles -e instructions -e L1-icache-load-misses -e L1-icache-loads ls

Performance counter stats for 'ls':

          17119496      cycles
           2867765      instructions              #    0.17  insn per cycle
             55929      L1-icache-load-misses     #    1.61% of all L1-icache accesses
           3467346      L1-icache-loads
       0.030156216 seconds time elapsed

       0.000000000 seconds user
       0.023496000 seconds sys

changes in v3:
- follow Atish's advice and implement an abstraction to not pollute the core pmu with cpu-specific variants

changes in v2:
- don't implement a separate interface but instead modify the sbi-pmu to allow the c9xx to use standard pmu interface

Quoted 7 lines

Heiko Stuebner (5):
lib: sbi: do platform-specific extension population earlier
lib: sbi_pmu: move pmu irq information into pmu itself
lib: sbi_pmu: add ability to override methods on non-standard
platforms
platform: generic: add extensions_init handler and platform-override
platform: generic: allwinner: add support for c9xx pmu

Quoted 3 lines

Please rebase this upon the latest OpenSBI. We already have "struct sbi_pmu_device" which platform can register and you can extend it in this series.

oh interesting, that sneaked past me it seems :-)
Will update to use this.

Thanks
Heiko

Quoted 20 lines
Quoted 19 lines
 include/sbi/sbi_pmu.h                        |  16 +++
 lib/sbi/sbi_hart.c                           |  21 ++-
 lib/sbi/sbi_pmu.c                            |  31 +++++
 platform/generic/allwinner/sun20i-d1.c       |  76 +++++++++++
 platform/generic/include/platform_override.h |   1 +
 platform/generic/include/thead_c9xx.h        | 127 +++++++++++++++++++
 platform/generic/platform.c                  |   9 ++
 7 files changed, 274 insertions(+), 7 deletions(-)
 create mode 100644 platform/generic/include/thead_c9xx.h

--
2.35.1

--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi