Skip to contentSkip to conversation

HSM implementation for Allwinner D1

PATCH New Samuel Holland 3 participants 13 messages Jun 13, 2022 Arm/Allwinner sunXi SoC support

lore
Share & tools
lore

Samuel Holland

author maintainer
#0 lore raw

v3 cover HSM implementation for Allwinner D1

This series adds a minimal working HSM implementation for the Allwinner D1 SoC. It works with the Linux sbi_cpuidle driver. Even with canned values for entry/exit latency, I am seeing 30-45 mW reduction in power usage at idle.

Since this is the first HSM driver submission, there is no example to follow. I have updated the series based on the comments to the RFCs. Still, comments welcome!

Regards,
Samuel

Changes in v3:
- Fix typo in commit message
- Move comment about priority range to function declaration
- Rename irqchip_plic_* functions to fdt_plic_*
- Add a "bool smode" parameter instead of hard coding the context ID

Changes in v2:
- Rebase on top of the current master branch
- Use array_size() from sbi_types.h
- Use a separate variable for each CSR instead of an array
- Use the PLIC helpers added in earlier patches
- Use the saved SIE bits to program the wakeup interrupt mask
- Set the entry address only once during cold boot
- Use the generic implementation for retentive suspend
- Clean up/add comments throughout the HSM implementation

Samuel Holland (9):
lib: sbi_hsm: Call a device hook during hart resume
lib: sbi_hsm: Assume a consistent resume address
lib: irqchip/plic: Constify plic_data pointers
lib: irqchip/plic: Factor out a context init function
lib: irqchip/plic: Add context save/restore helpers
lib: irqchip/plic: Add priority save/restore helpers
lib: utils/irqchip: Add FDT wrappers for PLIC save/restore functions
lib: utils/irqchip: Add wrapper for T-HEAD PLIC delegation
platform: Add HSM implementation for Allwinner D1

 include/sbi/sbi_hsm.h                        |  12 +-
 include/sbi_utils/irqchip/fdt_irqchip_plic.h |  22 ++
 include/sbi_utils/irqchip/plic.h             |  20 +-
 lib/sbi/sbi_hsm.c                            |  54 ++---
 lib/utils/irqchip/fdt_irqchip_plic.c         |  39 ++++
 lib/utils/irqchip/plic.c                     | 124 ++++++++---
 platform/fpga/ariane/platform.c              |  19 +-
 platform/fpga/openpiton/platform.c           |  19 +-
 platform/generic/allwinner/objects.mk        |   6 +
 platform/generic/allwinner/sun20i-d1.c       | 210 +++++++++++++++++++
 10 files changed, 442 insertions(+), 83 deletions(-)
 create mode 100644 include/sbi_utils/irqchip/fdt_irqchip_plic.h
 create mode 100644 platform/generic/allwinner/objects.mk
 create mode 100644 platform/generic/allwinner/sun20i-d1.c
Signature · 1 line
2.35.1
author maintainer
#1 lore raw

PATCH v3 1/9 lib: sbi_hsm: Call a device hook during hart resume

Non-retentive suspend states may require platform-specific actions during resume. For example, firmware may need to save and restore the values of custom CSRs. Add a hook to support this.

---

(no changes since v1)

 include/sbi/sbi_hsm.h | 8 ++++++++
 lib/sbi/sbi_hsm.c     | 8 ++++++++
 2 files changed, 16 insertions(+)
include/sbi/sbi_hsm.h +8 -0
index c16e871..6da8cee 100644
@@ -37,6 +37,14 @@ struct sbi_hsm_device {
 	 * specified resume address
 	 */
 	int (*hart_suspend)(u32 suspend_type, ulong raddr);
+
+	/**
+	 * Perform platform-specific actions to resume from a suspended state.
+	 *
+	 * This includes restoring any platform state that was lost during
+	 * non-retentive suspend.
+	 */
+	void (*hart_resume)(void);
 };
 
 struct sbi_domain;
lib/sbi/sbi_hsm.c +8 -0
index c4d2c6d..a7b6a80 100644
@@ -178,6 +178,12 @@ static int hsm_device_hart_suspend(u32 suspend_type, ulong raddr)
 	return SBI_ENOTSUPP;
 }
 
+static void hsm_device_hart_resume(void)
+{
+	if (hsm_dev && hsm_dev->hart_resume)
+		hsm_dev->hart_resume();
+}
+
 int sbi_hsm_init(struct sbi_scratch *scratch, u32 hartid, bool cold_boot)
 {
 	u32 i;
@@ -384,6 +390,8 @@ void sbi_hsm_hart_resume_start(struct sbi_scratch *scratch)
 			   __func__, oldstate);
 		sbi_hart_hang();
 	}
+
+	hsm_device_hart_resume();
 }
 
 void sbi_hsm_hart_resume_finish(struct sbi_scratch *scratch)
Signature · 1 line
2.35.1
author maintainer
#2 lore raw

PATCH v3 2/9 lib: sbi_hsm: Assume a consistent resume address

In reply to Samuel Holland

The suspend code needs to know the resume address for two reasons:
1) Programming some hardware register or management firmware. Here we assume the hardware/firmware maintains its state between suspends, so it only needs to be programmed once at startup.
2) When a non-retentive suspend request ends up being retentive, due to lack of hardware support, pending interrupt, or for some other reason. However, the behavior here is not platform-dependent, and this can be handled in the generic hart suspend function.

Since neither situation requires the platform-level suspend function to know the resume address, stop passing it to that function. Instead, handle the non-retentive to retentive situation generically.

---

Changes in v3:
- Fix typo in commit message

Changes in v2:
- New patch for v2

 include/sbi/sbi_hsm.h |  4 ++--
 lib/sbi/sbi_hsm.c     | 46 +++++++++++++++++++------------------------
 2 files changed, 22 insertions(+), 28 deletions(-)
include/sbi/sbi_hsm.h +2 -2
index 6da8cee..d6cc468 100644
@@ -34,9 +34,9 @@ struct sbi_hsm_device {
 	 * the hart resumes normal execution.
 	 *
 	 * For successful non-retentive suspend, the hart will resume from
-	 * specified resume address
+	 * the warm boot entry point.
 	 */
-	int (*hart_suspend)(u32 suspend_type, ulong raddr);
+	int (*hart_suspend)(u32 suspend_type);
 
 	/**
 	 * Perform platform-specific actions to resume from a suspended state.
lib/sbi/sbi_hsm.c +20 -26

77 lines not on this page · raw message

Signature · 1 line
2.35.1
author maintainer
#3 lore raw

PATCH v3 3/9 lib: irqchip/plic: Constify plic_data pointers

In reply to Samuel Holland

None of the functions modify the passed-in plic_data, so mark it const.

---

(no changes since v2)

Changes in v2:
- New patch for v2

 include/sbi_utils/irqchip/plic.h |  9 +++++----
 lib/utils/irqchip/plic.c         | 11 ++++++-----
 2 files changed, 11 insertions(+), 9 deletions(-)
include/sbi_utils/irqchip/plic.h +5 -4
index 0e56d80..50be5df 100644
@@ -17,13 +17,14 @@ struct plic_data {
 	unsigned long num_src;
 };
 
-int plic_warm_irqchip_init(struct plic_data *plic,
+int plic_warm_irqchip_init(const struct plic_data *plic,
 			   int m_cntx_id, int s_cntx_id);
 
-int plic_cold_irqchip_init(struct plic_data *plic);
+int plic_cold_irqchip_init(const struct plic_data *plic);
 
-void plic_set_thresh(struct plic_data *plic, u32 cntxid, u32 val);
+void plic_set_thresh(const struct plic_data *plic, u32 cntxid, u32 val);
 
-void plic_set_ie(struct plic_data *plic, u32 cntxid, u32 word_index, u32 val);
+void plic_set_ie(const struct plic_data *plic, u32 cntxid,
+		 u32 word_index, u32 val);
 
 #endif
lib/utils/irqchip/plic.c +6 -5
index 0f58f3e..4c334ec 100644
@@ -21,14 +21,14 @@
 #define PLIC_CONTEXT_BASE 0x200000
 #define PLIC_CONTEXT_STRIDE 0x1000
 
-static void plic_set_priority(struct plic_data *plic, u32 source, u32 val)
+static void plic_set_priority(const struct plic_data *plic, u32 source, u32 val)
 {
 	volatile void *plic_priority = (char *)plic->addr +
 			PLIC_PRIORITY_BASE + 4 * source;
 	writel(val, plic_priority);
 }
 
-void plic_set_thresh(struct plic_data *plic, u32 cntxid, u32 val)
+void plic_set_thresh(const struct plic_data *plic, u32 cntxid, u32 val)
 {
 	volatile void *plic_thresh;
 
@@ -40,7 +40,8 @@ void plic_set_thresh(struct plic_data *plic, u32 cntxid, u32 val)
 	writel(val, plic_thresh);
 }
 
-void plic_set_ie(struct plic_data *plic, u32 cntxid, u32 word_index, u32 val)
+void plic_set_ie(const struct plic_data *plic, u32 cntxid,
+		 u32 word_index, u32 val)
 {
 	volatile char *plic_ie;
 
@@ -52,7 +53,7 @@ void plic_set_ie(struct plic_data *plic, u32 cntxid, u32 word_index, u32 val)
 	writel(val, plic_ie + word_index * 4);
 }
 
-int plic_warm_irqchip_init(struct plic_data *plic,
+int plic_warm_irqchip_init(const struct plic_data *plic,
 			   int m_cntx_id, int s_cntx_id)
 {
 	size_t i, ie_words;
@@ -85,7 +86,7 @@ int plic_warm_irqchip_init(struct plic_data *plic,
 	return 0;
 }
 
-int plic_cold_irqchip_init(struct plic_data *plic)
+int plic_cold_irqchip_init(const struct plic_data *plic)
 {
 	int i;
 
Signature · 1 line
2.35.1
author maintainer
#4 lore raw

PATCH v3 4/9 lib: irqchip/plic: Factor out a context init function

In reply to Samuel Holland

This simplifies both the callers and the callees by removing duplicated code and consolidating the error handling. It also fixes two bugs in the process:
1) ie_words was one too large when plic->num_src was a multiple of 32.
2) plic_set_ie takes a 32-bit mask, not a Boolean value, so the FPGA platforms previously only enabled one out of every 32 interrupts.

---

(no changes since v2)

Changes in v2:
- New patch for v2

 include/sbi_utils/irqchip/plic.h   |  8 ++---
 lib/utils/irqchip/plic.c           | 55 ++++++++++++++++--------------
 platform/fpga/ariane/platform.c    | 19 +++++------
 platform/fpga/openpiton/platform.c | 19 +++++------
 4 files changed, 48 insertions(+), 53 deletions(-)
include/sbi_utils/irqchip/plic.h +3 -5
index 50be5df..8f21af6 100644
@@ -17,14 +17,12 @@ struct plic_data {
 	unsigned long num_src;
 };
 
+int plic_context_init(const struct plic_data *plic, int context_id,
+		      bool enable, u32 threshold);
+
 int plic_warm_irqchip_init(const struct plic_data *plic,
 			   int m_cntx_id, int s_cntx_id);
 
 int plic_cold_irqchip_init(const struct plic_data *plic);
 
-void plic_set_thresh(const struct plic_data *plic, u32 cntxid, u32 val);
-
-void plic_set_ie(const struct plic_data *plic, u32 cntxid,
-		 u32 word_index, u32 val);
-
 #endif
lib/utils/irqchip/plic.c +29 -26

96 lines not on this page · raw message

platform/fpga/ariane/platform.c +8 -11
index 3fdc03b..56a666b 100644
@@ -99,24 +99,21 @@ static int ariane_console_init(void)
 
 static int plic_ariane_warm_irqchip_init(int m_cntx_id, int s_cntx_id)
 {
-	size_t i, ie_words = ARIANE_PLIC_NUM_SOURCES / 32 + 1;
+	int ret;
 
 	/* By default, enable all IRQs for M-mode of target HART */
 	if (m_cntx_id > -1) {
-		for (i = 0; i < ie_words; i++)
-			plic_set_ie(&plic, m_cntx_id, i, 1);
+		ret = plic_context_init(&plic, m_cntx_id, true, 0x1);
+		if (ret)
+			return ret;
 	}
+
 	/* Enable all IRQs for S-mode of target HART */
 	if (s_cntx_id > -1) {
-		for (i = 0; i < ie_words; i++)
-			plic_set_ie(&plic, s_cntx_id, i, 1);
+		ret = plic_context_init(&plic, s_cntx_id, true, 0x0);
+		if (ret)
+			return ret;
 	}
-	/* By default, enable M-mode threshold */
-	if (m_cntx_id > -1)
-		plic_set_thresh(&plic, m_cntx_id, 1);
-	/* By default, disable S-mode threshold */
-	if (s_cntx_id > -1)
-		plic_set_thresh(&plic, s_cntx_id, 0);
 
 	return 0;
 }
platform/fpga/openpiton/platform.c +8 -11
index 15e289a..7ca2123 100644
@@ -134,24 +134,21 @@ static int openpiton_console_init(void)
 
 static int plic_openpiton_warm_irqchip_init(int m_cntx_id, int s_cntx_id)
 {
-	size_t i, ie_words = plic.num_src / 32 + 1;
+	int ret;
 
 	/* By default, enable all IRQs for M-mode of target HART */
 	if (m_cntx_id > -1) {
-		for (i = 0; i < ie_words; i++)
-			plic_set_ie(&plic, m_cntx_id, i, 1);
+		ret = plic_context_init(&plic, m_cntx_id, true, 0x1);
+		if (ret)
+			return ret;
 	}
+
 	/* Enable all IRQs for S-mode of target HART */
 	if (s_cntx_id > -1) {
-		for (i = 0; i < ie_words; i++)
-			plic_set_ie(&plic, s_cntx_id, i, 1);
+		ret = plic_context_init(&plic, s_cntx_id, true, 0x0);
+		if (ret)
+			return ret;
 	}
-	/* By default, enable M-mode threshold */
-	if (m_cntx_id > -1)
-		plic_set_thresh(&plic, m_cntx_id, 1);
-	/* By default, disable S-mode threshold */
-	if (s_cntx_id > -1)
-		plic_set_thresh(&plic, s_cntx_id, 0);
 
 	return 0;
 }
Signature · 1 line
2.35.1
author maintainer
#5 lore raw

PATCH v3 5/9 lib: irqchip/plic: Add context save/restore helpers

In reply to Samuel Holland

These can be used by platform code to save the PLIC context state, if it would otherwise be lost during non-retentive suspend. The platform is responsible for allocating all necessary storage.

---

(no changes since v2)

Changes in v2:
- New patch for v2

 include/sbi_utils/irqchip/plic.h |  6 ++++
 lib/utils/irqchip/plic.c         | 51 ++++++++++++++++++++++++++++++--
 2 files changed, 54 insertions(+), 3 deletions(-)
include/sbi_utils/irqchip/plic.h +6 -0
index 8f21af6..21b2266 100644
@@ -17,6 +17,12 @@ struct plic_data {
 	unsigned long num_src;
 };
 
+void plic_context_save(const struct plic_data *plic, int context_id,
+		       u32 *enable, u32 *threshold);
+
+void plic_context_restore(const struct plic_data *plic, int context_id,
+			  const u32 *enable, u32 threshold);
+
 int plic_context_init(const struct plic_data *plic, int context_id,
 		      bool enable, u32 threshold);
 
lib/utils/irqchip/plic.c +48 -3

68 lines not on this page · raw message

Signature · 1 line
2.35.1
author maintainer
#6 lore raw

PATCH v3 6/9 lib: irqchip/plic: Add priority save/restore helpers

In reply to Samuel Holland

These can be used by platform code to save the PLIC priority state, if it would otherwise be lost during non-retentive suspend. The platform is responsible for allocating all necessary storage.

As a space optimization, store the saved priority values as 8-bit integers, since that is large enough to hold any priority value on the relevant platforms.

---

Changes in v3:
- Move comment about priority range to function declaration

Changes in v2:
- New patch for v2

 include/sbi_utils/irqchip/plic.h |  5 +++++
 lib/utils/irqchip/plic.c         | 19 +++++++++++++++++++
 2 files changed, 24 insertions(+)
include/sbi_utils/irqchip/plic.h +5 -0
index 21b2266..48c24f0 100644
@@ -17,6 +17,11 @@ struct plic_data {
 	unsigned long num_src;
 };
 
+/* So far, priorities on all consumers of these functions fit in 8 bits. */
+void plic_priority_save(const struct plic_data *plic, u8 *priority);
+
+void plic_priority_restore(const struct plic_data *plic, const u8 *priority);
+
 void plic_context_save(const struct plic_data *plic, int context_id,
 		       u32 *enable, u32 *threshold);
 
lib/utils/irqchip/plic.c +19 -0
index 0c64078..8c25b9d 100644
@@ -22,6 +22,13 @@
 #define PLIC_CONTEXT_BASE 0x200000
 #define PLIC_CONTEXT_STRIDE 0x1000
 
+static u32 plic_get_priority(const struct plic_data *plic, u32 source)
+{
+	volatile void *plic_priority = (char *)plic->addr +
+			PLIC_PRIORITY_BASE + 4 * source;
+	return readl(plic_priority);
+}
+
 static void plic_set_priority(const struct plic_data *plic, u32 source, u32 val)
 {
 	volatile void *plic_priority = (char *)plic->addr +
@@ -29,6 +36,18 @@ static void plic_set_priority(const struct plic_data *plic, u32 source, u32 val)
 	writel(val, plic_priority);
 }
 
+void plic_priority_save(const struct plic_data *plic, u8 *priority)
+{
+	for (u32 i = 0; i < plic->num_src; i++)
+		priority[i] = plic_get_priority(plic, i);
+}
+
+void plic_priority_restore(const struct plic_data *plic, const u8 *priority)
+{
+	for (u32 i = 0; i < plic->num_src; i++)
+		plic_set_priority(plic, i, priority[i]);
+}
+
 static u32 plic_get_thresh(const struct plic_data *plic, u32 cntxid)
 {
 	volatile void *plic_thresh;
Signature · 1 line
2.35.1
author maintainer
#7 lore raw

PATCH v3 7/9 lib: utils/irqchip: Add FDT wrappers for PLIC save/restore functions

In reply to Samuel Holland

These functions save/restore the state of the PLIC associated with the current hart. The context save/restore functions only manipulate a single context, since most likely the M-mode context is unused and does not need to be saved.

---

Changes in v3:
- Rename irqchip_plic_* functions to fdt_plic_*
- Add a "bool smode" parameter instead of hard coding the context ID

Changes in v2:
- New patch for v2

 include/sbi_utils/irqchip/fdt_irqchip_plic.h | 20 ++++++++++++
 lib/utils/irqchip/fdt_irqchip_plic.c         | 32 ++++++++++++++++++++
 2 files changed, 52 insertions(+)
 create mode 100644 include/sbi_utils/irqchip/fdt_irqchip_plic.h
include/sbi_utils/irqchip/fdt_irqchip_plic.h new file +20 -0
new file mode 100644
index 0000000..f158b02
@@ -0,0 +1,20 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2022 Samuel Holland <[email protected]>
+ */
+
+#ifndef __IRQCHIP_FDT_IRQCHIP_PLIC_H__
+#define __IRQCHIP_FDT_IRQCHIP_PLIC_H__
+
+#include <sbi/sbi_types.h>
+
+void fdt_plic_priority_save(u8 *priority);
+
+void fdt_plic_priority_restore(const u8 *priority);
+
+void fdt_plic_context_save(bool smode, u32 *enable, u32 *threshold);
+
+void fdt_plic_context_restore(bool smode, const u32 *enable, u32 threshold);
+
+#endif
lib/utils/irqchip/fdt_irqchip_plic.c +32 -0
index b2d2a1b..89b12d4 100644
@@ -24,6 +24,38 @@ static struct plic_data plic[PLIC_MAX_NR];
 static struct plic_data *plic_hartid2data[SBI_HARTMASK_MAX_BITS];
 static int plic_hartid2context[SBI_HARTMASK_MAX_BITS][2];
 
+void fdt_plic_priority_save(u8 *priority)
+{
+	struct plic_data *plic = plic_hartid2data[current_hartid()];
+
+	plic_priority_save(plic, priority);
+}
+
+void fdt_plic_priority_restore(const u8 *priority)
+{
+	struct plic_data *plic = plic_hartid2data[current_hartid()];
+
+	plic_priority_restore(plic, priority);
+}
+
+void fdt_plic_context_save(bool smode, u32 *enable, u32 *threshold)
+{
+	u32 hartid = current_hartid();
+
+	plic_context_save(plic_hartid2data[hartid],
+			  plic_hartid2context[hartid][smode],
+			  enable, threshold);
+}
+
+void fdt_plic_context_restore(bool smode, const u32 *enable, u32 threshold)
+{
+	u32 hartid = current_hartid();
+
+	plic_context_restore(plic_hartid2data[hartid],
+			     plic_hartid2context[hartid][smode],
+			     enable, threshold);
+}
+
 static int irqchip_plic_warm_init(void)
 {
 	u32 hartid = current_hartid();
Signature · 1 line
2.35.1
#8 lore raw

PATCH v3 7/9 lib: utils/irqchip: Add FDT wrappers for PLIC save/restore functions

Quoted 7 lines · Samuel Holland

These functions save/restore the state of the PLIC associated with the current hart. The context save/restore functions only manipulate a single context, since most likely the M-mode context is unused and does not need to be saved.

Looks good to me.

Regards,
Anup

Quoted 91 lines

---

Changes in v3:
- Rename irqchip_plic_* functions to fdt_plic_*
- Add a "bool smode" parameter instead of hard coding the context ID

Changes in v2:
- New patch for v2

 include/sbi_utils/irqchip/fdt_irqchip_plic.h | 20 ++++++++++++
 lib/utils/irqchip/fdt_irqchip_plic.c         | 32 ++++++++++++++++++++
 2 files changed, 52 insertions(+)
 create mode 100644 include/sbi_utils/irqchip/fdt_irqchip_plic.h
include/sbi_utils/irqchip/fdt_irqchip_plic.h new file +20 -0
new file mode 100644
index 0000000..f158b02
@@ -0,0 +1,20 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2022 Samuel Holland <[email protected]>
+ */
+
+#ifndef __IRQCHIP_FDT_IRQCHIP_PLIC_H__
+#define __IRQCHIP_FDT_IRQCHIP_PLIC_H__
+
+#include <sbi/sbi_types.h>
+
+void fdt_plic_priority_save(u8 *priority);
+
+void fdt_plic_priority_restore(const u8 *priority);
+
+void fdt_plic_context_save(bool smode, u32 *enable, u32 *threshold);
+
+void fdt_plic_context_restore(bool smode, const u32 *enable, u32 threshold);
+
+#endif
lib/utils/irqchip/fdt_irqchip_plic.c +32 -1
index b2d2a1b..89b12d4 100644
@@ -24,6 +24,38 @@ static struct plic_data plic[PLIC_MAX_NR];
 static struct plic_data *plic_hartid2data[SBI_HARTMASK_MAX_BITS];
 static int plic_hartid2context[SBI_HARTMASK_MAX_BITS][2];
 
+void fdt_plic_priority_save(u8 *priority)
+{
+       struct plic_data *plic = plic_hartid2data[current_hartid()];
+
+       plic_priority_save(plic, priority);
+}
+
+void fdt_plic_priority_restore(const u8 *priority)
+{
+       struct plic_data *plic = plic_hartid2data[current_hartid()];
+
+       plic_priority_restore(plic, priority);
+}
+
+void fdt_plic_context_save(bool smode, u32 *enable, u32 *threshold)
+{
+       u32 hartid = current_hartid();
+
+       plic_context_save(plic_hartid2data[hartid],
+                         plic_hartid2context[hartid][smode],
+                         enable, threshold);
+}
+
+void fdt_plic_context_restore(bool smode, const u32 *enable, u32 threshold)
+{
+       u32 hartid = current_hartid();
+
+       plic_context_restore(plic_hartid2data[hartid],
+                            plic_hartid2context[hartid][smode],
+                            enable, threshold);
+}
+
 static int irqchip_plic_warm_init(void)
 {
        u32 hartid = current_hartid();
--

2.35.1

--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

author maintainer
#9 lore raw

PATCH v3 8/9 lib: utils/irqchip: Add wrapper for T-HEAD PLIC delegation

In reply to Samuel Holland

The delegation bit is lost along with the rest of the PLIC state when the CPU power domain in the Allwinner D1 is powered down, so the PLIC needs to be re-delegated to S-mode during the hart resume path.

---

(no changes since v2)

Changes in v2:
- New patch for v2

 include/sbi_utils/irqchip/fdt_irqchip_plic.h | 2 ++
 lib/utils/irqchip/fdt_irqchip_plic.c         | 7 +++++++
 2 files changed, 9 insertions(+)
include/sbi_utils/irqchip/fdt_irqchip_plic.h +2 -0
index f158b02..98d4de5 100644
@@ -17,4 +17,6 @@ void fdt_plic_context_save(bool smode, u32 *enable, u32 *threshold);
 
 void fdt_plic_context_restore(bool smode, const u32 *enable, u32 threshold);
 
+void thead_plic_restore(void);
+
 #endif
lib/utils/irqchip/fdt_irqchip_plic.c +7 -0
index 89b12d4..a6e185c 100644
@@ -151,6 +151,13 @@ static void thead_plic_plat_init(struct plic_data *pd)
 	writel_relaxed(BIT(0), (char *)pd->addr + THEAD_PLIC_CTRL_REG);
 }
 
+void thead_plic_restore(void)
+{
+	struct plic_data *plic = plic_hartid2data[current_hartid()];
+
+	thead_plic_plat_init(plic);
+}
+
 static const struct fdt_match irqchip_plic_match[] = {
 	{ .compatible = "riscv,plic0" },
 	{ .compatible = "sifive,plic-1.0.0" },
Signature · 1 line
2.35.1
author maintainer
#10 lore raw

PATCH v3 9/9 platform: Add HSM implementation for Allwinner D1

In reply to Samuel Holland

Allwinner D1 contains a "PPU" power domain controller which can automatically power down/up the CPU power domain. This power domain includes the C906 core along with its CLINT and PLIC.

This HSM implementation supports non-retentive hart suspend by:
1) Saving/restoring state that is lost during hart suspend,
2) Performing cache maintenance before/after hart suspend,
3) Configuring wakeup sources before hart suspend, and
4) Asking the PPU to power down the hart when it enters WFI.

Since this HSM implementation is for a single-core SoC, it does not need to worry about concurrency or saving multiple instances of state.

---

(no changes since v2)

Changes in v2:
- Rebase on top of the current master branch
- Use array_size() from sbi_types.h
- Use a separate variable for each CSR instead of an array
- Use the PLIC helpers added in earlier patches
- Use the saved SIE bits to program the wakeup interrupt mask
- Set the entry address only once during cold boot
- Use the generic implementation for retentive suspend
- Clean up/add comments throughout the HSM implementation

 platform/generic/allwinner/objects.mk  |   6 +
 platform/generic/allwinner/sun20i-d1.c | 210 +++++++++++++++++++++++++
 2 files changed, 216 insertions(+)
 create mode 100644 platform/generic/allwinner/objects.mk
 create mode 100644 platform/generic/allwinner/sun20i-d1.c
platform/generic/allwinner/objects.mk new file +6 -0
new file mode 100644
index 0000000..9e36ab6
@@ -0,0 +1,6 @@
+#
+# SPDX-License-Identifier: BSD-2-Clause
+#
+
+carray-platform_override_modules-y += sun20i_d1
+platform-objs-y += allwinner/sun20i-d1.o
platform/generic/allwinner/sun20i-d1.c new file +210 -0

210 lines not on this page · raw message

Signature · 1 line
2.35.1
#11 lore raw

PATCH v3 9/9 platform: Add HSM implementation for Allwinner D1

Quoted 198 lines · Samuel Holland

Allwinner D1 contains a "PPU" power domain controller which can automatically power down/up the CPU power domain. This power domain includes the C906 core along with its CLINT and PLIC.

This HSM implementation supports non-retentive hart suspend by:
1) Saving/restoring state that is lost during hart suspend,
2) Performing cache maintenance before/after hart suspend,
3) Configuring wakeup sources before hart suspend, and
4) Asking the PPU to power down the hart when it enters WFI.

Since this HSM implementation is for a single-core SoC, it does not need to worry about concurrency or saving multiple instances of state.

---

(no changes since v2)

Changes in v2:
- Rebase on top of the current master branch
- Use array_size() from sbi_types.h
- Use a separate variable for each CSR instead of an array
- Use the PLIC helpers added in earlier patches
- Use the saved SIE bits to program the wakeup interrupt mask
- Set the entry address only once during cold boot
- Use the generic implementation for retentive suspend
- Clean up/add comments throughout the HSM implementation

 platform/generic/allwinner/objects.mk  |   6 +
 platform/generic/allwinner/sun20i-d1.c | 210 +++++++++++++++++++++++++
 2 files changed, 216 insertions(+)
 create mode 100644 platform/generic/allwinner/objects.mk
 create mode 100644 platform/generic/allwinner/sun20i-d1.c
platform/generic/allwinner/objects.mk new file +6 -0
new file mode 100644
index 0000000..9e36ab6
@@ -0,0 +1,6 @@
+#
+# SPDX-License-Identifier: BSD-2-Clause
+#
+
+carray-platform_override_modules-y += sun20i_d1
+platform-objs-y += allwinner/sun20i-d1.o
platform/generic/allwinner/sun20i-d1.c new file +144 -0
new file mode 100644
index 0000000..2de12e4
@@ -0,0 +1,210 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2022 Samuel Holland <[email protected]>
+ */
+
+#include <platform_override.h>
+#include <sbi/riscv_io.h>
+#include <sbi/sbi_bitops.h>
+#include <sbi/sbi_ecall_interface.h>
+#include <sbi/sbi_error.h>
+#include <sbi/sbi_hsm.h>
+#include <sbi_utils/fdt/fdt_helper.h>
+#include <sbi_utils/irqchip/fdt_irqchip_plic.h>
+
+#define SUN20I_D1_CCU_BASE             ((void *)0x02001000)
+#define SUN20I_D1_RISCV_CFG_BASE       ((void *)0x06010000)
+#define SUN20I_D1_PPU_BASE             ((void *)0x07001000)
+#define SUN20I_D1_PRCM_BASE            ((void *)0x07010000)
+
+/*
+ * CCU
+ */
+
+#define CCU_BGR_ENABLE                 (BIT(16) | BIT(0))
+
+#define RISCV_CFG_BGR_REG              0xd0c
+#define PPU_BGR_REG                    0x1ac
+
+/*
+ * CSRs
+ */
+
+#define CSR_MXSTATUS                   0x7c0
+#define CSR_MHCR                       0x7c1
+#define CSR_MCOR                       0x7c2
+#define CSR_MHINT                      0x7c5
+
+static unsigned long csr_mxstatus;
+static unsigned long csr_mhcr;
+static unsigned long csr_mhint;
+
+static void sun20i_d1_csr_save(void)
+{
+       /* Save custom CSRs. */
+       csr_mxstatus    = csr_read(CSR_MXSTATUS);
+       csr_mhcr        = csr_read(CSR_MHCR);
+       csr_mhint       = csr_read(CSR_MHINT);
+
+       /* Flush and disable caches. */
+       csr_write(CSR_MCOR, 0x22);
+       csr_write(CSR_MHCR, 0x0);
+}
+
+static void sun20i_d1_csr_restore(void)
+{
+       /* Invalidate caches and the branch predictor. */
+       csr_write(CSR_MCOR, 0x70013);
+
+       /* Restore custom CSRs, including the cache state. */
+       csr_write(CSR_MXSTATUS, csr_mxstatus);
+       csr_write(CSR_MHCR,     csr_mhcr);
+       csr_write(CSR_MHINT,    csr_mhint);
+}
+
+/*
+ * PLIC
+ */
+
+#define PLIC_SOURCES                   176
+#define PLIC_IE_WORDS                  ((PLIC_SOURCES + 31) / 32)
+
+static u8 plic_priority[PLIC_SOURCES];
+static u32 plic_sie[PLIC_IE_WORDS];
+static u32 plic_threshold;
+
+static void sun20i_d1_plic_save(void)
+{
+       fdt_plic_context_save(true, plic_sie, &plic_threshold);
+       fdt_plic_priority_save(plic_priority);
+}
+
+static void sun20i_d1_plic_restore(void)
+{
+       thead_plic_restore();
+       fdt_plic_priority_restore(plic_priority);
+       fdt_plic_context_restore(true, plic_sie, plic_threshold);
+}
+
+/*
+ * PPU
+ */
+
+#define PPU_PD_ACTIVE_CTRL             0x2c
+
+static void sun20i_d1_ppu_save(void)
+{
+       /* Enable MMIO access. Do not assume S-mode leaves the clock enabled. */
+       writel_relaxed(CCU_BGR_ENABLE, SUN20I_D1_PRCM_BASE + PPU_BGR_REG);
+
+       /* Activate automatic power-down during the next WFI. */
+       writel_relaxed(1, SUN20I_D1_PPU_BASE + PPU_PD_ACTIVE_CTRL);
+}
+
+static void sun20i_d1_ppu_restore(void)
+{
+       /* Disable automatic power-down. */
+       writel_relaxed(0, SUN20I_D1_PPU_BASE + PPU_PD_ACTIVE_CTRL);
+}
+
+/*
+ * RISCV_CFG
+ */
+
+#define RESET_ENTRY_LO_REG             0x0004
+#define RESET_ENTRY_HI_REG             0x0008
+#define WAKEUP_EN_REG                  0x0020
+#define WAKEUP_MASK_REG(i)             (0x0024 + 4 * (i))
+
+static void sun20i_d1_riscv_cfg_save(void)
+{
+       /* Enable MMIO access. Do not assume S-mode leaves the clock enabled. */
+       writel_relaxed(CCU_BGR_ENABLE, SUN20I_D1_CCU_BASE + RISCV_CFG_BGR_REG);
+
+       /*
+        * Copy the SIE bits to the wakeup registers. D1 has 160 "real"
+        * interrupt sources, numbered 16-175. These are the ones that map to
+        * the wakeup mask registers (the offset is for GIC compatibility). So
+        * copying SIE to the wakeup mask needs some bit manipulation.
+        */
+       for (int i = 0; i < PLIC_IE_WORDS - 1; i++)
+               writel_relaxed(plic_sie[i] >> 16 | plic_sie[i + 1] << 16,
+                              SUN20I_D1_RISCV_CFG_BASE + WAKEUP_MASK_REG(i));
+
+       /* Enable PPU wakeup for interrupts. */
+       writel_relaxed(1, SUN20I_D1_RISCV_CFG_BASE + WAKEUP_EN_REG);
+}
+
+static void sun20i_d1_riscv_cfg_restore(void)
+{
+       /* Disable PPU wakeup for interrupts. */
+       writel_relaxed(0, SUN20I_D1_RISCV_CFG_BASE + WAKEUP_EN_REG);
+}
+

platform/generic/allwinner/sun20i-d1.c line 154

Quoted 10 lines
platform/generic/allwinner/sun20i-d1.c new file +10 -0
+static void sun20i_d1_riscv_cfg_init(void)
+{
+       unsigned long entry = sbi_hartid_to_scratch(0)->warmboot_addr;
+
+       /* Enable MMIO access. */
+       writel_relaxed(CCU_BGR_ENABLE, SUN20I_D1_CCU_BASE + RISCV_CFG_BGR_REG);
+
+       /* Program the reset entry address. */
+       writel_relaxed((u32)entry, SUN20I_D1_RISCV_CFG_BASE + RESET_ENTRY_LO_REG);
+       writel_relaxed(entry >> 32, SUN20I_D1_RISCV_CFG_BASE + RESET_ENTRY_HI_REG);

This line gives a compile error for RV32. I have updated it at the time of merging this patch.

Quoted 60 lines
platform/generic/allwinner/sun20i-d1.c new file +56 -1
+}
+
+static int sun20i_d1_hart_suspend(u32 suspend_type)
+{
+       /* Use the generic code for retentive suspend. */
+       if (!(suspend_type & SBI_HSM_SUSP_NON_RET_BIT))
+               return SBI_ENOTSUPP;
+
+       sun20i_d1_plic_save();
+       sun20i_d1_ppu_save();
+       sun20i_d1_riscv_cfg_save();
+       sun20i_d1_csr_save();
+
+       /*
+        * If no interrupt is pending, this will power down the CPU power
+        * domain. Otherwise, this will fall through, and the generic HSM
+        * code will jump to the resume address.
+        */
+       wfi();
+
+       return 0;
+}
+
+static void sun20i_d1_hart_resume(void)
+{
+       sun20i_d1_csr_restore();
+       sun20i_d1_riscv_cfg_restore();
+       sun20i_d1_ppu_restore();
+       sun20i_d1_plic_restore();
+}
+
+static const struct sbi_hsm_device sun20i_d1_ppu = {
+       .name           = "sun20i-d1-ppu",
+       .hart_suspend   = sun20i_d1_hart_suspend,
+       .hart_resume    = sun20i_d1_hart_resume,
+};
+
+static int sun20i_d1_final_init(bool cold_boot, const struct fdt_match *match)
+{
+       if (cold_boot) {
+               sun20i_d1_riscv_cfg_init();
+               sbi_hsm_set_device(&sun20i_d1_ppu);
+       }
+
+       return 0;
+}
+
+static const struct fdt_match sun20i_d1_match[] = {
+       { .compatible = "allwinner,sun20i-d1" },
+       { },
+};
+
+const struct platform_override sun20i_d1 = {
+       .match_table    = sun20i_d1_match,
+       .final_init     = sun20i_d1_final_init,
+};
--

2.35.1

Quoted 4 lines

--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

Regards,
Anup

#12 lore raw

v3 cover HSM implementation for Allwinner D1

In reply to Samuel Holland
Quoted 9 lines · Samuel Holland

This series adds a minimal working HSM implementation for the Allwinner D1 SoC. It works with the Linux sbi_cpuidle driver. Even with canned values for entry/exit latency, I am seeing 30-45 mW reduction in power usage at idle.

Since this is the first HSM driver submission, there is no example to follow. I have updated the series based on the comments to the RFCs. Still, comments welcome!

Thanks, this help us harden the OpenSBI HSM implementation.

Quoted 20 lines

Regards,
Samuel

Changes in v3:
- Fix typo in commit message
- Move comment about priority range to function declaration
- Rename irqchip_plic_* functions to fdt_plic_*
- Add a "bool smode" parameter instead of hard coding the context ID

Changes in v2:
- Rebase on top of the current master branch
- Use array_size() from sbi_types.h
- Use a separate variable for each CSR instead of an array
- Use the PLIC helpers added in earlier patches
- Use the saved SIE bits to program the wakeup interrupt mask
- Set the entry address only once during cold boot
- Use the generic implementation for retentive suspend
- Clean up/add comments throughout the HSM implementation

Quoted 10 lines

Samuel Holland (9):
lib: sbi_hsm: Call a device hook during hart resume
lib: sbi_hsm: Assume a consistent resume address
lib: irqchip/plic: Constify plic_data pointers
lib: irqchip/plic: Factor out a context init function
lib: irqchip/plic: Add context save/restore helpers
lib: irqchip/plic: Add priority save/restore helpers
lib: utils/irqchip: Add FDT wrappers for PLIC save/restore functions
lib: utils/irqchip: Add wrapper for T-HEAD PLIC delegation
platform: Add HSM implementation for Allwinner D1

I have tested this series on QEMU virt machine using a custom DTB.

Applied this series on the riscv/opensbi repo

Thanks,
Anup

Quoted 24 lines
 include/sbi/sbi_hsm.h                        |  12 +-
 include/sbi_utils/irqchip/fdt_irqchip_plic.h |  22 ++
 include/sbi_utils/irqchip/plic.h             |  20 +-
 lib/sbi/sbi_hsm.c                            |  54 ++---
 lib/utils/irqchip/fdt_irqchip_plic.c         |  39 ++++
 lib/utils/irqchip/plic.c                     | 124 ++++++++---
 platform/fpga/ariane/platform.c              |  19 +-
 platform/fpga/openpiton/platform.c           |  19 +-
 platform/generic/allwinner/objects.mk        |   6 +
 platform/generic/allwinner/sun20i-d1.c       | 210 +++++++++++++++++++
 10 files changed, 442 insertions(+), 83 deletions(-)
 create mode 100644 include/sbi_utils/irqchip/fdt_irqchip_plic.h
 create mode 100644 platform/generic/allwinner/objects.mk
 create mode 100644 platform/generic/allwinner/sun20i-d1.c

--
2.35.1

--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi