Skip to contentSkip to conversation

riscv: improve nommu and timer-clint

PATCH In review Jisheng Zhang 4 participants 8 messages Apr 10, 2024 – Apr 24, 2024

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v3 cover riscv: improve nommu and timer-clint

As is known, the sophgo CV1800B contains so called little core, which is C906 w/o MMU, so I want to run nommu linux on it. This series is the result of the bring up. After this series, w/ proper dts, we can run nommu linux on milkv duo's little core.

First of all, patch1 removes the PAGE_OFFSET hardcoding by introducing DRAM_BASE Kconfig option.

Secondly, to use the T-HEAD C9xx clint in RISCV-M NOMMU env, we need to take care two points:

1.The mtimecmp in T-Head C9xx clint only supports 32bit read/write, implement such support.

2. As pointed out by commit ca7810aecdba ("lib: utils/timer: mtimer:
add a quirk for lacking mtime register") of opensbi:

"T-Head developers surely have a different understanding of time CSR and CLINT's mtime register with SiFive ones, that they did not implement the mtime register at all -- as shown in openC906 source code, their time CSR value is just exposed at the top of their processor IP block and expects an external continous counter, which makes it not overrideable, and thus mtime register is not implemented, even not for reading. However, if CLINTEE is not enabled in T-Head's MXSTATUS extended CSR, these systems still rely on the mtimecmp registers to generate timer interrupts. This makes it necessary to implement T-Head C9xx CLINT support in OpenSBI MTIMER driver, which skips implementing reading mtime register and falls back to default code that reads time CSR."

So, we need to fall back to read time CSR instead of mtime register. Add riscv_csr_time_available static key for this purpose.

The second patch adds T-Head C9xxx clint support to timer-clint driver by taking care of above two points.

Since v2:
- drop CONFIG_CLINT_USE_CSR_INSTEADOF_MTIME, instead we use runtime code patch to dynamically enable CSR TIME code path or MTIME code path.

Since v1:
- fix c900_clint_timer_init_dt() defined but not used build warning
- add option CONFIG_CLINT_USE_CSR_INSTEADOF_MTIME instead of removing mtime usage for all platforms, since not all platforms implement the time CSR in HW in M mode.
- rebase on the timer-clint improvement series
https://lore.kernel.org/linux-riscv/[email protected]/T/#t

Jisheng Zhang (2):
riscv: nommu: remove PAGE_OFFSET hardcoding clocksource/drivers/timer-clint: Add T-Head C9xx clint

 arch/riscv/Kconfig                |  8 +++++-
 arch/riscv/include/asm/clint.h    |  2 ++
 arch/riscv/include/asm/timex.h    | 18 +++++++++---
 drivers/clocksource/timer-clint.c | 48 +++++++++++++++++++++++++++----
 4 files changed, 66 insertions(+), 10 deletions(-)
Signature · 1 line
2.43.0
#1 lore raw

PATCH v3 1/2 riscv: nommu: remove PAGE_OFFSET hardcoding

Currently, PAGE_OFFSET is hardcoded as 0x8000_0000, it works fine since there's only one nommu platform in the mainline. However, there are many cases where the (S)DRAM base address isn't 0x8000_0000, so remove the hardcoding value, and introduce DRAM_BASE which will be set by users during configuring. DRAM_BASE is 0x8000_0000 by default.

---
 arch/riscv/Kconfig | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
arch/riscv/Kconfig +7 -1
index 7895c77545f1..b4af1df86352 100644
@@ -247,10 +247,16 @@ config MMU
 	  Select if you want MMU-based virtualised addressing space
 	  support by paged memory management. If unsure, say 'Y'.
 
+if !MMU
+config DRAM_BASE
+	hex '(S)DRAM Base Address'
+	default 0x80000000
+endif
+
 config PAGE_OFFSET
 	hex
 	default 0xC0000000 if 32BIT && MMU
-	default 0x80000000 if !MMU
+	default DRAM_BASE if !MMU
 	default 0xff60000000000000 if 64BIT
 
 config KASAN_SHADOW_OFFSET
Signature · 1 line
2.43.0
#4 lore raw

PATCH v3 2/2 clocksource/drivers/timer-clint: Add T-Head C9xx clint

In reply to Jisheng Zhang

To use the T-HEAD C9xx clint in RISCV-M NOMMU env, we need to take care two points:

1.The mtimecmp in T-Head C9xx clint only supports 32bit read/write, implement such support.

2. As pointed out by commit ca7810aecdba ("lib: utils/timer: mtimer:
add a quirk for lacking mtime register") of opensbi:

"T-Head developers surely have a different understanding of time CSR and CLINT's mtime register with SiFive ones, that they did not implement the mtime register at all -- as shown in openC906 source code, their time CSR value is just exposed at the top of their processor IP block and expects an external continous counter, which makes it not overrideable, and thus mtime register is not implemented, even not for reading. However, if CLINTEE is not enabled in T-Head's MXSTATUS extended CSR, these systems still rely on the mtimecmp registers to generate timer interrupts. This makes it necessary to implement T-Head C9xx CLINT support in OpenSBI MTIMER driver, which skips implementing reading mtime register and falls back to default code that reads time CSR."

So, we need to fall back to read time CSR instead of mtime register. Add riscv_csr_time_available static key for this purpose.

---
 arch/riscv/include/asm/clint.h    |  2 ++
 arch/riscv/include/asm/timex.h    | 18 +++++++++---
 drivers/clocksource/timer-clint.c | 48 +++++++++++++++++++++++++++----
 3 files changed, 59 insertions(+), 9 deletions(-)
arch/riscv/include/asm/clint.h +2 -0
index 0789fd37b40a..c6057a182c5d 100644
@@ -10,6 +10,7 @@
 #include <asm/mmio.h>
 
 #ifdef CONFIG_RISCV_M_MODE
+#include <linux/jump_label.h>
 /*
  * This lives in the CLINT driver, but is accessed directly by timex.h to avoid
  * any overhead when accessing the MMIO timer.
@@ -21,6 +22,7 @@
  * like "riscv_mtime", to signify that these non-ISA assumptions must hold.
  */
 extern u64 __iomem *clint_time_val;
+DECLARE_STATIC_KEY_FALSE(riscv_csr_time_available);
 #endif
 
 #endif
arch/riscv/include/asm/timex.h +14 -4
index a06697846e69..007a15482d75 100644
@@ -17,18 +17,27 @@ typedef unsigned long cycles_t;
 #ifdef CONFIG_64BIT
 static inline cycles_t get_cycles(void)
 {
-	return readq_relaxed(clint_time_val);
+	if (static_branch_likely(&riscv_csr_time_available))
+		return csr_read(CSR_TIME);
+	else
+		return readq_relaxed(clint_time_val);
 }
 #else /* !CONFIG_64BIT */
 static inline u32 get_cycles(void)
 {
-	return readl_relaxed(((u32 *)clint_time_val));
+	if (static_branch_likely(&riscv_csr_time_available))
+		return csr_read(CSR_TIME);
+	else
+		return readl_relaxed(((u32 *)clint_time_val));
 }
 #define get_cycles get_cycles
 
 static inline u32 get_cycles_hi(void)
 {
-	return readl_relaxed(((u32 *)clint_time_val) + 1);
+	if (static_branch_likely(&riscv_csr_time_available))
+		return csr_read(CSR_TIMEH);
+	else
+		return readl_relaxed(((u32 *)clint_time_val) + 1);
 }
 #define get_cycles_hi get_cycles_hi
 #endif /* CONFIG_64BIT */
@@ -40,7 +49,8 @@ static inline u32 get_cycles_hi(void)
  */
 static inline unsigned long random_get_entropy(void)
 {
-	if (unlikely(clint_time_val == NULL))
+	if (!static_branch_likely(&riscv_csr_time_available) &&
+	   (unlikely(clint_time_val == NULL)))
 		return random_get_entropy_fallback();
 	return get_cycles();
 }
drivers/clocksource/timer-clint.c +43 -5

87 lines not on this page · raw message

Signature · 1 line
2.43.0
#2 lore raw

Re: [PATCH v3 1/2] riscv: nommu: remove PAGE_OFFSET hardcoding

In reply to Jisheng Zhang
Quoted 4 lines · Jisheng Zhang

Currently, PAGE_OFFSET is hardcoded as 0x8000_0000, it works fine since there's only one nommu platform in the mainline. However, there are many cases where the (S)DRAM base address isn't 0x8000_0000, so remove the hardcoding value, and introduce DRAM_BASE which will be set by

Hi Jisheng,

Typo: s/harcoding/hardcoded

Quoted 12 lines

users during configuring. DRAM_BASE is 0x8000_0000 by default.

---
 arch/riscv/Kconfig | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
arch/riscv/Kconfig +0 -0
index 7895c77545f1..b4af1df86352 100644

arch/riscv/Kconfig config MMU · line 254

Quoted 8 lines
arch/riscv/Kconfig +5 -0
@@ -247,10 +247,16 @@ config MMU
 	  Select if you want MMU-based virtualised addressing space
 	  support by paged memory management. If unsure, say 'Y'.
 
+if !MMU
+config DRAM_BASE
+	hex '(S)DRAM Base Address'
+	default 0x80000000
+endif

I'm not sure but it feels odd to have this at top level config menu. Maybe it would make more sense for this to be located under the "Platform Type" section ?

Thanks,

Clément

Quoted 9 lines
arch/riscv/Kconfig +2 -1
+
 config PAGE_OFFSET
 	hex
 	default 0xC0000000 if 32BIT && MMU
-	default 0x80000000 if !MMU
+	default DRAM_BASE if !MMU
 	default 0xff60000000000000 if 64BIT
 
 config KASAN_SHADOW_OFFSET
#3 lore raw

Re: [PATCH v3 1/2] riscv: nommu: remove PAGE_OFFSET hardcoding

Quoted 41 lines · Clément Léger
Quoted 4 lines · Jisheng Zhang

Currently, PAGE_OFFSET is hardcoded as 0x8000_0000, it works fine since there's only one nommu platform in the mainline. However, there are many cases where the (S)DRAM base address isn't 0x8000_0000, so remove the hardcoding value, and introduce DRAM_BASE which will be set by

Hi Jisheng,

Typo: s/harcoding/hardcoded

Quoted 12 lines

users during configuring. DRAM_BASE is 0x8000_0000 by default.

---
 arch/riscv/Kconfig | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
arch/riscv/Kconfig +0 -0
index 7895c77545f1..b4af1df86352 100644

arch/riscv/Kconfig config MMU · line 254

Quoted 8 lines
arch/riscv/Kconfig +5 -0
@@ -247,10 +247,16 @@ config MMU
 	  Select if you want MMU-based virtualised addressing space
 	  support by paged memory management. If unsure, say 'Y'.
 
+if !MMU
+config DRAM_BASE
+	hex '(S)DRAM Base Address'
+	default 0x80000000
+endif

I'm not sure but it feels odd to have this at top level config menu. Maybe it would make more sense for this to be located under the "Platform Type" section ?

Thanks,

Clément

Quoted 7 lines

arch/riscv/Kconfig config MMU · line 260

Quoted 7 lines
arch/riscv/Kconfig +2 -1
+
 config PAGE_OFFSET
 	hex
 	default 0xC0000000 if 32BIT && MMU
-	default 0x80000000 if !MMU
+	default DRAM_BASE if !MMU
 	default 0xff60000000000000 if 64BIT

By the way, you should probably rebase that on top of Samuel's work [1] in order to support !MMU is S-mode.

Thanks,

Clément

Link:
https://lore.kernel.org/lkml/[email protected]/
[1]

Quoted 2 lines
Quoted 2 lines
arch/riscv/Kconfig +0 -0
 config KASAN_SHADOW_OFFSET
#7 lore raw
In reply to Jisheng Zhang
Quoted 41 lines · Jisheng Zhang

As is known, the sophgo CV1800B contains so called little core, which is C906 w/o MMU, so I want to run nommu linux on it. This series is the result of the bring up. After this series, w/ proper dts, we can run nommu linux on milkv duo's little core.

First of all, patch1 removes the PAGE_OFFSET hardcoding by introducing DRAM_BASE Kconfig option.

Secondly, to use the T-HEAD C9xx clint in RISCV-M NOMMU env, we need to take care two points:

1.The mtimecmp in T-Head C9xx clint only supports 32bit read/write, implement such support.

2. As pointed out by commit ca7810aecdba ("lib: utils/timer: mtimer:
add a quirk for lacking mtime register") of opensbi:

"T-Head developers surely have a different understanding of time CSR and CLINT's mtime register with SiFive ones, that they did not implement the mtime register at all -- as shown in openC906 source code, their time CSR value is just exposed at the top of their processor IP block and expects an external continous counter, which makes it not overrideable, and thus mtime register is not implemented, even not for reading. However, if CLINTEE is not enabled in T-Head's MXSTATUS extended CSR, these systems still rely on the mtimecmp registers to generate timer interrupts. This makes it necessary to implement T-Head C9xx CLINT support in OpenSBI MTIMER driver, which skips implementing reading mtime register and falls back to default code that reads time CSR."

So, we need to fall back to read time CSR instead of mtime register. Add riscv_csr_time_available static key for this purpose.

The second patch adds T-Head C9xxx clint support to timer-clint driver by taking care of above two points.

Since v2:
- drop CONFIG_CLINT_USE_CSR_INSTEADOF_MTIME, instead we use runtime code patch to dynamically enable CSR TIME code path or MTIME code path.

Quoted 7 lines

Since v1:
- fix c900_clint_timer_init_dt() defined but not used build warning
- add option CONFIG_CLINT_USE_CSR_INSTEADOF_MTIME instead of removing mtime usage for all platforms, since not all platforms implement the time CSR in HW in M mode.
- rebase on the timer-clint improvement series
https://lore.kernel.org/linux-riscv/[email protected]/T/#t

Hi Palmer, Daniel,

This series itself relies on the timer-clint improvement series
https://lore.kernel.org/linux-riscv/[email protected]/T/#t

But as suggested by Clément, it's better to rebase the patch1 on the Samuel's work which has been merged into riscv for-next, So I have two dependencies now, how do I handle this series then?

Could I leave patch1 alone and move patch2 into the timer-clint improvement series? Any suggestion?

Thanks in advance

Quoted 20 lines

Jisheng Zhang (2):
riscv: nommu: remove PAGE_OFFSET hardcoding clocksource/drivers/timer-clint: Add T-Head C9xx clint

 arch/riscv/Kconfig                |  8 +++++-
 arch/riscv/include/asm/clint.h    |  2 ++
 arch/riscv/include/asm/timex.h    | 18 +++++++++---
 drivers/clocksource/timer-clint.c | 48 +++++++++++++++++++++++++++----
 4 files changed, 66 insertions(+), 10 deletions(-)
Signature · 7 lines
2.43.0


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
#5 lore raw

Re: [PATCH v3 2/2] clocksource/drivers/timer-clint: Add T-Head C9xx clint

In reply to Jisheng Zhang
Quoted 75 lines · Jisheng Zhang

To use the T-HEAD C9xx clint in RISCV-M NOMMU env, we need to take care two points:

1.The mtimecmp in T-Head C9xx clint only supports 32bit read/write, implement such support.

2. As pointed out by commit ca7810aecdba ("lib: utils/timer: mtimer:
add a quirk for lacking mtime register") of opensbi:

"T-Head developers surely have a different understanding of time CSR and CLINT's mtime register with SiFive ones, that they did not implement the mtime register at all -- as shown in openC906 source code, their time CSR value is just exposed at the top of their processor IP block and expects an external continous counter, which makes it not overrideable, and thus mtime register is not implemented, even not for reading. However, if CLINTEE is not enabled in T-Head's MXSTATUS extended CSR, these systems still rely on the mtimecmp registers to generate timer interrupts. This makes it necessary to implement T-Head C9xx CLINT support in OpenSBI MTIMER driver, which skips implementing reading mtime register and falls back to default code that reads time CSR."

So, we need to fall back to read time CSR instead of mtime register. Add riscv_csr_time_available static key for this purpose.

---
 arch/riscv/include/asm/clint.h    |  2 ++
 arch/riscv/include/asm/timex.h    | 18 +++++++++---
 drivers/clocksource/timer-clint.c | 48 +++++++++++++++++++++++++++----
 3 files changed, 59 insertions(+), 9 deletions(-)
arch/riscv/include/asm/clint.h +2 -0
index 0789fd37b40a..c6057a182c5d 100644
@@ -10,6 +10,7 @@
 #include <asm/mmio.h>
 
 #ifdef CONFIG_RISCV_M_MODE
+#include <linux/jump_label.h>
 /*
  * This lives in the CLINT driver, but is accessed directly by timex.h to avoid
  * any overhead when accessing the MMIO timer.
@@ -21,6 +22,7 @@
  * like "riscv_mtime", to signify that these non-ISA assumptions must hold.
  */
 extern u64 __iomem *clint_time_val;
+DECLARE_STATIC_KEY_FALSE(riscv_csr_time_available);
 #endif
 
 #endif
arch/riscv/include/asm/timex.h +8 -2
index a06697846e69..007a15482d75 100644
@@ -17,18 +17,27 @@ typedef unsigned long cycles_t;
 #ifdef CONFIG_64BIT
 static inline cycles_t get_cycles(void)
 {
-	return readq_relaxed(clint_time_val);
+	if (static_branch_likely(&riscv_csr_time_available))
+		return csr_read(CSR_TIME);
+	else
+		return readq_relaxed(clint_time_val);
 }
 #else /* !CONFIG_64BIT */
 static inline u32 get_cycles(void)
 {
-	return readl_relaxed(((u32 *)clint_time_val));
+	if (static_branch_likely(&riscv_csr_time_available))
+		return csr_read(CSR_TIME);
+	else
+		return readl_relaxed(((u32 *)clint_time_val));
 }

arch/riscv/include/asm/timex.h typedef unsigned long cycles_t; · line 41

Quoted 10 lines
arch/riscv/include/asm/timex.h +4 -1
 #define get_cycles get_cycles
 
 static inline u32 get_cycles_hi(void)
 {
-	return readl_relaxed(((u32 *)clint_time_val) + 1);
+	if (static_branch_likely(&riscv_csr_time_available))
+		return csr_read(CSR_TIMEH);
+	else
+		return readl_relaxed(((u32 *)clint_time_val) + 1);
 }

None of the else branches here need to actually be an else, since the other branch returns. Otherwise, looks aight to me, thanks for the update.

#6 lore raw

Re: [PATCH v3 2/2] clocksource/drivers/timer-clint: Add T-Head C9xx clint

Quoted 78 lines · Conor Dooley
Quoted 77 lines · Jisheng Zhang

To use the T-HEAD C9xx clint in RISCV-M NOMMU env, we need to take care two points:

1.The mtimecmp in T-Head C9xx clint only supports 32bit read/write, implement such support.

2. As pointed out by commit ca7810aecdba ("lib: utils/timer: mtimer:
add a quirk for lacking mtime register") of opensbi:

"T-Head developers surely have a different understanding of time CSR and CLINT's mtime register with SiFive ones, that they did not implement the mtime register at all -- as shown in openC906 source code, their time CSR value is just exposed at the top of their processor IP block and expects an external continous counter, which makes it not overrideable, and thus mtime register is not implemented, even not for reading. However, if CLINTEE is not enabled in T-Head's MXSTATUS extended CSR, these systems still rely on the mtimecmp registers to generate timer interrupts. This makes it necessary to implement T-Head C9xx CLINT support in OpenSBI MTIMER driver, which skips implementing reading mtime register and falls back to default code that reads time CSR."

So, we need to fall back to read time CSR instead of mtime register. Add riscv_csr_time_available static key for this purpose.

---
 arch/riscv/include/asm/clint.h    |  2 ++
 arch/riscv/include/asm/timex.h    | 18 +++++++++---
 drivers/clocksource/timer-clint.c | 48 +++++++++++++++++++++++++++----
 3 files changed, 59 insertions(+), 9 deletions(-)
arch/riscv/include/asm/clint.h +2 -0
index 0789fd37b40a..c6057a182c5d 100644
@@ -10,6 +10,7 @@
 #include <asm/mmio.h>
 
 #ifdef CONFIG_RISCV_M_MODE
+#include <linux/jump_label.h>
 /*
  * This lives in the CLINT driver, but is accessed directly by timex.h to avoid
  * any overhead when accessing the MMIO timer.
@@ -21,6 +22,7 @@
  * like "riscv_mtime", to signify that these non-ISA assumptions must hold.
  */
 extern u64 __iomem *clint_time_val;
+DECLARE_STATIC_KEY_FALSE(riscv_csr_time_available);
 #endif
 
 #endif
arch/riscv/include/asm/timex.h +8 -2
index a06697846e69..007a15482d75 100644
@@ -17,18 +17,27 @@ typedef unsigned long cycles_t;
 #ifdef CONFIG_64BIT
 static inline cycles_t get_cycles(void)
 {
-	return readq_relaxed(clint_time_val);
+	if (static_branch_likely(&riscv_csr_time_available))
+		return csr_read(CSR_TIME);
+	else
+		return readq_relaxed(clint_time_val);
 }
 #else /* !CONFIG_64BIT */
 static inline u32 get_cycles(void)
 {
-	return readl_relaxed(((u32 *)clint_time_val));
+	if (static_branch_likely(&riscv_csr_time_available))
+		return csr_read(CSR_TIME);
+	else
+		return readl_relaxed(((u32 *)clint_time_val));
 }
 #define get_cycles get_cycles
 
Quoted 12 lines

arch/riscv/include/asm/timex.h typedef unsigned long cycles_t; · line 41

Quoted 8 lines
arch/riscv/include/asm/timex.h +4 -1
 static inline u32 get_cycles_hi(void)
 {
-	return readl_relaxed(((u32 *)clint_time_val) + 1);
+	if (static_branch_likely(&riscv_csr_time_available))
+		return csr_read(CSR_TIMEH);
+	else
+		return readl_relaxed(((u32 *)clint_time_val) + 1);
 }

None of the else branches here need to actually be an else, since the other branch returns. Otherwise, looks aight to me, thanks for the update.

Also the static key is initialized to false and the commit message says only the C9xx cores need to use the CSR_TIME, so shouldn't it be

if (static_branch_unlikely(&riscv_csr_time_available))

instead, or am I reading this wrong?

/Emil