Skip to contentSkip to conversation

riscv: convert bottom half of exception handling to C

PATCH Superseded Jisheng Zhang 5 participants 24 messages Jun 16, 2024 – Jun 24, 2024

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover riscv: convert bottom half of exception handling to C

For readability, maintainability and future scalability, convert the bottom half of the exception handling to C.

During the conversion, I found Anton fixed a performance issue and my patches will touch the same exception asm code, so I include Anton's patch for completeness. I also cooked a similar patch to avoid corrupting the RAS in ret_from_fork() per the inspiration.

Mostly the assembly code is converted to C in a relatively straightforward manner.

However, there are two modifications I need to mention:

1. the CSR_CAUSE reg reading and saving is moved to the C code because we need the cause to dispatch the exception handling, if we keep the cause reading and saving, we either pass it to do_traps() via. 2nd param or get it from pt_regs which an extra memory load is needed, I don't like any of the two solutions becase the exception handling sits in hot code path, every instruction matters.

2.To cope with SIFIVE_CIP_453 errata, it looks like we don't need alternative mechanism any more after the asm->c convertion. Just replace the excp_vect_table two entries.

Anton Blanchard (1):
riscv: Improve exception and system call latency

Jisheng Zhang (5):
riscv: avoid corrupting the RAS
riscv: convert bottom half of exception handling to C
riscv: errata: remove ALT_INSN_FAULT and ALT_PAGE_FAULT
riscv: errata: sifive: remove NOMMU handling
riscv: remove asmlinkage from updated functions

 arch/riscv/errata/sifive/errata.c         | 25 +++++++---
 arch/riscv/errata/sifive/errata_cip_453.S |  4 --
 arch/riscv/include/asm/asm-prototypes.h   |  7 +--
 arch/riscv/include/asm/errata_list.h      | 21 ++------
 arch/riscv/kernel/entry.S                 | 61 ++---------------------
 arch/riscv/kernel/stacktrace.c            |  4 +-
 arch/riscv/kernel/traps.c                 | 57 ++++++++++++++++++---
 7 files changed, 81 insertions(+), 98 deletions(-)
Signature · 1 line
2.43.0
#1 lore raw

PATCH 1/6 riscv: Improve exception and system call latency

From: Anton Blanchard <[email protected]>

Many CPUs implement return address branch prediction as a stack. The RISCV architecture refers to this as a return address stack (RAS). If this gets corrupted then the CPU will mispredict at least one but potentally many function returns.

There are two issues with the current RISCV exception code:

- We are using the alternate link stack (x5/t0) for the indirect branch which makes the hardware think this is a function return. This will corrupt the RAS.

- We modify the return address of handle_exception to point to ret_from_exception. This will also corrupt the RAS.

Testing the null system call latency before and after the patch:

Visionfive2 (StarFive JH7110 / U74)

baseline: 189.87 ns
patched:  176.76 ns

Lichee pi 4a (T-Head TH1520 / C910)

baseline: 666.58 ns
patched:  636.90 ns

Just over 7% on the U74 and just over 4% on the C910.

---
 arch/riscv/kernel/entry.S      | 17 ++++++++++-------
 arch/riscv/kernel/stacktrace.c |  4 ++--
 2 files changed, 12 insertions(+), 9 deletions(-)
arch/riscv/kernel/entry.S +10 -7
index 68a24cf9481a..c933460ed3e9 100644
@@ -88,7 +88,6 @@ SYM_CODE_START(handle_exception)
 	call riscv_v_context_nesting_start
 #endif
 	move a0, sp /* pt_regs */
-	la ra, ret_from_exception
 
 	/*
 	 * MSB of cause differentiates between
@@ -97,7 +96,8 @@ SYM_CODE_START(handle_exception)
 	bge s4, zero, 1f
 
 	/* Handle interrupts */
-	tail do_irq
+	call do_irq
+	j ret_from_exception
 1:
 	/* Handle other exceptions */
 	slli t0, s4, RISCV_LGPTR
@@ -105,11 +105,14 @@ SYM_CODE_START(handle_exception)
 	la t2, excp_vect_table_end
 	add t0, t1, t0
 	/* Check if exception code lies within bounds */
-	bgeu t0, t2, 1f
-	REG_L t0, 0(t0)
-	jr t0
-1:
-	tail do_trap_unknown
+	bgeu t0, t2, 3f
+	REG_L t1, 0(t0)
+2:	jalr t1
+	j ret_from_exception
+3:
+
+	la t1, do_trap_unknown
+	j 2b
 SYM_CODE_END(handle_exception)
 ASM_NOKPROBE(handle_exception)
 
arch/riscv/kernel/stacktrace.c +2 -2
index 528ec7cc9a62..5eb3d135b717 100644
@@ -16,7 +16,7 @@
 
 #ifdef CONFIG_FRAME_POINTER
 
-extern asmlinkage void ret_from_exception(void);
+extern asmlinkage void handle_exception(void);
 
 static inline int fp_is_valid(unsigned long fp, unsigned long sp)
 {
@@ -70,7 +70,7 @@ void notrace walk_stackframe(struct task_struct *task, struct pt_regs *regs,
 			fp = frame->fp;
 			pc = ftrace_graph_ret_addr(current, NULL, frame->ra,
 						   &frame->ra);
-			if (pc == (unsigned long)ret_from_exception) {
+			if (pc == (unsigned long)handle_exception) {
 				if (unlikely(!__kernel_text_address(pc) || !fn(arg, pc)))
 					break;
 
Signature · 1 line
2.43.0
Reviewed-by on v1 1/6
#2 lore raw

Re: [PATCH 1/6] riscv: Improve exception and system call latency

Quoted 22 lines · Jisheng Zhang

From: Anton Blanchard <[email protected]>

Many CPUs implement return address branch prediction as a stack. The RISCV architecture refers to this as a return address stack (RAS). If this gets corrupted then the CPU will mispredict at least one but potentally many function returns.

There are two issues with the current RISCV exception code:

- We are using the alternate link stack (x5/t0) for the indirect branch which makes the hardware think this is a function return. This will corrupt the RAS.

- We modify the return address of handle_exception to point to ret_from_exception. This will also corrupt the RAS.

Testing the null system call latency before and after the patch:

Visionfive2 (StarFive JH7110 / U74)

baseline: 189.87 ns
patched:  176.76 ns
Quoted 8 lines

Lichee pi 4a (T-Head TH1520 / C910)

baseline: 666.58 ns
patched:  636.90 ns

Just over 7% on the U74 and just over 4% on the C910.

Do you need to sign this off since you're sending this Jisheng?

Quoted 32 lines
---
 arch/riscv/kernel/entry.S      | 17 ++++++++++-------
 arch/riscv/kernel/stacktrace.c |  4 ++--
 2 files changed, 12 insertions(+), 9 deletions(-)
arch/riscv/kernel/entry.S +2 -3
index 68a24cf9481a..c933460ed3e9 100644
@@ -88,7 +88,6 @@ SYM_CODE_START(handle_exception)
 	call riscv_v_context_nesting_start
 #endif
 	move a0, sp /* pt_regs */
-	la ra, ret_from_exception
 
 	/*
 	 * MSB of cause differentiates between
@@ -97,7 +96,8 @@ SYM_CODE_START(handle_exception)
 	bge s4, zero, 1f
 
 	/* Handle interrupts */
-	tail do_irq
+	call do_irq
+	j ret_from_exception
 1:
 	/* Handle other exceptions */
 	slli t0, s4, RISCV_LGPTR
@@ -105,11 +105,14 @@ SYM_CODE_START(handle_exception)
 	la t2, excp_vect_table_end
 	add t0, t1, t0
 	/* Check if exception code lies within bounds */
-	bgeu t0, t2, 1f

arch/riscv/kernel/entry.S SYM_CODE_START(handle_exception) · line 113

Quoted 10 lines
arch/riscv/kernel/entry.S +6 -4
-	REG_L t0, 0(t0)
-	jr t0
-1:
-	tail do_trap_unknown
+	bgeu t0, t2, 3f
+	REG_L t1, 0(t0)
+2:	jalr t1
+	j ret_from_exception
+3:
+

The whitespace is odd here, but nonetheless:

Quoted 35 lines
arch/riscv/kernel/entry.S +2 -0
+	la t1, do_trap_unknown
+	j 2b
 SYM_CODE_END(handle_exception)
 ASM_NOKPROBE(handle_exception)
 
arch/riscv/kernel/stacktrace.c +2 -2
index 528ec7cc9a62..5eb3d135b717 100644
@@ -16,7 +16,7 @@
 
 #ifdef CONFIG_FRAME_POINTER
 
-extern asmlinkage void ret_from_exception(void);
+extern asmlinkage void handle_exception(void);
 
 static inline int fp_is_valid(unsigned long fp, unsigned long sp)
 {
@@ -70,7 +70,7 @@ void notrace walk_stackframe(struct task_struct *task, struct pt_regs *regs,
 			fp = frame->fp;
 			pc = ftrace_graph_ret_addr(current, NULL, frame->ra,
 						   &frame->ra);
-			if (pc == (unsigned long)ret_from_exception) {
+			if (pc == (unsigned long)handle_exception) {
 				if (unlikely(!__kernel_text_address(pc) || !fn(arg, pc)))
 					break;
 
Signature · 7 lines
2.43.0


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
#3 lore raw

Re: [PATCH 1/6] riscv: Improve exception and system call latency

Quoted 27 lines · Charlie Jenkins
Quoted 26 lines · Jisheng Zhang

From: Anton Blanchard <[email protected]>

Many CPUs implement return address branch prediction as a stack. The RISCV architecture refers to this as a return address stack (RAS). If this gets corrupted then the CPU will mispredict at least one but potentally many function returns.

There are two issues with the current RISCV exception code:

- We are using the alternate link stack (x5/t0) for the indirect branch which makes the hardware think this is a function return. This will corrupt the RAS.

- We modify the return address of handle_exception to point to ret_from_exception. This will also corrupt the RAS.

Testing the null system call latency before and after the patch:

Visionfive2 (StarFive JH7110 / U74)

baseline: 189.87 ns
patched:  176.76 ns

Lichee pi 4a (T-Head TH1520 / C910)

baseline: 666.58 ns
patched:  636.90 ns
Quoted 6 lines
Quoted 4 lines

Just over 7% on the U74 and just over 4% on the C910.

Do you need to sign this off since you're sending this Jisheng?

will do in newer version. Thanks for reminding.

I'm sending out this for reference since we touched the same asm source code.

Quoted 83 lines
Quoted 33 lines
---
 arch/riscv/kernel/entry.S      | 17 ++++++++++-------
 arch/riscv/kernel/stacktrace.c |  4 ++--
 2 files changed, 12 insertions(+), 9 deletions(-)
arch/riscv/kernel/entry.S +2 -3
index 68a24cf9481a..c933460ed3e9 100644
@@ -88,7 +88,6 @@ SYM_CODE_START(handle_exception)
 	call riscv_v_context_nesting_start
 #endif
 	move a0, sp /* pt_regs */
-	la ra, ret_from_exception
 
 	/*
 	 * MSB of cause differentiates between
@@ -97,7 +96,8 @@ SYM_CODE_START(handle_exception)
 	bge s4, zero, 1f
 
 	/* Handle interrupts */
-	tail do_irq
+	call do_irq
+	j ret_from_exception
 1:
 	/* Handle other exceptions */
 	slli t0, s4, RISCV_LGPTR
@@ -105,11 +105,14 @@ SYM_CODE_START(handle_exception)
 	la t2, excp_vect_table_end
 	add t0, t1, t0
 	/* Check if exception code lies within bounds */
-	bgeu t0, t2, 1f

arch/riscv/kernel/entry.S SYM_CODE_START(handle_exception) · line 113

Quoted 10 lines
arch/riscv/kernel/entry.S +6 -4
-	REG_L t0, 0(t0)
-	jr t0
-1:
-	tail do_trap_unknown
+	bgeu t0, t2, 3f
+	REG_L t1, 0(t0)
+2:	jalr t1
+	j ret_from_exception
+3:
+

The whitespace is odd here, but nonetheless:

Quoted 35 lines
arch/riscv/kernel/entry.S +2 -0
+	la t1, do_trap_unknown
+	j 2b
 SYM_CODE_END(handle_exception)
 ASM_NOKPROBE(handle_exception)
 
arch/riscv/kernel/stacktrace.c +2 -2
index 528ec7cc9a62..5eb3d135b717 100644
@@ -16,7 +16,7 @@
 
 #ifdef CONFIG_FRAME_POINTER
 
-extern asmlinkage void ret_from_exception(void);
+extern asmlinkage void handle_exception(void);
 
 static inline int fp_is_valid(unsigned long fp, unsigned long sp)
 {
@@ -70,7 +70,7 @@ void notrace walk_stackframe(struct task_struct *task, struct pt_regs *regs,
 			fp = frame->fp;
 			pc = ftrace_graph_ret_addr(current, NULL, frame->ra,
 						   &frame->ra);
-			if (pc == (unsigned long)ret_from_exception) {
+			if (pc == (unsigned long)handle_exception) {
 				if (unlikely(!__kernel_text_address(pc) || !fn(arg, pc)))
 					break;
 
Signature · 7 lines
2.43.0


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
#4 lore raw

PATCH 2/6 riscv: avoid corrupting the RAS

In reply to Jisheng Zhang

Inspired by[1], "modifying the return address of to point to ret_from_exception will corrupt the RAS", so modify the code to remove the code of modifying ra.

---
 arch/riscv/kernel/entry.S | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/riscv/kernel/entry.S +2 -2
index c933460ed3e9..81dec627a8d4 100644
@@ -235,8 +235,8 @@ SYM_CODE_START(ret_from_fork)
 	jalr s0
 1:
 	move a0, sp /* pt_regs */
-	la ra, ret_from_exception
-	tail syscall_exit_to_user_mode
+	call syscall_exit_to_user_mode
+	j ret_from_exception
 SYM_CODE_END(ret_from_fork)
 
 #ifdef CONFIG_IRQ_STACKS
Signature · 1 line
2.43.0
Reviewed-by on v1 2/6
#5 lore raw

Re: [PATCH 2/6] riscv: avoid corrupting the RAS

Quoted 7 lines · Jisheng Zhang

Inspired by[1], "modifying the return address of to point to ret_from_exception will corrupt the RAS", so modify the code to remove the code of modifying ra.

I've run this and it is the same idea that Anton used in the previous patch

Quoted 27 lines
---
 arch/riscv/kernel/entry.S | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/riscv/kernel/entry.S +2 -3
index c933460ed3e9..81dec627a8d4 100644
@@ -235,8 +235,8 @@ SYM_CODE_START(ret_from_fork)
        jalr s0
 1:
        move a0, sp /* pt_regs */
-       la ra, ret_from_exception
-       tail syscall_exit_to_user_mode
+       call syscall_exit_to_user_mode
+       j ret_from_exception
 SYM_CODE_END(ret_from_fork)
 
 #ifdef CONFIG_IRQ_STACKS
--

2.43.0

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#6 lore raw

PATCH 3/6 riscv: convert bottom half of exception handling to C

In reply to Jisheng Zhang

For readability, maintainability and future scalability, convert the bottom half of the exception handling to C.

Mostly the assembly code is converted to C in a relatively straightforward manner.

However, there are two modifications I need to mention:

1. the CSR_CAUSE reg reading and saving is moved to the C code because we need the cause to dispatch the exception handling, if we keep the cause reading and saving, we either pass it to do_traps() via. 2nd param or get it from pt_regs which an extra memory load is needed, I don't like any of the two solutions becase the exception handling sits in hot code path, every instruction matters.

2.To cope with SIFIVE_CIP_453 errata, it looks like we don't need alternative mechanism any more after the asm->c convertion. Just replace the excp_vect_table two entries.

---
 arch/riscv/errata/sifive/errata.c       | 25 ++++++++---
 arch/riscv/include/asm/asm-prototypes.h |  1 +
 arch/riscv/include/asm/errata_list.h    |  5 +--
 arch/riscv/kernel/entry.S               | 58 +------------------------
 arch/riscv/kernel/traps.c               | 41 +++++++++++++++++
 5 files changed, 64 insertions(+), 66 deletions(-)
arch/riscv/errata/sifive/errata.c +19 -6
index 716cfedad3a2..bbba99f207ca 100644
@@ -10,9 +10,14 @@
 #include <linux/bug.h>
 #include <asm/patch.h>
 #include <asm/alternative.h>
+#include <asm/csr.h>
 #include <asm/vendorid_list.h>
 #include <asm/errata_list.h>
 
+extern void (*excp_vect_table[])(struct pt_regs *regs);
+extern void sifive_cip_453_insn_fault_trp(struct pt_regs *regs);
+extern void sifive_cip_453_page_fault_trp(struct pt_regs *regs);
+
 struct errata_info_t {
 	char name[32];
 	bool (*check_func)(unsigned long  arch_id, unsigned long impid);
@@ -20,6 +25,9 @@ struct errata_info_t {
 
 static bool errata_cip_453_check_func(unsigned long  arch_id, unsigned long impid)
 {
+	if (!IS_ENABLED(CONFIG_ERRATA_SIFIVE_CIP_453))
+		return false;
+
 	/*
 	 * Affected cores:
 	 * Architecture ID: 0x8000000000000007
@@ -51,10 +59,6 @@ static bool errata_cip_1200_check_func(unsigned long  arch_id, unsigned long imp
 }
 
 static struct errata_info_t errata_list[ERRATA_SIFIVE_NUMBER] = {
-	{
-		.name = "cip-453",
-		.check_func = errata_cip_453_check_func
-	},
 	{
 		.name = "cip-1200",
 		.check_func = errata_cip_1200_check_func
@@ -62,11 +66,20 @@ static struct errata_info_t errata_list[ERRATA_SIFIVE_NUMBER] = {
 };
 
 static u32 __init_or_module sifive_errata_probe(unsigned long archid,
-						unsigned long impid)
+						unsigned long impid,
+						unsigned int stage)
 {
 	int idx;
 	u32 cpu_req_errata = 0;
 
+	if (stage == RISCV_ALTERNATIVES_BOOT) {
+		if (IS_ENABLED(CONFIG_MMU) &&
+		    errata_cip_453_check_func(archid, impid)) {
+			excp_vect_table[EXC_INST_ACCESS] = sifive_cip_453_insn_fault_trp;
+			excp_vect_table[EXC_INST_PAGE_FAULT] = sifive_cip_453_page_fault_trp;
+		}
+	}
+
 	for (idx = 0; idx < ERRATA_SIFIVE_NUMBER; idx++)
 		if (errata_list[idx].check_func(archid, impid))
 			cpu_req_errata |= (1U << idx);
@@ -99,7 +112,7 @@ void sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
 	if (stage == RISCV_ALTERNATIVES_EARLY_BOOT)
 		return;
 
-	cpu_req_errata = sifive_errata_probe(archid, impid);
+	cpu_req_errata = sifive_errata_probe(archid, impid, stage);
 
 	for (alt = begin; alt < end; alt++) {
 		if (alt->vendor_id != SIFIVE_VENDOR_ID)
arch/riscv/include/asm/asm-prototypes.h +1 -0
index cd627ec289f1..81a1f27fa54f 100644
@@ -55,5 +55,6 @@ DECLARE_DO_ERROR_INFO(do_trap_break);
 asmlinkage void handle_bad_stack(struct pt_regs *regs);
 asmlinkage void do_page_fault(struct pt_regs *regs);
 asmlinkage void do_irq(struct pt_regs *regs);
+asmlinkage void do_traps(struct pt_regs *regs);
 
 #endif /* _ASM_RISCV_PROTOTYPES_H */
arch/riscv/include/asm/errata_list.h +2 -3
index 7c8a71a526a3..95b79afc4061 100644
@@ -17,9 +17,8 @@
 #endif
 
 #ifdef CONFIG_ERRATA_SIFIVE
-#define	ERRATA_SIFIVE_CIP_453 0
-#define	ERRATA_SIFIVE_CIP_1200 1
-#define	ERRATA_SIFIVE_NUMBER 2
+#define	ERRATA_SIFIVE_CIP_1200 0
+#define	ERRATA_SIFIVE_NUMBER 1
 #endif
 
 #ifdef CONFIG_ERRATA_THEAD
arch/riscv/kernel/entry.S +1 -57

83 lines not on this page · raw message

arch/riscv/kernel/traps.c +41 -0

47 lines not on this page · raw message

Signature · 1 line
2.43.0
#7 lore raw
Quoted 8 lines · Jisheng Zhang

For readability, maintainability and future scalability, convert the bottom half of the exception handling to C.

Mostly the assembly code is converted to C in a relatively straightforward manner.

However, there are two modifications I need to mention:

Quoted 7 lines

1. the CSR_CAUSE reg reading and saving is moved to the C code because we need the cause to dispatch the exception handling, if we keep the cause reading and saving, we either pass it to do_traps() via. 2nd param or get it from pt_regs which an extra memory load is needed, I don't like any of the two solutions becase the exception handling sits in hot code path, every instruction matters.

CC Clement.

I think its better to save away cause in pt_regs prior to calling do_traps. Once control is transferred to C code in do_traps, another trap can happen. It's a problem anyways today without CPU support.

Although with Ssdbltrp [1] extension and it kernel support [2] for it, I expect asm code would clear up SDT bit in mstatus. Whenever Ssdbltrp lands, I think do_traps should expect nesting of traps and thus cause should be saved away before it gets control so that safely traps can be nested.

[1] - https://github.com/riscv/riscv-double-trap/releases/download/v1.0-rc1/riscv-double-trap.pdf
[2] - https://lore.kernel.org/all/[email protected]/

Quoted 6 lines

2.To cope with SIFIVE_CIP_453 errata, it looks like we don't need alternative mechanism any more after the asm->c convertion. Just replace the excp_vect_table two entries.

#8 lore raw
Quoted 20 lines · Deepak Gupta
Quoted 8 lines · Jisheng Zhang

For readability, maintainability and future scalability, convert the bottom half of the exception handling to C.

Mostly the assembly code is converted to C in a relatively straightforward manner.

However, there are two modifications I need to mention:

Quoted 7 lines

1. the CSR_CAUSE reg reading and saving is moved to the C code because we need the cause to dispatch the exception handling, if we keep the cause reading and saving, we either pass it to do_traps() via. 2nd param or get it from pt_regs which an extra memory load is needed, I don't like any of the two solutions becase the exception handling sits in hot code path, every instruction matters.

CC Clement.
Quoted 9 lines

I think its better to save away cause in pt_regs prior to calling do_traps. Once control is transferred to C code in do_traps, another trap can happen. It's a problem anyways today without CPU support.

Although with Ssdbltrp [1] extension and it kernel support [2] for it, I expect asm code would clear up SDT bit in mstatus. Whenever Ssdbltrp lands, I think do_traps should expect nesting of traps and thus cause should be saved away before it gets control so that safely traps can be nested.

Is a possible solution to do both options Jisheng suggested? Save the cause before
calling do_traps but also pass it via second param?

Quoted 14 lines

[1] - https://github.com/riscv/riscv-double-trap/releases/download/v1.0-rc1/riscv-double-trap.pdf
[2] - https://lore.kernel.org/all/[email protected]/

Quoted 6 lines

2.To cope with SIFIVE_CIP_453 errata, it looks like we don't need alternative mechanism any more after the asm->c convertion. Just replace the excp_vect_table two entries.

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#9 lore raw
Quoted 21 lines · Cyril Bur
Quoted 20 lines · Deepak Gupta
Quoted 8 lines · Jisheng Zhang

For readability, maintainability and future scalability, convert the bottom half of the exception handling to C.

Mostly the assembly code is converted to C in a relatively straightforward manner.

However, there are two modifications I need to mention:

Quoted 7 lines

1. the CSR_CAUSE reg reading and saving is moved to the C code because we need the cause to dispatch the exception handling, if we keep the cause reading and saving, we either pass it to do_traps() via. 2nd param or get it from pt_regs which an extra memory load is needed, I don't like any of the two solutions becase the exception handling sits in hot code path, every instruction matters.

CC Clement.
Quoted 8 lines
Quoted 8 lines

I think its better to save away cause in pt_regs prior to calling do_traps. Once control is transferred to C code in do_traps, another trap can happen. It's a problem anyways today without CPU support.

Although with Ssdbltrp [1] extension and it kernel support [2] for it, I expect asm code would clear up SDT bit in mstatus. Whenever Ssdbltrp lands, I think do_traps should expect nesting of traps and thus cause should be saved away before it gets control so that safely traps can be nested.

Hi,

Indeed, every register that is "unique" to a trap and than can be overwritten by a second trap should be saved before reenabling them when using Ssdbltrp. So that would be nice to preserve that.

Quoted 5 lines
Quoted 1 line

Is a possible solution to do both options Jisheng suggested? Save the cause before
calling do_traps but also pass it via second param?

I guess so if it fits your performance requirements.

Thanks,

Clément

Quoted 15 lines
Quoted 15 lines

[1] - https://github.com/riscv/riscv-double-trap/releases/download/v1.0-rc1/riscv-double-trap.pdf
[2] - https://lore.kernel.org/all/[email protected]/

Quoted 6 lines

2.To cope with SIFIVE_CIP_453 errata, it looks like we don't need alternative mechanism any more after the asm->c convertion. Just replace the excp_vect_table two entries.

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#10 lore raw
Quoted 22 lines · Clément Léger
Quoted 19 lines · Cyril Bur
Quoted 18 lines · Deepak Gupta
Quoted 15 lines · Jisheng Zhang

For readability, maintainability and future scalability, convert the bottom half of the exception handling to C.

Mostly the assembly code is converted to C in a relatively straightforward manner.

However, there are two modifications I need to mention:

1. the CSR_CAUSE reg reading and saving is moved to the C code because we need the cause to dispatch the exception handling, if we keep the cause reading and saving, we either pass it to do_traps() via. 2nd param or get it from pt_regs which an extra memory load is needed, I don't like any of the two solutions becase the exception handling sits in hot code path, every instruction matters.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
CC Clement.

I think its better to save away cause in pt_regs prior to calling do_traps. Once control is transferred to C code in do_traps, another trap can happen. It's a problem anyways today without CPU support.

Although with Ssdbltrp [1] extension and it kernel support [2] for it, I expect asm code would clear up SDT bit in mstatus. Whenever Ssdbltrp lands,

Hi Deepak, Clément,

Currently, SR_IE bit is is set(setting means enable irq) in c, could the 'SDT' bit be cleared in c as well when Ssdbltrp lands?

Thanks

Quoted 36 lines
Quoted 2 lines
Quoted 2 lines

I think do_traps should expect nesting of traps and thus cause should be saved away before it gets control so that safely traps can be nested.

Hi,

Indeed, every register that is "unique" to a trap and than can be overwritten by a second trap should be saved before reenabling them when using Ssdbltrp. So that would be nice to preserve that.

Quoted 5 lines
Quoted 1 line

Is a possible solution to do both options Jisheng suggested? Save the cause before
calling do_traps but also pass it via second param?

I guess so if it fits your performance requirements.

Thanks,

Clément

Quoted 15 lines
Quoted 15 lines

[1] - https://github.com/riscv/riscv-double-trap/releases/download/v1.0-rc1/riscv-double-trap.pdf
[2] - https://lore.kernel.org/all/[email protected]/

Quoted 6 lines

2.To cope with SIFIVE_CIP_453 errata, it looks like we don't need alternative mechanism any more after the asm->c convertion. Just replace the excp_vect_table two entries.

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#11 lore raw
Quoted 29 lines · Jisheng Zhang
Quoted 28 lines · Clément Léger
Quoted 25 lines · Cyril Bur
Quoted 24 lines · Deepak Gupta
Quoted 8 lines · Jisheng Zhang

For readability, maintainability and future scalability, convert the bottom half of the exception handling to C.

Mostly the assembly code is converted to C in a relatively straightforward manner.

However, there are two modifications I need to mention:

Quoted 7 lines

1. the CSR_CAUSE reg reading and saving is moved to the C code because we need the cause to dispatch the exception handling, if we keep the cause reading and saving, we either pass it to do_traps() via. 2nd param or get it from pt_regs which an extra memory load is needed, I don't like any of the two solutions becase the exception handling sits in hot code path, every instruction matters.

CC Clement.

I think its better to save away cause in pt_regs prior to calling do_traps. Once control is transferred to C code in do_traps, another trap can happen. It's a problem anyways today without CPU support.

Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Although with Ssdbltrp [1] extension and it kernel support [2] for it, I expect asm code would clear up SDT bit in mstatus. Whenever Ssdbltrp lands,

Hi Deepak, Clément,

Currently, SR_IE bit is is set(setting means enable irq) in c, could the 'SDT' bit be cleared in c as well when Ssdbltrp lands?

SDT is placed in sstatus CSR. So yes its possible to clear it in C in do_traps. Although then you (and any future developer) will have to pay extra attention to this function because this function can be nested depending on when SDT is cleared or not. Maintainence (and introductions of error) wise it doesn't look ideal.

If we keep read of cause in asm code and pass it as parameter to do_traps, it cleanly defines the boundary of which functions can be nested and which can't. It helps features like SSE [1, 2] (which expect nesting of events and had to be creative) to implement cleaner logic.

[1] https://lists.riscv.org/g/tech-prs/message/515
[2] https://lpc.events/event/17/contributions/1479/attachments/1243/2526/SSE_Plumbers.pdf

Quoted 38 lines

Thanks

Quoted 36 lines
Quoted 2 lines
Quoted 2 lines

I think do_traps should expect nesting of traps and thus cause should be saved away before it gets control so that safely traps can be nested.

Hi,

Indeed, every register that is "unique" to a trap and than can be overwritten by a second trap should be saved before reenabling them when using Ssdbltrp. So that would be nice to preserve that.

Quoted 5 lines
Quoted 1 line

Is a possible solution to do both options Jisheng suggested? Save the cause before
calling do_traps but also pass it via second param?

I guess so if it fits your performance requirements.

Thanks,

Clément

Quoted 15 lines
Quoted 15 lines

[1] - https://github.com/riscv/riscv-double-trap/releases/download/v1.0-rc1/riscv-double-trap.pdf
[2] - https://lore.kernel.org/all/[email protected]/

Quoted 6 lines

2.To cope with SIFIVE_CIP_453 errata, it looks like we don't need alternative mechanism any more after the asm->c convertion. Just replace the excp_vect_table two entries.

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#12 lore raw
In reply to Jisheng Zhang
Quoted 16 lines · Jisheng Zhang

For readability, maintainability and future scalability, convert the bottom half of the exception handling to C.

Mostly the assembly code is converted to C in a relatively straightforward manner.

However, there are two modifications I need to mention:

1. the CSR_CAUSE reg reading and saving is moved to the C code because we need the cause to dispatch the exception handling, if we keep the cause reading and saving, we either pass it to do_traps() via. 2nd param or get it from pt_regs which an extra memory load is needed, I don't like any of the two solutions becase the exception handling sits in hot code path, every instruction matters.

Quoted 4 lines

2.To cope with SIFIVE_CIP_453 errata, it looks like we don't need alternative mechanism any more after the asm->c convertion. Just replace the excp_vect_table two entries.

Hi Jisheng,

Sorry I've been having email client problems. This email if from a few days ago.

I really like the look of the patch. I just have one question.

Quoted 126 lines
---
  arch/riscv/errata/sifive/errata.c       | 25 ++++++++---
  arch/riscv/include/asm/asm-prototypes.h |  1 +
  arch/riscv/include/asm/errata_list.h    |  5 +--
  arch/riscv/kernel/entry.S               | 58 +------------------------
  arch/riscv/kernel/traps.c               | 41 +++++++++++++++++
  5 files changed, 64 insertions(+), 66 deletions(-)
arch/riscv/errata/sifive/errata.c +19 -6
index 716cfedad3a2..bbba99f207ca 100644
@@ -10,9 +10,14 @@
  #include <linux/bug.h>
  #include <asm/patch.h>
  #include <asm/alternative.h>
+#include <asm/csr.h>
  #include <asm/vendorid_list.h>
  #include <asm/errata_list.h>
 
+extern void (*excp_vect_table[])(struct pt_regs *regs);
+extern void sifive_cip_453_insn_fault_trp(struct pt_regs *regs);
+extern void sifive_cip_453_page_fault_trp(struct pt_regs *regs);
+
  struct errata_info_t {
  	char name[32];
  	bool (*check_func)(unsigned long  arch_id, unsigned long impid);
@@ -20,6 +25,9 @@ struct errata_info_t {
 
  static bool errata_cip_453_check_func(unsigned long  arch_id, unsigned long impid)
  {
+	if (!IS_ENABLED(CONFIG_ERRATA_SIFIVE_CIP_453))
+		return false;
+
  	/*
  	 * Affected cores:
  	 * Architecture ID: 0x8000000000000007
@@ -51,10 +59,6 @@ static bool errata_cip_1200_check_func(unsigned long  arch_id, unsigned long imp
  }
 
  static struct errata_info_t errata_list[ERRATA_SIFIVE_NUMBER] = {
-	{
-		.name = "cip-453",
-		.check_func = errata_cip_453_check_func
-	},
  	{
  		.name = "cip-1200",
  		.check_func = errata_cip_1200_check_func
@@ -62,11 +66,20 @@ static struct errata_info_t errata_list[ERRATA_SIFIVE_NUMBER] = {
  };
 
  static u32 __init_or_module sifive_errata_probe(unsigned long archid,
-						unsigned long impid)
+						unsigned long impid,
+						unsigned int stage)
  {
  	int idx;
  	u32 cpu_req_errata = 0;
 
+	if (stage == RISCV_ALTERNATIVES_BOOT) {
+		if (IS_ENABLED(CONFIG_MMU) &&
+		    errata_cip_453_check_func(archid, impid)) {
+			excp_vect_table[EXC_INST_ACCESS] = sifive_cip_453_insn_fault_trp;
+			excp_vect_table[EXC_INST_PAGE_FAULT] = sifive_cip_453_page_fault_trp;
+		}
+	}
+
  	for (idx = 0; idx < ERRATA_SIFIVE_NUMBER; idx++)
  		if (errata_list[idx].check_func(archid, impid))
  			cpu_req_errata |= (1U << idx);
@@ -99,7 +112,7 @@ void sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
  	if (stage == RISCV_ALTERNATIVES_EARLY_BOOT)
  		return;
 
-	cpu_req_errata = sifive_errata_probe(archid, impid);
+	cpu_req_errata = sifive_errata_probe(archid, impid, stage);
 
  	for (alt = begin; alt < end; alt++) {
  		if (alt->vendor_id != SIFIVE_VENDOR_ID)
arch/riscv/include/asm/asm-prototypes.h +1 -0
index cd627ec289f1..81a1f27fa54f 100644
@@ -55,5 +55,6 @@ DECLARE_DO_ERROR_INFO(do_trap_break);
  asmlinkage void handle_bad_stack(struct pt_regs *regs);
  asmlinkage void do_page_fault(struct pt_regs *regs);
  asmlinkage void do_irq(struct pt_regs *regs);
+asmlinkage void do_traps(struct pt_regs *regs);
 
  #endif /* _ASM_RISCV_PROTOTYPES_H */
arch/riscv/include/asm/errata_list.h +2 -3
index 7c8a71a526a3..95b79afc4061 100644
@@ -17,9 +17,8 @@
  #endif
 
  #ifdef CONFIG_ERRATA_SIFIVE
-#define	ERRATA_SIFIVE_CIP_453 0
-#define	ERRATA_SIFIVE_CIP_1200 1
-#define	ERRATA_SIFIVE_NUMBER 2
+#define	ERRATA_SIFIVE_CIP_1200 0
+#define	ERRATA_SIFIVE_NUMBER 1
  #endif
 
  #ifdef CONFIG_ERRATA_THEAD
arch/riscv/kernel/entry.S +0 -2
index 81dec627a8d4..401bfe85a098 100644
@@ -62,13 +62,11 @@ SYM_CODE_START(handle_exception)
  	csrrc s1, CSR_STATUS, t0
  	csrr s2, CSR_EPC
  	csrr s3, CSR_TVAL
-	csrr s4, CSR_CAUSE
  	csrr s5, CSR_SCRATCH
  	REG_S s0, PT_SP(sp)
  	REG_S s1, PT_STATUS(sp)
  	REG_S s2, PT_EPC(sp)
  	REG_S s3, PT_BADADDR(sp)
-	REG_S s4, PT_CAUSE(sp)
  	REG_S s5, PT_TP(sp)
 
  	/*

arch/riscv/kernel/entry.S SYM_CODE_START(handle_exception) · line 89

Quoted 7 lines
arch/riscv/kernel/entry.S +0 -4
@@ -83,36 +81,9 @@ SYM_CODE_START(handle_exception)
  	/* Load the kernel shadow call stack pointer if coming from userspace */
  	scs_load_current_if_task_changed s5
 
-#ifdef CONFIG_RISCV_ISA_V_PREEMPTIVE
-	move a0, sp
-	call riscv_v_context_nesting_start
-#endif

Along with removing this, can the asmlinkage in asm-prototypes.h be removed? Or are you keeping the _start() in because the _end() needs to stay?

That leads me to think about leaving the call to riscv_context_nesting_start() in asm here for the symmetry of _start() and _end() being called from entry.S.

Quoted 116 lines
arch/riscv/kernel/entry.S +1 -51
  	move a0, sp /* pt_regs */
-
-	/*
-	 * MSB of cause differentiates between
-	 * interrupts and exceptions
-	 */
-	bge s4, zero, 1f
-
-	/* Handle interrupts */
-	call do_irq
-	j ret_from_exception
-1:
-	/* Handle other exceptions */
-	slli t0, s4, RISCV_LGPTR
-	la t1, excp_vect_table
-	la t2, excp_vect_table_end
-	add t0, t1, t0
-	/* Check if exception code lies within bounds */
-	bgeu t0, t2, 3f
-	REG_L t1, 0(t0)
-2:	jalr t1
+	call do_traps
  	j ret_from_exception
-3:
-
-	la t1, do_trap_unknown
-	j 2b
  SYM_CODE_END(handle_exception)
  ASM_NOKPROBE(handle_exception)
 
@@ -329,33 +300,6 @@ SYM_FUNC_START(__switch_to)
  	ret
  SYM_FUNC_END(__switch_to)
 
-#ifndef CONFIG_MMU
-#define do_page_fault do_trap_unknown
-#endif
-
-	.section ".rodata"
-	.align LGREG
-	/* Exception vector table */
-SYM_DATA_START_LOCAL(excp_vect_table)
-	RISCV_PTR do_trap_insn_misaligned
-	ALT_INSN_FAULT(RISCV_PTR do_trap_insn_fault)
-	RISCV_PTR do_trap_insn_illegal
-	RISCV_PTR do_trap_break
-	RISCV_PTR do_trap_load_misaligned
-	RISCV_PTR do_trap_load_fault
-	RISCV_PTR do_trap_store_misaligned
-	RISCV_PTR do_trap_store_fault
-	RISCV_PTR do_trap_ecall_u /* system call */
-	RISCV_PTR do_trap_ecall_s
-	RISCV_PTR do_trap_unknown
-	RISCV_PTR do_trap_ecall_m
-	/* instruciton page fault */
-	ALT_PAGE_FAULT(RISCV_PTR do_page_fault)
-	RISCV_PTR do_page_fault   /* load page fault */
-	RISCV_PTR do_trap_unknown
-	RISCV_PTR do_page_fault   /* store page fault */
-SYM_DATA_END_LABEL(excp_vect_table, SYM_L_LOCAL, excp_vect_table_end)
-
  #ifndef CONFIG_MMU
  SYM_DATA_START(__user_rt_sigreturn)
  	li a7, __NR_rt_sigreturn
arch/riscv/kernel/traps.c +41 -0
index 05a16b1f0aee..b44d4a8d4083 100644
@@ -390,6 +390,47 @@ asmlinkage void noinstr do_irq(struct pt_regs *regs)
  	irqentry_exit(regs, state);
  }
 
+void (*excp_vect_table[])(struct pt_regs *regs) __ro_after_init = {
+	do_trap_insn_misaligned,	/*  0 Instruction address misaligned */
+	do_trap_insn_fault,		/*  1 Instruction access fault */
+	do_trap_insn_illegal,		/*  2 Illegal instruction */
+	do_trap_break,			/*  3 Breakpoint */
+	do_trap_load_misaligned,	/*  4 Load address misaligned */
+	do_trap_load_fault,		/*  5 Load access fault */
+	do_trap_store_misaligned,	/*  6 Store/AMO address misaligned */
+	do_trap_store_fault,		/*  7 Store/AMO access fault */
+	do_trap_ecall_u,		/*  8 Environment call from U-mode */
+	do_trap_ecall_s,		/*  9 Environment call from S-mode */
+	do_trap_unknown,		/* 10 Reserved */
+	do_trap_ecall_m,		/* 11 Environment call from M-mode */
+#ifdef CONFIG_MMU
+	do_page_fault,			/* 12 Instruciton page fault */
+	do_page_fault,			/* 13 Load page fault */
+	do_trap_unknown,		/* 14 Reserved */
+	do_page_fault,			/* 15 Store/AMO page fault */
+#endif
+};
+
+asmlinkage void noinstr do_traps(struct pt_regs *regs)
+{
+	unsigned long cause = csr_read(CSR_CAUSE);
+
+	regs->cause = cause;
+
+#ifdef CONFIG_RISCV_ISA_V_PREEMPTIVE
+	riscv_v_context_nesting_start(regs);
+#endif
+	if (cause & CAUSE_IRQ_FLAG) {
+		do_irq(regs);
+	} else {
+		if (cause >= ARRAY_SIZE(excp_vect_table)) {
+			do_trap_unknown(regs);
+			return;
+		}
+		excp_vect_table[cause](regs);
+	}
+}
+
  #ifdef CONFIG_GENERIC_BUG
  int is_valid_bugaddr(unsigned long pc)
  {
#13 lore raw
Quoted 19 lines · Cyril Bur
Quoted 16 lines · Jisheng Zhang

For readability, maintainability and future scalability, convert the bottom half of the exception handling to C.

Mostly the assembly code is converted to C in a relatively straightforward manner.

However, there are two modifications I need to mention:

1. the CSR_CAUSE reg reading and saving is moved to the C code because we need the cause to dispatch the exception handling, if we keep the cause reading and saving, we either pass it to do_traps() via. 2nd param or get it from pt_regs which an extra memory load is needed, I don't like any of the two solutions becase the exception handling sits in hot code path, every instruction matters.

Quoted 6 lines
Quoted 4 lines

2.To cope with SIFIVE_CIP_453 errata, it looks like we don't need alternative mechanism any more after the asm->c convertion. Just replace the excp_vect_table two entries.

Hi Jisheng,

Hi Cyril,

Quoted 5 lines

Sorry I've been having email client problems. This email if from a few days ago.

I really like the look of the patch. I just have one question.

thanks!

Quoted 115 lines
Quoted 115 lines
---
  arch/riscv/errata/sifive/errata.c       | 25 ++++++++---
  arch/riscv/include/asm/asm-prototypes.h |  1 +
  arch/riscv/include/asm/errata_list.h    |  5 +--
  arch/riscv/kernel/entry.S               | 58 +------------------------
  arch/riscv/kernel/traps.c               | 41 +++++++++++++++++
  5 files changed, 64 insertions(+), 66 deletions(-)
arch/riscv/errata/sifive/errata.c +19 -6
index 716cfedad3a2..bbba99f207ca 100644
@@ -10,9 +10,14 @@
  #include <linux/bug.h>
  #include <asm/patch.h>
  #include <asm/alternative.h>
+#include <asm/csr.h>
  #include <asm/vendorid_list.h>
  #include <asm/errata_list.h>
+extern void (*excp_vect_table[])(struct pt_regs *regs);
+extern void sifive_cip_453_insn_fault_trp(struct pt_regs *regs);
+extern void sifive_cip_453_page_fault_trp(struct pt_regs *regs);
+
  struct errata_info_t {
  	char name[32];
  	bool (*check_func)(unsigned long  arch_id, unsigned long impid);
@@ -20,6 +25,9 @@ struct errata_info_t {
  static bool errata_cip_453_check_func(unsigned long  arch_id, unsigned long impid)
  {
+	if (!IS_ENABLED(CONFIG_ERRATA_SIFIVE_CIP_453))
+		return false;
+
  	/*
  	 * Affected cores:
  	 * Architecture ID: 0x8000000000000007
@@ -51,10 +59,6 @@ static bool errata_cip_1200_check_func(unsigned long  arch_id, unsigned long imp
  }
  static struct errata_info_t errata_list[ERRATA_SIFIVE_NUMBER] = {
-	{
-		.name = "cip-453",
-		.check_func = errata_cip_453_check_func
-	},
  	{
  		.name = "cip-1200",
  		.check_func = errata_cip_1200_check_func
@@ -62,11 +66,20 @@ static struct errata_info_t errata_list[ERRATA_SIFIVE_NUMBER] = {
  };
  static u32 __init_or_module sifive_errata_probe(unsigned long archid,
-						unsigned long impid)
+						unsigned long impid,
+						unsigned int stage)
  {
  	int idx;
  	u32 cpu_req_errata = 0;
+	if (stage == RISCV_ALTERNATIVES_BOOT) {
+		if (IS_ENABLED(CONFIG_MMU) &&
+		    errata_cip_453_check_func(archid, impid)) {
+			excp_vect_table[EXC_INST_ACCESS] = sifive_cip_453_insn_fault_trp;
+			excp_vect_table[EXC_INST_PAGE_FAULT] = sifive_cip_453_page_fault_trp;
+		}
+	}
+
  	for (idx = 0; idx < ERRATA_SIFIVE_NUMBER; idx++)
  		if (errata_list[idx].check_func(archid, impid))
  			cpu_req_errata |= (1U << idx);
@@ -99,7 +112,7 @@ void sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
  	if (stage == RISCV_ALTERNATIVES_EARLY_BOOT)
  		return;
-	cpu_req_errata = sifive_errata_probe(archid, impid);
+	cpu_req_errata = sifive_errata_probe(archid, impid, stage);
  	for (alt = begin; alt < end; alt++) {
  		if (alt->vendor_id != SIFIVE_VENDOR_ID)
arch/riscv/include/asm/asm-prototypes.h +1 -0
index cd627ec289f1..81a1f27fa54f 100644
@@ -55,5 +55,6 @@ DECLARE_DO_ERROR_INFO(do_trap_break);
  asmlinkage void handle_bad_stack(struct pt_regs *regs);
  asmlinkage void do_page_fault(struct pt_regs *regs);
  asmlinkage void do_irq(struct pt_regs *regs);
+asmlinkage void do_traps(struct pt_regs *regs);
  #endif /* _ASM_RISCV_PROTOTYPES_H */
arch/riscv/include/asm/errata_list.h +2 -3
index 7c8a71a526a3..95b79afc4061 100644
@@ -17,9 +17,8 @@
  #endif
  #ifdef CONFIG_ERRATA_SIFIVE
-#define	ERRATA_SIFIVE_CIP_453 0
-#define	ERRATA_SIFIVE_CIP_1200 1
-#define	ERRATA_SIFIVE_NUMBER 2
+#define	ERRATA_SIFIVE_CIP_1200 0
+#define	ERRATA_SIFIVE_NUMBER 1
  #endif
  #ifdef CONFIG_ERRATA_THEAD
arch/riscv/kernel/entry.S +0 -2
index 81dec627a8d4..401bfe85a098 100644
@@ -62,13 +62,11 @@ SYM_CODE_START(handle_exception)
  	csrrc s1, CSR_STATUS, t0
  	csrr s2, CSR_EPC
  	csrr s3, CSR_TVAL
-	csrr s4, CSR_CAUSE
  	csrr s5, CSR_SCRATCH
  	REG_S s0, PT_SP(sp)
  	REG_S s1, PT_STATUS(sp)
  	REG_S s2, PT_EPC(sp)
  	REG_S s3, PT_BADADDR(sp)
-	REG_S s4, PT_CAUSE(sp)
  	REG_S s5, PT_TP(sp)
  	/*
Quoted 8 lines

arch/riscv/kernel/entry.S SYM_CODE_START(handle_exception) · line 88

Quoted 7 lines
arch/riscv/kernel/entry.S +0 -4
@@ -83,36 +81,9 @@ SYM_CODE_START(handle_exception)
  	/* Load the kernel shadow call stack pointer if coming from userspace */
  	scs_load_current_if_task_changed s5
-#ifdef CONFIG_RISCV_ISA_V_PREEMPTIVE
-	move a0, sp
-	call riscv_v_context_nesting_start
-#endif

Along with removing this, can the asmlinkage in asm-prototypes.h be removed?

the asmlinkage is removed in patch6 of the series ;)

Quoted 5 lines

Or are you keeping the _start() in because the _end() needs to stay?

That leads me to think about leaving the call to riscv_context_nesting_start() in asm here for the symmetry of _start() and _end() being called from entry.S.

seems a good idea, symmetry matters. will do in new version

Quoted 114 lines
Quoted 114 lines
arch/riscv/kernel/entry.S +1 -51
  	move a0, sp /* pt_regs */
-
-	/*
-	 * MSB of cause differentiates between
-	 * interrupts and exceptions
-	 */
-	bge s4, zero, 1f
-
-	/* Handle interrupts */
-	call do_irq
-	j ret_from_exception
-1:
-	/* Handle other exceptions */
-	slli t0, s4, RISCV_LGPTR
-	la t1, excp_vect_table
-	la t2, excp_vect_table_end
-	add t0, t1, t0
-	/* Check if exception code lies within bounds */
-	bgeu t0, t2, 3f
-	REG_L t1, 0(t0)
-2:	jalr t1
+	call do_traps
  	j ret_from_exception
-3:
-
-	la t1, do_trap_unknown
-	j 2b
  SYM_CODE_END(handle_exception)
  ASM_NOKPROBE(handle_exception)
@@ -329,33 +300,6 @@ SYM_FUNC_START(__switch_to)
  	ret
  SYM_FUNC_END(__switch_to)
-#ifndef CONFIG_MMU
-#define do_page_fault do_trap_unknown
-#endif
-
-	.section ".rodata"
-	.align LGREG
-	/* Exception vector table */
-SYM_DATA_START_LOCAL(excp_vect_table)
-	RISCV_PTR do_trap_insn_misaligned
-	ALT_INSN_FAULT(RISCV_PTR do_trap_insn_fault)
-	RISCV_PTR do_trap_insn_illegal
-	RISCV_PTR do_trap_break
-	RISCV_PTR do_trap_load_misaligned
-	RISCV_PTR do_trap_load_fault
-	RISCV_PTR do_trap_store_misaligned
-	RISCV_PTR do_trap_store_fault
-	RISCV_PTR do_trap_ecall_u /* system call */
-	RISCV_PTR do_trap_ecall_s
-	RISCV_PTR do_trap_unknown
-	RISCV_PTR do_trap_ecall_m
-	/* instruciton page fault */
-	ALT_PAGE_FAULT(RISCV_PTR do_page_fault)
-	RISCV_PTR do_page_fault   /* load page fault */
-	RISCV_PTR do_trap_unknown
-	RISCV_PTR do_page_fault   /* store page fault */
-SYM_DATA_END_LABEL(excp_vect_table, SYM_L_LOCAL, excp_vect_table_end)
-
  #ifndef CONFIG_MMU
  SYM_DATA_START(__user_rt_sigreturn)
  	li a7, __NR_rt_sigreturn
arch/riscv/kernel/traps.c +41 -0
index 05a16b1f0aee..b44d4a8d4083 100644
@@ -390,6 +390,47 @@ asmlinkage void noinstr do_irq(struct pt_regs *regs)
  	irqentry_exit(regs, state);
  }
+void (*excp_vect_table[])(struct pt_regs *regs) __ro_after_init = {
+	do_trap_insn_misaligned,	/*  0 Instruction address misaligned */
+	do_trap_insn_fault,		/*  1 Instruction access fault */
+	do_trap_insn_illegal,		/*  2 Illegal instruction */
+	do_trap_break,			/*  3 Breakpoint */
+	do_trap_load_misaligned,	/*  4 Load address misaligned */
+	do_trap_load_fault,		/*  5 Load access fault */
+	do_trap_store_misaligned,	/*  6 Store/AMO address misaligned */
+	do_trap_store_fault,		/*  7 Store/AMO access fault */
+	do_trap_ecall_u,		/*  8 Environment call from U-mode */
+	do_trap_ecall_s,		/*  9 Environment call from S-mode */
+	do_trap_unknown,		/* 10 Reserved */
+	do_trap_ecall_m,		/* 11 Environment call from M-mode */
+#ifdef CONFIG_MMU
+	do_page_fault,			/* 12 Instruciton page fault */
+	do_page_fault,			/* 13 Load page fault */
+	do_trap_unknown,		/* 14 Reserved */
+	do_page_fault,			/* 15 Store/AMO page fault */
+#endif
+};
+
+asmlinkage void noinstr do_traps(struct pt_regs *regs)
+{
+	unsigned long cause = csr_read(CSR_CAUSE);
+
+	regs->cause = cause;
+
+#ifdef CONFIG_RISCV_ISA_V_PREEMPTIVE
+	riscv_v_context_nesting_start(regs);
+#endif
+	if (cause & CAUSE_IRQ_FLAG) {
+		do_irq(regs);
+	} else {
+		if (cause >= ARRAY_SIZE(excp_vect_table)) {
+			do_trap_unknown(regs);
+			return;
+		}
+		excp_vect_table[cause](regs);
+	}
+}
+
  #ifdef CONFIG_GENERIC_BUG
  int is_valid_bugaddr(unsigned long pc)
  {
#14 lore raw
In reply to Jisheng Zhang
Quoted 2 lines · Jisheng Zhang

For readability, maintainability and future scalability, convert the bottom half of the exception handling to C.

I would be interested to see the performance impact of this, since this mostly eliminates the code from patch 1 and replaces it in C. Anton/Cyril can you share the test case you used so that these changes can also be benchmarked?

Quoted 247 lines

Mostly the assembly code is converted to C in a relatively straightforward manner.

However, there are two modifications I need to mention:

1. the CSR_CAUSE reg reading and saving is moved to the C code because we need the cause to dispatch the exception handling, if we keep the cause reading and saving, we either pass it to do_traps() via. 2nd param or get it from pt_regs which an extra memory load is needed, I don't like any of the two solutions becase the exception handling sits in hot code path, every instruction matters.

2.To cope with SIFIVE_CIP_453 errata, it looks like we don't need alternative mechanism any more after the asm->c convertion. Just replace the excp_vect_table two entries.

---
 arch/riscv/errata/sifive/errata.c       | 25 ++++++++---
 arch/riscv/include/asm/asm-prototypes.h |  1 +
 arch/riscv/include/asm/errata_list.h    |  5 +--
 arch/riscv/kernel/entry.S               | 58 +------------------------
 arch/riscv/kernel/traps.c               | 41 +++++++++++++++++
 5 files changed, 64 insertions(+), 66 deletions(-)
arch/riscv/errata/sifive/errata.c +19 -6
index 716cfedad3a2..bbba99f207ca 100644
@@ -10,9 +10,14 @@
 #include <linux/bug.h>
 #include <asm/patch.h>
 #include <asm/alternative.h>
+#include <asm/csr.h>
 #include <asm/vendorid_list.h>
 #include <asm/errata_list.h>
 
+extern void (*excp_vect_table[])(struct pt_regs *regs);
+extern void sifive_cip_453_insn_fault_trp(struct pt_regs *regs);
+extern void sifive_cip_453_page_fault_trp(struct pt_regs *regs);
+
 struct errata_info_t {
 	char name[32];
 	bool (*check_func)(unsigned long  arch_id, unsigned long impid);
@@ -20,6 +25,9 @@ struct errata_info_t {
 
 static bool errata_cip_453_check_func(unsigned long  arch_id, unsigned long impid)
 {
+	if (!IS_ENABLED(CONFIG_ERRATA_SIFIVE_CIP_453))
+		return false;
+
 	/*
 	 * Affected cores:
 	 * Architecture ID: 0x8000000000000007
@@ -51,10 +59,6 @@ static bool errata_cip_1200_check_func(unsigned long  arch_id, unsigned long imp
 }
 
 static struct errata_info_t errata_list[ERRATA_SIFIVE_NUMBER] = {
-	{
-		.name = "cip-453",
-		.check_func = errata_cip_453_check_func
-	},
 	{
 		.name = "cip-1200",
 		.check_func = errata_cip_1200_check_func
@@ -62,11 +66,20 @@ static struct errata_info_t errata_list[ERRATA_SIFIVE_NUMBER] = {
 };
 
 static u32 __init_or_module sifive_errata_probe(unsigned long archid,
-						unsigned long impid)
+						unsigned long impid,
+						unsigned int stage)
 {
 	int idx;
 	u32 cpu_req_errata = 0;
 
+	if (stage == RISCV_ALTERNATIVES_BOOT) {
+		if (IS_ENABLED(CONFIG_MMU) &&
+		    errata_cip_453_check_func(archid, impid)) {
+			excp_vect_table[EXC_INST_ACCESS] = sifive_cip_453_insn_fault_trp;
+			excp_vect_table[EXC_INST_PAGE_FAULT] = sifive_cip_453_page_fault_trp;
+		}
+	}
+
 	for (idx = 0; idx < ERRATA_SIFIVE_NUMBER; idx++)
 		if (errata_list[idx].check_func(archid, impid))
 			cpu_req_errata |= (1U << idx);
@@ -99,7 +112,7 @@ void sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
 	if (stage == RISCV_ALTERNATIVES_EARLY_BOOT)
 		return;
 
-	cpu_req_errata = sifive_errata_probe(archid, impid);
+	cpu_req_errata = sifive_errata_probe(archid, impid, stage);
 
 	for (alt = begin; alt < end; alt++) {
 		if (alt->vendor_id != SIFIVE_VENDOR_ID)
arch/riscv/include/asm/asm-prototypes.h +1 -0
index cd627ec289f1..81a1f27fa54f 100644
@@ -55,5 +55,6 @@ DECLARE_DO_ERROR_INFO(do_trap_break);
 asmlinkage void handle_bad_stack(struct pt_regs *regs);
 asmlinkage void do_page_fault(struct pt_regs *regs);
 asmlinkage void do_irq(struct pt_regs *regs);
+asmlinkage void do_traps(struct pt_regs *regs);
 
 #endif /* _ASM_RISCV_PROTOTYPES_H */
arch/riscv/include/asm/errata_list.h +2 -3
index 7c8a71a526a3..95b79afc4061 100644
@@ -17,9 +17,8 @@
 #endif
 
 #ifdef CONFIG_ERRATA_SIFIVE
-#define	ERRATA_SIFIVE_CIP_453 0
-#define	ERRATA_SIFIVE_CIP_1200 1
-#define	ERRATA_SIFIVE_NUMBER 2
+#define	ERRATA_SIFIVE_CIP_1200 0
+#define	ERRATA_SIFIVE_NUMBER 1
 #endif
 
 #ifdef CONFIG_ERRATA_THEAD
arch/riscv/kernel/entry.S +1 -57
index 81dec627a8d4..401bfe85a098 100644
@@ -62,13 +62,11 @@ SYM_CODE_START(handle_exception)
 	csrrc s1, CSR_STATUS, t0
 	csrr s2, CSR_EPC
 	csrr s3, CSR_TVAL
-	csrr s4, CSR_CAUSE
 	csrr s5, CSR_SCRATCH
 	REG_S s0, PT_SP(sp)
 	REG_S s1, PT_STATUS(sp)
 	REG_S s2, PT_EPC(sp)
 	REG_S s3, PT_BADADDR(sp)
-	REG_S s4, PT_CAUSE(sp)
 	REG_S s5, PT_TP(sp)
 
 	/*
@@ -83,36 +81,9 @@ SYM_CODE_START(handle_exception)
 	/* Load the kernel shadow call stack pointer if coming from userspace */
 	scs_load_current_if_task_changed s5
 
-#ifdef CONFIG_RISCV_ISA_V_PREEMPTIVE
-	move a0, sp
-	call riscv_v_context_nesting_start
-#endif
 	move a0, sp /* pt_regs */
-
-	/*
-	 * MSB of cause differentiates between
-	 * interrupts and exceptions
-	 */
-	bge s4, zero, 1f
-
-	/* Handle interrupts */
-	call do_irq
-	j ret_from_exception
-1:
-	/* Handle other exceptions */
-	slli t0, s4, RISCV_LGPTR
-	la t1, excp_vect_table
-	la t2, excp_vect_table_end
-	add t0, t1, t0
-	/* Check if exception code lies within bounds */
-	bgeu t0, t2, 3f
-	REG_L t1, 0(t0)
-2:	jalr t1
+	call do_traps
 	j ret_from_exception
-3:
-
-	la t1, do_trap_unknown
-	j 2b
 SYM_CODE_END(handle_exception)
 ASM_NOKPROBE(handle_exception)
 
@@ -329,33 +300,6 @@ SYM_FUNC_START(__switch_to)
 	ret
 SYM_FUNC_END(__switch_to)
 
-#ifndef CONFIG_MMU
-#define do_page_fault do_trap_unknown
-#endif
-
-	.section ".rodata"
-	.align LGREG
-	/* Exception vector table */
-SYM_DATA_START_LOCAL(excp_vect_table)
-	RISCV_PTR do_trap_insn_misaligned
-	ALT_INSN_FAULT(RISCV_PTR do_trap_insn_fault)
-	RISCV_PTR do_trap_insn_illegal
-	RISCV_PTR do_trap_break
-	RISCV_PTR do_trap_load_misaligned
-	RISCV_PTR do_trap_load_fault
-	RISCV_PTR do_trap_store_misaligned
-	RISCV_PTR do_trap_store_fault
-	RISCV_PTR do_trap_ecall_u /* system call */
-	RISCV_PTR do_trap_ecall_s
-	RISCV_PTR do_trap_unknown
-	RISCV_PTR do_trap_ecall_m
-	/* instruciton page fault */
-	ALT_PAGE_FAULT(RISCV_PTR do_page_fault)
-	RISCV_PTR do_page_fault   /* load page fault */
-	RISCV_PTR do_trap_unknown
-	RISCV_PTR do_page_fault   /* store page fault */
-SYM_DATA_END_LABEL(excp_vect_table, SYM_L_LOCAL, excp_vect_table_end)
-
 #ifndef CONFIG_MMU
 SYM_DATA_START(__user_rt_sigreturn)
 	li a7, __NR_rt_sigreturn
arch/riscv/kernel/traps.c +24 -0
index 05a16b1f0aee..b44d4a8d4083 100644
@@ -390,6 +390,47 @@ asmlinkage void noinstr do_irq(struct pt_regs *regs)
 	irqentry_exit(regs, state);
 }
 
+void (*excp_vect_table[])(struct pt_regs *regs) __ro_after_init = {
+	do_trap_insn_misaligned,	/*  0 Instruction address misaligned */
+	do_trap_insn_fault,		/*  1 Instruction access fault */
+	do_trap_insn_illegal,		/*  2 Illegal instruction */
+	do_trap_break,			/*  3 Breakpoint */
+	do_trap_load_misaligned,	/*  4 Load address misaligned */
+	do_trap_load_fault,		/*  5 Load access fault */
+	do_trap_store_misaligned,	/*  6 Store/AMO address misaligned */
+	do_trap_store_fault,		/*  7 Store/AMO access fault */
+	do_trap_ecall_u,		/*  8 Environment call from U-mode */
+	do_trap_ecall_s,		/*  9 Environment call from S-mode */
+	do_trap_unknown,		/* 10 Reserved */
+	do_trap_ecall_m,		/* 11 Environment call from M-mode */
+#ifdef CONFIG_MMU
+	do_page_fault,			/* 12 Instruciton page fault */
+	do_page_fault,			/* 13 Load page fault */
+	do_trap_unknown,		/* 14 Reserved */
+	do_page_fault,			/* 15 Store/AMO page fault */
+#endif
+};
+
+asmlinkage void noinstr do_traps(struct pt_regs *regs)
+{
+	unsigned long cause = csr_read(CSR_CAUSE);

arch/riscv/kernel/traps.c asmlinkage void noinstr do_irq(struct pt_regs *regs) · line 426

Quoted 10 lines
arch/riscv/kernel/traps.c +10 -0
+
+	regs->cause = cause;
+
+#ifdef CONFIG_RISCV_ISA_V_PREEMPTIVE
+	riscv_v_context_nesting_start(regs);
+#endif
+	if (cause & CAUSE_IRQ_FLAG) {
+		do_irq(regs);
+	} else {
+		if (cause >= ARRAY_SIZE(excp_vect_table)) {

Using unlikely here may optimize the hotpath here slightly.

- Charlie

Quoted 18 lines
arch/riscv/kernel/traps.c +7 -0
+			do_trap_unknown(regs);
+			return;
+		}
+		excp_vect_table[cause](regs);
+	}
+}
+
 #ifdef CONFIG_GENERIC_BUG
 int is_valid_bugaddr(unsigned long pc)
 {
Signature · 7 lines
2.43.0


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
#15 lore raw
Quoted 8 lines · Charlie Jenkins
Quoted 2 lines · Jisheng Zhang

For readability, maintainability and future scalability, convert the bottom half of the exception handling to C.

I would be interested to see the performance impact of this, since this mostly eliminates the code from patch 1 and replaces it in C. Anton/Cyril can you share the test case you used so that these changes can also be benchmarked?

I ran the series as a whole and it keeps the performance improvement. I think that makes sense because C shouldn't be modifying return addresses.

The benchmark is a tweaked (for obvious reasons) of tools/testing/selftests/powerpc/benchmarks/null_syscall.c.

Quoted 281 lines
Quoted 248 lines

Mostly the assembly code is converted to C in a relatively straightforward manner.

However, there are two modifications I need to mention:

1. the CSR_CAUSE reg reading and saving is moved to the C code because we need the cause to dispatch the exception handling, if we keep the cause reading and saving, we either pass it to do_traps() via. 2nd param or get it from pt_regs which an extra memory load is needed, I don't like any of the two solutions becase the exception handling sits in hot code path, every instruction matters.

2.To cope with SIFIVE_CIP_453 errata, it looks like we don't need alternative mechanism any more after the asm->c convertion. Just replace the excp_vect_table two entries.

---
  arch/riscv/errata/sifive/errata.c       | 25 ++++++++---
  arch/riscv/include/asm/asm-prototypes.h |  1 +
  arch/riscv/include/asm/errata_list.h    |  5 +--
  arch/riscv/kernel/entry.S               | 58 +------------------------
  arch/riscv/kernel/traps.c               | 41 +++++++++++++++++
  5 files changed, 64 insertions(+), 66 deletions(-)
arch/riscv/errata/sifive/errata.c +19 -6
index 716cfedad3a2..bbba99f207ca 100644
@@ -10,9 +10,14 @@
  #include <linux/bug.h>
  #include <asm/patch.h>
  #include <asm/alternative.h>
+#include <asm/csr.h>
  #include <asm/vendorid_list.h>
  #include <asm/errata_list.h>
 
+extern void (*excp_vect_table[])(struct pt_regs *regs);
+extern void sifive_cip_453_insn_fault_trp(struct pt_regs *regs);
+extern void sifive_cip_453_page_fault_trp(struct pt_regs *regs);
+
  struct errata_info_t {
  	char name[32];
  	bool (*check_func)(unsigned long  arch_id, unsigned long impid);
@@ -20,6 +25,9 @@ struct errata_info_t {
 
  static bool errata_cip_453_check_func(unsigned long  arch_id, unsigned long impid)
  {
+	if (!IS_ENABLED(CONFIG_ERRATA_SIFIVE_CIP_453))
+		return false;
+
  	/*
  	 * Affected cores:
  	 * Architecture ID: 0x8000000000000007
@@ -51,10 +59,6 @@ static bool errata_cip_1200_check_func(unsigned long  arch_id, unsigned long imp
  }
 
  static struct errata_info_t errata_list[ERRATA_SIFIVE_NUMBER] = {
-	{
-		.name = "cip-453",
-		.check_func = errata_cip_453_check_func
-	},
  	{
  		.name = "cip-1200",
  		.check_func = errata_cip_1200_check_func
@@ -62,11 +66,20 @@ static struct errata_info_t errata_list[ERRATA_SIFIVE_NUMBER] = {
  };
 
  static u32 __init_or_module sifive_errata_probe(unsigned long archid,
-						unsigned long impid)
+						unsigned long impid,
+						unsigned int stage)
  {
  	int idx;
  	u32 cpu_req_errata = 0;
 
+	if (stage == RISCV_ALTERNATIVES_BOOT) {
+		if (IS_ENABLED(CONFIG_MMU) &&
+		    errata_cip_453_check_func(archid, impid)) {
+			excp_vect_table[EXC_INST_ACCESS] = sifive_cip_453_insn_fault_trp;
+			excp_vect_table[EXC_INST_PAGE_FAULT] = sifive_cip_453_page_fault_trp;
+		}
+	}
+
  	for (idx = 0; idx < ERRATA_SIFIVE_NUMBER; idx++)
  		if (errata_list[idx].check_func(archid, impid))
  			cpu_req_errata |= (1U << idx);
@@ -99,7 +112,7 @@ void sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
  	if (stage == RISCV_ALTERNATIVES_EARLY_BOOT)
  		return;
 
-	cpu_req_errata = sifive_errata_probe(archid, impid);
+	cpu_req_errata = sifive_errata_probe(archid, impid, stage);
 
  	for (alt = begin; alt < end; alt++) {
  		if (alt->vendor_id != SIFIVE_VENDOR_ID)
arch/riscv/include/asm/asm-prototypes.h +1 -0
index cd627ec289f1..81a1f27fa54f 100644
@@ -55,5 +55,6 @@ DECLARE_DO_ERROR_INFO(do_trap_break);
  asmlinkage void handle_bad_stack(struct pt_regs *regs);
  asmlinkage void do_page_fault(struct pt_regs *regs);
  asmlinkage void do_irq(struct pt_regs *regs);
+asmlinkage void do_traps(struct pt_regs *regs);
 
  #endif /* _ASM_RISCV_PROTOTYPES_H */
arch/riscv/include/asm/errata_list.h +2 -3
index 7c8a71a526a3..95b79afc4061 100644
@@ -17,9 +17,8 @@
  #endif
 
  #ifdef CONFIG_ERRATA_SIFIVE
-#define	ERRATA_SIFIVE_CIP_453 0
-#define	ERRATA_SIFIVE_CIP_1200 1
-#define	ERRATA_SIFIVE_NUMBER 2
+#define	ERRATA_SIFIVE_CIP_1200 0
+#define	ERRATA_SIFIVE_NUMBER 1
  #endif
 
  #ifdef CONFIG_ERRATA_THEAD
arch/riscv/kernel/entry.S +1 -57
index 81dec627a8d4..401bfe85a098 100644
@@ -62,13 +62,11 @@ SYM_CODE_START(handle_exception)
  	csrrc s1, CSR_STATUS, t0
  	csrr s2, CSR_EPC
  	csrr s3, CSR_TVAL
-	csrr s4, CSR_CAUSE
  	csrr s5, CSR_SCRATCH
  	REG_S s0, PT_SP(sp)
  	REG_S s1, PT_STATUS(sp)
  	REG_S s2, PT_EPC(sp)
  	REG_S s3, PT_BADADDR(sp)
-	REG_S s4, PT_CAUSE(sp)
  	REG_S s5, PT_TP(sp)
 
  	/*
@@ -83,36 +81,9 @@ SYM_CODE_START(handle_exception)
  	/* Load the kernel shadow call stack pointer if coming from userspace */
  	scs_load_current_if_task_changed s5
 
-#ifdef CONFIG_RISCV_ISA_V_PREEMPTIVE
-	move a0, sp
-	call riscv_v_context_nesting_start
-#endif
  	move a0, sp /* pt_regs */
-
-	/*
-	 * MSB of cause differentiates between
-	 * interrupts and exceptions
-	 */
-	bge s4, zero, 1f
-
-	/* Handle interrupts */
-	call do_irq
-	j ret_from_exception
-1:
-	/* Handle other exceptions */
-	slli t0, s4, RISCV_LGPTR
-	la t1, excp_vect_table
-	la t2, excp_vect_table_end
-	add t0, t1, t0
-	/* Check if exception code lies within bounds */
-	bgeu t0, t2, 3f
-	REG_L t1, 0(t0)
-2:	jalr t1
+	call do_traps
  	j ret_from_exception
-3:
-
-	la t1, do_trap_unknown
-	j 2b
  SYM_CODE_END(handle_exception)
  ASM_NOKPROBE(handle_exception)
 
@@ -329,33 +300,6 @@ SYM_FUNC_START(__switch_to)
  	ret
  SYM_FUNC_END(__switch_to)
 
-#ifndef CONFIG_MMU
-#define do_page_fault do_trap_unknown
-#endif
-
-	.section ".rodata"
-	.align LGREG
-	/* Exception vector table */
-SYM_DATA_START_LOCAL(excp_vect_table)
-	RISCV_PTR do_trap_insn_misaligned
-	ALT_INSN_FAULT(RISCV_PTR do_trap_insn_fault)
-	RISCV_PTR do_trap_insn_illegal
-	RISCV_PTR do_trap_break
-	RISCV_PTR do_trap_load_misaligned
-	RISCV_PTR do_trap_load_fault
-	RISCV_PTR do_trap_store_misaligned
-	RISCV_PTR do_trap_store_fault
-	RISCV_PTR do_trap_ecall_u /* system call */
-	RISCV_PTR do_trap_ecall_s
-	RISCV_PTR do_trap_unknown
-	RISCV_PTR do_trap_ecall_m
-	/* instruciton page fault */
-	ALT_PAGE_FAULT(RISCV_PTR do_page_fault)
-	RISCV_PTR do_page_fault   /* load page fault */
-	RISCV_PTR do_trap_unknown
-	RISCV_PTR do_page_fault   /* store page fault */
-SYM_DATA_END_LABEL(excp_vect_table, SYM_L_LOCAL, excp_vect_table_end)
-
  #ifndef CONFIG_MMU
  SYM_DATA_START(__user_rt_sigreturn)
  	li a7, __NR_rt_sigreturn
arch/riscv/kernel/traps.c +24 -0
index 05a16b1f0aee..b44d4a8d4083 100644
@@ -390,6 +390,47 @@ asmlinkage void noinstr do_irq(struct pt_regs *regs)
  	irqentry_exit(regs, state);
  }
 
+void (*excp_vect_table[])(struct pt_regs *regs) __ro_after_init = {
+	do_trap_insn_misaligned,	/*  0 Instruction address misaligned */
+	do_trap_insn_fault,		/*  1 Instruction access fault */
+	do_trap_insn_illegal,		/*  2 Illegal instruction */
+	do_trap_break,			/*  3 Breakpoint */
+	do_trap_load_misaligned,	/*  4 Load address misaligned */
+	do_trap_load_fault,		/*  5 Load access fault */
+	do_trap_store_misaligned,	/*  6 Store/AMO address misaligned */
+	do_trap_store_fault,		/*  7 Store/AMO access fault */
+	do_trap_ecall_u,		/*  8 Environment call from U-mode */
+	do_trap_ecall_s,		/*  9 Environment call from S-mode */
+	do_trap_unknown,		/* 10 Reserved */
+	do_trap_ecall_m,		/* 11 Environment call from M-mode */
+#ifdef CONFIG_MMU
+	do_page_fault,			/* 12 Instruciton page fault */
+	do_page_fault,			/* 13 Load page fault */
+	do_trap_unknown,		/* 14 Reserved */
+	do_page_fault,			/* 15 Store/AMO page fault */
+#endif
+};
+
+asmlinkage void noinstr do_traps(struct pt_regs *regs)
+{
+	unsigned long cause = csr_read(CSR_CAUSE);

arch/riscv/kernel/traps.c asmlinkage void noinstr do_irq(struct pt_regs *regs) · line 426

Quoted 10 lines
arch/riscv/kernel/traps.c +10 -0
+
+	regs->cause = cause;
+
+#ifdef CONFIG_RISCV_ISA_V_PREEMPTIVE
+	riscv_v_context_nesting_start(regs);
+#endif
+	if (cause & CAUSE_IRQ_FLAG) {
+		do_irq(regs);
+	} else {
+		if (cause >= ARRAY_SIZE(excp_vect_table)) {

Using unlikely here may optimize the hotpath here slightly.

- Charlie

Quoted 18 lines
arch/riscv/kernel/traps.c +7 -0
+			do_trap_unknown(regs);
+			return;
+		}
+		excp_vect_table[cause](regs);
+	}
+}
+
  #ifdef CONFIG_GENERIC_BUG
  int is_valid_bugaddr(unsigned long pc)
  {
Signature · 7 lines
2.43.0


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
#16 lore raw

PATCH 4/6 riscv: errata: remove ALT_INSN_FAULT and ALT_PAGE_FAULT

In reply to Jisheng Zhang

They are used for SIFIVE_CIP_453 errata, which has been solved by replacing the excp_vect_table[] two entries in last commit. So these two macros are useless now, remove them.

---
 arch/riscv/include/asm/errata_list.h | 16 +---------------
 1 file changed, 1 insertion(+), 15 deletions(-)
arch/riscv/include/asm/errata_list.h +1 -15
index 95b79afc4061..46bf00c4a57a 100644
@@ -27,21 +27,7 @@
 #define	ERRATA_THEAD_NUMBER 2
 #endif
 
-#ifdef __ASSEMBLY__
-
-#define ALT_INSN_FAULT(x)						\
-ALTERNATIVE(__stringify(RISCV_PTR do_trap_insn_fault),			\
-	    __stringify(RISCV_PTR sifive_cip_453_insn_fault_trp),	\
-	    SIFIVE_VENDOR_ID, ERRATA_SIFIVE_CIP_453,			\
-	    CONFIG_ERRATA_SIFIVE_CIP_453)
-
-#define ALT_PAGE_FAULT(x)						\
-ALTERNATIVE(__stringify(RISCV_PTR do_page_fault),			\
-	    __stringify(RISCV_PTR sifive_cip_453_page_fault_trp),	\
-	    SIFIVE_VENDOR_ID, ERRATA_SIFIVE_CIP_453,			\
-	    CONFIG_ERRATA_SIFIVE_CIP_453)
-#else /* !__ASSEMBLY__ */
-
+#ifndef __ASSEMBLY__
 #define ALT_SFENCE_VMA_ASID(asid)					\
 asm(ALTERNATIVE("sfence.vma x0, %0", "sfence.vma", SIFIVE_VENDOR_ID,	\
 		ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200)	\
Signature · 1 line
2.43.0
#17 lore raw

PATCH 5/6 riscv: errata: sifive: remove NOMMU handling

In reply to Jisheng Zhang

Since NOMMU is now properly handling in generic do_traps() which will call do_trap_unknown() for instruciton page fault for NOMMU.

---
 arch/riscv/errata/sifive/errata_cip_453.S | 4 ----
 1 file changed, 4 deletions(-)
arch/riscv/errata/sifive/errata_cip_453.S +0 -4
index f1b9623fe1de..b8ca2ebc2652 100644
@@ -23,11 +23,7 @@
 
 ENTRY(sifive_cip_453_page_fault_trp)
 	ADD_SIGN_EXT a0, t0, t1
-#ifdef CONFIG_MMU
 	la t0, do_page_fault
-#else
-	la t0, do_trap_unknown
-#endif
 	jr t0
 END(sifive_cip_453_page_fault_trp)
 
Signature · 1 line
2.43.0
#18 lore raw

PATCH 6/6 riscv: remove asmlinkage from updated functions

In reply to Jisheng Zhang

Now that the callers of these functions have moved into C, they no longer need the asmlinkage annotation. Remove it

---
 arch/riscv/include/asm/asm-prototypes.h |  6 +++---
 arch/riscv/kernel/traps.c               | 16 ++++++++--------
 2 files changed, 11 insertions(+), 11 deletions(-)
arch/riscv/include/asm/asm-prototypes.h +3 -3
index 81a1f27fa54f..70b86a825922 100644
@@ -37,7 +37,7 @@ asmlinkage void riscv_v_context_nesting_end(struct pt_regs *regs);
 
 #endif /* CONFIG_RISCV_ISA_V */
 
-#define DECLARE_DO_ERROR_INFO(name)	asmlinkage void name(struct pt_regs *regs)
+#define DECLARE_DO_ERROR_INFO(name)	void name(struct pt_regs *regs)
 
 DECLARE_DO_ERROR_INFO(do_trap_unknown);
 DECLARE_DO_ERROR_INFO(do_trap_insn_misaligned);
@@ -53,8 +53,8 @@ DECLARE_DO_ERROR_INFO(do_trap_ecall_m);
 DECLARE_DO_ERROR_INFO(do_trap_break);
 
 asmlinkage void handle_bad_stack(struct pt_regs *regs);
-asmlinkage void do_page_fault(struct pt_regs *regs);
-asmlinkage void do_irq(struct pt_regs *regs);
+void do_page_fault(struct pt_regs *regs);
+void do_irq(struct pt_regs *regs);
 asmlinkage void do_traps(struct pt_regs *regs);
 
 #endif /* _ASM_RISCV_PROTOTYPES_H */
arch/riscv/kernel/traps.c +8 -8
index b44d4a8d4083..ddca8e74fb72 100644
@@ -147,7 +147,7 @@ static void do_trap_error(struct pt_regs *regs, int signo, int code,
 #define __trap_section noinstr
 #endif
 #define DO_ERROR_INFO(name, signo, code, str)					\
-asmlinkage __visible __trap_section void name(struct pt_regs *regs)		\
+__visible __trap_section void name(struct pt_regs *regs)			\
 {										\
 	if (user_mode(regs)) {							\
 		irqentry_enter_from_user_mode(regs);				\
@@ -167,7 +167,7 @@ DO_ERROR_INFO(do_trap_insn_misaligned,
 DO_ERROR_INFO(do_trap_insn_fault,
 	SIGSEGV, SEGV_ACCERR, "instruction access fault");
 
-asmlinkage __visible __trap_section void do_trap_insn_illegal(struct pt_regs *regs)
+__visible __trap_section void do_trap_insn_illegal(struct pt_regs *regs)
 {
 	bool handled;
 
@@ -198,7 +198,7 @@ asmlinkage __visible __trap_section void do_trap_insn_illegal(struct pt_regs *re
 DO_ERROR_INFO(do_trap_load_fault,
 	SIGSEGV, SEGV_ACCERR, "load access fault");
 
-asmlinkage __visible __trap_section void do_trap_load_misaligned(struct pt_regs *regs)
+__visible __trap_section void do_trap_load_misaligned(struct pt_regs *regs)
 {
 	if (user_mode(regs)) {
 		irqentry_enter_from_user_mode(regs);
@@ -219,7 +219,7 @@ asmlinkage __visible __trap_section void do_trap_load_misaligned(struct pt_regs
 	}
 }
 
-asmlinkage __visible __trap_section void do_trap_store_misaligned(struct pt_regs *regs)
+__visible __trap_section void do_trap_store_misaligned(struct pt_regs *regs)
 {
 	if (user_mode(regs)) {
 		irqentry_enter_from_user_mode(regs);
@@ -294,7 +294,7 @@ void handle_break(struct pt_regs *regs)
 		die(regs, "Kernel BUG");
 }
 
-asmlinkage __visible __trap_section void do_trap_break(struct pt_regs *regs)
+__visible __trap_section void do_trap_break(struct pt_regs *regs)
 {
 	if (user_mode(regs)) {
 		irqentry_enter_from_user_mode(regs);
@@ -311,7 +311,7 @@ asmlinkage __visible __trap_section void do_trap_break(struct pt_regs *regs)
 	}
 }
 
-asmlinkage __visible __trap_section  __no_stack_protector
+__visible __trap_section  __no_stack_protector
 void do_trap_ecall_u(struct pt_regs *regs)
 {
 	if (user_mode(regs)) {
@@ -355,7 +355,7 @@ void do_trap_ecall_u(struct pt_regs *regs)
 }
 
 #ifdef CONFIG_MMU
-asmlinkage __visible noinstr void do_page_fault(struct pt_regs *regs)
+__visible noinstr void do_page_fault(struct pt_regs *regs)
 {
 	irqentry_state_t state = irqentry_enter(regs);
 
@@ -378,7 +378,7 @@ static void noinstr handle_riscv_irq(struct pt_regs *regs)
 	irq_exit_rcu();
 }
 
-asmlinkage void noinstr do_irq(struct pt_regs *regs)
+void noinstr do_irq(struct pt_regs *regs)
 {
 	irqentry_state_t state = irqentry_enter(regs);
 
Signature · 1 line
2.43.0
#19 lore raw

Re: [PATCH 6/6] riscv: remove asmlinkage from updated functions

Quoted 2 lines · Jisheng Zhang

Now that the callers of these functions have moved into C, they no longer need the asmlinkage annotation. Remove it

A couple of the functions here can be inlined as well now that they are not called by assembly, this will allow the compiler to get rid of the stack setup/tear down code and eliminate a handful of instructions. The functions that can be inlined are riscv_v_context_nesting_start() and do_irq().

- Charlie

Quoted 115 lines
---
 arch/riscv/include/asm/asm-prototypes.h |  6 +++---
 arch/riscv/kernel/traps.c               | 16 ++++++++--------
 2 files changed, 11 insertions(+), 11 deletions(-)
arch/riscv/include/asm/asm-prototypes.h +3 -3
index 81a1f27fa54f..70b86a825922 100644
@@ -37,7 +37,7 @@ asmlinkage void riscv_v_context_nesting_end(struct pt_regs *regs);
 
 #endif /* CONFIG_RISCV_ISA_V */
 
-#define DECLARE_DO_ERROR_INFO(name)	asmlinkage void name(struct pt_regs *regs)
+#define DECLARE_DO_ERROR_INFO(name)	void name(struct pt_regs *regs)
 
 DECLARE_DO_ERROR_INFO(do_trap_unknown);
 DECLARE_DO_ERROR_INFO(do_trap_insn_misaligned);
@@ -53,8 +53,8 @@ DECLARE_DO_ERROR_INFO(do_trap_ecall_m);
 DECLARE_DO_ERROR_INFO(do_trap_break);
 
 asmlinkage void handle_bad_stack(struct pt_regs *regs);
-asmlinkage void do_page_fault(struct pt_regs *regs);
-asmlinkage void do_irq(struct pt_regs *regs);
+void do_page_fault(struct pt_regs *regs);
+void do_irq(struct pt_regs *regs);
 asmlinkage void do_traps(struct pt_regs *regs);
 
 #endif /* _ASM_RISCV_PROTOTYPES_H */
arch/riscv/kernel/traps.c +8 -8
index b44d4a8d4083..ddca8e74fb72 100644
@@ -147,7 +147,7 @@ static void do_trap_error(struct pt_regs *regs, int signo, int code,
 #define __trap_section noinstr
 #endif
 #define DO_ERROR_INFO(name, signo, code, str)					\
-asmlinkage __visible __trap_section void name(struct pt_regs *regs)		\
+__visible __trap_section void name(struct pt_regs *regs)			\
 {										\
 	if (user_mode(regs)) {							\
 		irqentry_enter_from_user_mode(regs);				\
@@ -167,7 +167,7 @@ DO_ERROR_INFO(do_trap_insn_misaligned,
 DO_ERROR_INFO(do_trap_insn_fault,
 	SIGSEGV, SEGV_ACCERR, "instruction access fault");
 
-asmlinkage __visible __trap_section void do_trap_insn_illegal(struct pt_regs *regs)
+__visible __trap_section void do_trap_insn_illegal(struct pt_regs *regs)
 {
 	bool handled;
 
@@ -198,7 +198,7 @@ asmlinkage __visible __trap_section void do_trap_insn_illegal(struct pt_regs *re
 DO_ERROR_INFO(do_trap_load_fault,
 	SIGSEGV, SEGV_ACCERR, "load access fault");
 
-asmlinkage __visible __trap_section void do_trap_load_misaligned(struct pt_regs *regs)
+__visible __trap_section void do_trap_load_misaligned(struct pt_regs *regs)
 {
 	if (user_mode(regs)) {
 		irqentry_enter_from_user_mode(regs);
@@ -219,7 +219,7 @@ asmlinkage __visible __trap_section void do_trap_load_misaligned(struct pt_regs
 	}
 }
 
-asmlinkage __visible __trap_section void do_trap_store_misaligned(struct pt_regs *regs)
+__visible __trap_section void do_trap_store_misaligned(struct pt_regs *regs)
 {
 	if (user_mode(regs)) {
 		irqentry_enter_from_user_mode(regs);
@@ -294,7 +294,7 @@ void handle_break(struct pt_regs *regs)
 		die(regs, "Kernel BUG");
 }
 
-asmlinkage __visible __trap_section void do_trap_break(struct pt_regs *regs)
+__visible __trap_section void do_trap_break(struct pt_regs *regs)
 {
 	if (user_mode(regs)) {
 		irqentry_enter_from_user_mode(regs);
@@ -311,7 +311,7 @@ asmlinkage __visible __trap_section void do_trap_break(struct pt_regs *regs)
 	}
 }
 
-asmlinkage __visible __trap_section  __no_stack_protector
+__visible __trap_section  __no_stack_protector
 void do_trap_ecall_u(struct pt_regs *regs)
 {
 	if (user_mode(regs)) {
@@ -355,7 +355,7 @@ void do_trap_ecall_u(struct pt_regs *regs)
 }
 
 #ifdef CONFIG_MMU
-asmlinkage __visible noinstr void do_page_fault(struct pt_regs *regs)
+__visible noinstr void do_page_fault(struct pt_regs *regs)
 {
 	irqentry_state_t state = irqentry_enter(regs);
 
@@ -378,7 +378,7 @@ static void noinstr handle_riscv_irq(struct pt_regs *regs)
 	irq_exit_rcu();
 }
 
-asmlinkage void noinstr do_irq(struct pt_regs *regs)
+void noinstr do_irq(struct pt_regs *regs)
 {
 	irqentry_state_t state = irqentry_enter(regs);
 
Signature · 7 lines
2.43.0


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
#20 lore raw

cover riscv: convert bottom half of exception handling to C

In reply to Jisheng Zhang
Quoted 22 lines · Jisheng Zhang

For readability, maintainability and future scalability, convert the bottom half of the exception handling to C.

During the conversion, I found Anton fixed a performance issue and my patches will touch the same exception asm code, so I include Anton's patch for completeness. I also cooked a similar patch to avoid corrupting the RAS in ret_from_fork() per the inspiration.

Mostly the assembly code is converted to C in a relatively straightforward manner.

However, there are two modifications I need to mention:

1. the CSR_CAUSE reg reading and saving is moved to the C code because we need the cause to dispatch the exception handling, if we keep the cause reading and saving, we either pass it to do_traps() via. 2nd param or get it from pt_regs which an extra memory load is needed, I don't like any of the two solutions becase the exception handling sits in hot code path, every instruction matters.

Quoted 9 lines

2.To cope with SIFIVE_CIP_453 errata, it looks like we don't need alternative mechanism any more after the asm->c convertion. Just replace the excp_vect_table two entries.

Anton Blanchard (1):
riscv: Improve exception and system call latency

I've retested this patch with the rest of the series applied. I can confirm that the performance improvement is still there. Definitely thumbs up on my end.

Quoted 24 lines

Jisheng Zhang (5):
riscv: avoid corrupting the RAS
riscv: convert bottom half of exception handling to C
riscv: errata: remove ALT_INSN_FAULT and ALT_PAGE_FAULT
riscv: errata: sifive: remove NOMMU handling
riscv: remove asmlinkage from updated functions

 arch/riscv/errata/sifive/errata.c         | 25 +++++++---
 arch/riscv/errata/sifive/errata_cip_453.S |  4 --
 arch/riscv/include/asm/asm-prototypes.h   |  7 +--
 arch/riscv/include/asm/errata_list.h      | 21 ++------
 arch/riscv/kernel/entry.S                 | 61 ++---------------------
 arch/riscv/kernel/stacktrace.c            |  4 +-
 arch/riscv/kernel/traps.c                 | 57 ++++++++++++++++++---
 7 files changed, 81 insertions(+), 98 deletions(-)

--
2.43.0

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#21 lore raw
Quoted 29 lines · Cyril Bur
Quoted 28 lines · Jisheng Zhang

For readability, maintainability and future scalability, convert the bottom half of the exception handling to C.

During the conversion, I found Anton fixed a performance issue and my patches will touch the same exception asm code, so I include Anton's patch for completeness. I also cooked a similar patch to avoid corrupting the RAS in ret_from_fork() per the inspiration.

Mostly the assembly code is converted to C in a relatively straightforward manner.

However, there are two modifications I need to mention:

1. the CSR_CAUSE reg reading and saving is moved to the C code because we need the cause to dispatch the exception handling, if we keep the cause reading and saving, we either pass it to do_traps() via. 2nd param or get it from pt_regs which an extra memory load is needed, I don't like any of the two solutions becase the exception handling sits in hot code path, every instruction matters.

2.To cope with SIFIVE_CIP_453 errata, it looks like we don't need alternative mechanism any more after the asm->c convertion. Just replace the excp_vect_table two entries.

Quoted 5 lines
Quoted 3 lines

Anton Blanchard (1):
riscv: Improve exception and system call latency

I've retested this patch with the rest of the series applied. I can confirm that the performance improvement is still there. Definitely thumbs up on my end.

Thanks! The 2nd patch is inspired by Anton's patch. The remainings are just to convert the asm to c in straight forward style.

If possible, could you please add your Tested-by tag? Or even better review the series ;)

Thanks

Quoted 25 lines
Quoted 25 lines

Jisheng Zhang (5):
riscv: avoid corrupting the RAS
riscv: convert bottom half of exception handling to C
riscv: errata: remove ALT_INSN_FAULT and ALT_PAGE_FAULT
riscv: errata: sifive: remove NOMMU handling
riscv: remove asmlinkage from updated functions

 arch/riscv/errata/sifive/errata.c         | 25 +++++++---
 arch/riscv/errata/sifive/errata_cip_453.S |  4 --
 arch/riscv/include/asm/asm-prototypes.h   |  7 +--
 arch/riscv/include/asm/errata_list.h      | 21 ++------
 arch/riscv/kernel/entry.S                 | 61 ++---------------------
 arch/riscv/kernel/stacktrace.c            |  4 +-
 arch/riscv/kernel/traps.c                 | 57 ++++++++++++++++++---
 7 files changed, 81 insertions(+), 98 deletions(-)

--
2.43.0

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#22 lore raw
Quoted 34 lines · Jisheng Zhang
Quoted 32 lines · Cyril Bur
Quoted 31 lines · Jisheng Zhang

For readability, maintainability and future scalability, convert the bottom half of the exception handling to C.

During the conversion, I found Anton fixed a performance issue and my patches will touch the same exception asm code, so I include Anton's patch for completeness. I also cooked a similar patch to avoid corrupting the RAS in ret_from_fork() per the inspiration.

Mostly the assembly code is converted to C in a relatively straightforward manner.

However, there are two modifications I need to mention:

1. the CSR_CAUSE reg reading and saving is moved to the C code because we need the cause to dispatch the exception handling, if we keep the cause reading and saving, we either pass it to do_traps() via. 2nd param or get it from pt_regs which an extra memory load is needed, I don't like any of the two solutions becase the exception handling sits in hot code path, every instruction matters.

2.To cope with SIFIVE_CIP_453 errata, it looks like we don't need alternative mechanism any more after the asm->c convertion. Just replace the excp_vect_table two entries.

Anton Blanchard (1):
riscv: Improve exception and system call latency

Quoted 9 lines
Quoted 2 lines

I've retested this patch with the rest of the series applied. I can confirm that the performance improvement is still there. Definitely thumbs up on my end.

Thanks! The 2nd patch is inspired by Anton's patch. The remainings are just to convert the asm to c in straight forward style.

If possible, could you please add your Tested-by tag? Or even better review the series ;)

I only saw this response after having a look at 3/6. I'm a little nervous about a full review
as I don't know all the riscv catches yet.

I'll say that the testing of the series wasn't massive. I just booted it enough to exercise
that codepath and get some numbers. Having said that I'm fine putting a Tested-by
for the series, it did boot to userspace after all :).

Quoted 26 lines

Thanks

Quoted 25 lines
Quoted 25 lines

Jisheng Zhang (5):
riscv: avoid corrupting the RAS
riscv: convert bottom half of exception handling to C
riscv: errata: remove ALT_INSN_FAULT and ALT_PAGE_FAULT
riscv: errata: sifive: remove NOMMU handling
riscv: remove asmlinkage from updated functions

 arch/riscv/errata/sifive/errata.c         | 25 +++++++---
 arch/riscv/errata/sifive/errata_cip_453.S |  4 --
 arch/riscv/include/asm/asm-prototypes.h   |  7 +--
 arch/riscv/include/asm/errata_list.h      | 21 ++------
 arch/riscv/kernel/entry.S                 | 61 ++---------------------
 arch/riscv/kernel/stacktrace.c            |  4 +-
 arch/riscv/kernel/traps.c                 | 57 ++++++++++++++++++---
 7 files changed, 81 insertions(+), 98 deletions(-)

--
2.43.0

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#23 lore raw
In reply to Jisheng Zhang
Quoted 7 lines · Jisheng Zhang

For readability, maintainability and future scalability, convert the bottom half of the exception handling to C.

During the conversion, I found Anton fixed a performance issue and my patches will touch the same exception asm code, so I include Anton's patch for completeness. I also cooked a similar patch to avoid corrupting the RAS in ret_from_fork() per the inspiration.

nit: Probably corruption is wrong word here giving the notion that software got some capability to corrupt uarch structures. It's simply mismatched # of call and # of rets. Imbalance (instead of calling it corruption) in return address stack (RAS) leading to incorret predictions on return.

Quoted 43 lines

Mostly the assembly code is converted to C in a relatively straightforward manner.

However, there are two modifications I need to mention:

1. the CSR_CAUSE reg reading and saving is moved to the C code because we need the cause to dispatch the exception handling, if we keep the cause reading and saving, we either pass it to do_traps() via. 2nd param or get it from pt_regs which an extra memory load is needed, I don't like any of the two solutions becase the exception handling sits in hot code path, every instruction matters.

2.To cope with SIFIVE_CIP_453 errata, it looks like we don't need alternative mechanism any more after the asm->c convertion. Just replace the excp_vect_table two entries.

Anton Blanchard (1):
riscv: Improve exception and system call latency

Jisheng Zhang (5):
riscv: avoid corrupting the RAS
riscv: convert bottom half of exception handling to C
riscv: errata: remove ALT_INSN_FAULT and ALT_PAGE_FAULT
riscv: errata: sifive: remove NOMMU handling
riscv: remove asmlinkage from updated functions

arch/riscv/errata/sifive/errata.c         | 25 +++++++---
arch/riscv/errata/sifive/errata_cip_453.S |  4 --
arch/riscv/include/asm/asm-prototypes.h   |  7 +--
arch/riscv/include/asm/errata_list.h      | 21 ++------
arch/riscv/kernel/entry.S                 | 61 ++---------------------
arch/riscv/kernel/stacktrace.c            |  4 +-
arch/riscv/kernel/traps.c                 | 57 ++++++++++++++++++---
7 files changed, 81 insertions(+), 98 deletions(-)
Signature · 1 line
2.43.0