Skip to contentSkip to conversation

Add a dummy register to sbi_trap_regs

PATCH New Chao-ying Fu 8 participants 11 messages Jan 9, 2025 – Feb 12, 2025

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Add a dummy register to sbi_trap_regs

SBI_TRAP_CONTEXT_SIZE becomes a multiple of 8 bytes for RV32 or 16 bytes for RV64, so we have sp aligned to 8 or 16 bytes to have better performance on some platforms,

---
 include/sbi/sbi_trap.h | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
include/sbi/sbi_trap.h +5 -1
index d5182bf..df99d8e 100644
@@ -84,8 +84,10 @@
 #define SBI_TRAP_REGS_mstatus			33
 /** Index of mstatusH member in sbi_trap_regs */
 #define SBI_TRAP_REGS_mstatusH			34
+/** Index of a dummy register */
+#define SBI_TRAP_REGS_dummy			35
 /** Last member index in sbi_trap_regs */
-#define SBI_TRAP_REGS_last			35
+#define SBI_TRAP_REGS_last			36
 
 /** Index of cause member in sbi_trap_info */
 #define SBI_TRAP_INFO_cause			0
@@ -194,6 +196,8 @@ struct sbi_trap_regs {
 	unsigned long mstatus;
 	/** mstatusH register state (only for 32-bit) */
 	unsigned long mstatusH;
+	/** a dummy register */
+	unsigned long dummy;
 };
 
 /** Representation of trap details */
Signature · 1 line
2.47.1
#1 lore raw

PATCH 1/1 Add a dummy register to sbi_trap_regs

Quoted 25 lines · Chao-ying Fu

SBI_TRAP_CONTEXT_SIZE becomes a multiple of 8 bytes for RV32 or 16 bytes for RV64, so we have sp aligned to 8 or 16 bytes to have better performance on some platforms,
---
include/sbi/sbi_trap.h | 6 +++++-

1 file changed, 5 insertions(+), 1 deletion(-)
include/sbi/sbi_trap.h +3 -1
index d5182bf..df99d8e 100644
@@ -84,8 +84,10 @@
 #define SBI_TRAP_REGS_mstatus 33
 /** Index of mstatusH member in sbi_trap_regs */
 #define SBI_TRAP_REGS_mstatusH 34
+/** Index of a dummy register */
+#define SBI_TRAP_REGS_dummy 35
 /** Last member index in sbi_trap_regs */
-#define SBI_TRAP_REGS_last 35
+#define SBI_TRAP_REGS_last 36
 
 /** Index of cause member in sbi_trap_info */
 #define SBI_TRAP_INFO_cause 0

include/sbi/sbi_trap.h struct sbi_trap_regs { · line 200

Quoted 5 lines
include/sbi/sbi_trap.h +2 -0
@@ -194,6 +196,8 @@ struct sbi_trap_regs {
 unsigned long mstatus;
 /** mstatusH register state (only for 32-bit) */
 unsigned long mstatusH;
+ /** a dummy register */
+ unsigned long dummy;

This is ugly and fragile for when changes are made to the structure. Either make SBI_TRAP_CONTEXT_SIZE round up the size or, if for some reason that doesn?t work, you can always make the struct type overaligned.

Also, sp is aligned to 16 bytes anyway, that?s mandated by the ABI. So I don?t understand your justification.

Strong NAK.

Jess

#2 lore raw

PATCH 1/1 Add a dummy register to sbi_trap_regs

Quoted 38 lines · On Thu, Jan 9, 2025 at 4:30?PM Jessica Clarke <[email protected]> wrote:
Quoted 25 lines · Chao-ying Fu

SBI_TRAP_CONTEXT_SIZE becomes a multiple of 8 bytes for RV32 or 16 bytes for RV64, so we have sp aligned to 8 or 16 bytes to have better performance on some platforms,
---
include/sbi/sbi_trap.h | 6 +++++-

1 file changed, 5 insertions(+), 1 deletion(-)
include/sbi/sbi_trap.h +3 -1
index d5182bf..df99d8e 100644
@@ -84,8 +84,10 @@
 #define SBI_TRAP_REGS_mstatus 33
 /** Index of mstatusH member in sbi_trap_regs */
 #define SBI_TRAP_REGS_mstatusH 34
+/** Index of a dummy register */
+#define SBI_TRAP_REGS_dummy 35
 /** Last member index in sbi_trap_regs */
-#define SBI_TRAP_REGS_last 35
+#define SBI_TRAP_REGS_last 36
 
 /** Index of cause member in sbi_trap_info */
 #define SBI_TRAP_INFO_cause 0

include/sbi/sbi_trap.h struct sbi_trap_regs { · line 200

Quoted 5 lines
include/sbi/sbi_trap.h +2 -0
@@ -194,6 +196,8 @@ struct sbi_trap_regs {
 unsigned long mstatus;
 /** mstatusH register state (only for 32-bit) */
 unsigned long mstatusH;
+ /** a dummy register */
+ unsigned long dummy;

This is ugly and fragile for when changes are made to the structure. Either make SBI_TRAP_CONTEXT_SIZE round up the size or, if for some reason that doesn?t work, you can always make the struct type overaligned.

Quoted 7 lines

Also, sp is aligned to 16 bytes anyway, that?s mandated by the ABI. So I don?t understand your justification.

Strong NAK.

Jess

Yes. The patch can be revised to be more solid. We can just change the define for SBI_TRAP_CONTEXT_SIZE to 16*x bytes for RV64, for example. Note that the original sp is aligned to 16 bytes. However, there is code in fw_base.S as follows.
/*
* Set T0 to appropriate exception stack
*
* Came_From_M_Mode = ((MSTATUS.MPP < PRV_M) ? 1 : 0) - 1;
* Exception_Stack = TP ^ (Came_From_M_Mode & (SP ^ TP)) *
* Came_From_M_Mode = 0 ==> Exception_Stack = TP
* Came_From_M_Mode = -1 ==> Exception_Stack = SP
*/
csrr t0, CSR_MSTATUS
srl t0, t0, MSTATUS_MPP_SHIFT
and t0, t0, PRV_M
slti t0, t0, PRV_M
add t0, t0, -1
xor sp, sp, tp
and t0, t0, sp
xor sp, sp, tp
xor t0, tp, t0

/* Save original SP on exception stack */
REG_S sp, (SBI_TRAP_REGS_OFFSET(sp) - SBI_TRAP_CONTEXT_SIZE)(t0)

/* Set SP to exception stack and make room for trap context */
add sp, t0, -(SBI_TRAP_CONTEXT_SIZE)

After this add instruction, sp may not be aligned to 16 bytes any more. Thanks!

#3 lore raw

PATCH v2 1/1 Align SBI trap registers to a nice boundary for performance reasons.

In reply to Chao-ying Fu

Align SBI_TRAP_CONTEXT_SIZE to a multiple of 16 bytes. If it is not aligned to 16 bytes for RV64, it can create performance problems. Aligning it correctly can fix the performance issues.

--
Test: Performance issues seen and fix Verified on FPGA. Other methods.
Run qemu with monitor to check the SP during cpu_in and cpu_out.
Add sbi_printf to the function sbi_trap_handler to check the alignment of sbi_trap_context

---
 include/sbi/sbi_trap.h | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
include/sbi/sbi_trap.h +7 -4
index d5182bf..82e9c08 100644
@@ -112,13 +112,16 @@
 /** Size (in bytes) of sbi_trap_info */
 #define SBI_TRAP_INFO_SIZE SBI_TRAP_INFO_OFFSET(last)
 
+/** Stack pointer is aligned to 16 bytes */
+#define STACK_BOUNDARY            16
+#define ALIGN_TO_BOUNDARY(x,a) (((x) + (a) - 1) & ~((a) - 1))
+
 /** Size (in bytes) of sbi_trap_context */
-#define SBI_TRAP_CONTEXT_SIZE (SBI_TRAP_REGS_SIZE + \
-			       SBI_TRAP_INFO_SIZE + \
-			       __SIZEOF_POINTER__)
+#define SBI_TRAP_CONTEXT_SIZE ALIGN_TO_BOUNDARY((SBI_TRAP_REGS_SIZE + \
+                               SBI_TRAP_INFO_SIZE + \
+                               __SIZEOF_POINTER__),STACK_BOUNDARY)
 
 #ifndef __ASSEMBLER__
-
 #include <sbi/sbi_types.h>
 #include <sbi/sbi_scratch.h>
 
Signature · 1 line
2.43.0
#4 lore raw

PATCH v2 1/1 Align SBI trap registers to a nice boundary for performance reasons.

Hi Raj,

This looks like the right approach. A few comments below on the mechanics of the patch.

Quoted 21 lines · Raj Vishwanathan

Align SBI_TRAP_CONTEXT_SIZE to a multiple of 16 bytes. If it is not aligned to 16 bytes for RV64, it can create performance problems. Aligning it correctly can fix the performance issues.

--
Test: Performance issues seen and fix Verified on FPGA. Other methods.
Run qemu with monitor to check the SP during cpu_in and cpu_out.
Add sbi_printf to the function sbi_trap_handler to check the alignment of sbi_trap_context

---
 include/sbi/sbi_trap.h | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
include/sbi/sbi_trap.h +0 -0
index d5182bf..82e9c08 100644

include/sbi/sbi_trap.h line 117

Quoted 6 lines
include/sbi/sbi_trap.h +3 -0
@@ -112,13 +112,16 @@
 /** Size (in bytes) of sbi_trap_info */
 #define SBI_TRAP_INFO_SIZE SBI_TRAP_INFO_OFFSET(last)
 
+/** Stack pointer is aligned to 16 bytes */
+#define STACK_BOUNDARY            16
+#define ALIGN_TO_BOUNDARY(x,a) (((x) + (a) - 1) & ~((a) - 1))

This is the same as the BIT_ALIGN macro that already exists in sbi_bitops.h.

include/sbi/sbi_trap.h line 122

Quoted 8 lines
include/sbi/sbi_trap.h +4 -3
+
 /** Size (in bytes) of sbi_trap_context */
-#define SBI_TRAP_CONTEXT_SIZE (SBI_TRAP_REGS_SIZE + \
-			       SBI_TRAP_INFO_SIZE + \
-			       __SIZEOF_POINTER__)
+#define SBI_TRAP_CONTEXT_SIZE ALIGN_TO_BOUNDARY((SBI_TRAP_REGS_SIZE + \
+                               SBI_TRAP_INFO_SIZE + \
+                               __SIZEOF_POINTER__),STACK_BOUNDARY)

Please follow the existing style (spaces between arguments).

include/sbi/sbi_trap.h line 121

Quoted 3 lines
include/sbi/sbi_trap.h +0 -1
 #ifndef __ASSEMBLER__
-

Please don't include unrelated changes in the patch.

Regards,
Samuel

Quoted 3 lines
include/sbi/sbi_trap.h +0 -0
 #include <sbi/sbi_types.h>
 #include <sbi/sbi_scratch.h>
 
#5 lore raw

[EXTERNAL]Re: [PATCH v2] Align SBI trap registers to a nice boundary for performance reasons.

Hi Samuel

Thanks for your review.
1. did look at the BIT_ALIGN macro. The sbi_bitops.h seems to be available for 'C' files and not for assembly files. The alignment I needed is used fw_base.S 2 &3 . I have added the space and deleted the unnecessary line

I have sent an updated patch.

Raj

-----Original Message-----
From: Samuel Holland <[email protected]>
Sent: Friday, February 7, 2025 9:34 PM
To: Raj Vishwanathan <[email protected]>; opensbi at lists.infradead.org
Cc: Raj Vishwanathan <[email protected]>
Subject: [EXTERNAL]Re: [PATCH v2] Align SBI trap registers to a nice boundary for performance reasons.

[You don't often get email from samuel.holland at sifive.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

Hi Raj,

This looks like the right approach. A few comments below on the mechanics of the patch.

Quoted 21 lines · Raj Vishwanathan

Align SBI_TRAP_CONTEXT_SIZE to a multiple of 16 bytes. If it is not aligned to 16 bytes for RV64, it can create performance problems. Aligning it correctly can fix the performance issues.

--
Test: Performance issues seen and fix Verified on FPGA. Other methods.
Run qemu with monitor to check the SP during cpu_in and cpu_out.
Add sbi_printf to the function sbi_trap_handler to check the alignment of sbi_trap_context

---
 include/sbi/sbi_trap.h | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
include/sbi/sbi_trap.h index renamed from include/sbi/sbi_trap.h +0 -0

d5182bf..82e9c08 100644

include/sbi/sbi_trap.h line 117

Quoted 6 lines
include/sbi/sbi_trap.h +3 -0
@@ -112,13 +112,16 @@
 /** Size (in bytes) of sbi_trap_info */  #define SBI_TRAP_INFO_SIZE 
 SBI_TRAP_INFO_OFFSET(last)
 
+/** Stack pointer is aligned to 16 bytes */
+#define STACK_BOUNDARY            16
+#define ALIGN_TO_BOUNDARY(x,a) (((x) + (a) - 1) & ~((a) - 1))

This is the same as the BIT_ALIGN macro that already exists in sbi_bitops.h.

include/sbi/sbi_trap.h line 122

Quoted 8 lines
include/sbi/sbi_trap.h +4 -2
+
 /** Size (in bytes) of sbi_trap_context */ -#define 
 SBI_TRAP_CONTEXT_SIZE (SBI_TRAP_REGS_SIZE + \
-                            SBI_TRAP_INFO_SIZE + \
-                            __SIZEOF_POINTER__)
+#define SBI_TRAP_CONTEXT_SIZE ALIGN_TO_BOUNDARY((SBI_TRAP_REGS_SIZE + \
+                               SBI_TRAP_INFO_SIZE + \
+                               __SIZEOF_POINTER__),STACK_BOUNDARY)

Please follow the existing style (spaces between arguments).

include/sbi/sbi_trap.h line 121

Quoted 3 lines
include/sbi/sbi_trap.h +0 -1
 #ifndef __ASSEMBLER__
-

Please don't include unrelated changes in the patch.

Regards,
Samuel

Quoted 3 lines
include/sbi/sbi_trap.h +0 -0
 #include <sbi/sbi_types.h>
 #include <sbi/sbi_scratch.h>
 
#6 lore raw

PATCH v2 1/1 Align SBI trap registers to a nice boundary for performance reasons.

In reply to Chao-ying Fu

Align SBI_TRAP_CONTEXT_SIZE to a multiple of 16 bytes. If it is not aligned to 16 bytes for RV64, it can create performance problems. Aligning it correctly can fix the performance issues.

---
Test: Performance issues seen and fix Verified on FPGA. Other methods.
Run qemu with monitor to check the SP during cpu_in and cpu_out.
Add sbi_printf to the function sbi_trap_handler to check the alignment of sbi_trap_context

---
 include/sbi/sbi_trap.h | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
include/sbi/sbi_trap.h +5 -2
index d5182bf..5eec4da 100644
@@ -112,10 +112,13 @@
 /** Size (in bytes) of sbi_trap_info */
 #define SBI_TRAP_INFO_SIZE SBI_TRAP_INFO_OFFSET(last)
 
+#define STACK_BOUNDARY 16
+#define ALIGN_TO_BOUNDARY(x, a) (((x) + (a) - 1) & ~((a) - 1))
+
 /** Size (in bytes) of sbi_trap_context */
-#define SBI_TRAP_CONTEXT_SIZE (SBI_TRAP_REGS_SIZE + \
+#define SBI_TRAP_CONTEXT_SIZE ALIGN_TO_BOUNDARY((SBI_TRAP_REGS_SIZE + \
 			       SBI_TRAP_INFO_SIZE + \
-			       __SIZEOF_POINTER__)
+			       __SIZEOF_POINTER__), STACK_BOUNDARY)
 
 #ifndef __ASSEMBLER__
 
Signature · 1 line
2.43.0
#7 lore raw

PATCH v2 1/1 Align SBI trap registers to a nice boundary for performance reasons.

? 2025-02-10?? 14:47 -0800?Raj Vishwanathan???

Quoted 24 lines

Align SBI_TRAP_CONTEXT_SIZE to a multiple of 16 bytes. If it is not aligned to 16 bytes for RV64, it can create performance problems. Aligning it correctly can fix the performance issues.

---
Test: Performance issues seen and fix Verified on FPGA. Other methods.
??? Run qemu with monitor to check the SP during cpu_in ??? and cpu_out.
??? Add sbi_printf to the function sbi_trap_handler to check ??? the alignment of sbi_trap_context
---
?include/sbi/sbi_trap.h | 7 +++++--
?1 file changed, 5 insertions(+), 2 deletions(-)

include/sbi/sbi_trap.h +0 -0
index d5182bf..5eec4da 100644
@@ -112,10 +112,13 @@

?/** Size (in bytes) of sbi_trap_info */ ?#define SBI_TRAP_INFO_SIZE SBI_TRAP_INFO_OFFSET(last) ?

include/sbi/sbi_trap.h line 116

Quoted 2 lines
include/sbi/sbi_trap.h +2 -0
+#define STACK_BOUNDARY 16
+#define ALIGN_TO_BOUNDARY(x, a) (((x) + (a) - 1) & ~((a) - 1))

This can be replace with BOUNDUP in sbi_types.h

Regards,
Xiang W

Quoted 14 lines

+
?/** Size (in bytes) of sbi_trap_context */

include/sbi/sbi_trap.h +1 -1
-#define SBI_TRAP_CONTEXT_SIZE (SBI_TRAP_REGS_SIZE + \
+#define SBI_TRAP_CONTEXT_SIZE ALIGN_TO_BOUNDARY((SBI_TRAP_REGS_SIZE + \

? ?????? SBI_TRAP_INFO_SIZE + \

include/sbi/sbi_trap.h +1 -1
-			?????? __SIZEOF_POINTER__)
+			?????? __SIZEOF_POINTER__), STACK_BOUNDARY)

?
?#ifndef __ASSEMBLER__
?

Signature · 1 line
2.43.0
#8 lore raw

PATCH v2 1/1 Align SBI trap registers to a nice boundary for performance reasons.

In reply to Raj Vishwanathan
Quoted 6 lines · On Tue, Feb 11, 2025 at 4:41?AM Raj Vishwanathan <[email protected]>…

Align SBI_TRAP_CONTEXT_SIZE to a multiple of 16 bytes. If it is not aligned to 16 bytes for RV64, it can create performance problems. Aligning it correctly can fix the performance issues.

Simplify the patch subject and use "include: sbi:" as subject prefix.

Regards,
Anup

Quoted 39 lines

---
Test: Performance issues seen and fix Verified on FPGA. Other methods.
Run qemu with monitor to check the SP during cpu_in and cpu_out.
Add sbi_printf to the function sbi_trap_handler to check the alignment of sbi_trap_context

---
 include/sbi/sbi_trap.h | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
include/sbi/sbi_trap.h +5 -3
index d5182bf..5eec4da 100644
@@ -112,10 +112,13 @@
 /** Size (in bytes) of sbi_trap_info */
 #define SBI_TRAP_INFO_SIZE SBI_TRAP_INFO_OFFSET(last)
 
+#define STACK_BOUNDARY 16
+#define ALIGN_TO_BOUNDARY(x, a) (((x) + (a) - 1) & ~((a) - 1))
+
 /** Size (in bytes) of sbi_trap_context */
-#define SBI_TRAP_CONTEXT_SIZE (SBI_TRAP_REGS_SIZE + \
+#define SBI_TRAP_CONTEXT_SIZE ALIGN_TO_BOUNDARY((SBI_TRAP_REGS_SIZE + \
                               SBI_TRAP_INFO_SIZE + \
-                              __SIZEOF_POINTER__)
+                              __SIZEOF_POINTER__), STACK_BOUNDARY)
 
 #ifndef __ASSEMBLER__
 
--

2.43.0

--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

#9 lore raw

PATCH v2 1/1 include: sbi: Align SBI trap registers to a nice boundary

In reply to Chao-ying Fu

Align SBI_TRAP_CONTEXT_SIZE to a multiple of 16 bytes. If it is not aligned to 16 bytes for RV64, it can create performance problems. Aligning it correctly can fix the performance issues.

---
Test: Performance issues seen and fix Verified on FPGA. Other methods.
Run qemu with monitor to check the SP during cpu_in and cpu_out.
Add sbi_printf to the function sbi_trap_handler to check the alignment of sbi_trap_context

---
 include/sbi/sbi_trap.h | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
include/sbi/sbi_trap.h +5 -2
index d5182bf..5eec4da 100644
@@ -112,10 +112,13 @@
 /** Size (in bytes) of sbi_trap_info */
 #define SBI_TRAP_INFO_SIZE SBI_TRAP_INFO_OFFSET(last)
 
+#define STACK_BOUNDARY 16
+#define ALIGN_TO_BOUNDARY(x, a) (((x) + (a) - 1) & ~((a) - 1))
+
 /** Size (in bytes) of sbi_trap_context */
-#define SBI_TRAP_CONTEXT_SIZE (SBI_TRAP_REGS_SIZE + \
+#define SBI_TRAP_CONTEXT_SIZE ALIGN_TO_BOUNDARY((SBI_TRAP_REGS_SIZE + \
 			       SBI_TRAP_INFO_SIZE + \
-			       __SIZEOF_POINTER__)
+			       __SIZEOF_POINTER__), STACK_BOUNDARY)
 
 #ifndef __ASSEMBLER__
 
Signature · 1 line
2.43.0
Applied
#10 lore raw

PATCH v2 1/1 include: sbi: Align SBI trap registers to a nice boundary

Quoted 6 lines · On Wed, Feb 12, 2025 at 3:16?AM Raj Vishwanathan <[email protected]>…

Align SBI_TRAP_CONTEXT_SIZE to a multiple of 16 bytes. If it is not aligned to 16 bytes for RV64, it can create performance problems. Aligning it correctly can fix the performance issues.

LGTM.

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

Quoted 39 lines

---
Test: Performance issues seen and fix Verified on FPGA. Other methods.
Run qemu with monitor to check the SP during cpu_in and cpu_out.
Add sbi_printf to the function sbi_trap_handler to check the alignment of sbi_trap_context

---
 include/sbi/sbi_trap.h | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
include/sbi/sbi_trap.h +5 -3
index d5182bf..5eec4da 100644
@@ -112,10 +112,13 @@
 /** Size (in bytes) of sbi_trap_info */
 #define SBI_TRAP_INFO_SIZE SBI_TRAP_INFO_OFFSET(last)
 
+#define STACK_BOUNDARY 16
+#define ALIGN_TO_BOUNDARY(x, a) (((x) + (a) - 1) & ~((a) - 1))
+
 /** Size (in bytes) of sbi_trap_context */
-#define SBI_TRAP_CONTEXT_SIZE (SBI_TRAP_REGS_SIZE + \
+#define SBI_TRAP_CONTEXT_SIZE ALIGN_TO_BOUNDARY((SBI_TRAP_REGS_SIZE + \
                               SBI_TRAP_INFO_SIZE + \
-                              __SIZEOF_POINTER__)
+                              __SIZEOF_POINTER__), STACK_BOUNDARY)
 
 #ifndef __ASSEMBLER__
 
--

2.43.0

--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi