riscv: smp: Align secondary_start_sbi to 4 bytes
PATCH v2 1/1 riscv: smp: Align secondary_start_sbi to 4 bytes
From: Chen Pei <[email protected]>
During SMP boot, the secondary_start_sbi address is passed to the slave core via sbi_hsm_hart_start. In OpenSBI, this address is written to STVEC in sbi_hart_switch_mode.
According to the privileged specification, the BASE field of STVEC must always be aligned on a 4-byte boundary. However, System.map reveals that secondary_start_sbi is not a 4-byte aligned address.
For example, the address of secondary_start_sbi is 0xffffffff80001066, and the disassembly is as follows:
Dump of assembler code from 0xffffffff80001052 to 0xffffffff8000107a:
0xffffffff80001052 <_start+4178>: c.nop -11 0xffffffff80001054 <_start+4180>: auipc gp,0x1a1f 0xffffffff80001058 <_start+4184>: addi gp,gp,84 0xffffffff8000105c <_start+4188>: csrw satp,a2 0xffffffff80001060 <_start+4192>: sfence.vma 0xffffffff80001064 <_start+4196>: ret 0xffffffff80001066 <_start+4198>: csrw sie,zero 0xffffffff8000106a <_start+4202>: csrw sip,zero 0xffffffff8000106e <_start+4206>: li t0,2 0xffffffff80001070 <_start+4208>: csrw scounteren,t0 0xffffffff80001074 <_start+4212>: auipc gp,0x1a1f 0xffffffff80001078 <_start+4216>: addi gp,gp,52
When writing to STVEC at address 0xffffffff80001066, the actual write address is 0xffffffff80001064, corresponding to the address of the previous ret instruction. This is unexpected, and if an interrupt occurs at this point, it will cause unpredictable results.
However, secondary_start_sbi immediately masks all interrupts and updates STVEC, so no problems occurred.
In summary, it is more reasonable to make secondary_start_sbi satisfy 4-byte alignment.
Changes in v2:
- Place .align 2 inside #ifdef CONFIG_SMP, above the tag.
- Add two Reviewed-by tags.
- Based on Linux 7.0.
--- arch/riscv/kernel/head.S | 1 + 1 file changed, 1 insertion(+)
arch/riscv/kernel/head.S +1 -0
@@ -127,6 +127,7 @@ relocate_enable_mmu: #endif /* CONFIG_MMU */ #ifdef CONFIG_SMP .global secondary_start_sbi + .align 2 secondary_start_sbi: /* Mask all interrupts */ csrw CSR_IE, zero
Signature · 1 line
2.50.1
Quoted 5 lines · On 13/4/2026 23:20, [email protected] wrote:
From: Chen Pei <[email protected]>
During SMP boot, the
secondary_start_sbiaddress is passed to the slave core viasbi_hsm_hart_start. In OpenSBI, this address is written to STVEC insbi_hart_switch_mode.
...
arch/riscv/kernel/head.S relocate_enable_mmu: · line 133
Quoted 12 lines
arch/riscv/kernel/head.S +1
-0@@ -127,6 +127,7 @@ relocate_enable_mmu: #endif /* CONFIG_MMU */ #ifdef CONFIG_SMP .global secondary_start_sbi + .align 2 secondary_start_sbi: /* Mask all interrupts */ csrw CSR_IE, zero
Minor nit, but IMHO .balign is preferable in new code. It always byte aligns, whereas .align has different meanings across architectures, and on some arches (including riscv) requires the reader to do the math to convert to a byte alignment.
These days there's also the SYM_CODE_START etc. macros defined via linkage.h, which use __ALIGN from arch/riscv/include/asm/linkage.h, which is already .balign 4.
cheers
Quoted 19 lines · On Wed, 15 Apr 2026 12:06:28 +1000, [email protected] wrote:
arch/riscv/kernel/head.S relocate_enable_mmu: · line 133
Quoted 12 lines
arch/riscv/kernel/head.S +1
-0@@ -127,6 +127,7 @@ relocate_enable_mmu: #endif /* CONFIG_MMU */ #ifdef CONFIG_SMP .global secondary_start_sbi + .align 2 secondary_start_sbi: /* Mask all interrupts */ csrw CSR_IE, zeroMinor nit, but IMHO .balign is preferable in new code. It always byte aligns, whereas .align has different meanings across architectures, and on some arches (including riscv) requires the reader to do the math to convert to a byte alignment.
Quoted 3 lines
These days there's also the
SYM_CODE_STARTetc. macros defined vialinkage.h, which use__ALIGNfromarch/riscv/include/asm/linkage.h, which is already .balign 4.
Hi Michael,
Thank you for your review and reminder. If this patch is ultimately approved (because the behavior of OpenSBI still needs clarification based on the new reviewer's comments), I will update it using .balign.
Thanks,
Pei
[ +cc opensbi, Anup: What was the deal with setting stvec in sbi_hart_switch_mode? ]
Quoted 5 lines · On 4/13/26 21:20, [email protected] wrote:
From: Chen Pei <[email protected]>
During SMP boot, the
secondary_start_sbiaddress is passed to the slave core viasbi_hsm_hart_start. In OpenSBI, this address is written to STVEC insbi_hart_switch_mode.
This is certainly odd (OpenSBI git blames to initial git commit. No idea why this was needed.) but technically permitted by the SBI HSM spec (see below).
* Anup and other opensbi folks: Do you know/remember why it was there? It's not really a bug, but some historical context would be appreciated.
Quoted 21 lines
According to the privileged specification, the BASE field of STVEC must always be aligned on a 4-byte boundary. However, System.map reveals that
secondary_start_sbiis not a 4-byte aligned address.For example, the address of
secondary_start_sbiis0xffffffff80001066, and the disassembly is as follows:Dump of assembler code from
0xffffffff80001052to0xffffffff8000107a:0xffffffff80001052 <_start+4178>: c.nop -11 0xffffffff80001054 <_start+4180>: auipc gp,0x1a1f 0xffffffff80001058 <_start+4184>: addi gp,gp,84 0xffffffff8000105c <_start+4188>: csrw satp,a2 0xffffffff80001060 <_start+4192>: sfence.vma 0xffffffff80001064 <_start+4196>: ret 0xffffffff80001066 <_start+4198>: csrw sie,zero 0xffffffff8000106a <_start+4202>: csrw sip,zero 0xffffffff8000106e <_start+4206>: li t0,2 0xffffffff80001070 <_start+4208>: csrw scounteren,t0 0xffffffff80001074 <_start+4212>: auipc gp,0x1a1f 0xffffffff80001078 <_start+4216>: addi gp,gp,52
Quoted 3 lines
When writing to STVEC at address
0xffffffff80001066, the actual write address is0xffffffff80001064, corresponding to the address of the previous ret instruction.
Also technically permitted.
Quoted 5 lines
This is unexpected, and if an
interrupt occurs at this point, it will cause unpredictable results.However,
secondary_start_sbiimmediately masks all interrupts and updates STVEC, so no problems occurred.
An interrupt cannot trap in between secondary_start_sbi starting and the interrupts getting masked individually. The only arch state guaranteed by HSM is [1]:
|Register Name | Register Value |satp | 0 |sstatus.SIE | 0 |a0 | hartid |a1 | `opaque` parameter |All other registers remain in an undefined state.
Of which sstatus.SIE = 0 means that interrupts are masked in Supervisor mode.
Quoted 2 lines
In summary, it is more reasonable to make
secondary_start_sbisatisfy 4-byte alignment.
Thus, it is unnecessary.
If this fixes a bug where an interrupt can trap into the wrong stvec for you, then your firmware is broken.
Michael's suggestion about SYM_CODE_START still stands, but if you do that, that's just a really minor refactoring, and in any case the current commit message cannot stand.
Vivian "dramforever" Wang
[1]:
https://github.com/riscv-non-isa/riscv-sbi-doc/blob/v3.0/src/ext-hsm.adoc#function-hart-start-fid-0
Quoted 10 lines · On Wed, 15 Apr 2026 11:39:22 +0800, [email protected] wrote:
Quoted 3 lines
During SMP boot, the
secondary_start_sbiaddress is passed to the slave core viasbi_hsm_hart_start. In OpenSBI, this address is written to STVEC insbi_hart_switch_mode.This is certainly odd (OpenSBI git blames to initial git commit. No idea why this was needed.) but technically permitted by the SBI HSM spec (see below).
* Anup and other opensbi folks: Do you know/remember why it was there? It's not really a bug, but some historical context would be appreciated.
Yes, looking forward to responses from Anup and other opensbi folks.
Quoted 21 lines
Quoted 21 lines
According to the privileged specification, the BASE field of STVEC must always be aligned on a 4-byte boundary. However, System.map reveals that
secondary_start_sbiis not a 4-byte aligned address.For example, the address of
secondary_start_sbiis0xffffffff80001066, and the disassembly is as follows:Dump of assembler code from
0xffffffff80001052to0xffffffff8000107a:0xffffffff80001052 <_start+4178>: c.nop -11 0xffffffff80001054 <_start+4180>: auipc gp,0x1a1f 0xffffffff80001058 <_start+4184>: addi gp,gp,84 0xffffffff8000105c <_start+4188>: csrw satp,a2 0xffffffff80001060 <_start+4192>: sfence.vma 0xffffffff80001064 <_start+4196>: ret 0xffffffff80001066 <_start+4198>: csrw sie,zero 0xffffffff8000106a <_start+4202>: csrw sip,zero 0xffffffff8000106e <_start+4206>: li t0,2 0xffffffff80001070 <_start+4208>: csrw scounteren,t0 0xffffffff80001074 <_start+4212>: auipc gp,0x1a1f 0xffffffff80001078 <_start+4216>: addi gp,gp,52
Quoted 4 lines
Quoted 3 lines
When writing to STVEC at address
0xffffffff80001066, the actual write address is0xffffffff80001064, corresponding to the address of the previous ret instruction.Also technically permitted.
Yes, but this is unintended behavior, and we should avoid it.
Quoted 19 lines
Quoted 5 lines
This is unexpected, and if an
interrupt occurs at this point, it will cause unpredictable results.However,
secondary_start_sbiimmediately masks all interrupts and updates STVEC, so no problems occurred.An interrupt cannot trap in between
secondary_start_sbistarting and the interrupts getting masked individually. The only arch state guaranteed by HSM is [1]:|Register Name | Register Value |satp | 0 |sstatus.SIE | 0 |a0 | hartid |a1 | `opaque` parameter |All other registers remain in an undefined state.Of which sstatus.SIE = 0 means that interrupts are masked in Supervisor mode.
Quoted 7 lines
Quoted 2 lines
In summary, it is more reasonable to make
secondary_start_sbisatisfy 4-byte alignment.Thus, it is unnecessary.
If this fixes a bug where an interrupt can trap into the wrong stvec for you, then your firmware is broken.
This isn't about fixing a bug (as mentioned earlier, it's highly unlikely to happen), but rather the STVEC check mechanism added to QEMU discovered this issue. Regarding the problem itself, I would appreciate any clarification or modifications from OpenSBI, but currently this modification is the simplest and most feasible.
Quoted 8 lines
Michael's suggestion about
SYM_CODE_STARTstill stands, but if you do that, that's just a really minor refactoring, and in any case the current commit message cannot stand.Vivian "dramforever" Wang
[1]:
https://github.com/riscv-non-isa/riscv-sbi-doc/blob/v3.0/src/ext-hsm.adoc#function-hart-start-fid-0
Thanks,
Pei
On 4/15/26 20:37, Chen Pei wrote:
[...]
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
In summary, it is more reasonable to make
secondary_start_sbisatisfy 4-byte alignment.Thus, it is unnecessary.
If this fixes a bug where an interrupt can trap into the wrong stvec for you, then your firmware is broken.
This isn't about fixing a bug (as mentioned earlier, it's highly unlikely to happen),
It is *impossible* given correct HW and FW, but okay...
Quoted 4 lines
but rather the STVEC check mechanism added to QEMU discovered this issue. Regarding the problem itself, I would appreciate any clarification or modifications from OpenSBI, but currently this modification is the simplest and most feasible.
The stvec CSR is WARL. If the "STVEC check mechanism" is complaining about usage of CSRs as specified, then the check is a false positive and is not something fixable.
Vivian "dramforever" Wang