LoongArch: initial 32-bit UAPI
Subsystems (2)
cover LoongArch: initial 32-bit UAPI
This series defines the UAPI for LoongArch32, marking my initial step towards upstreaming support for the architecture. Once the UAPI is ratified, we can proceed to scrutinise various kernel components to enable 32-bit support while simultaneously addressing user-space porting.
Why am I upstreaming LoongArch32?
================================
Although 32-bit systems are experiencing declining adoption in general computing, LoongArch32 remains highly relevant within specific niches. Beyond embedded applications, several vendors are actively developing application-level LoongArch32 processors. Loongson, for example, has released two open-source reference hardware implementations: openLA500 and openLA1000 [6].
The architecture also holds considerable educational value, having been integrated into China's national computer architecture curricula and embedded systems courses. Additionally, the National Student Computer System Capability Challenge (NSCSCC) [1] features LoongArch32 CPUs, where hundreds of students design Linux-capable hardware implementations and compete on performance. This initiative has resulted in several exciting high-performance LoongArch32 cores, including LainCore[2], Wired[3], NOP-Core[4], NagiCore[5]....
From an upstream perspective, we will largely reuse the infrastructure already established for LoongArch64, ensuring that the maintenance burden remains minimal.
Porting Status
==============
The LoongArch32 port has been available downstream for some time, with various system components hosted on Loongson's Gitee[6]. However, these components utilise an older downstream ABI and fall short of upstream quality.
On the upstream front, LLVM-19 now includes experimental support for LoongArch32 (ILP32 ABI) under the loongarch32* triple, and efforts are underway to enable GNU toolchain support. My upstream-ready kernel port and musl libc port can successfully boot into a minimal Buildroot environment and execute test cases on QEMU virt machine with clang toolchain.
Thank you for reading. I look forward to your comments and feedback.
[1]: https://www.tsinghua.edu.cn/en/info/1245/13802.htm
[2]: https://github.com/LainChip/LainCore
[3]: https://github.com/gmlayer0/wired
[4]: https://github.com/NOP-Processor/NOP-Core
[5]: https://github.com/MrAMS/NagiCore
[6]: https://gitee.com/loongson-edu
---
Jiaxun Yang (3):
loongarch: Wire up 32 bit syscalls
loongarch: Introduce sys_loongarch_flush_icache syscall
loongarch: vdso: Introduce __vdso_flush_icache function
arch/loongarch/include/asm/Kbuild | 1 + arch/loongarch/include/asm/cacheflush.h | 6 ++++ arch/loongarch/include/asm/syscall.h | 2 ++ arch/loongarch/include/asm/vdso/vdso.h | 10 ++++++ arch/loongarch/include/asm/vdso/vsyscall.h | 1 + arch/loongarch/include/uapi/asm/Kbuild | 1 + arch/loongarch/include/uapi/asm/unistd.h | 6 ++++ arch/loongarch/kernel/Makefile.syscalls | 3 +- arch/loongarch/kernel/syscall.c | 49 +++++++++++++++++++++++++++++ arch/loongarch/kernel/vdso.c | 2 ++ arch/loongarch/mm/cache.c | 3 ++ arch/loongarch/vdso/Makefile | 2 +- arch/loongarch/vdso/flush_icache.c | 50 ++++++++++++++++++++++++++++++ arch/loongarch/vdso/vdso.lds.S | 5 +++ scripts/syscall.tbl | 2 ++ 15 files changed, 140 insertions(+), 3 deletions(-)
---
base-commit: 8155b4ef3466f0e289e8fcc9e6e62f3f4dceeac2
change-id: 20250102-la32-uapi-8395e83a4e88
Best regards,
Signature · 1 line
Jiaxun Yang <[email protected]>
PATCH 1/3 loongarch: Wire up 32 bit syscalls
LoongArch 32-bit UAPI will be using generic syscall table, mostly identical with 64-bit one. It will follow the convention set by RISC-V, being the second 32 bit architecture without time32.
Generate unisted_32.h and syscall_table_32.h as necessary, expose united_32.h in unisted.h UAPI, and implement mmap2 which is a part of 32 bit syscalls.
--- arch/loongarch/include/asm/Kbuild | 1 + arch/loongarch/include/uapi/asm/Kbuild | 1 + arch/loongarch/include/uapi/asm/unistd.h | 6 ++++++ arch/loongarch/kernel/syscall.c | 21 +++++++++++++++++++++ 4 files changed, 29 insertions(+)
arch/loongarch/include/asm/Kbuild +1 -0
@@ -1,4 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 +syscall-y += syscall_table_32.h syscall-y += syscall_table_64.h generated-y += orc_hash.h
arch/loongarch/include/uapi/asm/Kbuild +1 -0
@@ -1,2 +1,3 @@ # SPDX-License-Identifier: GPL-2.0 +syscall-y += unistd_32.h syscall-y += unistd_64.h
arch/loongarch/include/uapi/asm/unistd.h +6 -0
@@ -1,3 +1,9 @@/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +#include <asm/bitsperlong.h> + +#if __BITS_PER_LONG ==64 #include <asm/unistd_64.h> +#else +#include <asm/unistd_32.h> +#endif
arch/loongarch/kernel/syscall.c +21 -0
@@ -34,9 +34,30 @@ SYSCALL_DEFINE6(mmap, unsigned long, addr, unsigned long, len, unsigned long,return ksys_mmap_pgoff (addr, len, prot, flags, fd, offset >>PAGE_SHIFT ); } +#ifdef CONFIG_32BIT +SYSCALL_DEFINE6(mmap2, unsigned long, addr, unsigned long, len, unsigned long, + prot, unsigned long, flags, unsigned long, fd, unsigned long, offset) +{ +/* +* Note that the shift for mmap2 is constant (12), +* regardless of PAGE_SIZE +*/ + +if (offset & (~PAGE_MASK >>12 )) +return -EINVAL ; + +return ksys_mmap_pgoff (addr, len, prot, flags, fd, + offset >> (PAGE_SHIFT -12 )); +} +#endif + void *sys_call_table[__NR_syscalls] = { [0 ... __NR_syscalls - 1] = sys_ni_syscall, +#ifdef CONFIG_64BIT#include <asm/syscall_table_64.h> +#else +#include <asm/syscall_table_32.h> +#endif }; typedef long (*sys_call_fn)(unsigned long, unsigned long,
Signature · 1 line
2.43.0
Arnd Bergmann
maintainerRe: [PATCH 1/3] loongarch: Wire up 32 bit syscalls
Quoted 7 lines · Jiaxun Yang
arch/loongarch/kernel/syscall.c +6
-0+#ifdef CONFIG_32BIT +SYSCALL_DEFINE6(mmap2, unsigned long, addr, unsigned long, len, unsigned long, + prot, unsigned long, flags, unsigned long, fd, unsigned long, offset) +{ +/* +* Note that the shift for mmap2 is constant (12),
arch/loongarch/kernel/syscall.c SYSCALL_DEFINE6(mmap, unsigned long, addr, unsigned long, len, unsigned long, · line 52
Quoted 10 lines
arch/loongarch/kernel/syscall.c +10
-0+* regardless of PAGE_SIZE +*/ + +if (offset & (~PAGE_MASK >>12 )) +return -EINVAL ; + +return ksys_mmap_pgoff (addr, len, prot, flags, fd, + offset >> (PAGE_SHIFT - 12)); +} +#endif
I think it's time we move this into mm/mmap.c and agree on the calling conventions across architectures. I'm currently travelling, but I can dig out a patch I made a while ago to convert most 32-bit architectures over to a common mmap2() implementation.
As far as I can tell, the only architectures that actually want mmap_pgoff() are m68k, arc and hexagon. Everything else either has a fixed 4KB page size (so mmap_pgoff and mmap2 are the same), or they already enforce the mmap2 semantics.
There are some smaller differences between architectures at the moment that I think shouldn't really exist: sparc32/m68k/arm32/parisc skips the alignment check, sparc64 and alpha add an overflow check (on sys_mmap) and powerpc adds a pgprot argument check. I think the version you have is fine for common code (including the ones that don't check alignment today), not sure about the additional checks.
Arnd
PATCH 2/3 loongarch: Introduce sys_loongarch_flush_icache syscall
In reply to Jiaxun YangOn LoongArch CPUs with ICACHET, writes automatically sync to both local and remote instruction caches. CPUs without this feature lack userspace cache flush instructions, requiring a syscall to maintain I/D cache coherence and propagate to remote caches.
sys_loongarch_flush_icache() is defined to flush the instruction cache over an address range, with the flush applying to either all threads or just the caller.
Currently all LoongArch64 implementations from Loongson comes with ICACHET, however most LoongArch32 implementations including openLA500 and emerging third party LoongArch64 implementations such as WiredNG are coming without ICACHET.
Sadly many user space applications are assuming ICACHET support, we can't recall those binaries. So we'd better get UAPI for cacheflush ready soonish and encourage application to start using it.
The syscall resolves to a ibar for now, it should be revised when we have actual non-ICACHET support in kernel.
--- arch/loongarch/include/asm/cacheflush.h | 6 ++++++ arch/loongarch/include/asm/syscall.h | 2 ++ arch/loongarch/kernel/Makefile.syscalls | 3 +-- arch/loongarch/kernel/syscall.c | 28 ++++++++++++++++++++++++++++ scripts/syscall.tbl | 2 ++ 5 files changed, 39 insertions(+), 2 deletions(-)
arch/loongarch/include/asm/cacheflush.h +6 -0
@@ -80,6 +80,12 @@ static inline void flush_cache_line(int leaf, unsigned long addr) } } +/* +* Bits in sys_loongarch_flush_icache()'s flags argument. +*/ +#define SYS_LOONGARCH_FLUSH_ICACHE_LOCAL 1UL +#define SYS_LOONGARCH_FLUSH_ICACHE_ALL (SYS_LOONGARCH_FLUSH_ICACHE_LOCAL) +#include <asm-generic/cacheflush.h> #endif /* _ASM_CACHEFLUSH_H */
arch/loongarch/include/asm/syscall.h +2 -0
@@ -71,4 +71,6 @@ static inline bool arch_syscall_is_vdso_sigreturn(struct pt_regs *regs) return false; } +asmlinkage long sys_loongarch_flush_icache (uintptr_t ,uintptr_t ,uintptr_t ); +#endif /* __ASM_LOONGARCH_SYSCALL_H */
arch/loongarch/kernel/Makefile.syscalls +1 -2
@@ -1,4 +1,3 @@ # SPDX-License-Identifier: GPL-2.0 -# No special ABIs on loongarch so far -syscall_abis_64 += +syscall_abis_64 += loongarch
arch/loongarch/kernel/syscall.c +28 -0
@@ -15,6 +15,7 @@#include <linux/unistd.h> #include <asm/asm.h> +#include <asm/cacheflush.h> #include <asm/exception.h> #include <asm/loongarch.h> #include <asm/signal.h>
@@ -51,6 +52,33 @@ SYSCALL_DEFINE6(mmap2, unsigned long, addr, unsigned long, len, unsigned long, }#endif +/* +* On LoongArch CPUs with ICACHET, writes automatically sync to both local and +* remote instruction caches. CPUs without this feature lack userspace cache +* flush instructions, requiring a syscall to maintain I/D cache coherence and +* propagate to remote caches. +* +* sys_loongarch_flush_icache() is defined to flush the instruction cache +* over an address range, with the flush applying to either all threads or +* just the caller. +*/ +SYSCALL_DEFINE3 (loongarch_flush_icache, uintptr_t, start, uintptr_t, end, + uintptr_t, flags) +{ +/* Check the reserved flags. */ +if (unlikely (flags & ~SYS_LOONGARCH_FLUSH_ICACHE_ALL )) +return -EINVAL ; + +/* +* SYS_LOONGARCH_FLUSH_ICACHE_LOCAL is not handled so far, needs +* to be realized when non-ICACHET CPUs are supported. +*/ + +flush_icache_user_range (start, end); + +return 0 ; +} +void *sys_call_table[__NR_syscalls] = { [0 ... __NR_syscalls -1 ] = sys_ni_syscall, #ifdef CONFIG_64BIT
scripts/syscall.tbl +2 -0
@@ -298,6 +298,8 @@ 244 csky set_thread_area sys_set_thread_area 245 csky cacheflush sys_cacheflush +259 loongarch loongarch_flush_icache sys_loongarch_flush_icache + 244 nios2 cacheflush sys_cacheflush 244 or1k or1k_atomic sys_or1k_atomic
Signature · 1 line
2.43.0
Re: [PATCH 2/3] loongarch: Introduce sys_loongarch_flush_icache syscall
Hi, Jiaxun,
Quoted 63 lines · Jiaxun Yang
On LoongArch CPUs with ICACHET, writes automatically sync to both local and remote instruction caches. CPUs without this feature lack userspace cache flush instructions, requiring a syscall to maintain I/D cache coherence and propagate to remote caches.
sys_loongarch_flush_icache()is defined to flush the instruction cache over an address range, with the flush applying to either all threads or just the caller.Currently all LoongArch64 implementations from Loongson comes with ICACHET, however most LoongArch32 implementations including openLA500 and emerging third party LoongArch64 implementations such as WiredNG are coming without ICACHET.
Sadly many user space applications are assuming ICACHET support, we can't recall those binaries. So we'd better get UAPI for cacheflush ready soonish and encourage application to start using it.
The syscall resolves to a ibar for now, it should be revised when we have actual non-ICACHET support in kernel.
Signed-off-by Jiaxun Yang <[email protected]>--- arch/loongarch/include/asm/cacheflush.h | 6 ++++++ arch/loongarch/include/asm/syscall.h | 2 ++ arch/loongarch/kernel/Makefile.syscalls | 3 +-- arch/loongarch/kernel/syscall.c | 28 ++++++++++++++++++++++++++++ scripts/syscall.tbl | 2 ++ 5 files changed, 39 insertions(+), 2 deletions(-)arch/loongarch/include/asm/cacheflush.h +6
-0@@ -80,6 +80,12 @@ static inline void flush_cache_line(int leaf, unsigned long addr) } } +/* +* Bits in sys_loongarch_flush_icache()'s flags argument. +*/ +#define SYS_LOONGARCH_FLUSH_ICACHE_LOCAL 1UL +#define SYS_LOONGARCH_FLUSH_ICACHE_ALL (SYS_LOONGARCH_FLUSH_ICACHE_LOCAL) +#include <asm-generic/cacheflush.h> #endif /* _ASM_CACHEFLUSH_H */ arch/loongarch/include/asm/syscall.h +2
-0@@ -71,4 +71,6 @@ static inline bool arch_syscall_is_vdso_sigreturn(struct pt_regs *regs) return false; } +asmlinkage long sys_loongarch_flush_icache (uintptr_t ,uintptr_t ,uintptr_t ); +#endif /* __ASM_LOONGARCH_SYSCALL_H */ arch/loongarch/kernel/Makefile.syscalls +0
-0
arch/loongarch/kernel/Makefile.syscalls line 3
Quoted 5 lines
arch/loongarch/kernel/Makefile.syscalls +1
-2@@ -1,4 +1,3 @@ # SPDX-License-Identifier: GPL-2.0 -# No special ABIs on loongarch so far -syscall_abis_64 += +syscall_abis_64 += loongarch
LoongArch64 need arch-specific syscall, but LoongArch32 needn't?
Quoted 51 lines
arch/loongarch/kernel/syscall.c +28
-0@@ -15,6 +15,7 @@#include <linux/unistd.h> #include <asm/asm.h> +#include <asm/cacheflush.h> #include <asm/exception.h> #include <asm/loongarch.h> #include <asm/signal.h> @@ -51,6 +52,33 @@ SYSCALL_DEFINE6(mmap2, unsigned long, addr, unsigned long, len, unsigned long, }#endif +/* +* On LoongArch CPUs with ICACHET, writes automatically sync to both local and +* remote instruction caches. CPUs without this feature lack userspace cache +* flush instructions, requiring a syscall to maintain I/D cache coherence and +* propagate to remote caches. +* +* sys_loongarch_flush_icache() is defined to flush the instruction cache +* over an address range, with the flush applying to either all threads or +* just the caller. +*/ +SYSCALL_DEFINE3 (loongarch_flush_icache, uintptr_t, start, uintptr_t, end, + uintptr_t, flags) +{ +/* Check the reserved flags. */ +if (unlikely (flags & ~SYS_LOONGARCH_FLUSH_ICACHE_ALL )) +return -EINVAL ; + +/* +* SYS_LOONGARCH_FLUSH_ICACHE_LOCAL is not handled so far, needs +* to be realized when non-ICACHET CPUs are supported. +*/ + +flush_icache_user_range (start, end); + +return 0 ; +} +void *sys_call_table[__NR_syscalls] = { [0 ... __NR_syscalls -1 ] = sys_ni_syscall, #ifdef CONFIG_64BITscripts/syscall.tbl +0
-0
scripts/syscall.tbl line 301
Quoted 4 lines
scripts/syscall.tbl +1
-0@@ -298,6 +298,8 @@ 244 csky set_thread_area sys_set_thread_area 245 csky cacheflush sys_cacheflush +259 loongarch loongarch_flush_icache sys_loongarch_flush_icache
Can we use cacheflush as arc, csky and nios2?
Jinyang
Quoted 5 lines
scripts/syscall.tbl +1
-0+ 244 nios2 cacheflush sys_cacheflush 244 or1k or1k_atomic sys_or1k_atomic
Re: [PATCH 2/3] loongarch: Introduce sys_loongarch_flush_icache syscall
在2025年1月4日一月 上午9:04,Jinyang Shen写道:
[...]
Quoted 6 lines
arch/loongarch/kernel/Makefile.syscalls line 3
Quoted 4 lines
arch/loongarch/kernel/Makefile.syscalls +1
-2-# No special ABIs on loongarch so far -syscall_abis_64 += +syscall_abis_64 += loongarchLoongArch64 need arch-specific syscall, but LoongArch32 needn't?
My bad, lost in rebasing :-)
Quoted 1 line
[...]
Quoted 11 lines
scripts/syscall.tbl line 301
Quoted 9 lines
scripts/syscall.tbl +1
-0@@ -298,6 +298,8 @@ 244 csky set_thread_area sys_set_thread_area 245 csky cacheflush sys_cacheflush +259 loongarch loongarch_flush_icache sys_loongarch_flush_icacheCan we use cacheflush as arc, csky and nios2?
I think cacheflush syscall is more or less an outdated design inherited from...MIPS...
Exposing flush of other cache levels to user space is not wise in security perspective. The design of cacheflush syscall is also not vDSO friendly. riscv_flush_icache is designed to avoid those drawbacks, and we should follow.
Thanks
Quoted 3 lines
Jinyang
[...]
Signature · 1 line
- Jiaxun
Arnd Bergmann
maintainerRe: [PATCH 2/3] loongarch: Introduce sys_loongarch_flush_icache syscall
In reply to Jinyang ShenQuoted 13 lines · Jinyang Shen
arch/loongarch/kernel/syscall.c SYSCALL_DEFINE6(mmap2, unsigned long, addr, unsigned long, len, unsigned long, · line 66
Quoted 12 lines · Jiaxun Yang
arch/loongarch/kernel/syscall.c +12
-0+/* +* On LoongArch CPUs with ICACHET, writes automatically sync to both local and +* remote instruction caches. CPUs without this feature lack userspace cache +* flush instructions, requiring a syscall to maintain I/D cache coherence and +* propagate to remote caches. +* +* sys_loongarch_flush_icache() is defined to flush the instruction cache +* over an address range, with the flush applying to either all threads or +* just the caller. +*/ +SYSCALL_DEFINE3 (loongarch_flush_icache, uintptr_t, start, uintptr_t, end, + uintptr_t, flags)
I think for consistency with other architectures, we want start/length/flags instead of start/end/flags.
The meaning of the third argument is rather inconsistent between architectures already, but at least the second argument is always length so far.
Quoted 11 lines
scripts/syscall.tbl line 301
Quoted 9 lines
scripts/syscall.tbl +1
-0@@ -298,6 +298,8 @@ 244 csky set_thread_area sys_set_thread_area 245 csky cacheflush sys_cacheflush +259 loongarch loongarch_flush_icache sys_loongarch_flush_icacheCan we use cacheflush as arc, csky and nios2?
Agreed. I would also use the number 244 instead of 259 here.
Arnd
Re: [PATCH 2/3] loongarch: Introduce sys_loongarch_flush_icache syscall
在2025年1月4日一月 下午3:07,Arnd Bergmann写道:
[...]
Hi Arnd,
Thanks for your comments!
Quoted 7 lines
I think for consistency with other architectures, we want start/length/flags instead of start/end/flags.
The meaning of the third argument is rather inconsistent between architectures already, but at least the second argument is always length so far.
So this is actually designed to be aligned with RISC-V's semantics, and thus all arguments are aligned with RISC-V.
IMO RISC-V's semantics is a better design that we should take, as I replied to Jinyang above.
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
scripts/syscall.tbl +0
-0@@ -298,6 +298,8 @@ 244 csky set_thread_area sys_set_thread_area 245 csky cacheflush sys_cacheflush
Quoted 5 lines
Quoted 3 lines
scripts/syscall.tbl line 301
Quoted 1 line
scripts/syscall.tbl +1
-0+259 loongarch loongarch_flush_icache sys_loongarch_flush_icacheCan we use cacheflush as arc, csky and nios2?
Agreed. I would also use the number 244 instead of 259 here.
259 is also selected to be aligned with RISC-V.
Thanks
Quoted 2 lines
Arnd
Signature · 1 line
- Jiaxun
Re: [PATCH 2/3] loongarch: Introduce sys_loongarch_flush_icache syscall
In reply to Jiaxun YangOn Thu, 2025-01-02 at 18:34 +0000, Jiaxun Yang wrote:
/* snip */
Quoted 3 lines
Sadly many user space applications are assuming ICACHET support, we can't recall those binaries. So we'd better get UAPI for cacheflush ready soonish and encourage application to start using it.
To encourage the developers changing ibar to loongarch_flush_icache, we should minimize the extra overhead on mainstream systems. We can add an vDSO layer so if the CPU has ICACHET:
int vdso_loongarch_flush_icache(...)
{
asm ("ibar 0");
return 0;
}
And otherwise the vDSO wrapper invokes the real syscall. I've implemented the boot-time alternative runtime patching for vDSO at
https://lore.kernel.org/loongarch/[email protected]/.
Quoted 2 lines
The syscall resolves to a ibar for now, it should be revised when we have actual non-ICACHET support in kernel.
/* snip */
arch/loongarch/include/asm/cacheflush.h static inline void flush_cache_line(int leaf, unsigned long addr) · line 87
Quoted 13 lines
arch/loongarch/include/asm/cacheflush.h +5
-0@@ -80,6 +80,12 @@ static inline void flush_cache_line(int leaf, unsigned long addr) } } +/* +* Bits in sys_loongarch_flush_icache()'s flags argument. +*/ +#define SYS_LOONGARCH_FLUSH_ICACHE_LOCAL 1UL +#define SYS_LOONGARCH_FLUSH_ICACHE_ALL (SYS_LOONGARCH_FLUSH_ICACHE_LOCAL)
Not a UAPI header so not usable by the user? How would they specify flags then?
If you meant to add them for UAPI, it would be very problematic. When a new cache type emerges in the hardware implementations, we need to grow SYS_LOONGARCH_FLUSH_ICACHE_ALL in the UAPI header, but we cannot change the already compiled JIT applications. Thus all JIT applications have to be recompiled with the latest UAPI header. This just seems an unnecessary severe burden to the packagers.
Instead IMO it's better not to expose so much details to the userspace. Just remove the flags argument and flush all the icaches the kernel knows, so with a new cache type the user (and distro) just need to update or patch their kernel, w/o recompiling all JIT apps.
Signature · 2 lines
Xi Ruoyao <[email protected]> School of Aerospace Science and Technology, Xidian University
Re: [PATCH 2/3] loongarch: Introduce sys_loongarch_flush_icache syscall
Quoted 11 lines · 在2025年1月4日一月 上午9:31,Xi Ruoyao写道:
On Thu, 2025-01-02 at 18:34 +0000, Jiaxun Yang wrote:
/* snip */
Quoted 3 lines
Sadly many user space applications are assuming ICACHET support, we can't recall those binaries. So we'd better get UAPI for cacheflush ready soonish and encourage application to start using it.
To encourage the developers changing ibar to
loongarch_flush_icache, we should minimize the extra overhead on mainstream systems. We can add an vDSO layer so if the CPU has ICACHET:
I'm a little bit confused as that's exactly what I'm doing in PATCH 3.
Quoted 10 lines
int
vdso_loongarch_flush_icache(...)
{
asm ("ibar 0");
return 0;
}And otherwise the vDSO wrapper invokes the real syscall. I've implemented the boot-time alternative runtime patching for vDSO at
https://lore.kernel.org/loongarch/[email protected]/.
Thanks! Noted.
Quoted 14 lines
Quoted 3 lines
The syscall resolves to a ibar for now, it should be revised when we have actual non-ICACHET support in kernel.
/* snip */
Quoted 8 lines
arch/loongarch/include/asm/cacheflush.h +0
-0@@ -80,6 +80,12 @@ static inline void flush_cache_line(int leaf, unsigned long addr) } }
Quoted 8 lines
arch/loongarch/include/asm/cacheflush.h static inline void flush_cache_line(int leaf, unsigned long addr) · line 87
Quoted 5 lines
arch/loongarch/include/asm/cacheflush.h +5
-0+/* +* Bits in sys_loongarch_flush_icache()'s flags argument. +*/ +#define SYS_LOONGARCH_FLUSH_ICACHE_LOCAL 1UL +#define SYS_LOONGARCH_FLUSH_ICACHE_ALL (SYS_LOONGARCH_FLUSH_ICACHE_LOCAL)Not a UAPI header so not usable by the user? How would they specify flags then?
We are following the RISC-V's convention on not exposing flags in UAPI header for now as it's not really ready.
Quoted 7 lines
If you meant to add them for UAPI, it would be very problematic. When a new cache type emerges in the hardware implementations, we need to grow
SYS_LOONGARCH_FLUSH_ICACHE_ALLin the UAPI header, but we cannot change the already compiled JIT applications. Thus all JIT applications have to be recompiled with the latest UAPI header. This just seems an unnecessary severe burden to the packagers.
The _LOCAL flag not meant to be hardware cache level but the scope. (i.e. all threads or just the caller). Vast majority of applications shouldn't need this level of granularity, so just setting flags to zero.
However, for application want fine-grained optimisations they should probe availability of flags before using it. Thus kernel should reject all unknown flags to assist application probing.
Quoted 5 lines
Instead IMO it's better not to expose so much details to the userspace. Just remove the flags argument and flush all the icaches the kernel knows, so with a new cache type the user (and distro) just need to update or patch their kernel, w/o recompiling all JIT apps.
There is no need to change anything in user space usage when a new cache type emerge. See explanations above.
Thanks
Quoted 4 lines
Signature · 3 lines
-- Xi Ruoyao <[email protected]> School of Aerospace Science and Technology, Xidian University
Signature · 1 line
- Jiaxun
PATCH 3/3 loongarch: vdso: Introduce __vdso_flush_icache function
In reply to Jiaxun YangIntroduce __vdso_flush_icache function for performing cache flush in userspace whenever possible.
It will use ibar in userspace if CPU comes with ICACHET support, and fallback to syscall if not. It also made infra ready for possible future userspace cache flush instruction.
--- arch/loongarch/include/asm/vdso/vdso.h | 10 ++++++ arch/loongarch/include/asm/vdso/vsyscall.h | 1 + arch/loongarch/kernel/vdso.c | 2 ++ arch/loongarch/mm/cache.c | 3 ++ arch/loongarch/vdso/Makefile | 2 +- arch/loongarch/vdso/flush_icache.c | 50 ++++++++++++++++++++++++++++++ arch/loongarch/vdso/vdso.lds.S | 5 +++ 7 files changed, 72 insertions(+), 1 deletion(-)
arch/loongarch/include/asm/vdso/vdso.h +10 -0
@@ -13,6 +13,15 @@#include <asm/page.h> #include <asm/vdso.h> +enum vdso_icacle_flush_mode { +VDSO_ICACLE_FLUSH_IBAR , +VDSO_ICACLE_FLUSH_FALLBACK , +}; + +struct vdso_icache_flush_data { +enum vdso_icacle_flush_mode mode; +}; +struct vdso_pcpu_data {u32 node; }____cacheline_aligned_in_smp ;
@@ -20,6 +29,7 @@ struct vdso_pcpu_data {struct loongarch_vdso_data {struct vdso_pcpu_data pdata[NR_CPUS ];struct vdso_rng_data rng_data; +struct vdso_icache_flush_data icache_flush_data; };/*
arch/loongarch/include/asm/vdso/vsyscall.h +1 -0
@@ -8,6 +8,7 @@extern struct vdso_data *vdso_data;extern struct vdso_rng_data *vdso_rng_data; +extern struct vdso_icache_flush_data *vdso_icache_flush_data;static __always_inline struct vdso_data *__loongarch_get_k_vdso_data(void)
arch/loongarch/kernel/vdso.c +2 -0
@@ -36,6 +36,8 @@ static union {struct vdso_data *vdso_data = generic_vdso_data.data;struct vdso_pcpu_data *vdso_pdata = loongarch_vdso_data.vdata.pdata;struct vdso_rng_data *vdso_rng_data = &loongarch_vdso_data.vdata.rng_data; +struct vdso_icache_flush_data *vdso_icache_flush_data = + &loongarch_vdso_data.vdata.icache_flush_data;static int vdso_mremap (const struct vm_special_mapping *sm,struct vm_area_struct *new_vma) {
arch/loongarch/mm/cache.c +3 -0
@@ -24,6 +24,7 @@#include <asm/numa.h> #include <asm/processor.h> #include <asm/setup.h> +#include <asm/vdso/vsyscall.h> void cache_error_setup (void ) {
@@ -156,6 +157,8 @@ void cpu_cache_init(void) current_cpu_data.cache_leaves_present = leaf; current_cpu_data.options |=LOONGARCH_CPU_PREFETCH ; + + vdso_icache_flush_data->mode =VDSO_ICACLE_FLUSH_IBAR ; }static const pgprot_t protection_map[16 ] = {
arch/loongarch/vdso/Makefile +1 -1
@@ -4,7 +4,7 @@ # Include the generic Makefile to check the built vdso. include $(srctree)/lib/vdso/Makefile -obj-vdso-y := elf.o vgetcpu.o vgettimeofday.o vgetrandom.o \ +obj-vdso-y := elf.o flush_icache.o vgetcpu.o vgettimeofday.o vgetrandom.o \ vgetrandom-chacha.o sigreturn.o # Common compiler flags between ABIs.
arch/loongarch/vdso/flush_icache.c new file +50 -0
50 lines not on this page · raw message
arch/loongarch/vdso/vdso.lds.S +5 -0
@@ -72,6 +72,11 @@ VERSION __vdso_rt_sigreturn; local: *; }; + LINUX_6.14 { + global: + __vdso_flush_icache; + local: *; + }; } /*
Signature · 1 line
2.43.0
Quoted 73 lines · Jiaxun Yang
This series defines the UAPI for LoongArch32, marking my initial step towards upstreaming support for the architecture. Once the UAPI is ratified, we can proceed to scrutinise various kernel components to enable 32-bit support while simultaneously addressing user-space porting.
Why am I upstreaming LoongArch32?
================================
Although 32-bit systems are experiencing declining adoption in general computing, LoongArch32 remains highly relevant within specific niches. Beyond embedded applications, several vendors are actively developing application-level LoongArch32 processors. Loongson, for example, has released two open-source reference hardware implementations: openLA500 and openLA1000 [6].The architecture also holds considerable educational value, having been integrated into China's national computer architecture curricula and embedded systems courses. Additionally, the National Student Computer System Capability Challenge (NSCSCC) [1] features LoongArch32 CPUs, where hundreds of students design Linux-capable hardware implementations and compete on performance. This initiative has resulted in several exciting high-performance LoongArch32 cores, including LainCore[2], Wired[3], NOP-Core[4], NagiCore[5]....
Quoted 1 line
From an upstream perspective, we will largely reuse the infrastructure
already established for LoongArch64, ensuring that the maintenance burden remains minimal.
Porting Status
==============
The LoongArch32 port has been available downstream for some time, with various system components hosted on Loongson's Gitee[6]. However, these components utilise an older downstream ABI and fall short of upstream quality.On the upstream front, LLVM-19 now includes experimental support for LoongArch32 (ILP32 ABI) under the loongarch32* triple, and efforts are underway to enable GNU toolchain support. My upstream-ready kernel port and musl libc port can successfully boot into a minimal Buildroot environment and execute test cases on QEMU virt machine with clang toolchain.
Thank you for reading. I look forward to your comments and feedback.
[1]: https://www.tsinghua.edu.cn/en/info/1245/13802.htm
[2]: https://github.com/LainChip/LainCore
[3]: https://github.com/gmlayer0/wired
[4]: https://github.com/NOP-Processor/NOP-Core
[5]: https://github.com/MrAMS/NagiCore
[6]: https://gitee.com/loongson-eduSigned-off-by Jiaxun Yang <[email protected]>---
Jiaxun Yang (3):
loongarch: Wire up 32 bit syscalls
loongarch: Introducesys_loongarch_flush_icachesyscall
loongarch: vdso: Introduce__vdso_flush_icachefunctionarch/loongarch/include/asm/Kbuild | 1 + arch/loongarch/include/asm/cacheflush.h | 6 ++++ arch/loongarch/include/asm/syscall.h | 2 ++ arch/loongarch/include/asm/vdso/vdso.h | 10 ++++++ arch/loongarch/include/asm/vdso/vsyscall.h | 1 + arch/loongarch/include/uapi/asm/Kbuild | 1 + arch/loongarch/include/uapi/asm/unistd.h | 6 ++++ arch/loongarch/kernel/Makefile.syscalls | 3 +- arch/loongarch/kernel/syscall.c | 49 +++++++++++++++++++++++++++++ arch/loongarch/kernel/vdso.c | 2 ++ arch/loongarch/mm/cache.c | 3 ++ arch/loongarch/vdso/Makefile | 2 +- arch/loongarch/vdso/flush_icache.c | 50 ++++++++++++++++++++++++++++++ arch/loongarch/vdso/vdso.lds.S | 5 +++ scripts/syscall.tbl | 2 ++ 15 files changed, 140 insertions(+), 3 deletions(-)
Quoted 5 lines
---
base-commit:8155b4ef3466f0e289e8fcc9e6e62f3f4dceeac2
change-id: 20250102-la32-uapi-8395e83a4e88Best regards,
Hi, Jiaxun,
Thank you for your hard work, I'm also working on LoongArch32 kernel side [1], I hope we can make it upstream together.
[1]: https://github.com/shenjinyang/la32r-Linux
Jinyang
Arnd Bergmann
maintainerQuoted 9 lines · Jiaxun Yang
Why am I upstreaming LoongArch32?
================================
Although 32-bit systems are experiencing declining adoption in general computing, LoongArch32 remains highly relevant within specific niches. Beyond embedded applications, several vendors are actively developing application-level LoongArch32 processors. Loongson, for example, has released two open-source reference hardware implementations: openLA500 and openLA1000 [6].
Quoted 8 lines
The architecture also holds considerable educational value, having been integrated into China's national computer architecture curricula and embedded systems courses. Additionally, the National Student Computer System Capability Challenge (NSCSCC) [1] features LoongArch32 CPUs, where hundreds of students design Linux-capable hardware implementations and compete on performance. This initiative has resulted in several exciting high-performance LoongArch32 cores, including LainCore[2], Wired[3], NOP-Core[4], NagiCore[5]....
I'm surprised that so many resources get put into 32-bit hardware implementations on loongarch, when this has mostly stopped on riscv and arm, where new hardware is practically all either 64-bit Linux or 32-bit NOMMU microcontrollers.
Quoted 11 lines
From an upstream perspective, we will largely reuse the infrastructure already established for LoongArch64, ensuring that the maintenance burden remains minimal.
Porting Status
==============
The LoongArch32 port has been available downstream for some time, with various system components hosted on Loongson's Gitee[6]. However, these components utilise an older downstream ABI and fall short of upstream quality.
Quoted 6 lines
On the upstream front, LLVM-19 now includes experimental support for LoongArch32 (ILP32 ABI) under the loongarch32* triple, and efforts are underway to enable GNU toolchain support. My upstream-ready kernel port and musl libc port can successfully boot into a minimal Buildroot environment and execute test cases on QEMU virt machine with clang toolchain.
I assume the MIPS legacy means that a 64-bit kernel is going to be able to run the same ILP32 binaries as a 32-bit kernel running on pure 32-bit hardware, similar to powerpc/s390/x86, but unlike riscv/arm?
We need to be careful in defining the ABI to ensure that this covers all the corner cases, such as defining a signal stack layout with room to save 64-bit user register contents if there is a chance that a 32-bit userspace will end up using the wide registers when running on a 64-bit kernel, but also avoid any dependency on 64-bit registers in the ABI itself.
Arnd
Quoted 43 lines · Arnd Bergmann
Quoted 9 lines · Jiaxun Yang
Why am I upstreaming LoongArch32?
================================
Although 32-bit systems are experiencing declining adoption in general computing, LoongArch32 remains highly relevant within specific niches. Beyond embedded applications, several vendors are actively developing application-level LoongArch32 processors. Loongson, for example, has released two open-source reference hardware implementations: openLA500 and openLA1000 [6].Quoted 8 lines
The architecture also holds considerable educational value, having been integrated into China's national computer architecture curricula and embedded systems courses. Additionally, the National Student Computer System Capability Challenge (NSCSCC) [1] features LoongArch32 CPUs, where hundreds of students design Linux-capable hardware implementations and compete on performance. This initiative has resulted in several exciting high-performance LoongArch32 cores, including LainCore[2], Wired[3], NOP-Core[4], NagiCore[5]....
I'm surprised that so many resources get put into 32-bit hardware implementations on loongarch, when this has mostly stopped on riscv and arm, where new hardware is practically all either 64-bit Linux or 32-bit NOMMU microcontrollers.
Quoted 11 lines
From an upstream perspective, we will largely reuse the infrastructure already established for LoongArch64, ensuring that the maintenance burden remains minimal.
Porting Status
==============
The LoongArch32 port has been available downstream for some time, with various system components hosted on Loongson's Gitee[6]. However, these components utilise an older downstream ABI and fall short of upstream quality.Quoted 6 lines
On the upstream front, LLVM-19 now includes experimental support for LoongArch32 (ILP32 ABI) under the loongarch32* triple, and efforts are underway to enable GNU toolchain support. My upstream-ready kernel port and musl libc port can successfully boot into a minimal Buildroot environment and execute test cases on QEMU virt machine with clang toolchain.
Quoted 4 lines
I assume the MIPS legacy means that a 64-bit kernel is going to be able to run the same ILP32 binaries as a 32-bit kernel running on pure 32-bit hardware, similar to powerpc/s390/x86, but unlike riscv/arm?
LoongArch has instructions like addi.d/addi.w, instead of addi/addi.w, thus on 32-bit implementation it's simply addi.d is missing, not the semantic of addi is changed. So I cannot see a real reason we cannot support the same ILP32 userspace binaries compiled for 32-bit hardware on 64-bit hardware.
Quoted 6 lines
We need to be careful in defining the ABI to ensure that this covers all the corner cases, such as defining a signal stack layout with room to save 64-bit user register contents if there is a chance that a 32-bit userspace will end up using the wide registers when running on a 64-bit kernel, but also avoid any dependency on 64-bit registers in the ABI itself.
Yes such issues are nasty, we'd already need something in the calling convention like "on 64-bit hardware, in ILP32 ABI the saved registers may be unchanged or changed to the sign-extension from the lower 32 bits of the original value."
Signature · 2 lines
Xi Ruoyao <[email protected]> School of Aerospace Science and Technology, Xidian University
在2025年1月4日一月 下午3:13,Xi Ruoyao写道:
[...]
Quoted 11 lines
Quoted 5 lines
I assume the MIPS legacy means that a 64-bit kernel is going to be able to run the same ILP32 binaries as a 32-bit kernel running on pure 32-bit hardware, similar to powerpc/s390/x86, but unlike riscv/arm?
LoongArch has instructions like addi.d/addi.w, instead of addi/addi.w, thus on 32-bit implementation it's simply addi.d is missing, not the semantic of addi is changed. So I cannot see a real reason we cannot support the same ILP32 userspace binaries compiled for 32-bit hardware on 64-bit hardware.
The only concern is the behaviour of PC relative instructions will change in VA32 mode for LoongArch64 systems, i.e. address will be signed extended. However I think this serves ILP32 purpose well.
Quoted 8 lines
Quoted 7 lines
We need to be careful in defining the ABI to ensure that this covers all the corner cases, such as defining a signal stack layout with room to save 64-bit user register contents if there is a chance that a 32-bit userspace will end up using the wide registers when running on a 64-bit kernel, but also avoid any dependency on 64-bit registers in the ABI itself.
Quoted 4 lines
Yes such issues are nasty, we'd already need something in the calling convention like "on 64-bit hardware, in ILP32 ABI the saved registers may be unchanged or changed to the sign-extension from the lower 32 bits of the original value."
Makes sense to me. For MIPS the n32 (ILP32 for 64bit) ABI has a new set of UAPI definition (also mandate 64bit GPR). While the vanilla o32 ABI is 32bit only, which disallows any 64bit instruction in user space.
When I'm designing current LA32 ABI I actually have o32 ABI in mind. However LoongArch64 hardware is not capable to disable 32bit instructions alone. So if we end up doing something like o32 the limitation of 32bit instruction needs to be enforced at compiler side.
So I think the question would do we want to allow 64bit instructions for LoongArch's ILP32 kernel UAPI. We can either go through MIPS's o32 PATH, making 32bit ABI truly 32bit, or maybe reusing the UAPI for ILP32 on 64.
From Guo Ren's RISC-V's compat work and arm64ilp32 I can certainly see the benefit of ILP32 on 64. Maybe we can bring that to LoongArch as well.
Thanks
Quoted 4 lines
Signature · 3 lines
-- Xi Ruoyao <[email protected]> School of Aerospace Science and Technology, Xidian University
Signature · 1 line
- Jiaxun
Arnd Bergmann
maintainerQuoted 18 lines · Jiaxun Yang
Quoted 8 lines · 在2025年1月4日一月 下午3:13,Xi Ruoyao写道:
Quoted 7 lines
We need to be careful in defining the ABI to ensure that this covers all the corner cases, such as defining a signal stack layout with room to save 64-bit user register contents if there is a chance that a 32-bit userspace will end up using the wide registers when running on a 64-bit kernel, but also avoid any dependency on 64-bit registers in the ABI itself.
Quoted 4 lines
Yes such issues are nasty, we'd already need something in the calling convention like "on 64-bit hardware, in ILP32 ABI the saved registers may be unchanged or changed to the sign-extension from the lower 32 bits of the original value."
Makes sense to me. For MIPS the n32 (ILP32 for 64bit) ABI has a new set of UAPI definition (also mandate 64bit GPR). While the vanilla o32 ABI is 32bit only, which disallows any 64bit instruction in user space.
Quoted 8 lines
When I'm designing current LA32 ABI I actually have o32 ABI in mind. However LoongArch64 hardware is not capable to disable 32bit instructions alone. So if we end up doing something like o32 the limitation of 32bit instruction needs to be enforced at compiler side.
So I think the question would do we want to allow 64bit instructions for LoongArch's ILP32 kernel UAPI. We can either go through MIPS's o32 PATH, making 32bit ABI truly 32bit, or maybe reusing the UAPI for ILP32 on 64.
If at all possible, I think both the kernel's UAPI and the user side ELF psABI should be defined as compatible with 32-bit hardware and with userspace running on 64-bit kernels.
Quoted 2 lines
From Guo Ren's RISC-V's compat work and arm64ilp32 I can certainly see the benefit of ILP32 on 64. Maybe we can bring that to LoongArch as well.
I would not take these as examples, since something went wrong for each of them:
- RISC-V defined rv64 to not be a superset of rv32, so arithmetic instructions behave differently unless you switch modes
- aarch64 and aarch32 modes are completely different instruction sets, so aarch64ilp32 is by definition incompatible
- mips o32 as I understand it could work with 64-bit at the ISA level, as n32 does, but the ELF ABI does not allow using 64-bit registers, while n32 requires the use of 64-bit registers and does not work on 32-bit hardware.
If both the ISA and the ABI get it right, it should be possible to build 32-bit userspace that is compatible with both when targeting a 32-bit hardware, but still use 64-bit registers inside a single function when the compiler is building for a 64-bit capable CPU (e.g. "-march=la464 -m32"). There is a small cost in the calling conventions for passing u64 arguments in pairs of registers (unlike n32/x32/aarch64ilp32/rv64ilp32), but a huge benefit in not maintaining two incompatible ABIs.
Arnd
在2025年1月5日一月 上午4:43,Arnd Bergmann写道:
[...]
Quoted 8 lines
If both the ISA and the ABI get it right, it should be possible to build 32-bit userspace that is compatible with both when targeting a 32-bit hardware, but still use 64-bit registers inside a single function when the compiler is building for a 64-bit capable CPU (e.g. "-march=la464 -m32"). There is a small cost in the calling conventions for passing u64 arguments in pairs of registers (unlike n32/x32/aarch64ilp32/rv64ilp32), but a huge benefit in not maintaining two incompatible ABIs.
Thanks Arnd for elaborating this!
I actually more or less have this in mind when I was designing this ABI, thus GRs were designed to be 64bit in sigcontext. But I never look into that closely.
I'll try to explore that option, maybe come up with a COMPAT implementation first.
Thanks
Quoted 2 lines
Arnd
Signature · 1 line
- Jiaxun
Quoted 11 lines · 在2025年1月5日一月 上午10:27,Jiaxun Yang写道:
在2025年1月5日一月 上午4:43,Arnd Bergmann写道:
[...]Quoted 8 lines
If both the ISA and the ABI get it right, it should be possible to build 32-bit userspace that is compatible with both when targeting a 32-bit hardware, but still use 64-bit registers inside a single function when the compiler is building for a 64-bit capable CPU (e.g. "-march=la464 -m32"). There is a small cost in the calling conventions for passing u64 arguments in pairs of registers (unlike n32/x32/aarch64ilp32/rv64ilp32), but a huge benefit in not maintaining two incompatible ABIs.
Upon having a closer look, I think there's an issue regarding having uniformed ABI for LA32 and LA64. I'll call them ILP32GRLEN32 and ILP32GRLEN64 below.
If we allow interlinking, we must treat the upper 32 bits of all GPRs as caller-saved, since an ILP32GRLEN32 callee would be unaware of them. This could incur significant performance overhead.
Alternatively, we can disallow interlinking. However, this effectively creates a new, incompatible ABI. I'm not sure if it still fits our design goal.
Thanks
Quoted 4 lines
Quoted 1 line
Arnd
Signature · 1 line
- Jiaxun
Signature · 1 line
- Jiaxun