LoongArch: Add 128-bit atomic cmpxchg support (v3)
v3 cover LoongArch: Add 128-bit atomic cmpxchg support (v3)
This patch series adds 128-bit atomic compare-and-exchange support for LoongArch architecture, which fixes BPF scheduler test failures caused by missing 128-bit atomics support.
The series consists of two patches:
1. "LoongArch: Add 128-bit atomic cmpxchg support"
- Implements 128-bit atomic compare-and-exchange using LoongArch's LL.D/SC.Q instructions
- Fixes BPF scheduler test failures (scx_central scx_qmap) where kmalloc_nolock_noprof returns NULL due to missing 128-bit atomics, leading to -ENOMEM errors during scheduler initialization
2. "LoongArch: Enable 128-bit atomics cmpxchg support"
- Adds select HAVE_CMPXCHG_DOUBLE and select HAVE_ALIGNED_STRUCT_PAGE in Kconfig to enable 128-bit atomic cmpxchg support
The issue was identified through BPF scheduler test failures where scx_central and scx_qmap schedulers would fail to initialize. Testing was performed using the scx_qmap scheduler from tools/sched_ext/, confirming that the patches resolve the initialization failures.
---
Changes in v3:
- dbar 0 -> __WEAK_LLSC_MB
- =ZB" (__ptr[0]) -> "r" (__ptr)
- Link to v2: https://lore.kernel.org/r/[email protected]
Changes in v2:
- Use a normal ld.d for the high word instead of ll.d to avoid race condition
- Insert a dbar between ll.d and ld.d to prevent reordering
- Simply __cmpxchg128_asm("ll.d", "sc.q", ptr, o, n) to __cmpxchg128_asm(ptr, o, n)
- Fix address operand constraints after testing different approaches:
* ld.d with "m"
* ll.d with "ZC",
* sc.q with "ZB"(alternative constraints caused issues:
- "r" caused system hang
- "ZC" caused compiler error:
{standard input}: Assembler messages:
{standard input}:10037: Fatal error: Immediate overflow.
format: u0:0 )
- Link to v1: https://lore.kernel.org/r/[email protected]
---
George Guo (2):
LoongArch: Add 128-bit atomic cmpxchg support
LoongArch: Enable 128-bit atomics cmpxchg support
arch/loongarch/Kconfig | 2 ++ arch/loongarch/include/asm/cmpxchg.h | 47 ++++++++++++++++++++++++++++++++++++ 2 files changed, 49 insertions(+)
---
base-commit: d5ae5ac32615e4af729f0610fdc11ff4f4798aef
change-id: 20251120-2-d03862b2cf6d
Best regards,
Signature · 1 line
George Guo <[email protected]>
PATCH v3 1/2 LoongArch: Add 128-bit atomic cmpxchg support
From: George Guo <[email protected]>
Implement 128-bit atomic compare-and-exchange using LoongArch's LL.D/SC.Q instructions.
At the same time, fix BPF scheduler test failures (scx_central scx_qmap) caused by kmalloc_nolock_noprof returning NULL due to missing 128-bit atomics. The NULL returns led to -ENOMEM errors during scheduler initialization, causing test cases to fail.
Verified by testing with the scx_qmap scheduler (located in tools/sched_ext/). Building with make and running ./tools/sched_ext/build/bin/scx_qmap.
--- arch/loongarch/include/asm/cmpxchg.h | 47 ++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+)
arch/loongarch/include/asm/cmpxchg.h +47 -0
@@ -111,6 +111,44 @@ __arch_xchg(volatile void *ptr, unsigned long x, int size) __ret; \ }) +union __u128_halves { +u128 full; +struct { +u64 low; +u64 high; + }; +}; + +#define __cmpxchg128_asm (ptr, old, new) \ +({ \ + union __u128_halves __old, __new, __ret; \ + volatile u64 *__ptr = (volatile u64 *)(ptr); \ + \ + __old.full = (old); \ + __new.full = (new); \ + \ + __asm__ __volatile__( \ + "1: ll.d %0, %3 # 128-bit cmpxchg low \n" \ + __WEAK_LLSC_MB \ + " ld.d %1, %4 # 128-bit cmpxchg high \n" \ + " bne %0, %z5, 2f \n" \ + " bne %1, %z6, 2f \n" \ + " move $t0, %z7 \n" \ + " move $t1, %z8 \n" \ + " sc.q $t0, $t1, %2 \n" \ + " beqz $t0, 1b \n" \ + "2: \n" \ + __WEAK_LLSC_MB \ + : "=&r" (__ret.low), "=&r" (__ret.high) \ + : "r" (__ptr), \ + "ZC" (__ptr[0]), "m" (__ptr[1]), \ + "Jr" (__old.low), "Jr" (__old.high), \ + "Jr" (__new.low), "Jr" (__new.high) \ + : "t0", "t1", "memory"); \ + \ + __ret.full; \ +}) +static inline unsigned int __cmpxchg_small (volatile void *ptr,unsigned int old,unsigned int new,unsigned int size) {
@@ -198,6 +236,15 @@ __cmpxchg(volatile void *ptr, unsigned long old, unsigned long new, unsigned int __res; \ }) +/* cmpxchg128 */ +#define system_has_cmpxchg128 () 1 + +#define arch_cmpxchg128 (ptr, o, n) \ +({ \ + BUILD_BUG_ON(sizeof(*(ptr)) != 16); \ + __cmpxchg128_asm(ptr, o, n); \ +}) +#ifdef CONFIG_64BIT #define arch_cmpxchg64_local (ptr, o, n) \ ({ \
Signature · 1 line
2.48.1
PATCH v3 2/2 LoongArch: Enable 128-bit atomics cmpxchg support
In reply to George GuoFrom: George Guo <[email protected]>
Add select HAVE_CMPXCHG_DOUBLE and select HAVE_ALIGNED_STRUCT_PAGE in Kconfig to enable 128-bit atomic cmpxchg support on LoongArch.
--- arch/loongarch/Kconfig | 2 ++ 1 file changed, 2 insertions(+)
arch/loongarch/Kconfig +2 -0
@@ -114,6 +114,7 @@ config LOONGARCH select GENERIC_TIME_VSYSCALL select GPIOLIB select HAS_IOPORT + select HAVE_ALIGNED_STRUCT_PAGE select HAVE_ARCH_AUDITSYSCALL select HAVE_ARCH_JUMP_LABEL select HAVE_ARCH_JUMP_LABEL_RELATIVE
@@ -140,6 +141,7 @@ config LOONGARCH select HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS select HAVE_DYNAMIC_FTRACE_WITH_REGS select HAVE_EBPF_JIT + select HAVE_CMPXCHG_DOUBLE select HAVE_EFFICIENT_UNALIGNED_ACCESS if !ARCH_STRICT_ALIGN select HAVE_EXIT_THREAD select HAVE_GENERIC_TIF_BITS
Signature · 1 line
2.48.1
Huacai Chen
maintainerHi, George,
Quoted 4 lines · George Guo
This patch series adds 128-bit atomic compare-and-exchange support for LoongArch architecture, which fixes BPF scheduler test failures caused by missing 128-bit atomics support.
Have you tested your code on Loongson-3A5000/3C5000?
Huacai
Quoted 58 lines
The series consists of two patches:
1. "LoongArch: Add 128-bit atomic cmpxchg support"
- Implements 128-bit atomic compare-and-exchange using LoongArch's LL.D/SC.Q instructions
- Fixes BPF scheduler test failures (scx_centralscx_qmap) wherekmalloc_nolock_noprofreturns NULL due to missing 128-bit atomics, leading to-ENOMEMerrors during scheduler initialization2. "LoongArch: Enable 128-bit atomics cmpxchg support"
- Adds selectHAVE_CMPXCHG_DOUBLEand selectHAVE_ALIGNED_STRUCT_PAGEinKconfigto enable 128-bit atomic cmpxchg supportThe issue was identified through BPF scheduler test failures where
scx_centralandscx_qmapschedulers would fail to initialize. Testing was performed using thescx_qmapscheduler fromtools/sched_ext/, confirming that the patches resolve the initialization failures.Signed-off-by George Guo <[email protected]>---
Changes in v3:
- dbar 0 ->__WEAK_LLSC_MB
- =ZB" (__ptr[0]) -> "r" (__ptr)
- Link to v2: https://lore.kernel.org/r/[email protected]Changes in v2:
- Use a normal ld.d for the high word instead of ll.d to avoid race condition
- Insert a dbar between ll.d and ld.d to prevent reordering
- Simply__cmpxchg128_asm("ll.d", "sc.q", ptr, o, n)to__cmpxchg128_asm(ptr, o, n)
- Fix address operand constraints after testing different approaches:
* ld.d with "m"
* ll.d with "ZC",
* sc.q with "ZB"(alternative constraints caused issues:
- "r" caused system hang
- "ZC" caused compiler error:
{standard input}: Assembler messages:
{standard input}:10037: Fatal error: Immediate overflow.
format: u0:0 )
- Link to v1: https://lore.kernel.org/r/[email protected]---
George Guo (2):
LoongArch: Add 128-bit atomic cmpxchg support
LoongArch: Enable 128-bit atomics cmpxchg supportarch/loongarch/Kconfig | 2 ++ arch/loongarch/include/asm/cmpxchg.h | 47 ++++++++++++++++++++++++++++++++++++ 2 files changed, 49 insertions(+)---
base-commit:d5ae5ac32615e4af729f0610fdc11ff4f4798aef
change-id: 20251120-2-d03862b2cf6dBest regards,
--
George Guo <[email protected]>
Quoted 5 lines · On Wed, 26 Nov 2025 12:44:44 +0800 Huacai Chen <[email protected]> wrote:
Hi, George,
Quoted 1 line · George Guo
Quoted 7 lines
Quoted 3 lines
This patch series adds 128-bit atomic compare-and-exchange support for LoongArch architecture, which fixes BPF scheduler test failures caused by missing 128-bit atomics support.
Have you tested your code on Loongson-3A5000/3C5000?
Huacai
Hi Huacai,
I have tested it on a virtual machine with fedora-42.
Quoted 64 lines
Quoted 64 lines
The series consists of two patches:
1. "LoongArch: Add 128-bit atomic cmpxchg support"
- Implements 128-bit atomic compare-and-exchange using
LoongArch's LL.D/SC.Q instructions
- Fixes BPF scheduler test failures (scx_centralscx_qmap) wherekmalloc_nolock_noprofreturns NULL due to missing 128-bit
atomics, leading to-ENOMEMerrors during scheduler initialization2. "LoongArch: Enable 128-bit atomics cmpxchg support"
- Adds selectHAVE_CMPXCHG_DOUBLEand selectHAVE_ALIGNED_STRUCT_PAGEinKconfigto enable 128-bit atomic cmpxchg supportThe issue was identified through BPF scheduler test failures where
scx_centralandscx_qmapschedulers would fail to initialize. Testing was performed using thescx_qmapscheduler fromtools/sched_ext/, confirming that the patches resolve the initialization failures.Signed-off-by George Guo <[email protected]>---
Changes in v3:
- dbar 0 ->__WEAK_LLSC_MB
- =ZB" (__ptr[0]) -> "r" (__ptr)
- Link to v2:
https://lore.kernel.org/r/[email protected]Changes in v2:
- Use a normal ld.d for the high word instead of ll.d to avoid race condition
- Insert a dbar between ll.d and ld.d to prevent reordering
- Simply__cmpxchg128_asm("ll.d", "sc.q", ptr, o, n)to__cmpxchg128_asm(ptr, o, n)
- Fix address operand constraints after testing different approaches:
* ld.d with "m"
* ll.d with "ZC",
* sc.q with "ZB"(alternative constraints caused issues:
- "r" caused system hang
- "ZC" caused compiler error:
{standard input}: Assembler messages:
{standard input}:10037: Fatal error: Immediate overflow.
format: u0:0 )
- Link to v1:
https://lore.kernel.org/r/[email protected]---
George Guo (2):
LoongArch: Add 128-bit atomic cmpxchg support
LoongArch: Enable 128-bit atomics cmpxchg supportarch/loongarch/Kconfig | 2 ++ arch/loongarch/include/asm/cmpxchg.h | 47++++++++++++++++++++++++++++++++++++ 2 files changed, 49 insertions(+) ---
base-commit:d5ae5ac32615e4af729f0610fdc11ff4f4798aef
change-id: 20251120-2-d03862b2cf6dBest regards,
--
George Guo <[email protected]>
Quoted 5 lines · George Guo
This patch series adds 128-bit atomic compare-and-exchange support for LoongArch architecture, which fixes BPF scheduler test failures caused by missing 128-bit atomics support.
Quoted 9 lines
The series consists of two patches:
1. "LoongArch: Add 128-bit atomic cmpxchg support"
- Implements 128-bit atomic compare-and-exchange using LoongArch's LL.D/SC.Q instructions
- Fixes BPF scheduler test failures (scx_centralscx_qmap) wherekmalloc_nolock_noprofreturns NULL due to missing 128-bit atomics, leading to-ENOMEMerrors during scheduler initialization
This kmalloc_nolock_noprof() was introduced in v6.18-rc1 and has no caller for now.
Why is this related to the sched_ext failure ?
Quoted 49 lines
2. "LoongArch: Enable 128-bit atomics cmpxchg support"
- Adds selectHAVE_CMPXCHG_DOUBLEand selectHAVE_ALIGNED_STRUCT_PAGEinKconfigto enable 128-bit atomic cmpxchg supportThe issue was identified through BPF scheduler test failures where
scx_centralandscx_qmapschedulers would fail to initialize. Testing was performed using thescx_qmapscheduler fromtools/sched_ext/, confirming that the patches resolve the initialization failures.Signed-off-by George Guo <[email protected]>---
Changes in v3:
- dbar 0 ->__WEAK_LLSC_MB
- =ZB" (__ptr[0]) -> "r" (__ptr)
- Link to v2: https://lore.kernel.org/r/[email protected]Changes in v2:
- Use a normal ld.d for the high word instead of ll.d to avoid race condition
- Insert a dbar between ll.d and ld.d to prevent reordering
- Simply__cmpxchg128_asm("ll.d", "sc.q", ptr, o, n)to__cmpxchg128_asm(ptr, o, n)
- Fix address operand constraints after testing different approaches:
* ld.d with "m"
* ll.d with "ZC",
* sc.q with "ZB"(alternative constraints caused issues:
- "r" caused system hang
- "ZC" caused compiler error:
{standard input}: Assembler messages:
{standard input}:10037: Fatal error: Immediate overflow.
format: u0:0 )
- Link to v1: https://lore.kernel.org/r/[email protected]---
George Guo (2):
LoongArch: Add 128-bit atomic cmpxchg support
LoongArch: Enable 128-bit atomics cmpxchg supportarch/loongarch/Kconfig | 2 ++ arch/loongarch/include/asm/cmpxchg.h | 47 ++++++++++++++++++++++++++++++++++++ 2 files changed, 49 insertions(+)---
base-commit:d5ae5ac32615e4af729f0610fdc11ff4f4798aef
change-id: 20251120-2-d03862b2cf6dBest regards,
--
George Guo <[email protected]>
Quoted 17 lines · On Wed, 26 Nov 2025 13:23:57 +0800 Hengqi Chen <[email protected]> wrote:
Quoted 5 lines · George Guo
This patch series adds 128-bit atomic compare-and-exchange support for LoongArch architecture, which fixes BPF scheduler test failures caused by missing 128-bit atomics support.
Quoted 9 lines
The series consists of two patches:
1. "LoongArch: Add 128-bit atomic cmpxchg support"
- Implements 128-bit atomic compare-and-exchange using
LoongArch's LL.D/SC.Q instructions
- Fixes BPF scheduler test failures (scx_centralscx_qmap) wherekmalloc_nolock_noprofreturns NULL due to missing 128-bit
atomics, leading to-ENOMEMerrors during scheduler initialization
Quoted 4 lines
This
kmalloc_nolock_noprof()was introduced in v6.18-rc1 and has no caller for now.
Why is this related to thesched_extfailure ?
Hi Hengqi,
When running scx_central, function call chain as below:central_init->bpf_timer_init->__bpf_async_init->bpf_map_kmalloc_nolock->kmalloc_nolock ->kmalloc_nolock_noprof
The function kmalloc_nolock_noprof returns NULL due to the following condition:
if (!(s->flags & __CMPXCHG_DOUBLE) && !kmem_cache_debug(s))
/*
* kmalloc_nolock() is not supported on architectures that
* don't implement cmpxchg16b, but debug caches don't use
* per-cpu slab and per-cpu partial slabs. They rely on
* kmem_cache_node->list_lock, so kmalloc_nolock() can
* attempt to allocate from debug caches by
* spin_trylock_irqsave(&n->list_lock, ...)
*/
return NULL;
The NULL return occurs because kmalloc_nolock is not supported on Loongarch, which don't implement cmpxchg16b. So I am giving the patch.
Also I tried with debug caches(CONFIG_SLUB_DEBUG_ON=y), it works, but not a good idea.
Quoted 55 lines
Quoted 55 lines
2. "LoongArch: Enable 128-bit atomics cmpxchg support"
- Adds selectHAVE_CMPXCHG_DOUBLEand selectHAVE_ALIGNED_STRUCT_PAGEinKconfigto enable 128-bit atomic cmpxchg supportThe issue was identified through BPF scheduler test failures where
scx_centralandscx_qmapschedulers would fail to initialize. Testing was performed using thescx_qmapscheduler fromtools/sched_ext/, confirming that the patches resolve the initialization failures.Signed-off-by George Guo <[email protected]>---
Changes in v3:
- dbar 0 ->__WEAK_LLSC_MB
- =ZB" (__ptr[0]) -> "r" (__ptr)
- Link to v2:
https://lore.kernel.org/r/[email protected]Changes in v2:
- Use a normal ld.d for the high word instead of ll.d to avoid race condition
- Insert a dbar between ll.d and ld.d to prevent reordering
- Simply__cmpxchg128_asm("ll.d", "sc.q", ptr, o, n)to__cmpxchg128_asm(ptr, o, n)
- Fix address operand constraints after testing different approaches:
* ld.d with "m"
* ll.d with "ZC",
* sc.q with "ZB"(alternative constraints caused issues:
- "r" caused system hang
- "ZC" caused compiler error:
{standard input}: Assembler messages:
{standard input}:10037: Fatal error: Immediate overflow.
format: u0:0 )
- Link to v1:
https://lore.kernel.org/r/[email protected]---
George Guo (2):
LoongArch: Add 128-bit atomic cmpxchg support
LoongArch: Enable 128-bit atomics cmpxchg supportarch/loongarch/Kconfig | 2 ++ arch/loongarch/include/asm/cmpxchg.h | 47++++++++++++++++++++++++++++++++++++ 2 files changed, 49 insertions(+) ---
base-commit:d5ae5ac32615e4af729f0610fdc11ff4f4798aef
change-id: 20251120-2-d03862b2cf6dBest regards,
--
George Guo <[email protected]>
Quoted 21 lines · George Guo
Quoted 17 lines · On Wed, 26 Nov 2025 13:23:57 +0800 Hengqi Chen <[email protected]> wrote:
Quoted 14 lines · George Guo
This patch series adds 128-bit atomic compare-and-exchange support for LoongArch architecture, which fixes BPF scheduler test failures caused by missing 128-bit atomics support.
The series consists of two patches:
1. "LoongArch: Add 128-bit atomic cmpxchg support"
- Implements 128-bit atomic compare-and-exchange using
LoongArch's LL.D/SC.Q instructions
- Fixes BPF scheduler test failures (scx_centralscx_qmap) wherekmalloc_nolock_noprofreturns NULL due to missing 128-bit
atomics, leading to-ENOMEMerrors during scheduler initialization
Quoted 2 lines
Quoted 2 lines
This
kmalloc_nolock_noprof()was introduced in v6.18-rc1 and has no caller for now.
OK, it does have a caller in rc-2 [1].
[1]: https://lore.kernel.org/bpf/[email protected]/
Quoted 8 lines
Quoted 2 lines
Why is this related to the
sched_extfailure ?Hi Hengqi,
When running
scx_central, function call chain as below:central_init->bpf_timer_init->__bpf_async_init->bpf_map_kmalloc_nolock->kmalloc_nolock->kmalloc_nolock_noprof
Thanks, will test this series.
Quoted 76 lines
The function
kmalloc_nolock_noprofreturns NULL due to the following condition:if (!(s->flags &
__CMPXCHG_DOUBLE) && !kmem_cache_debug(s))
/*
*kmalloc_nolock()is not supported on architectures that
* don't implement cmpxchg16b, but debug caches don't use
* per-cpu slab and per-cpu partial slabs. They rely on
*kmem_cache_node->list_lock, sokmalloc_nolock()can
* attempt to allocate from debug caches by
*spin_trylock_irqsave(&n->list_lock, ...)
*/
return NULL;The NULL return occurs because
kmalloc_nolockis not supported on Loongarch, which don't implement cmpxchg16b. So I am giving the patch.Also I tried with debug
caches(CONFIG_SLUB_DEBUG_ON=y), it works, but not a good idea.Quoted 55 lines
Quoted 55 lines
2. "LoongArch: Enable 128-bit atomics cmpxchg support"
- Adds selectHAVE_CMPXCHG_DOUBLEand selectHAVE_ALIGNED_STRUCT_PAGEinKconfigto enable 128-bit atomic cmpxchg supportThe issue was identified through BPF scheduler test failures where
scx_centralandscx_qmapschedulers would fail to initialize. Testing was performed using thescx_qmapscheduler fromtools/sched_ext/, confirming that the patches resolve the initialization failures.Signed-off-by George Guo <[email protected]>---
Changes in v3:
- dbar 0 ->__WEAK_LLSC_MB
- =ZB" (__ptr[0]) -> "r" (__ptr)
- Link to v2:
https://lore.kernel.org/r/[email protected]Changes in v2:
- Use a normal ld.d for the high word instead of ll.d to avoid race condition
- Insert a dbar between ll.d and ld.d to prevent reordering
- Simply__cmpxchg128_asm("ll.d", "sc.q", ptr, o, n)to__cmpxchg128_asm(ptr, o, n)
- Fix address operand constraints after testing different approaches:
* ld.d with "m"
* ll.d with "ZC",
* sc.q with "ZB"(alternative constraints caused issues:
- "r" caused system hang
- "ZC" caused compiler error:
{standard input}: Assembler messages:
{standard input}:10037: Fatal error: Immediate overflow.
format: u0:0 )
- Link to v1:
https://lore.kernel.org/r/[email protected]---
George Guo (2):
LoongArch: Add 128-bit atomic cmpxchg support
LoongArch: Enable 128-bit atomics cmpxchg supportarch/loongarch/Kconfig | 2 ++ arch/loongarch/include/asm/cmpxchg.h | 47++++++++++++++++++++++++++++++++++++ 2 files changed, 49 insertions(+) ---
base-commit:d5ae5ac32615e4af729f0610fdc11ff4f4798aef
change-id: 20251120-2-d03862b2cf6dBest regards,
--
George Guo <[email protected]>
Quoted 32 lines · Hengqi Chen
Quoted 21 lines · George Guo
Quoted 17 lines · On Wed, 26 Nov 2025 13:23:57 +0800 Hengqi Chen <[email protected]> wrote:
Quoted 14 lines · George Guo
This patch series adds 128-bit atomic compare-and-exchange support for LoongArch architecture, which fixes BPF scheduler test failures caused by missing 128-bit atomics support.
The series consists of two patches:
1. "LoongArch: Add 128-bit atomic cmpxchg support"
- Implements 128-bit atomic compare-and-exchange using
LoongArch's LL.D/SC.Q instructions
- Fixes BPF scheduler test failures (scx_centralscx_qmap) wherekmalloc_nolock_noprofreturns NULL due to missing 128-bit
atomics, leading to-ENOMEMerrors during scheduler initializationQuoted 2 lines
Quoted 2 lines
This
kmalloc_nolock_noprof()was introduced in v6.18-rc1 and has no caller for now.OK, it does have a caller in rc-2 [1].
[1]: https://lore.kernel.org/bpf/[email protected]/
Quoted 2 lines
Quoted 2 lines
Why is this related to the
sched_extfailure ?
Quoted 8 lines
Quoted 6 lines
Hi Hengqi,
When running
scx_central, function call chain as below:central_init->bpf_timer_init->__bpf_async_init->bpf_map_kmalloc_nolock->kmalloc_nolock->kmalloc_nolock_noprofThanks, will test this series.
I tried with qemu, but it seems the kernel can't even boot.
Quoted 77 lines
Quoted 77 lines
The function
kmalloc_nolock_noprofreturns NULL due to the following condition:if (!(s->flags &
__CMPXCHG_DOUBLE) && !kmem_cache_debug(s))
/*
*kmalloc_nolock()is not supported on architectures that
* don't implement cmpxchg16b, but debug caches don't use
* per-cpu slab and per-cpu partial slabs. They rely on
*kmem_cache_node->list_lock, sokmalloc_nolock()can
* attempt to allocate from debug caches by
*spin_trylock_irqsave(&n->list_lock, ...)
*/
return NULL;The NULL return occurs because
kmalloc_nolockis not supported on Loongarch, which don't implement cmpxchg16b. So I am giving the patch.Also I tried with debug
caches(CONFIG_SLUB_DEBUG_ON=y), it works, but not a good idea.Quoted 55 lines
Quoted 55 lines
2. "LoongArch: Enable 128-bit atomics cmpxchg support"
- Adds selectHAVE_CMPXCHG_DOUBLEand selectHAVE_ALIGNED_STRUCT_PAGEinKconfigto enable 128-bit atomic cmpxchg supportThe issue was identified through BPF scheduler test failures where
scx_centralandscx_qmapschedulers would fail to initialize. Testing was performed using thescx_qmapscheduler fromtools/sched_ext/, confirming that the patches resolve the initialization failures.Signed-off-by George Guo <[email protected]>---
Changes in v3:
- dbar 0 ->__WEAK_LLSC_MB
- =ZB" (__ptr[0]) -> "r" (__ptr)
- Link to v2:
https://lore.kernel.org/r/[email protected]Changes in v2:
- Use a normal ld.d for the high word instead of ll.d to avoid race condition
- Insert a dbar between ll.d and ld.d to prevent reordering
- Simply__cmpxchg128_asm("ll.d", "sc.q", ptr, o, n)to__cmpxchg128_asm(ptr, o, n)
- Fix address operand constraints after testing different approaches:
* ld.d with "m"
* ll.d with "ZC",
* sc.q with "ZB"(alternative constraints caused issues:
- "r" caused system hang
- "ZC" caused compiler error:
{standard input}: Assembler messages:
{standard input}:10037: Fatal error: Immediate overflow.
format: u0:0 )
- Link to v1:
https://lore.kernel.org/r/[email protected]---
George Guo (2):
LoongArch: Add 128-bit atomic cmpxchg support
LoongArch: Enable 128-bit atomics cmpxchg supportarch/loongarch/Kconfig | 2 ++ arch/loongarch/include/asm/cmpxchg.h | 47++++++++++++++++++++++++++++++++++++ 2 files changed, 49 insertions(+) ---
base-commit:d5ae5ac32615e4af729f0610fdc11ff4f4798aef
change-id: 20251120-2-d03862b2cf6dBest regards,
--
George Guo <[email protected]>