fork: Honor task_struct's declared alignment
Subsystems (4)
PATCH v2 1/1 fork: Honor task_struct's declared alignment
Since commit cb7ca40a3882 ("x86/fpu: Make task_struct::thread constant size"), struct task_struct is declared __attribute__((aligned(64))) on all architectures.
But fork_init() sets the task_struct slab cache's alignment to
align = max(L1_CACHE_BYTES, ARCH_MIN_TASKALIGN)
which is smaller than 64 on architectures whose cache lines are below 64 bytes: e.g. 32 on ARMv5.
In practice plain SLUB happens to hand out 64-byte-aligned objects anyway. With CONFIG_SLUB_DEBUG_ON the red-zone padding shifts objects to the requested alignment.
With CONFIG_UBSAN_ALIGNMENT=y a boot on QEMU versatilepb (ARM926EJ-S, v7.2-rc2, gcc 13.3) floods the console with reports like:
UBSAN: misaligned-access in include/linux/sched.h:2087:9 member access within misaligned address c295d7e0 for type 'struct task_struct' which requires 64 byte alignment CPU: 0 UID: 0 PID: 15 Comm: pr/ttyAMA-1 Not tainted 7.2.0-rc2 #1 VOLUNTARY
Set the slab alignment to at least the type's declared alignment. Replace the hardcoded L1_CACHE_BYTES with SLAB_HWCACHE_ALIGN so the allocator applies cache_line_size(). On x86, arm and arm64 that is the cache line size of the booted CPU. Everywhere else it falls back to L1_CACHE_BYTES and nothing changes. ARCH_MIN_TASKALIGN (e.g. 4096 with x86 VSMP) still applies through the explicit align argument.
---
Notes (format-patch):
v2:
- fold in Vlastimil's suggestion: let SLAB_HWCACHE_ALIGN replace the
hardcoded L1_CACHE_BYTES
v1: https://lore.kernel.org/all/[email protected]/kernel/fork.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
kernel/fork.c +3 -2
@@ -857,14 +857,15 @@ void __init fork_init(void)#ifndef ARCH_MIN_TASKALIGN #define ARCH_MIN_TASKALIGN 0#endif -int align =max_t (int,L1_CACHE_BYTES ,ARCH_MIN_TASKALIGN ); +int align =max (ARCH_MIN_TASKALIGN , + __alignof__(struct task_struct ));unsigned long useroffset, usersize;/* create a slab on which task_structs can be allocated */ task_struct_whitelist (&useroffset, &usersize); task_struct_cachep =kmem_cache_create_usercopy ("task_struct" , arch_task_struct_size, align, -SLAB_PANIC |SLAB_ACCOUNT , +SLAB_PANIC |SLAB_ACCOUNT |SLAB_HWCACHE_ALIGN , useroffset, usersize,NULL );/* do the arch specific task caches init */
base-commit: af5e34a41cd607c00ef752e00331736570992354
Signature · 1 line
2.39.5 (Apple Git-154)
Quoted 30 lines · Karl Mehltretter
Since commit
cb7ca40a3882("x86/fpu: Maketask_struct::threadconstant size"), structtask_structis declared __attribute__((aligned(64))) on all architectures.But
fork_init()sets thetask_structslab cache's alignment toalign =
max(L1_CACHE_BYTES, ARCH_MIN_TASKALIGN)which is smaller than 64 on architectures whose cache lines are below 64 bytes: e.g. 32 on ARMv5.
In practice plain SLUB happens to hand out 64-byte-aligned objects anyway. With
CONFIG_SLUB_DEBUG_ONthe red-zone padding shifts objects to the requested alignment.With
CONFIG_UBSAN_ALIGNMENT=ya boot on QEMU versatilepb (ARM926EJ-S, v7.2-rc2, gcc 13.3) floods the console with reports like:UBSAN: misaligned-access in include/linux/sched.h:2087:9 member access within misaligned address c295d7e0 for type 'struct task_struct' which requires 64 byte alignment CPU: 0 UID: 0 PID: 15 Comm: pr/ttyAMA-1 Not tainted 7.2.0-rc2 #1 VOLUNTARYSet the slab alignment to at least the type's declared alignment. Replace the hardcoded
L1_CACHE_BYTESwithSLAB_HWCACHE_ALIGNso the allocator appliescache_line_size(). On x86, arm and arm64 that is the cache line size of the booted CPU. Everywhere else it falls back toL1_CACHE_BYTESand nothing changes.ARCH_MIN_TASKALIGN(e.g. 4096 with x86 VSMP) still applies through the explicit align argument.
Quoted 2 lines
Fixes cb7ca40a3882 ("x86/fpu: Make task_struct::thread constant size")Suggested-by Vlastimil Babka (SUSE) <[email protected]>
Quoted 36 lines
Assisted-by Claude:claude-fable-5Signed-off-by Karl Mehltretter <[email protected]>---
Notes (format-patch):
v2: - fold in Vlastimil's suggestion: let SLAB_HWCACHE_ALIGN replace the hardcoded L1_CACHE_BYTES v1: https://lore.kernel.org/all/[email protected]/
kernel/fork.c| 5 +++--1 file changed, 3 insertions(+), 2 deletions(-)kernel/fork.c +3
-2@@ -857,14 +857,15 @@ void __init fork_init(void)#ifndef ARCH_MIN_TASKALIGN #define ARCH_MIN_TASKALIGN 0#endif -int align =max_t (int,L1_CACHE_BYTES ,ARCH_MIN_TASKALIGN ); +int align =max (ARCH_MIN_TASKALIGN , + __alignof__(struct task_struct ));unsigned long useroffset, usersize;/* create a slab on which task_structs can be allocated */ task_struct_whitelist (&useroffset, &usersize); task_struct_cachep =kmem_cache_create_usercopy ("task_struct" , arch_task_struct_size, align, -SLAB_PANIC |SLAB_ACCOUNT , +SLAB_PANIC |SLAB_ACCOUNT |SLAB_HWCACHE_ALIGN , useroffset, usersize,NULL );/* do the arch specific task caches init */
Quoted 2 lines
base-commit:
af5e34a41cd607c00ef752e00331736570992354
Thanks!
Quoted 30 lines · Karl Mehltretter
Since commit
cb7ca40a3882("x86/fpu: Maketask_struct::threadconstant size"), structtask_structis declared __attribute__((aligned(64))) on all architectures.But
fork_init()sets thetask_structslab cache's alignment toalign =
max(L1_CACHE_BYTES, ARCH_MIN_TASKALIGN)which is smaller than 64 on architectures whose cache lines are below 64 bytes: e.g. 32 on ARMv5.
In practice plain SLUB happens to hand out 64-byte-aligned objects anyway. With
CONFIG_SLUB_DEBUG_ONthe red-zone padding shifts objects to the requested alignment.With
CONFIG_UBSAN_ALIGNMENT=ya boot on QEMU versatilepb (ARM926EJ-S, v7.2-rc2, gcc 13.3) floods the console with reports like:UBSAN: misaligned-access in include/linux/sched.h:2087:9 member access within misaligned address c295d7e0 for type 'struct task_struct' which requires 64 byte alignment CPU: 0 UID: 0 PID: 15 Comm: pr/ttyAMA-1 Not tainted 7.2.0-rc2 #1 VOLUNTARYSet the slab alignment to at least the type's declared alignment. Replace the hardcoded
L1_CACHE_BYTESwithSLAB_HWCACHE_ALIGNso the allocator appliescache_line_size(). On x86, arm and arm64 that is the cache line size of the booted CPU. Everywhere else it falls back toL1_CACHE_BYTESand nothing changes.ARCH_MIN_TASKALIGN(e.g. 4096 with x86 VSMP) still applies through the explicit align argument.
Quoted 2 lines
Fixes cb7ca40a3882 ("x86/fpu: Make task_struct::thread constant size")Suggested-by Vlastimil Babka (SUSE) <[email protected]>
Thanks but I suggested only SLAB_HWCACHE_ALIGN, not the whole thing :) Can be removed when a maintainer applies this.
Quoted 2 lines
Assisted-by Claude:claude-fable-5Signed-off-by Karl Mehltretter <[email protected]>
Quoted 35 lines
---
Notes (format-patch):
v2: - fold in Vlastimil's suggestion: let SLAB_HWCACHE_ALIGN replace the hardcoded L1_CACHE_BYTES v1: https://lore.kernel.org/all/[email protected]/kernel/fork.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)kernel/fork.c +3
-2@@ -857,14 +857,15 @@ void __init fork_init(void)#ifndef ARCH_MIN_TASKALIGN #define ARCH_MIN_TASKALIGN 0#endif -int align =max_t (int,L1_CACHE_BYTES ,ARCH_MIN_TASKALIGN ); +int align =max (ARCH_MIN_TASKALIGN , + __alignof__(struct task_struct ));unsigned long useroffset, usersize;/* create a slab on which task_structs can be allocated */ task_struct_whitelist (&useroffset, &usersize); task_struct_cachep =kmem_cache_create_usercopy ("task_struct" , arch_task_struct_size, align, -SLAB_PANIC |SLAB_ACCOUNT , +SLAB_PANIC |SLAB_ACCOUNT |SLAB_HWCACHE_ALIGN , useroffset, usersize,NULL );/* do the arch specific task caches init */ base-commit:
af5e34a41cd607c00ef752e00331736570992354
Quoted 6 lines · Vlastimil Babka (SUSE)
Quoted 2 lines
Fixes cb7ca40a3882 ("x86/fpu: Make task_struct::thread constant size")Suggested-by Vlastimil Babka (SUSE) <[email protected]>Thanks but I suggested only
SLAB_HWCACHE_ALIGN, not the whole thing :) Can be removed when a maintainer applies this.
Ok, I believe Suggested-by doesn't necessarily mean the whole thing, but I agree, here it maybe was too much. Fine to drop it.
Thanks,
Karl