Skip to contentSkip to conversation

fork: Honor task_struct's declared alignment

PATCH In review Karl Mehltretter 3 participants 4 messages Aug 4

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 fork: Honor task_struct's declared alignment

Since commit cb7ca40a3882 ("x86/fpu: Make task_struct::thread constant size"), struct task_struct is declared __attribute__((aligned(64))) on all architectures.

But fork_init() sets the task_struct slab cache's alignment to

align = max(L1_CACHE_BYTES, ARCH_MIN_TASKALIGN)

which is smaller than 64 on architectures whose cache lines are below 64 bytes: e.g. 32 on ARMv5.

In practice plain SLUB happens to hand out 64-byte-aligned objects anyway. With CONFIG_SLUB_DEBUG_ON the red-zone padding shifts objects to the requested alignment.

With CONFIG_UBSAN_ALIGNMENT=y a boot on QEMU versatilepb (ARM926EJ-S, v7.2-rc2, gcc 13.3) floods the console with reports like:

  UBSAN: misaligned-access in include/linux/sched.h:2087:9
  member access within misaligned address c295d7e0 for type 'struct task_struct'
  which requires 64 byte alignment
  CPU: 0 UID: 0 PID: 15 Comm: pr/ttyAMA-1 Not tainted 7.2.0-rc2 #1 VOLUNTARY

Set the slab alignment to at least the type's declared alignment. Replace the hardcoded L1_CACHE_BYTES with SLAB_HWCACHE_ALIGN so the allocator applies cache_line_size(). On x86, arm and arm64 that is the cache line size of the booted CPU. Everywhere else it falls back to L1_CACHE_BYTES and nothing changes. ARCH_MIN_TASKALIGN (e.g. 4096 with x86 VSMP) still applies through the explicit align argument.

Fixes cb7ca40a3882 ("x86/fpu: Make task_struct::thread constant size")
Assisted-by Claude:claude-fable-5

---

Notes (format-patch):

    v2:
     - fold in Vlastimil's suggestion: let SLAB_HWCACHE_ALIGN replace the
       hardcoded L1_CACHE_BYTES
    v1: https://lore.kernel.org/all/[email protected]/
 kernel/fork.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
kernel/fork.c +3 -2
index f0e2e131a9a5..3a0093efb29f 100644
@@ -857,14 +857,15 @@ void __init fork_init(void)
 #ifndef ARCH_MIN_TASKALIGN
 #define ARCH_MIN_TASKALIGN	0
 #endif
-	int align = max_t(int, L1_CACHE_BYTES, ARCH_MIN_TASKALIGN);
+	int align = max(ARCH_MIN_TASKALIGN,
+			__alignof__(struct task_struct));
 	unsigned long useroffset, usersize;
 
 	/* create a slab on which task_structs can be allocated */
 	task_struct_whitelist(&useroffset, &usersize);
 	task_struct_cachep = kmem_cache_create_usercopy("task_struct",
 			arch_task_struct_size, align,
-			SLAB_PANIC|SLAB_ACCOUNT,
+			SLAB_PANIC|SLAB_ACCOUNT|SLAB_HWCACHE_ALIGN,
 			useroffset, usersize, NULL);
 
 	/* do the arch specific task caches init */

base-commit: af5e34a41cd607c00ef752e00331736570992354

Signature · 1 line
2.39.5 (Apple Git-154)
Reviewed-by on v2 1/1
#1 lore raw
Quoted 30 lines · Karl Mehltretter

Since commit cb7ca40a3882 ("x86/fpu: Make task_struct::thread constant size"), struct task_struct is declared __attribute__((aligned(64))) on all architectures.

But fork_init() sets the task_struct slab cache's alignment to

align = max(L1_CACHE_BYTES, ARCH_MIN_TASKALIGN)

which is smaller than 64 on architectures whose cache lines are below 64 bytes: e.g. 32 on ARMv5.

In practice plain SLUB happens to hand out 64-byte-aligned objects anyway. With CONFIG_SLUB_DEBUG_ON the red-zone padding shifts objects to the requested alignment.

With CONFIG_UBSAN_ALIGNMENT=y a boot on QEMU versatilepb (ARM926EJ-S, v7.2-rc2, gcc 13.3) floods the console with reports like:

 UBSAN: misaligned-access in include/linux/sched.h:2087:9
 member access within misaligned address c295d7e0 for type 'struct task_struct'
 which requires 64 byte alignment
 CPU: 0 UID: 0 PID: 15 Comm: pr/ttyAMA-1 Not tainted 7.2.0-rc2 #1 VOLUNTARY

Set the slab alignment to at least the type's declared alignment. Replace the hardcoded L1_CACHE_BYTES with SLAB_HWCACHE_ALIGN so the allocator applies cache_line_size(). On x86, arm and arm64 that is the cache line size of the booted CPU. Everywhere else it falls back to L1_CACHE_BYTES and nothing changes. ARCH_MIN_TASKALIGN (e.g. 4096 with x86 VSMP) still applies through the explicit align argument.

Quoted 2 lines
Fixes cb7ca40a3882 ("x86/fpu: Make task_struct::thread constant size")
Quoted 36 lines
Assisted-by Claude:claude-fable-5

---

Notes (format-patch):

   v2:
    - fold in Vlastimil's suggestion: let SLAB_HWCACHE_ALIGN replace the
      hardcoded L1_CACHE_BYTES
   v1: https://lore.kernel.org/all/[email protected]/

kernel/fork.c | 5 +++--

1 file changed, 3 insertions(+), 2 deletions(-)
kernel/fork.c +3 -2
index f0e2e131a9a5..3a0093efb29f 100644
@@ -857,14 +857,15 @@ void __init fork_init(void)
 #ifndef ARCH_MIN_TASKALIGN
 #define ARCH_MIN_TASKALIGN	0
 #endif
-	int align = max_t(int, L1_CACHE_BYTES, ARCH_MIN_TASKALIGN);
+	int align = max(ARCH_MIN_TASKALIGN,
+			__alignof__(struct task_struct));
 	unsigned long useroffset, usersize;
 
 	/* create a slab on which task_structs can be allocated */
 	task_struct_whitelist(&useroffset, &usersize);
 	task_struct_cachep = kmem_cache_create_usercopy("task_struct",
 			arch_task_struct_size, align,
-			SLAB_PANIC|SLAB_ACCOUNT,
+			SLAB_PANIC|SLAB_ACCOUNT|SLAB_HWCACHE_ALIGN,
 			useroffset, usersize, NULL);
 
 	/* do the arch specific task caches init */
Quoted 2 lines

base-commit: af5e34a41cd607c00ef752e00331736570992354

Thanks!

reviewer Reviewed-by on v2 1/1
#2 lore raw
In reply to Karl Mehltretter
Quoted 30 lines · Karl Mehltretter

Since commit cb7ca40a3882 ("x86/fpu: Make task_struct::thread constant size"), struct task_struct is declared __attribute__((aligned(64))) on all architectures.

But fork_init() sets the task_struct slab cache's alignment to

align = max(L1_CACHE_BYTES, ARCH_MIN_TASKALIGN)

which is smaller than 64 on architectures whose cache lines are below 64 bytes: e.g. 32 on ARMv5.

In practice plain SLUB happens to hand out 64-byte-aligned objects anyway. With CONFIG_SLUB_DEBUG_ON the red-zone padding shifts objects to the requested alignment.

With CONFIG_UBSAN_ALIGNMENT=y a boot on QEMU versatilepb (ARM926EJ-S, v7.2-rc2, gcc 13.3) floods the console with reports like:

  UBSAN: misaligned-access in include/linux/sched.h:2087:9
  member access within misaligned address c295d7e0 for type 'struct task_struct'
  which requires 64 byte alignment
  CPU: 0 UID: 0 PID: 15 Comm: pr/ttyAMA-1 Not tainted 7.2.0-rc2 #1 VOLUNTARY

Set the slab alignment to at least the type's declared alignment. Replace the hardcoded L1_CACHE_BYTES with SLAB_HWCACHE_ALIGN so the allocator applies cache_line_size(). On x86, arm and arm64 that is the cache line size of the booted CPU. Everywhere else it falls back to L1_CACHE_BYTES and nothing changes. ARCH_MIN_TASKALIGN (e.g. 4096 with x86 VSMP) still applies through the explicit align argument.

Quoted 2 lines
Fixes cb7ca40a3882 ("x86/fpu: Make task_struct::thread constant size")

Thanks but I suggested only SLAB_HWCACHE_ALIGN, not the whole thing :) Can be removed when a maintainer applies this.

Quoted 2 lines
Assisted-by Claude:claude-fable-5
Quoted 35 lines

---

Notes (format-patch):

    v2:
     - fold in Vlastimil's suggestion: let SLAB_HWCACHE_ALIGN replace the
       hardcoded L1_CACHE_BYTES
    v1: https://lore.kernel.org/all/[email protected]/
 kernel/fork.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
kernel/fork.c +3 -2
index f0e2e131a9a5..3a0093efb29f 100644
@@ -857,14 +857,15 @@ void __init fork_init(void)
 #ifndef ARCH_MIN_TASKALIGN
 #define ARCH_MIN_TASKALIGN	0
 #endif
-	int align = max_t(int, L1_CACHE_BYTES, ARCH_MIN_TASKALIGN);
+	int align = max(ARCH_MIN_TASKALIGN,
+			__alignof__(struct task_struct));
 	unsigned long useroffset, usersize;
 
 	/* create a slab on which task_structs can be allocated */
 	task_struct_whitelist(&useroffset, &usersize);
 	task_struct_cachep = kmem_cache_create_usercopy("task_struct",
 			arch_task_struct_size, align,
-			SLAB_PANIC|SLAB_ACCOUNT,
+			SLAB_PANIC|SLAB_ACCOUNT|SLAB_HWCACHE_ALIGN,
 			useroffset, usersize, NULL);
 
 	/* do the arch specific task caches init */

base-commit: af5e34a41cd607c00ef752e00331736570992354

#3 lore raw
Quoted 6 lines · Vlastimil Babka (SUSE)
Quoted 2 lines
Fixes cb7ca40a3882 ("x86/fpu: Make task_struct::thread constant size")

Thanks but I suggested only SLAB_HWCACHE_ALIGN, not the whole thing :) Can be removed when a maintainer applies this.

Ok, I believe Suggested-by doesn't necessarily mean the whole thing, but I agree, here it maybe was too much. Fine to drop it.

Thanks,
Karl