init: Align init_task to avoid conflict with MUTEX_FLAGS
PATCH 1/1 init: Align init_task to avoid conflict with MUTEX_FLAGS
When booting on 32-bit machines (seen on OpenRISC) I saw this warning with CONFIG_DEBUG_MUTEXES turned on.
------------[ cut here ]------------
WARNING: CPU: 0 PID: 0 at kernel/locking/mutex.c:1242 __mutex_unlock_slowpath+0x328/0x3ec
DEBUG_LOCKS_WARN_ON(__owner_task(owner) != current)
Modules linked in:
CPU: 0 PID: 0 Comm: swapper Not tainted 5.8.0-rc1-simple-smp-00005-g2864e2171db4-dirty #179
Call trace:
[<(ptrval)>] dump_stack+0x34/0x48
[<(ptrval)>] __warn+0x104/0x158
[<(ptrval)>] ? __mutex_unlock_slowpath+0x328/0x3ec
[<(ptrval)>] warn_slowpath_fmt+0x7c/0x94
[<(ptrval)>] __mutex_unlock_slowpath+0x328/0x3ec
[<(ptrval)>] mutex_unlock+0x18/0x28
[<(ptrval)>] __cpuhp_setup_state_cpuslocked.part.0+0x29c/0x2f4
[<(ptrval)>] ? page_alloc_cpu_dead+0x0/0x30
[<(ptrval)>] ? start_kernel+0x0/0x684
[<(ptrval)>] __cpuhp_setup_state+0x4c/0x5c
[<(ptrval)>] page_alloc_init+0x34/0x68
[<(ptrval)>] ? start_kernel+0x1a0/0x684
[<(ptrval)>] ? early_init_dt_scan_nodes+0x60/0x70
irq event stamp: 0I traced this to kernel/locking/mutex.c storing 3 bits of MUTEX_FLAGS in the task_struct pointer (mutex.owner). There is a comment saying that task_structs are always aligned to L1_CACHE_BYTES. This is not true for the init_task.
On 64-bit machines this is not a problem because symbol addresses are naturally aligned to 64-bits providing 3 bits for MUTEX_FLAGS. Howerver, for 32-bit machines the symbol address only has 2 bits available.
Fix this by setting init_task alignment to at least L1_CACHE_BYTES.
--- init/init_task.c | 2 ++ 1 file changed, 2 insertions(+)
init/init_task.c +2 -0
@@ -64,6 +64,8 @@ unsigned long init_shadow_call_stack[SCS_SIZE / sizeof(long)] struct task_struct init_task#ifdef CONFIG_ARCH_TASK_STRUCT_ON_STACK __init_task_data +#else +__aligned(L1_CACHE_BYTES) #endif = {#ifdef CONFIG_THREAD_INFO_IN_TASK
Signature · 1 line
2.26.2
Quoted 8 lines · Stafford Horne
When booting on 32-bit machines (seen on OpenRISC) I saw this warning with
CONFIG_DEBUG_MUTEXESturned on.I traced this to
kernel/locking/mutex.cstoring 3 bits ofMUTEX_FLAGSin thetask_structpointer (mutex.owner). There is a comment saying thattask_structsare always aligned toL1_CACHE_BYTES. This is not true for theinit_task.
Quoted 5 lines
On 64-bit machines this is not a problem because symbol addresses are naturally aligned to 64-bits providing 3 bits for
MUTEX_FLAGS. Howerver, for 32-bit machines the symbol address only has 2 bits available.Fix this by setting
init_taskalignment to at leastL1_CACHE_BYTES.
Whoopsie, sorry about that.
Quoted 11 lines
Cc Peter Zijlstra <[email protected]>Signed-off-by Stafford Horne <[email protected]>--- init/init_task.c | 2 ++ 1 file changed, 2 insertions(+)init/init_task.c +0
-0
init/init_task.c unsigned long init_shadow_call_stack[SCS_SIZE / sizeof(long)] · line 69
Quoted 6 lines
init/init_task.c +2
-0@@ -64,6 +64,8 @@ unsigned long init_shadow_call_stack[SCS_SIZE / sizeof(long)] struct task_struct init_task#ifdef CONFIG_ARCH_TASK_STRUCT_ON_STACK __init_task_data +#else +__aligned(L1_CACHE_BYTES) #endif
Why make this conditional? task_struct_cachep (in kernel/fork.c) has max_t(int, L1_CACHE_BYTES, ARCH_MIN_TASKALIGN) alignment, so this really should be aligned on L1_CACHE_BYTES at least.
Quoted 23 lines · Peter Zijlstra
Quoted 8 lines · Stafford Horne
When booting on 32-bit machines (seen on OpenRISC) I saw this warning with
CONFIG_DEBUG_MUTEXESturned on.I traced this to
kernel/locking/mutex.cstoring 3 bits ofMUTEX_FLAGSin thetask_structpointer (mutex.owner). There is a comment saying thattask_structsare always aligned toL1_CACHE_BYTES. This is not true for theinit_task.Quoted 5 lines
On 64-bit machines this is not a problem because symbol addresses are naturally aligned to 64-bits providing 3 bits for
MUTEX_FLAGS. Howerver, for 32-bit machines the symbol address only has 2 bits available.Fix this by setting
init_taskalignment to at leastL1_CACHE_BYTES.Whoopsie, sorry about that.
Quoted 6 lines
Cc Peter Zijlstra <[email protected]>Signed-off-by Stafford Horne <[email protected]>--- init/init_task.c | 2 ++ 1 file changed, 2 insertions(+)
Quoted 15 lines
init/init_task.c unsigned long init_shadow_call_stack[SCS_SIZE / sizeof(long)] · line 69
Quoted 11 lines
init/init_task.c +2
-0@@ -64,6 +64,8 @@ unsigned long init_shadow_call_stack[SCS_SIZE / sizeof(long)] struct task_struct init_task#ifdef CONFIG_ARCH_TASK_STRUCT_ON_STACK __init_task_data +#else +__aligned(L1_CACHE_BYTES) #endif Why make this conditional?
task_struct_cachep(inkernel/fork.c) hasmax_t(int, L1_CACHE_BYTES, ARCH_MIN_TASKALIGN)alignment, so this really should be aligned onL1_CACHE_BYTESat least.
I think we can make this unconditional. The only reason I used the condition is because the only architecture that sets CONFIG_ARCH_TASK_STRUCT_ON_STACK is ia64 which already has 64-bit alignment.
I can change it to (not incorporating max_t(ARCH_MIN_TASKALIGN) as not sure how):
init/init_task.c +1 -0
@@ -65,6 +65,7 @@ struct task_struct init_task#ifdef CONFIG_ARCH_TASK_STRUCT_ON_STACK __init_task_data #endif +__aligned(L1_CACHE_BYTES) = { #ifdef CONFIG_THREAD_INFO_IN_TASK .thread_info = INIT_THREAD_INFO(init_task),
-Stafford
Peter Zijlstra
Acked-byQuoted 5 lines · Stafford Horne
I think we can make this unconditional. The only reason I used the condition is because the only architecture that sets
CONFIG_ARCH_TASK_STRUCT_ON_STACKis ia64 which already has 64-bit alignment.I can change it to (not incorporating
max_t(ARCH_MIN_TASKALIGN)as not sure how):
The only arch that has ARCH_MIN_TASKALIGN larger than L1_CACHE_SIZE is some daft x86 config and it shouldn't be fatal to them.
So with this:
Quoted 14 lines
init/init_task.c +1
-1@@ -65,6 +65,7 @@ struct task_struct init_task#ifdef CONFIG_ARCH_TASK_STRUCT_ON_STACK __init_task_data #endif +__aligned(L1_CACHE_BYTES) = { #ifdef CONFIG_THREAD_INFO_IN_TASK .thread_info = INIT_THREAD_INFO(init_task), -Stafford