powerpc: Fix kuap warnings
Shrikanth Hegde
reviewercover powerpc: Fix kuap warnings
Recently stumbled upon these kuap warnings. This happens with preempt=full/lazy kernel with function tracing enabled. What irked me was kernel compilation was getting failed when i had tracing enabled. It doesn't fail everytime. While running stress-ng memory class it threw same warnings. So that helped to narrow it down.
So one possible way is to disable tracing for these enter/exit vmx_usercopy. That seems to fix the bug/warnings. But that will make them as non trace-able. If there is a better way to fix these warning while keeping them as trace-able, please let me know.
Anyone with insights on amr, vmx and tracing, please advise.
Shrikanth Hegde (1):
powerpc: Fix kuap warnings on lazy/full preemption with tracing
arch/powerpc/kernel/process.c | 10 +++++----- arch/powerpc/lib/vmx-helper.c | 4 ++-- 2 files changed, 7 insertions(+), 7 deletions(-)
Signature · 1 line
2.47.3
Shrikanth Hegde
reviewerPATCH 1/1 powerpc: Fix kuap warnings on lazy/full preemption with tracing
These KUAP bugs/Warnings were seen often when tracing was enabled. It happens with preempt=full/lazy. It is easily hit.
How to trigger:
echo lazy > /sys/kernel/debug/sched/preempt echo function > /sys/kernel/debug/tracing/current_tracer stress-ng --class memory --all 1 -t 3
Bug: Write fault blocked by KUAP!
WARNING: [] arch/powerpc/mm/fault.c:231 at bad_kernel_fault.constprop.0+0x1a8/0x2c8, CPU#9: stress-ng-vm-rw/5477
NIP [c00000000008cdec] bad_kernel_fault.constprop.0+0x1a8/0x2c8
Call Trace: bad_kernel_fault.constprop.0+0x1a4/0x2c8 (unreliable) ___do_page_fault+0x688/0xa54 do_page_fault+0x30/0x70 data_access_common_virt+0x210/0x220 ---- interrupt: 300 at __copy_tofrom_user_power7+0x410/0x7ac NIP [c0000000000b3b44] __copy_tofrom_user_power7+0x410/0x7ac LR [c0000000009a7d78] _copy_to_iter+0x134/0x9c4
Enabled CONFIG_PPC_KUAP_DEBUG=y, which prints out below warnings.
WARNING: ./arch/powerpc/include/asm/book3s/64/kup.h:93 at _switch+0x80/0x12c, CPU#9: stress-ng-vm-rw/5477 NIP [c000000000013ce4] _switch+0x80/0x12c LR [c00000000001f968] __switch_to+0x148/0x230 Call Trace: __switch_to+0x148/0x230 __schedule+0x270/0x700 preempt_schedule_notrace+0x64/0xd8 function_trace_call+0x180/0x204 ftrace_call+0x4/0x4c enter_vmx_usercopy+0x10/0x74 __copy_tofrom_user_power7+0x278/0x7ac _copy_to_iter+0x134/0x9c4 copy_page_to_iter+0xe4/0x1c4 process_vm_rw_single_vec.constprop.0+0x1cc/0x3b4 process_vm_rw_core.constprop.0+0x168/0x30c process_vm_rw+0x128/0x184 system_call_exception+0x128/0x390 system_call_vectored_common+0x15c/0x2ec
enter/exit vmx_usercopy clearly says it shouldn't call schedule. Doing so will end up corrupting AMR registers. When function tracer is enabled, the entry point, i.e enter_vmx_usercopy could be in preemptible context. First thing enter_vmx_usercopy does is, preempt_disable and again function exit of exit_vmx_usercopy maybe preemptible too. So make these as notrace to avoid these bug reports.
WARNING: [amr != AMR_KUAP_BLOCKED] ./arch/powerpc/include/asm/book3s/64/kup.h:293 at arch_local_irq_restore.part.0+0x1e8/0x224, CPU#15: stress-ng-pipe/11623 NIP [c000000000038830] arch_local_irq_restore.part.0+0x1e8/0x224
LR [c00000000003871c] arch_local_irq_restore.part.0+0xd4/0x224
Call Trace: return_to_handler+0x0/0x4c (unreliable) __rb_reserve_next+0x198/0x4f8 ring_buffer_lock_reserve+0x1a8/0x51c trace_buffer_lock_reserve+0x30/0x80 __graph_entry.isra.0+0x118/0x140 function_graph_enter_regs+0x1ec/0x408 ftrace_graph_func+0x50/0xcc ftrace_call+0x4/0x4c enable_kernel_altivec+0x10/0xd0 enter_vmx_usercopy+0x58/0x74 return_to_handler+0x0/0x4c (__copy_tofrom_user_power7+0x278/0x7ac) _copy_from_iter+0x134/0x9bc copy_page_from_iter+0xd4/0x1a0
Since AMR registers aren't set to BLOCKED state, warnings could be seen if there is any unlock involved, which gets triggered via arch_local_irq_restore. So had to for that enable_kernel_altivec too. Similarly for check_if_tm_restore_required, giveup_altivec.
--- arch/powerpc/kernel/process.c | 10 +++++----- arch/powerpc/lib/vmx-helper.c | 4 ++-- 2 files changed, 7 insertions(+), 7 deletions(-)
arch/powerpc/kernel/process.c +5 -5
@@ -80,7 +80,7 @@*/ bool tm_suspend_disabled__ro_after_init = false; -static void check_if_tm_restore_required (struct task_struct *tsk) +notrace static void check_if_tm_restore_required(struct task_struct *tsk) {/* * If we are saving the current thread's registers, and the
@@ -98,7 +98,7 @@ static void check_if_tm_restore_required(struct task_struct *tsk) }#else -static inline void check_if_tm_restore_required (struct task_struct *tsk) { } +static __always_inline void check_if_tm_restore_required(struct task_struct *tsk) { }#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */ bool strict_msr_control;
@@ -231,7 +231,7 @@ static inline void __giveup_fpu(struct task_struct *tsk) { }#endif /* CONFIG_PPC_FPU */ #ifdef CONFIG_ALTIVEC -static void __giveup_altivec (struct task_struct *tsk) +notrace static void __giveup_altivec(struct task_struct *tsk) {unsigned long msr;
@@ -243,7 +243,7 @@ static void __giveup_altivec(struct task_struct *tsk)regs_set_return_msr (tsk->thread.regs, msr); } -void giveup_altivec (struct task_struct *tsk) +notrace void giveup_altivec(struct task_struct *tsk) {check_if_tm_restore_required (tsk);
@@ -253,7 +253,7 @@ void giveup_altivec(struct task_struct *tsk) }EXPORT_SYMBOL (giveup_altivec); -void enable_kernel_altivec (void ) +notrace void enable_kernel_altivec(void) {unsigned long cpumsr;
arch/powerpc/lib/vmx-helper.c +2 -2
@@ -10,7 +10,7 @@#include <linux/hardirq.h> #include <asm/switch_to.h> -int enter_vmx_usercopy (void ) +notrace int enter_vmx_usercopy(void) {if (in_interrupt ())return 0 ;
@@ -32,7 +32,7 @@ int enter_vmx_usercopy(void)* This function must return 0 because we tail call optimise when calling * from __copy_tofrom_user_power7 which returns 0 on success. */ -int exit_vmx_usercopy (void ) +notrace int exit_vmx_usercopy(void) {disable_kernel_altivec ();pagefault_enable ();
Signature · 1 line
2.47.3
Christophe Leroy (CS GROUP)
reviewerLe 09/01/2026 à 07:49, Shrikanth Hegde a écrit :
Quoted 6 lines
Recently stumbled upon these kuap warnings. This happens with preempt=full/lazy kernel with function tracing enabled. What irked me was kernel compilation was getting failed when i had tracing enabled. It doesn't fail everytime. While running stress-ng memory class it threw same warnings. So that helped to narrow it down.
Quoted 6 lines
So one possible way is to disable tracing for these enter/exit
vmx_usercopy. That seems to fix the bug/warnings. But that will make them as non trace-able. If there is a better way to fix these warning while keeping them as trace-able, please let me know.Anyone with insights on amr, vmx and tracing, please advise.
The main principle with KUAP is to not call subfunctions once userspace access enabled. There are a few exceptions like __copy_tofrom_user() that are allowed in order to optimise large copies. However this needs to be handled very carefully, and in principle we don't expect __copy_tofrom_user() to call other functions.
So it might require wider rework but we should narrow as much as possible the period during which access to userspace is opened, with something like:
raw_coy_to_user_power7()
{
enter_vmx_usercopy();
allow_write_to_user(to, n);
ret = __copy_tofrom_user_power7();
prevent_write_to_user(to, n);
exit_vmx_usercopy();
return ret;
}raw_copy_to_user()
{
if (cpu_has_feature(CPU_FTR_VMX_COPY))
raw_copy_to_user_power7();
allow_write_to_user(to, n);
ret = __copy_tofrom_user(to, (__force const void __user *)from, n);
prevent_write_to_user(to, n);
return ret;
}Shrikanth Hegde
reviewerHi Christophe.
Quoted 13 lines · Christophe Leroy (CS GROUP)
Le 09/01/2026 à 07:49, Shrikanth Hegde a écrit :
Quoted 10 lines
Recently stumbled upon these kuap warnings. This happens with preempt=full/lazy kernel with function tracing enabled. What irked me was kernel compilation was getting failed when i had tracing enabled. It doesn't fail everytime. While running stress-ng memory class it threw same warnings. So that helped to narrow it down.
So one possible way is to disable tracing for these enter/exitvmx_usercopy. That seems to fix the bug/warnings. But that will make them as non trace-able. If there is a better way to fix these warning while keeping them as trace-able, please let me know.
Quoted 4 lines
Quoted 1 line
Anyone with insights on amr, vmx and tracing, please advise.
The main principle with KUAP is to not call subfunctions once userspace access enabled. There are a few exceptions like
__copy_tofrom_user()that are allowed in order to optimise large copies. However this needs to be handled very carefully, and in principle we don't expect__copy_tofrom_user()to call other functions.
I didn't understand. My knowledge is quite limited in this space. Could you please explain how this will help us avoid the warnings? or are you saying we have more callsites which needs to worked upon.
Quoted 5 lines
So it might require wider rework but we should narrow as much as possible the period during which access to userspace is opened, with something like:
raw_coy_to_user_power7()
{
enter_vmx_usercopy();
I think the problem is when it comes here, it has some AMR state, but it is preemptible. So shouldn't call schedule IIUC.
Quoted 17 lines
allow_write_to_user(to, n);
ret =__copy_tofrom_user_power7();
prevent_write_to_user(to, n);
exit_vmx_usercopy();
return ret;
}
raw_copy_to_user()
{
if (cpu_has_feature(CPU_FTR_VMX_COPY))
raw_copy_to_user_power7();allow_write_to_user(to, n);
ret =__copy_tofrom_user(to, (__force const void __user *)from, n);prevent_write_to_user(to, n);
return ret;
}
Christophe Leroy (CS GROUP)
reviewerHi Shrikanth,
Le 09/01/2026 à 13:19, Shrikanth Hegde a écrit :
Quoted 16 lines
Hi Christophe.
Quoted 13 lines · Christophe Leroy (CS GROUP)
Le 09/01/2026 à 07:49, Shrikanth Hegde a écrit :
Quoted 10 lines
Recently stumbled upon these kuap warnings. This happens with preempt=full/lazy kernel with function tracing enabled. What irked me was kernel compilation was getting failed when i had tracing enabled. It doesn't fail everytime. While running stress-ng memory class it threw same warnings. So that helped to narrow it down.
So one possible way is to disable tracing for these enter/exitvmx_usercopy. That seems to fix the bug/warnings. But that will make them as non trace-able. If there is a better way to fix these warning while keeping them as trace-able, please let me know.
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
Anyone with insights on amr, vmx and tracing, please advise.
The main principle with KUAP is to not call subfunctions once userspace access enabled. There are a few exceptions like
__copy_tofrom_user()that are allowed in order to optimise large copies. However this needs to be handled very carefully, and in principle we don't expect__copy_tofrom_user()to call other functions.I didn't understand. My knowledge is quite limited in this space. Could you please explain how this will help us avoid the warnings? or are you saying we have more callsites which needs to worked upon.
Read tools/objtool/Documentation/objtool.txt section "Objtool warning" item 9.
Unfortunately powerpc doesn't yet implement objtool to detect it, but the principle applies anyway.
Quoted 9 lines
Quoted 6 lines
So it might require wider rework but we should narrow as much as possible the period during which access to userspace is opened, with something like:
raw_coy_to_user_power7()
{
enter_vmx_usercopy();I think the problem is when it comes here, it has some AMR state, but it is preemptible. So shouldn't call schedule IIUC.
See commit 00ff1eaac129 ("powerpc: Fix reschedule bug in KUAP-unlocked user copy")
The problem is because enter_vmx_usercopy() is called _after_ allow_write_to_user() which changes AMR.
If you call enter_vmx_usercopy() _before_ allow_write_to_user() and call exit_vmx_usercopy() _after_ prevent_write_to_user() the problem is solved.
Quoted 19 lines
Quoted 18 lines
allow_write_to_user(to, n);
ret =__copy_tofrom_user_power7();
prevent_write_to_user(to, n);
exit_vmx_usercopy();
return ret;
}
raw_copy_to_user()
{
if (cpu_has_feature(CPU_FTR_VMX_COPY))
raw_copy_to_user_power7();allow_write_to_user(to, n);
ret =__copy_tofrom_user(to, (__force const void __user *)from, n);prevent_write_to_user(to, n);
return ret;
}
Shrikanth Hegde
reviewerHi Christophe.
Quoted 39 lines · Christophe Leroy (CS GROUP)
Hi Shrikanth,
Le 09/01/2026 à 13:19, Shrikanth Hegde a écrit :
Quoted 16 lines
Hi Christophe.
Quoted 13 lines · Christophe Leroy (CS GROUP)
Le 09/01/2026 à 07:49, Shrikanth Hegde a écrit :
Quoted 10 lines
Recently stumbled upon these kuap warnings. This happens with preempt=full/lazy kernel with function tracing enabled. What irked me was kernel compilation was getting failed when i had tracing enabled. It doesn't fail everytime. While running stress-ng memory class it threw same warnings. So that helped to narrow it down.
So one possible way is to disable tracing for these enter/exitvmx_usercopy. That seems to fix the bug/warnings. But that will make them as non trace-able. If there is a better way to fix these warning while keeping them as trace-able, please let me know.Quoted 8 lines
Quoted 4 lines
Quoted 1 line
Anyone with insights on amr, vmx and tracing, please advise.
The main principle with KUAP is to not call subfunctions once userspace access enabled. There are a few exceptions like
__copy_tofrom_user()that are allowed in order to optimise large copies. However this needs to be handled very carefully, and in principle we don't expect__copy_tofrom_user()to call other functions.I didn't understand. My knowledge is quite limited in this space. Could you please explain how this will help us avoid the warnings? or are you saying we have more callsites which needs to worked upon.
Read
tools/objtool/Documentation/objtool.txtsection "Objtool warning" item 9.Unfortunately powerpc doesn't yet implement objtool to detect it, but the principle applies anyway.
Quoted 7 lines
Quoted 6 lines
So it might require wider rework but we should narrow as much as possible the period during which access to userspace is opened, with something like:
raw_coy_to_user_power7()
{
enter_vmx_usercopy();
Quoted 29 lines
Quoted 2 lines
I think the problem is when it comes here, it has some AMR state, but it is preemptible. So shouldn't call schedule IIUC.
See commit
00ff1eaac129("powerpc: Fix reschedule bug in KUAP-unlocked user copy")The problem is because
enter_vmx_usercopy()is called _after_allow_write_to_user()which changes AMR.If you call
enter_vmx_usercopy()_before_allow_write_to_user()and callexit_vmx_usercopy()_after_prevent_write_to_user()the problem is solved.Quoted 19 lines
Quoted 18 lines
allow_write_to_user(to, n);
ret =__copy_tofrom_user_power7();
prevent_write_to_user(to, n);
exit_vmx_usercopy();
return ret;
}
raw_copy_to_user()
{
if (cpu_has_feature(CPU_FTR_VMX_COPY))
raw_copy_to_user_power7();allow_write_to_user(to, n);
ret =__copy_tofrom_user(to, (__force const void __user *)from, n);prevent_write_to_user(to, n);
return ret;
}
Sorry, I forgot to update.
I spoke to ritesh a while ago and someone with better knowledge in this area will work on a proper fix for this.
Ritesh Harjani (IBM)
reviewerQuoted 5 lines · Shrikanth Hegde
Sorry, I forgot to update.
I spoke to ritesh a while ago and someone with better knowledge in this area will work on a proper fix for this.
Sorry about the delay in getting back to this. Sayali from our team has been looking into this. She is soon going to post the fix which is basd upon Christophe suggestion.
-ritesh