s390: Exception based WARN() / WARN_ONCE()
Subsystems (2)
Heiko Carstens
maintainercover s390: Exception based WARN() / WARN_ONCE()
Use the generic infrastructure introduced by Peter Zijlstra [1] to implement an exception based WARN() and WARN_ONCE() similar to x86.
Due to some compiler oddities on s390 this requires to raise the minimum gcc version to 9. Maybe there are ways to avoid this, but I failed to find a working solution. Details are in the patch descriptions.
Just posting this now to also get some compile bot testing, since I'm afraid there might be some compiler version / config option around where even this new approach breaks.
Peter, since you were wondering: your generic infrastructure pieces work very nice. Looking at the x86 and s390 implementation: it might be possible to make things even more generic since both __WARN_printf(), and WARN_ONCE() are identical; it looks like only __WARN_print_arg() needs to be provided.
Arnd, just adding you so you know that there may be an architecture which requires gcc-9 instead of gcc-8 in the near future.
Series is based ongit://git.kernel.org/pub/scm/linux/kernel/git/s390/linux.git features
...and not yet meant to be included anywhere.
Thanks,
Heiko
[1] https://lore.kernel.org/all/[email protected]/
Heiko Carstens (9):
kbuild: Require gcc-9 for s390
s390/bug: Convert to inline assembly with input operands s390/bug: Use BUG_FORMAT for DEBUG_BUGVERBOSE_DETAILED s390/bug: Introduce and use monitor code macro s390/traps: Copy monitor code to pt_regs s390/bug: Implement __WARN_printf()
s390/bug: Implement WARN_ONCE()
s390/bug: Skip __WARN_trap() in call traces s390/bug: Prevent tail-call optimization
arch/s390/include/asm/bug.h | 141 ++++++++++++++++++++++++--------- arch/s390/include/asm/ptrace.h | 5 +- arch/s390/kernel/entry.S | 7 ++ arch/s390/kernel/traps.c | 42 +++++++++- scripts/min-tool-version.sh | 2 + 5 files changed, 156 insertions(+), 41 deletions(-)
base-commit: 70075e3d0ca0b72cc983d03f7cd9796e43492980
Signature · 1 line
2.51.0
Heiko Carstens
maintainerPATCH 1/9 kbuild: Require gcc-9 for s390
Upcoming changes to s390 specific inline assemblies require the usage of strings as immediate input operands. This works only with gcc-9 and newer compilers. With gcc-8 this leads to a compile error:
void bar(void) { asm volatile("" :: "i" ("foo")); }
Results in:
In function 'bar':
warning: asm operand 0 probably doesn't match constraints
asm volatile("" :: "i" ("foo"));
^~~
error: impossible constraint in 'asm'
Bump the minimal gcc version from gcc-8 to gcc-9 for s390 to solve this problem.
--- scripts/min-tool-version.sh | 2 ++ 1 file changed, 2 insertions(+)
scripts/min-tool-version.sh +2 -0
@@ -19,6 +19,8 @@ binutils) gcc) if [ "$ARCH" = parisc64 ]; then echo 12.0.0 + elif [ "$ARCH" = s390 ]; then + echo 9.1.0 else echo 8.1.0 fi
Signature · 1 line
2.51.0
Heiko Carstens
maintainerPATCH 2/9 s390/bug: Convert to inline assembly with input operands
In reply to Heiko CarstensRewrite the bug inline assembly so it uses input operands again instead of pure macro replacements. This more or less reverts the conversion done when 'cond_str' support was added [1].
Reason for this is that the upcoming __WARN_printf() implementation requires an inline assembly with an output operand. At the same time input strings (format specifier and condition string) may contain the special '%' character. As soon as an inline assembly is specified to have input/output operands the '%' has a special meaning: e.g. converting the existing
#define __BUG_FLAGS(cond_str, flags) \
asm_inline volatile(__stringify(ASM_BUG_FLAGS(cond_str, flags)));
to
#define __BUG_FLAGS(cond_str, flags) \
asm_inline volatile(__stringify(ASM_BUG_FLAGS(cond_str, flags))::);
will result in a compile error as soon as 'cond_str' contains a '%' character:
net/core/neighbour.c: In function ‘neigh_table_init’:././include/linux/compiler_types.h:546:20: error: invalid 'asm': invalid %-code
...net/core/neighbour.c:1838:17: note: in expansion of macro ‘WARN_ON’
1838 | WARN_ON(tbl->entry_size % NEIGH_PRIV_ALIGN);
| ^~~~~~~Convert the code, use immediate operands, and also add comments similar to x86 which are emitted to the generated assembly file, which makes debugging much easier.
[1] 6584ff203aec ("bugs/s390: Use 'cond_str' in __EMIT_BUG()")
--- arch/s390/include/asm/bug.h | 71 ++++++++++++++++--------------------- 1 file changed, 30 insertions(+), 41 deletions(-)
arch/s390/include/asm/bug.h +30 -41
@@ -2,55 +2,44 @@#ifndef _ASM_S390_BUG_H#define _ASM_S390_BUG_H -#include <linux/stringify.h> +#include <linux/compiler.h> #ifdef CONFIG_BUG -#ifndef CONFIG_DEBUG_BUGVERBOSE -#define _BUGVERBOSE_LOCATION (file, line) +#ifdef CONFIG_DEBUG_BUGVERBOSE +#define __BUG_ENTRY_VERBOSE (file, line) \ + " .long " file " - . # bug_entry::file\n" \ + " .short " line " # bug_entry::line\n"#else -#define __BUGVERBOSE_LOCATION (file, line) \ - .pushsection .rodata.str, "aMS", @progbits, 1; \ - .align 2; \ - 10002: .ascii file "\0"; \ - .popsection; \ - \ - .long 10002b - .; \ - .short line; -#define _BUGVERBOSE_LOCATION (file, line) __BUGVERBOSE_LOCATION(file, line) +#define __BUG_ENTRY_VERBOSE (file, line)#endif -#ifndef CONFIG_GENERIC_BUG -#define __BUG_ENTRY (cond_str, flags) -#else -#define __BUG_ENTRY (cond_str, flags) \ - .pushsection __bug_table, "aw"; \ - .align 4; \ - 10000: .long 10001f - .; \ - _BUGVERBOSE_LOCATION(WARN_CONDITION_STR(cond_str) __FILE__, __LINE__) \ - .short flags; \ - .popsection; \ - 10001: -#endif - -#define ASM_BUG_FLAGS (cond_str, flags) \ - __BUG_ENTRY(cond_str, flags) \ - mc 0,0 - -#define ASM_BUG () ASM_BUG_FLAGS("", 0) - -#define __BUG_FLAGS (cond_str, flags) \ - asm_inline volatile(__stringify(ASM_BUG_FLAGS(cond_str, flags))); - -#define __WARN_FLAGS (cond_str, flags) \ -do { \ - __BUG_FLAGS(cond_str, BUGFLAG_WARNING|(flags)); \ +#define __BUG_ASM (cond_str, flags) \ +do { \ + asm_inline volatile("\n" \ + "0: mc 0,0\n" \ + " .section __bug_table,\"aw\"\n" \ + "1: .long 0b - . # bug_entry::bug_addr\n" \ + __BUG_ENTRY_VERBOSE("%[file]", "%[line]") \ + " .short %[flgs] # bug_entry::flags\n" \ + " .org 1b+%[size]\n" \ + " .previous" \ + : \ + : [file] "i" (WARN_CONDITION_STR(cond_str) __FILE__), \ + [line] "i" (__LINE__), \ + [flgs] "i" (flags), \ + [size] "i" (sizeof(struct bug_entry))); \ } while (0) -#define BUG () \ -do { \ - __BUG_FLAGS("", 0); \ - unreachable(); \ +#define BUG () \ +do { \ + __BUG_ASM("", 0); \ + unreachable(); \ +} while (0) + +#define __WARN_FLAGS (cond_str, flags) \ +do { \ + __BUG_ASM(cond_str, BUGFLAG_WARNING | (flags)); \ } while (0)#define HAVE_ARCH_BUG
Signature · 1 line
2.51.0
Heiko Carstens
maintainerPATCH 3/9 s390/bug: Use BUG_FORMAT for DEBUG_BUGVERBOSE_DETAILED
In reply to Heiko CarstensThis is just the s390 variant of commit 4f1b701f24be ("x86/bug: Use BUG_FORMAT for DEBUG_BUGVERBOSE_DETAILED").
--- arch/s390/include/asm/bug.h | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-)
arch/s390/include/asm/bug.h +13 -4
@@ -7,11 +7,18 @@#ifdef CONFIG_BUG #ifdef CONFIG_DEBUG_BUGVERBOSE -#define __BUG_ENTRY_VERBOSE (file, line) \ +#define __BUG_ENTRY_VERBOSE (format, file, line) \ + " .long " format " - . # bug_entry::format\n" \ " .long " file " - . # bug_entry::file\n" \ " .short " line " # bug_entry::line\n"#else -#define __BUG_ENTRY_VERBOSE (file, line) +#define __BUG_ENTRY_VERBOSE (format, file, line) +#endif + +#ifdef CONFIG_DEBUG_BUGVERBOSE_DETAILED +#define WARN_CONDITION_STR (cond_str) cond_str +#else +#define WARN_CONDITION_STR (cond_str) ""#endif #define __BUG_ASM (cond_str, flags) \
@@ -20,12 +27,13 @@ do { \"0: mc 0,0\n" \" .section __bug_table,\"aw\"\n" \"1: .long 0b - . # bug_entry::bug_addr\n" \ - __BUG_ENTRY_VERBOSE("%[file]", "%[line]") \ + __BUG_ENTRY_VERBOSE("%[frmt]", "%[file]", "%[line]") \" .short %[flgs] # bug_entry::flags\n" \" .org 1b+%[size]\n" \" .previous" \ : \ - : [file] "i" (WARN_CONDITION_STR(cond_str) __FILE__), \ + : [frmt] "i" (WARN_CONDITION_STR(cond_str)), \ + [file] "i" (__FILE__), \[line] " i" (__LINE__), \ [flgs] " i" (flags), \ [size] " i" (sizeof(struct bug_entry))); \
@@ -43,6 +51,7 @@ do { \ } while (0)#define HAVE_ARCH_BUG +#define HAVE_ARCH_BUG_FORMAT #endif /* CONFIG_BUG */
Signature · 1 line
2.51.0
Heiko Carstens
maintainerPATCH 4/9 s390/bug: Introduce and use monitor code macro
In reply to Heiko CarstensThe first operand address of the monitor call (mc) instruction is the monitor code. Currently the monitor code is ignored, but this will change. Therefore add and use MONCODE_BUG instead of a hardcoded zero.
--- arch/s390/include/asm/bug.h | 10 ++++++++-- arch/s390/kernel/traps.c | 5 +++-- 2 files changed, 11 insertions(+), 4 deletions(-)
arch/s390/include/asm/bug.h +8 -2
@@ -3,7 +3,11 @@#define _ASM_S390_BUG_H#include <linux/compiler.h> +#include <linux/const.h> +#define MONCODE_BUG _AC(0, U) + +#ifndef __ASSEMBLER__#ifdef CONFIG_BUG #ifdef CONFIG_DEBUG_BUGVERBOSE
@@ -24,7 +28,7 @@#define __BUG_ASM (cond_str, flags) \ do { \ asm_inline volatile("\n" \ - "0: mc 0,0\n" \ + "0: mc %[monc](%%r0),0\n" \ " .section __bug_table,\"aw\"\n" \ "1: .long 0b - . # bug_entry::bug_addr\n" \ __BUG_ENTRY_VERBOSE("%[frmt]", "%[file]", "%[line]") \
@@ -32,7 +36,8 @@ do { \" .org 1b+%[size]\n" \" .previous" \ : \ - : [frmt] "i" (WARN_CONDITION_STR(cond_str)), \ + : [monc] "i" (MONCODE_BUG), \ + [frmt] "i" (WARN_CONDITION_STR(cond_str)), \[file] " i" (__FILE__), \ [line] " i" (__LINE__), \ [flgs] " i" (flags), \
@@ -54,6 +59,7 @@ do { \#define HAVE_ARCH_BUG_FORMAT #endif /* CONFIG_BUG */ +#endif /* __ASSEMBLER__ */ #include <asm-generic/bug.h>
arch/s390/kernel/traps.c +3 -2
@@ -258,11 +258,12 @@ static void __init test_monitor_call(void)if (!IS_ENABLED (CONFIG_BUG ))return ; asm_inline volatile( -" mc 0,0\n" +" mc %[monc](%%r0),0\n" "0: lhi %[val],0\n" "1:\n" EX_TABLE (0b ,1b ) - : [val] "+d" (val)); + : [val] "+d" (val) + : [monc] "i" (MONCODE_BUG));if (!val)panic ("Monitor call doesn't work!\n" ); }
Signature · 1 line
2.51.0
Heiko Carstens
maintainerPATCH 5/9 s390/traps: Copy monitor code to pt_regs
In reply to Heiko CarstensIn case of a monitor call program check the CPU stores the monitor code to lowcore. Let the program check handler copy it to the pt_regs structure so it can be used by the monitor call exception handler.
Instead of increasing the pt_regs size add a union which contains both orig_gpr2 and monitor_code, since orig_gpr2 is not used in case of a program check.
--- arch/s390/include/asm/ptrace.h | 5 ++++- arch/s390/kernel/traps.c | 1 + 2 files changed, 5 insertions(+), 1 deletion(-)
arch/s390/include/asm/ptrace.h +4 -1
@@ -120,7 +120,10 @@ struct pt_regs {unsigned long gprs[NUM_GPRS ]; }; }; -unsigned long orig_gpr2; +union { +unsigned long orig_gpr2; +unsigned long monitor_code; + }; union { struct { unsigned int int_code;
arch/s390/kernel/traps.c +1 -0
@@ -298,6 +298,7 @@ void noinstr __do_pgm_check(struct pt_regs *regs) teid.val = lc->trans_exc_code; regs->int_code = lc->pgm_int_code; regs->int_parm_long = teid.val; + regs->monitor_code = lc->monitor_code;/* * In case of a guest fault, short-circuit the fault handler and return. * This way the sie64a() function will return 0; fault address and
Signature · 1 line
2.51.0
Heiko Carstens
maintainerPATCH 6/9 s390/bug: Implement __WARN_printf()
In reply to Heiko CarstensThis is the s390 variant of commit 5b472b6e5bd9 ("x86_64/bug: Implement __WARN_printf()"). See the x86 commit for the general idea; there are only implementation details which are different.
With the new exception based __WARN_printf() implementation the generated code for a simple WARN() is simplified.
For example:
void foo(int a) { WARN(a, "bar"); }
Before this change the generated code looks like this:
0000000000000210 <foo>:
210: c0 04 00 00 00 00 jgnop 210 <foo>
216: ec 26 00 06 00 7c cgijne %r2,0,222 <foo+0x12>
21c: c0 f4 00 00 00 00 jg 21c <foo+0xc>
21e: R_390_PC32DBL __s390_indirect_jump_r14+0x2
222: eb ef f0 88 00 24 stmg %r14,%r15,136(%r15)
228: b9 04 00 ef lgr %r14,%r15
22c: e3 f0 ff e8 ff 71 lay %r15,-24(%r15)
232: e3 e0 f0 98 00 24 stg %r14,152(%r15)
238: c0 20 00 00 00 00 larl %r2,238 <foo+0x28>
23a: R_390_PC32DBL .LC48+0x2
23e: c0 e5 00 00 00 00 brasl %r14,23e <foo+0x2e>
240: R_390_PLT32DBL __warn_printk+0x2
244: af 00 00 00 mc 0,0
248: eb ef f0 a0 00 04 lmg %r14,%r15,160(%r15)
24e: c0 f4 00 00 00 00 jg 24e <foo+0x3e>
250: R_390_PC32DBL __s390_indirect_jump_r14+0x2With this change the generated code looks like this:
0000000000000210 <foo>:
210: c0 04 00 00 00 00 jgnop 210 <foo>
216: ec 26 00 06 00 7c cgijne %r2,0,222 <foo+0x12>
21c: c0 f4 00 00 00 00 jg 21c <foo+0xc>
21e: R_390_PC32DBL __s390_indirect_jump_r14+0x2
222: c0 20 00 00 00 00 larl %r2,222 <foobar+0x12>
224: R_390_PC32DBL __bug_table+0x2
228: c0 f4 00 00 00 00 jg 228 <foobar+0x18>
22a: R_390_PLT32DBL __WARN_trap+0x2Downside is that the call trace now starts at __WARN_trap():
------------[ cut here ]------------
bar
WARNING: arch/s390/kernel/setup.c:1017 at 0x0, CPU#0: swapper/0/0
...
Krnl PSW : 0704c00180000000 000003ffe0f6a3b4 (__WARN_trap+0x4/0x10)
...
Krnl Code: 000003ffe0f6a3ac: 0707 bcr 0,%r7
000003ffe0f6a3ae: 0707 bcr 0,%r7
*000003ffe0f6a3b0: af000001 mc 1,0
>000003ffe0f6a3b4: 07fe bcr 15,%r14
000003ffe0f6a3b6: 47000700 bc 0,1792
000003ffe0f6a3ba: 0707 bcr 0,%r7
000003ffe0f6a3bc: 0707 bcr 0,%r7
000003ffe0f6a3be: 0707 bcr 0,%r7
Call Trace:
[<000003ffe0f6a3b4>] __WARN_trap+0x4/0x10
([<000003ffe185a54c>] start_kernel+0x53c/0x5d8)
[<000003ffe010002e>] startup_continue+0x2e/0x40Which isn't too helpful. This can be addressed by just skipping __WARN_trap(), which will be addressed in a later patch.
--- arch/s390/include/asm/bug.h | 62 +++++++++++++++++++++++++++++++++---- arch/s390/kernel/entry.S | 7 +++++ arch/s390/kernel/traps.c | 34 +++++++++++++++++++- 3 files changed, 96 insertions(+), 7 deletions(-)
arch/s390/include/asm/bug.h +56 -6
@@ -6,6 +6,7 @@#include <linux/const.h> #define MONCODE_BUG _AC(0, U) +#define MONCODE_BUG_ARG _AC(1, U)#ifndef __ASSEMBLER__#ifdef CONFIG_BUG
@@ -25,16 +26,20 @@#define WARN_CONDITION_STR (cond_str) ""#endif +#define __BUG_ENTRY (format, file, line, flags, size) \ + " .section __bug_table,\"aw\"\n" \ + "1: .long 0b - . # bug_entry::bug_addr\n" \ + __BUG_ENTRY_VERBOSE(format, file, line) \ + " .short "flags" # bug_entry::flags\n" \ + " .org 1b+"size"\n" \ + " .previous" +#define __BUG_ASM (cond_str, flags) \ do { \ asm_inline volatile("\n" \ "0: mc %[monc](%%r0),0\n" \ - " .section __bug_table,\"aw\"\n" \ - "1: .long 0b - . # bug_entry::bug_addr\n" \ - __BUG_ENTRY_VERBOSE("%[frmt]", "%[file]", "%[line]") \ - " .short %[flgs] # bug_entry::flags\n" \ - " .org 1b+%[size]\n" \ - " .previous" \ + __BUG_ENTRY("%[frmt]", "%[file]", "%[line]", \ + "%[flgs]", "%[size]") \ : \ : [monc] "i" (MONCODE_BUG), \ [frmt] "i" (WARN_CONDITION_STR(cond_str)), \
@@ -55,8 +60,53 @@ do { \__BUG_ASM (cond_str,BUGFLAG_WARNING | (flags)); \ } while (0) +#define __WARN_bug_entry (flags, format) \ +({ \ + struct bug_entry *bug; \ + \ + asm_inline volatile("\n" \ + "0: larl %[bug],1f\n" \ + __BUG_ENTRY("%[frmt]", "%[file]", "%[line]", \ + "%[flgs]", "%[size]") \ + : [bug] "=d" (bug) \ + : [frmt] "i" (format), \ + [file] "i" (__FILE__), \ + [line] "i" (__LINE__), \ + [flgs] "i" (flags), \ + [size] "i" (sizeof(struct bug_entry))); \ + bug; \ +}) + +/* +* Variable Argument List (va_list) as defined in ELF Application +* Binary Interface s390x Supplement documentation. +*/ +struct arch_va_list { +long __gpr; +long __fpr; +void *__overflow_arg_area; +void *__reg_save_area; +}; + +struct bug_entry ; +struct pt_regs ; + +void *__warn_args (struct arch_va_list *args,struct pt_regs *regs); +void __WARN_trap (struct bug_entry *bug, ...); + +#define __WARN_print_arg(flags, format, arg...) \ +do { \ + int __flags = (flags) | BUGFLAG_WARNING | BUGFLAG_ARGS; \ + \ + __WARN_trap(__WARN_bug_entry(__flags, format), ## arg); \ +} while (0) + +#define __WARN_printf(taint, fmt, arg...) \ + __WARN_print_arg(BUGFLAG_TAINT(taint), fmt, ## arg) +#define HAVE_ARCH_BUG #define HAVE_ARCH_BUG_FORMAT +#define HAVE_ARCH_BUG_FORMAT_ARGS #endif /* CONFIG_BUG */ #endif /* __ASSEMBLER__ */
arch/s390/kernel/entry.S +7 -0
@@ -23,6 +23,7 @@ #include <asm/unistd.h> #include <asm/page.h> #include <asm/sigp.h> +#include <asm/bug.h> #include <asm/irq.h> #include <asm/fpu-insn.h> #include <asm/setup.h>
@@ -173,6 +174,12 @@ SYM_FUNC_START(__switch_to_asm) BR_EX %r14 SYM_FUNC_END(__switch_to_asm) +SYM_FUNC_START(__WARN_trap) + mc MONCODE_BUG_ARG(%r0),0 + BR_EX %r14 +SYM_FUNC_END(__WARN_trap) +EXPORT_SYMBOL(__WARN_trap) + #if IS_ENABLED(CONFIG_KVM) /* * __sie64a calling convention:
arch/s390/kernel/traps.c +33 -1
@@ -23,6 +23,7 @@#include <linux/cpu.h> #include <linux/entry-common.h> #include <linux/kmsan.h> +#include <linux/bug.h> #include <asm/asm-extable.h> #include <asm/irqflags.h> #include <asm/ptrace.h>
@@ -220,11 +221,42 @@ static void space_switch_exception(struct pt_regs *regs)do_trap (regs,SIGILL ,ILL_PRVOPC ,"space switch event" ); } +void *__warn_args (struct arch_va_list *args,struct pt_regs *regs) +{ +struct stack_frame *stack_frame; + +/* +* Generate va_list from pt_regs. See ELF Application Binary Interface +* s390x Supplement documentation for details. +* +* - __overflow_arg_area needs to point to the parameter area, which +* is right above the standard stack frame (160 bytes) +* +* - __reg_save_area needs to point to a register save area where +* general registers (%r2 - %r6) can be found at offset 16. Which +* means that the gprs save area of pt_regs can be used +* +* - __gpr must be set to one, since the first parameter has been +* processed (pointer to bug_entry) +*/ + stack_frame = (struct stack_frame *)regs->gprs[15 ]; + args->__overflow_arg_area = stack_frame +1 ; + args->__reg_save_area = regs->gprs; + args->__gpr =1 ; +return args; +} +static void monitor_event_exception (struct pt_regs *regs) { +enum bug_trap_type btt; +if (user_mode (regs))return ; -switch (report_bug (regs->psw.addr - (regs->int_code >>16 ), regs)) { +if (regs->monitor_code ==MONCODE_BUG_ARG ) + btt =report_bug_entry ((struct bug_entry *)regs->gprs[2 ], regs); +else + btt =report_bug (regs->psw.addr - (regs->int_code >>16 ), regs); +switch (btt) {case BUG_TRAP_TYPE_NONE :fixup_exception (regs);break ;
Signature · 1 line
2.51.0
Re: [PATCH 6/9] s390/bug: Implement __WARN_printf()
arch/s390/include/asm/bug.h do { \ · line 102
Quoted 6 lines · Heiko Carstens
arch/s390/include/asm/bug.h +6
-0+#define __WARN_print_arg(flags, format, arg...) \ +do { \ + int __flags = (flags) | BUGFLAG_WARNING | BUGFLAG_ARGS; \ + \ + __WARN_trap(__WARN_bug_entry(__flags, format), ## arg); \ +} while (0)
So on x86 I had to add:
asm("");after the __WARN_trap() call above, to inhibit tail call optimization, because:
Quoted 14 lines
arch/s390/kernel/traps.c +14
-0+void *__warn_args (struct arch_va_list *args,struct pt_regs *regs) +{ +struct stack_frame *stack_frame; + +/* +* Generate va_list from pt_regs. See ELF Application Binary Interface +* s390x Supplement documentation for details. +* +* - __overflow_arg_area needs to point to the parameter area, which +* is right above the standard stack frame (160 bytes) +* +* - __reg_save_area needs to point to a register save area where +* general registers (%r2 - %r6) can be found at offset 16. Which +* means that the gprs save area of pt_regs can be used
arch/s390/kernel/traps.c static void space_switch_exception(struct pt_regs *regs) · line 247
Quoted 10 lines
arch/s390/kernel/traps.c +10
-0+* +* - __gpr must be set to one, since the first parameter has been +* processed (pointer to bug_entry) +*/ + stack_frame = (struct stack_frame *)regs->gprs[15 ]; + args->__overflow_arg_area = stack_frame +1 ; + args->__reg_save_area = regs->gprs; + args->__gpr =1 ; +return args; +}
that would affect the stack layout here. You don't suffer this because you have a link register like setup?
Heiko Carstens
maintainerRe: [PATCH 6/9] s390/bug: Implement __WARN_printf()
Quoted 18 lines · Peter Zijlstra
arch/s390/include/asm/bug.h do { \ · line 102
Quoted 6 lines · Heiko Carstens
arch/s390/include/asm/bug.h +6
-0+#define __WARN_print_arg(flags, format, arg...) \ +do { \ + int __flags = (flags) | BUGFLAG_WARNING | BUGFLAG_ARGS; \ + \ + __WARN_trap(__WARN_bug_entry(__flags, format), ## arg); \ +} while (0)So on x86 I had to add:
asm("");after the
__WARN_trap()call above, to inhibit tail call optimization, because:Quoted 3 lines
arch/s390/kernel/traps.c +3
-0+void *__warn_args (struct arch_va_list *args,struct pt_regs *regs) +{ +struct stack_frame *stack_frame;
...
Quoted 9 lines
arch/s390/kernel/traps.c static void space_switch_exception(struct pt_regs *regs) · line 247
Quoted 6 lines
arch/s390/kernel/traps.c +6
-0+ stack_frame = (struct stack_frame *)regs->gprs[15 ]; + args->__overflow_arg_area = stack_frame +1 ; + args->__reg_save_area = regs->gprs; + args->__gpr =1 ; +return args; +}that would affect the stack layout here. You don't suffer this because you have a link register like setup?
Yes, in case of tail call optimization everything which needs to be known to setup va_list is passed in registers. __overflow_arg_area will then point to garbage, but it doesn't matter since it is unused for such cases.
kernel test robot
build botRe: [PATCH 6/9] s390/bug: Implement __WARN_printf()
In reply to Heiko CarstensAll warnings (new ones prefixed by >>):
All warnings (new ones prefixed by >>):
>> lib/bug.c:187:22: warning: diagnostic behavior may be improved by adding the 'format(printf, 1, 0)' attribute to the declaration of '__warn_printf' [-Wmissing-format-attribute]
176 | static void __warn_printf(const char *fmt, struct pt_regs *regs)
| __attribute__((format(printf, 1, 0)))
177 | {
178 | if (!fmt)
179 | return;
180 |
181 | #ifdef HAVE_ARCH_BUG_FORMAT_ARGS
182 | if (regs) {
183 | struct arch_va_list _args;
184 | va_list *args = __warn_args(&_args, regs);
185 |
186 | if (args) {
187 | vprintk(fmt, *args);
| ^ lib/bug.c:176:13: note: '__warn_printf' declared here
176 | static void __warn_printf(const char *fmt, struct pt_regs *regs)
| ^
1 warning generated.
vim +187 lib/bug.c
7664c5a1da4711b Jeremy Fitzhardinge 2006-12-08 175
5c47b7f3d1a9d75 Peter Zijlstra 2025-06-07 176 static void __warn_printf(const char *fmt, struct pt_regs *regs)
d292dbb5640c5b7 Peter Zijlstra 2025-06-02 177 {
d292dbb5640c5b7 Peter Zijlstra 2025-06-02 178 if (!fmt)
d292dbb5640c5b7 Peter Zijlstra 2025-06-02 179 return;
d292dbb5640c5b7 Peter Zijlstra 2025-06-02 180
5c47b7f3d1a9d75 Peter Zijlstra 2025-06-07 181 #ifdef HAVE_ARCH_BUG_FORMAT_ARGS
5c47b7f3d1a9d75 Peter Zijlstra 2025-06-07 182 if (regs) {
5c47b7f3d1a9d75 Peter Zijlstra 2025-06-07 183 struct arch_va_list _args;
5c47b7f3d1a9d75 Peter Zijlstra 2025-06-07 184 va_list *args = __warn_args(&_args, regs);
5c47b7f3d1a9d75 Peter Zijlstra 2025-06-07 185
5c47b7f3d1a9d75 Peter Zijlstra 2025-06-07 186 if (args) {
5c47b7f3d1a9d75 Peter Zijlstra 2025-06-07 @187 vprintk(fmt, *args);
5c47b7f3d1a9d75 Peter Zijlstra 2025-06-07 188 return;
5c47b7f3d1a9d75 Peter Zijlstra 2025-06-07 189 }
5c47b7f3d1a9d75 Peter Zijlstra 2025-06-07 190 }
5c47b7f3d1a9d75 Peter Zijlstra 2025-06-07 191 #endif
5c47b7f3d1a9d75 Peter Zijlstra 2025-06-07 192
d292dbb5640c5b7 Peter Zijlstra 2025-06-02 193 printk("%s", fmt);
d292dbb5640c5b7 Peter Zijlstra 2025-06-02 194 }
d292dbb5640c5b7 Peter Zijlstra 2025-06-02 195 Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
kernel test robot
build botRe: [PATCH 6/9] s390/bug: Implement __WARN_printf()
In reply to Heiko Carstenskernel test robot noticed the following build warnings:
Hi Heiko,
kernel test robot noticed the following build warnings:
[auto build test WARNING on 70075e3d0ca0b72cc983d03f7cd9796e43492980]
url: https://github.com/intel-lab-lkp/linux/commits/Heiko-Carstens/kbuild-Require-gcc-9-for-s390/20251209-202647 base: 70075e3d0ca0b72cc983d03f7cd9796e43492980 patch link: https://lore.kernel.org/r/20251209121701.1856271-7-hca%40linux.ibm.com patch subject: [PATCH 6/9] s390/bug: Implement __WARN_printf() config: s390-randconfig-r062-20251210 (https://download.01.org/0day-ci/archive/20251210/[email protected]/config) compiler: s390-linux-gcc (GCC) 14.3.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20251210/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
>> arch/s390/kernel/traps.c:224:26: warning: 'struct arch_va_list' declared inside parameter list will not be visible outside of this definition or declaration
224 | void *__warn_args(struct arch_va_list *args, struct pt_regs *regs)
| ^~~~~~~~~~~~
>> arch/s390/kernel/traps.c:224:7: warning: no previous prototype for '__warn_args' [-Wmissing-prototypes]
224 | void *__warn_args(struct arch_va_list *args, struct pt_regs *regs)
| ^~~~~~~~~~~
arch/s390/kernel/traps.c: In function '__warn_args':
arch/s390/kernel/traps.c:243:13: error: invalid use of undefined type 'struct arch_va_list'
243 | args->__overflow_arg_area = stack_frame + 1;
| ^~
arch/s390/kernel/traps.c:244:13: error: invalid use of undefined type 'struct arch_va_list'
244 | args->__reg_save_area = regs->gprs;
| ^~
arch/s390/kernel/traps.c:245:13: error: invalid use of undefined type 'struct arch_va_list'
245 | args->__gpr = 1;
| ^~vim +224 arch/s390/kernel/traps.c
223
> 224 void *__warn_args(struct arch_va_list *args, struct pt_regs *regs)
225 {
226 struct stack_frame *stack_frame;
227
228 /*
229 * Generate va_list from pt_regs. See ELF Application Binary Interface
230 * s390x Supplement documentation for details.
231 *
232 * - __overflow_arg_area needs to point to the parameter area, which
233 * is right above the standard stack frame (160 bytes)
234 *
235 * - __reg_save_area needs to point to a register save area where
236 * general registers (%r2 - %r6) can be found at offset 16. Which
237 * means that the gprs save area of pt_regs can be used
238 *
239 * - __gpr must be set to one, since the first parameter has been
240 * processed (pointer to bug_entry)
241 */
242 stack_frame = (struct stack_frame *)regs->gprs[15];
243 args->__overflow_arg_area = stack_frame + 1;
244 args->__reg_save_area = regs->gprs;
245 args->__gpr = 1;
246 return args;
247 }
248
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
kernel test robot
build botRe: [PATCH 6/9] s390/bug: Implement __WARN_printf()
In reply to Heiko Carstenskernel test robot noticed the following build errors:
Hi Heiko,
kernel test robot noticed the following build errors:
[auto build test ERROR on 70075e3d0ca0b72cc983d03f7cd9796e43492980]
url: https://github.com/intel-lab-lkp/linux/commits/Heiko-Carstens/kbuild-Require-gcc-9-for-s390/20251209-202647 base: 70075e3d0ca0b72cc983d03f7cd9796e43492980 patch link: https://lore.kernel.org/r/20251209121701.1856271-7-hca%40linux.ibm.com patch subject: [PATCH 6/9] s390/bug: Implement __WARN_printf() config: s390-randconfig-r062-20251210 (https://download.01.org/0day-ci/archive/20251210/[email protected]/config) compiler: s390-linux-gcc (GCC) 14.3.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20251210/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
arch/s390/kernel/traps.c:224:26: warning: 'struct arch_va_list' declared inside parameter list will not be visible outside of this definition or declaration
224 | void *__warn_args(struct arch_va_list *args, struct pt_regs *regs)
| ^~~~~~~~~~~~
arch/s390/kernel/traps.c:224:7: warning: no previous prototype for '__warn_args' [-Wmissing-prototypes]
224 | void *__warn_args(struct arch_va_list *args, struct pt_regs *regs)
| ^~~~~~~~~~~
arch/s390/kernel/traps.c: In function '__warn_args':
>> arch/s390/kernel/traps.c:243:13: error: invalid use of undefined type 'struct arch_va_list'
243 | args->__overflow_arg_area = stack_frame + 1;
| ^~
arch/s390/kernel/traps.c:244:13: error: invalid use of undefined type 'struct arch_va_list'
244 | args->__reg_save_area = regs->gprs;
| ^~
arch/s390/kernel/traps.c:245:13: error: invalid use of undefined type 'struct arch_va_list'
245 | args->__gpr = 1;
| ^~vim +243 arch/s390/kernel/traps.c
223
> 224 void *__warn_args(struct arch_va_list *args, struct pt_regs *regs)
225 {
226 struct stack_frame *stack_frame;
227
228 /*
229 * Generate va_list from pt_regs. See ELF Application Binary Interface
230 * s390x Supplement documentation for details.
231 *
232 * - __overflow_arg_area needs to point to the parameter area, which
233 * is right above the standard stack frame (160 bytes)
234 *
235 * - __reg_save_area needs to point to a register save area where
236 * general registers (%r2 - %r6) can be found at offset 16. Which
237 * means that the gprs save area of pt_regs can be used
238 *
239 * - __gpr must be set to one, since the first parameter has been
240 * processed (pointer to bug_entry)
241 */
242 stack_frame = (struct stack_frame *)regs->gprs[15];
> 243 args->__overflow_arg_area = stack_frame + 1; 244 args->__reg_save_area = regs->gprs;
245 args->__gpr = 1;
246 return args;
247 }
248
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Heiko Carstens
maintainerPATCH 7/9 s390/bug: Implement WARN_ONCE()
In reply to Heiko CarstensThis is the s390 variant of commit 11bb4944f014 ("x86/bug: Implement WARN_ONCE()").
--- arch/s390/include/asm/bug.h | 11 +++++++++++ 1 file changed, 11 insertions(+)
arch/s390/include/asm/bug.h +11 -0
@@ -104,6 +104,17 @@ do { \ #define __WARN_printf(taint, fmt, arg...) \ __WARN_print_arg(BUGFLAG_TAINT(taint), fmt, ## arg) +#define WARN_ONCE(cond, format, arg...) \ +({ \ + int __ret_warn_on = !!(cond); \ + \ + if (unlikely(__ret_warn_on)) { \ + __WARN_print_arg(BUGFLAG_ONCE|BUGFLAG_TAINT(TAINT_WARN),\ + format, ## arg); \ + } \ + __ret_warn_on; \ +}) +#define HAVE_ARCH_BUG #define HAVE_ARCH_BUG_FORMAT #define HAVE_ARCH_BUG_FORMAT_ARGS
Signature · 1 line
2.51.0
Heiko Carstens
maintainerPATCH 8/9 s390/bug: Skip __WARN_trap() in call traces
In reply to Heiko CarstensIn order to avoid rather pointless warning disassemblies of __WARN_trap() set the PSW address to the return address of the function which called __WARN_trap(). This is the address to which __WARN_trap() would return in any case.
The result is a disassembly of the function which called __WARN_trap(), which is much more helpful.
Before:
WARNING: arch/s390/kernel/setup.c:1017 at foobar+0x2c/0x20, CPU#0: swapper/0/0
...
Krnl PSW : 0704c00180000000 000003ffe0f675f4 (__WARN_trap+0x4/0x10)
...
Krnl Code: 000003ffe0f675ec: 0707 bcr 0,%r7
000003ffe0f675ee: 0707 bcr 0,%r7
*000003ffe0f675f0: af000001 mc 1,0
>000003ffe0f675f4: 07fe bcr 15,%r14
000003ffe0f675f6: 47000700 bc 0,1792
000003ffe0f675fa: 0707 bcr 0,%r7
000003ffe0f675fc: 0707 bcr 0,%r7
000003ffe0f675fe: 0707 bcr 0,%r7
Call Trace:
[<000003ffe0f675f4>] __WARN_trap+0x4/0x10
[<000003ffe185bc2e>] arch_cpu_finalize_init+0x26/0x60
[<000003ffe185654c>] start_kernel+0x53c/0x5d8
[<000003ffe010002e>] startup_continue+0x2e/0x40Afterwards:
WARNING: arch/s390/kernel/setup.c:1017 at foobar+0x12/0x30, CPU#0: swapper/0/0
...
Krnl PSW : 0704c00180000000 000003ffe185bc2e (arch_cpu_finalize_init+0x26/0x60)
...
Krnl Code: 000003ffe185bc1c: e3f0ff98ff71 lay %r15,-104(%r15)
000003ffe185bc22: e3e0f0980024 stg %r14,152(%r15)
*000003ffe185bc28: c0e5ff45ed94 brasl %r14,000003ffe0119750
>000003ffe185bc2e: c0e5ffa052b9 brasl %r14,000003ffe0c661a0
000003ffe185bc34: c020fffe86d6 larl %r2,000003ffe182c9e0
000003ffe185bc3a: e548f0a80006 mvghi 168(%r15),6
000003ffe185bc40: e548f0a00005 mvghi 160(%r15),5
000003ffe185bc46: a7690004 lghi %r6,4
Call Trace:
[<000003ffe185bc2e>] arch_cpu_finalize_init+0x26/0x60
[<000003ffe185654c>] start_kernel+0x53c/0x5d8
[<000003ffe010002e>] startup_continue+0x2e/0x40--- arch/s390/kernel/traps.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
arch/s390/kernel/traps.c +4 -2
@@ -252,10 +252,12 @@ static void monitor_event_exception(struct pt_regs *regs)if (user_mode (regs))return ; -if (regs->monitor_code ==MONCODE_BUG_ARG ) +if (regs->monitor_code ==MONCODE_BUG_ARG ) { + regs->psw.addr = regs->gprs[14 ]; btt =report_bug_entry ((struct bug_entry *)regs->gprs[2 ], regs); -else + }else { btt =report_bug (regs->psw.addr - (regs->int_code >>16 ), regs); + }switch (btt) {case BUG_TRAP_TYPE_NONE :fixup_exception (regs);
Signature · 1 line
2.51.0
Heiko Carstens
maintainerPATCH 9/9 s390/bug: Prevent tail-call optimization
In reply to Heiko CarstensFor the exception based __WARN_trap() implementation it is technically not necessary to prevent tail-call optimization, however it may be confusing to see warning messages like:
WARNING: arch/s390/kernel/setup.c:1017 at foobar+0x2c/0x50, CPU#0: swapper/0/0
together with a disassembly of a different function caused by tail-call optimaziation for the __WARN_trap() call. Prevent that by adding an empty asm statement. This generates slightly worse code, but should hopefully avoid confusion.
With this the output looks like:
WARNING: arch/s390/kernel/setup.c:1017 at foobar+0x2c/0x50, CPU#0: swapper/0/0
...
Krnl PSW : 0704c00180000000 000003ffe0119788 (foobar+0x38/0x50)
...
Krnl Code: 000003ffe0119776: e3e0f0980024 stg %r14,152(%r15)
000003ffe011977c: c02000b8992a larl %r2,000003ffe182c9d0
*000003ffe0119782: c0e5007270b7 brasl %r14,000003ffe0f678f0
>000003ffe0119788: ebeff0a00004 lmg %r14,%r15,160(%r15)
000003ffe011978e: 07fe bcr 15,%r14
000003ffe0119790: 47000700 bc 0,1792
000003ffe0119794: 0707 bcr 0,%r7
000003ffe0119796: 0707 bcr 0,%r7
Call Trace:
[<000003ffe0119788>] foobar+0x38/0x50
[<000003ffe185bc2e>] arch_cpu_finalize_init+0x26/0x60
[<000003ffe185654c>] start_kernel+0x53c/0x5d8
[<000003ffe010002e>] startup_continue+0x2e/0x40A better solution would be to replace or patch the branch instruction to __WARN_trap() with the monitor call instruction, similar to what is done for x86 [1]. However s390 does not support static_cond_calls(). Therefore use the simple approach for the time being.
[1] commit 860238af7a33 ("x86_64/bug: Inline the UD1")
--- arch/s390/include/asm/bug.h | 2 ++ 1 file changed, 2 insertions(+)
arch/s390/include/asm/bug.h +2 -0
@@ -99,6 +99,8 @@ do { \int __flags = (flags) |BUGFLAG_WARNING |BUGFLAG_ARGS ; \ \ __WARN_trap(__WARN_bug_entry(__flags, format), ## arg); \ +/* prevent tail-call optimization */ \ + asm("" ); \ }while (0 ) #define __WARN_printf(taint, fmt, arg...) \
Signature · 1 line
2.51.0
Re: [PATCH 9/9] s390/bug: Prevent tail-call optimization
Quoted 10 lines · Heiko Carstens
For the exception based
__WARN_trap()implementation it is technically not necessary to prevent tail-call optimization, however it may be confusing to see warning messages like:WARNING: arch/s390/kernel/setup.c:1017 at foobar+0x2c/0x50, CPU#0: swapper/0/0together with a disassembly of a different function caused by tail-call optimaziation for the
__WARN_trap()call. Prevent that by adding an empty asm statement. This generates slightly worse code, but should hopefully avoid confusion.
Aah, because:
bar() foo() __WARN_trap()
when foo() does a tail-call, your link reg points to bar() and not the expected foo().
And at this point you don't have enough clues to conditionally do that psw/r14 fixup either.
Oh well.
Quoted 20 lines
With this the output looks like:
WARNING: arch/s390/kernel/setup.c:1017 at foobar+0x2c/0x50, CPU#0: swapper/0/0 ... Krnl PSW : 0704c00180000000 000003ffe0119788 (foobar+0x38/0x50) ... Krnl Code: 000003ffe0119776: e3e0f0980024 stg %r14,152(%r15) 000003ffe011977c: c02000b8992a larl %r2,000003ffe182c9d0 *000003ffe0119782: c0e5007270b7 brasl %r14,000003ffe0f678f0 >000003ffe0119788: ebeff0a00004 lmg %r14,%r15,160(%r15) 000003ffe011978e: 07fe bcr 15,%r14 000003ffe0119790: 47000700 bc 0,1792 000003ffe0119794: 0707 bcr 0,%r7 000003ffe0119796: 0707 bcr 0,%r7 Call Trace: [<000003ffe0119788>] foobar+0x38/0x50 [<000003ffe185bc2e>] arch_cpu_finalize_init+0x26/0x60 [<000003ffe185654c>] start_kernel+0x53c/0x5d8 [<000003ffe010002e>] startup_continue+0x2e/0x40
Quoted 4 lines
A better solution would be to replace or patch the branch instruction to
__WARN_trap()with the monitor call instruction, similar to what is done for x86 [1]. However s390 does not supportstatic_cond_calls(). Therefore use the simple approach for the time being.
Right, and no objtool for you either :/ Because all you need is something to find all the __WARN_trap() callsites and stick them in a section.
Quoted 23 lines
[1] commit
860238af7a33("x86_64/bug: Inline the UD1")Signed-off-by Heiko Carstens <[email protected]>--- arch/s390/include/asm/bug.h | 2 ++ 1 file changed, 2 insertions(+)arch/s390/include/asm/bug.h +2
-0@@ -99,6 +99,8 @@ do { \int __flags = (flags) |BUGFLAG_WARNING |BUGFLAG_ARGS ; \ \ __WARN_trap(__WARN_bug_entry(__flags, format), ## arg); \ +/* prevent tail-call optimization */ \ + asm("" ); \ }while (0 ) #define __WARN_printf(taint, fmt, arg...) \Signature · 1 line
2.51.0
Quoted 7 lines · Heiko Carstens
Use the generic infrastructure introduced by Peter Zijlstra [1] to implement an exception based
WARN()andWARN_ONCE()similar to x86.Due to some compiler oddities on s390 this requires to raise the minimum gcc version to 9. Maybe there are ways to avoid this, but I failed to find a working solution. Details are in the patch descriptions.
Quoted 8 lines
Just posting this now to also get some compile bot testing, since I'm afraid there might be some compiler version / config option around where even this new approach breaks.
Peter, since you were wondering: your generic infrastructure pieces work very nice. Looking at the x86 and s390 implementation: it might be possible to make things even more generic since both
__WARN_printf(), andWARN_ONCE()are identical; it looks like only__WARN_print_arg()needs to be provided.
Nice! and yeah, perhaps we can unify that later.