Skip to contentSkip to conversation

objtool: Fix STACK_FRAME_NON_STANDARD reloc type

PATCH Superseded Peter Zijlstra 3 participants 10 messages Apr 29, 2022 – May 4, 2022 objtool

lore
Share & tools
lore

Peter Zijlstra

author maintainer
#0 lore raw

PATCH 1/1 objtool: Fix STACK_FRAME_NON_STANDARD reloc type

STACK_FRAME_NON_STANDARD results in inconsistent relocation types depending on .c or .S usage:

Relocation section '.rela.discard.func_stack_frame_non_standard' at offset 0x3c01090 contains 5 entries:
Offset Info Type Symbol's Value Symbol's Name + Addend

  0000000000000000  00020c2200000002 R_X86_64_PC32          0000000000047b40 do_suspend_lowlevel + 0
  0000000000000008  0002461e00000001 R_X86_64_64            00000000000480a0 machine_real_restart + 0
  0000000000000010  0000001400000001 R_X86_64_64            0000000000000000 .rodata + b3d4
  0000000000000018  0002444600000002 R_X86_64_PC32          00000000000678a0 __efi64_thunk + 0
  0000000000000020  0002659d00000001 R_X86_64_64            0000000000113160 __crash_kexec + 0
---
 include/linux/objtool.h |    6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
include/linux/objtool.h +5 -1
@@ -137,7 +137,11 @@ struct unwind_hint {
 
 .macro STACK_FRAME_NON_STANDARD func:req
 	.pushsection .discard.func_stack_frame_non_standard, "aw"
-		.long \func - .
+#ifdef CONFIG_64BIT
+		.quad \func
+#else
+		.long \func
+#endif
 	.popsection
 .endm
 
author maintainer
#1 lore raw
Quoted 11 lines · Peter Zijlstra

STACK_FRAME_NON_STANDARD results in inconsistent relocation types depending on .c or .S usage:

Relocation section '.rela.discard.func_stack_frame_non_standard' at offset 0x3c01090 contains 5 entries:
Offset Info Type Symbol's Value Symbol's Name + Addend

  0000000000000000  00020c2200000002 R_X86_64_PC32          0000000000047b40 do_suspend_lowlevel + 0
  0000000000000008  0002461e00000001 R_X86_64_64            00000000000480a0 machine_real_restart + 0
  0000000000000010  0000001400000001 R_X86_64_64            0000000000000000 .rodata + b3d4
  0000000000000018  0002444600000002 R_X86_64_PC32          00000000000678a0 __efi64_thunk + 0
  0000000000000020  0002659d00000001 R_X86_64_64            0000000000113160 __crash_kexec + 0

So that weird .rodata entry is optprobe_template_func.

It being in .rodata also means it's not validated and there is no ORC data generated, is that all intentional? The changelog for:

877b145f0f47 ("x86/kprobes: Move trampoline code into RODATA")

doesn't really say anything useful about any of that :/

I also don't see any kprobe/optprobe hooks in unwind.h, so what happens if we hit an optprobe?

#2 lore raw
Quoted 18 lines · Peter Zijlstra
Quoted 11 lines · Peter Zijlstra

STACK_FRAME_NON_STANDARD results in inconsistent relocation types depending on .c or .S usage:

Relocation section '.rela.discard.func_stack_frame_non_standard' at offset 0x3c01090 contains 5 entries:
Offset Info Type Symbol's Value Symbol's Name + Addend

  0000000000000000  00020c2200000002 R_X86_64_PC32          0000000000047b40 do_suspend_lowlevel + 0
  0000000000000008  0002461e00000001 R_X86_64_64            00000000000480a0 machine_real_restart + 0
  0000000000000010  0000001400000001 R_X86_64_64            0000000000000000 .rodata + b3d4
  0000000000000018  0002444600000002 R_X86_64_PC32          00000000000678a0 __efi64_thunk + 0
  0000000000000020  0002659d00000001 R_X86_64_64            0000000000113160 __crash_kexec + 0

So that weird .rodata entry is optprobe_template_func.

It being in .rodata also means it's not validated and there is no ORC data generated, is that all intentional? The changelog for:

Quoted 6 lines

877b145f0f47 ("x86/kprobes: Move trampoline code into RODATA")

doesn't really say anything useful about any of that :/

I also don't see any kprobe/optprobe hooks in unwind.h, so what happens if we hit an optprobe?

Same as for any other generated code, the unwinder will try to fall back to frame pointers, and if that doesn't work, the unwind stops.

That commit didn't change anything since it was already not being directly executed anyway, but rather used to generate code on the fly.

And before that commit it was being ignored by ORC anyway, thanks to STACK_FRAME_NON_STANDARD. Which can now be removed since this code is now data and objtool will no longer try to understand it.

Signature · 1 line
Josh
author maintainer
#3 lore raw
Quoted 29 lines · Josh Poimboeuf
Quoted 18 lines · Peter Zijlstra
Quoted 11 lines · Peter Zijlstra

STACK_FRAME_NON_STANDARD results in inconsistent relocation types depending on .c or .S usage:

Relocation section '.rela.discard.func_stack_frame_non_standard' at offset 0x3c01090 contains 5 entries:
Offset Info Type Symbol's Value Symbol's Name + Addend

  0000000000000000  00020c2200000002 R_X86_64_PC32          0000000000047b40 do_suspend_lowlevel + 0
  0000000000000008  0002461e00000001 R_X86_64_64            00000000000480a0 machine_real_restart + 0
  0000000000000010  0000001400000001 R_X86_64_64            0000000000000000 .rodata + b3d4
  0000000000000018  0002444600000002 R_X86_64_PC32          00000000000678a0 __efi64_thunk + 0
  0000000000000020  0002659d00000001 R_X86_64_64            0000000000113160 __crash_kexec + 0

So that weird .rodata entry is optprobe_template_func.

It being in .rodata also means it's not validated and there is no ORC data generated, is that all intentional? The changelog for:

Quoted 6 lines

877b145f0f47 ("x86/kprobes: Move trampoline code into RODATA")

doesn't really say anything useful about any of that :/

I also don't see any kprobe/optprobe hooks in unwind.h, so what happens if we hit an optprobe?

Same as for any other generated code, the unwinder will try to fall back to frame pointers, and if that doesn't work, the unwind stops.

Quoted 6 lines

That commit didn't change anything since it was already not being directly executed anyway, but rather used to generate code on the fly.

And before that commit it was being ignored by ORC anyway, thanks to STACK_FRAME_NON_STANDARD. Which can now be removed since this code is now data and objtool will no longer try to understand it.

Right; but I suppose I'm wondering if we should fix this. It seems a rather sub-optimal state of affairs.

#4 lore raw
Quoted 9 lines · Peter Zijlstra
Quoted 9 lines
Quoted 2 lines

I also don't see any kprobe/optprobe hooks in unwind.h, so what happens if we hit an optprobe?

Same as for any other generated code, the unwinder will try to fall back to frame pointers, and if that doesn't work, the unwind stops.

That commit didn't change anything since it was already not being directly executed anyway, but rather used to generate code on the fly.

Quoted 6 lines
Quoted 3 lines

And before that commit it was being ignored by ORC anyway, thanks to STACK_FRAME_NON_STANDARD. Which can now be removed since this code is now data and objtool will no longer try to understand it.

Right; but I suppose I'm wondering if we should fix this. It seems a rather sub-optimal state of affairs.

Masami recently fixed some kprobes ORC issues but I don't know if this one was fixed.

As to the whether it's worth fixing, I dunno. There are trade offs.

Depends on how common the stack trace is -- I'm guessing not very, since I've never seen a bug report -- and how important it is to get to full ORC coverage. If our goal is full coverage, we'd need a way for generated code to add/remove ORC entries.

Signature · 1 line
Josh
#5 lore raw
Quoted 9 lines · On Mon, 2 May 2022 10:59:21 -0700 Josh Poimboeuf <[email protected]> wrote:
Quoted 8 lines · Peter Zijlstra
Quoted 8 lines
Quoted 2 lines

I also don't see any kprobe/optprobe hooks in unwind.h, so what happens if we hit an optprobe?

Same as for any other generated code, the unwinder will try to fall back to frame pointers, and if that doesn't work, the unwind stops.

That commit didn't change anything since it was already not being directly executed anyway, but rather used to generate code on the fly.

Ah, OK. So ORC will not work on the dynamically generated trampoline code. Can we generate ORC information entry dynamically?
(E.g. copying ORC data from the original code)

Quoted 10 lines
Quoted 7 lines
Quoted 4 lines

And before that commit it was being ignored by ORC anyway, thanks to STACK_FRAME_NON_STANDARD. Which can now be removed since this code is now data and objtool will no longer try to understand it.

Right; but I suppose I'm wondering if we should fix this. It seems a rather sub-optimal state of affairs.

Masami recently fixed some kprobes ORC issues but I don't know if this one was fixed.

I've fixed the kretprobe ORC unwinder issue. I need to check the optprobe case too.

Quoted 7 lines

As to the whether it's worth fixing, I dunno. There are trade offs.

Depends on how common the stack trace is -- I'm guessing not very, since I've never seen a bug report -- and how important it is to get to full ORC coverage. If our goal is full coverage, we'd need a way for generated code to add/remove ORC entries.

Agreed, if I can copy the ORC entries for the original code to the entries for generated code, I can fix it.

Thank you,

Quoted 4 lines
Signature · 2 lines
-- 
Josh
Signature · 1 line
Masami Hiramatsu <[email protected]>
#6 lore raw
In reply to Peter Zijlstra
Quoted 13 lines · On Fri, 29 Apr 2022 14:00:44 +0200 Peter Zijlstra <[email protected]> wrote:
Quoted 11 lines · Peter Zijlstra

STACK_FRAME_NON_STANDARD results in inconsistent relocation types depending on .c or .S usage:

Relocation section '.rela.discard.func_stack_frame_non_standard' at offset 0x3c01090 contains 5 entries:
Offset Info Type Symbol's Value Symbol's Name + Addend

  0000000000000000  00020c2200000002 R_X86_64_PC32          0000000000047b40 do_suspend_lowlevel + 0
  0000000000000008  0002461e00000001 R_X86_64_64            00000000000480a0 machine_real_restart + 0
  0000000000000010  0000001400000001 R_X86_64_64            0000000000000000 .rodata + b3d4
  0000000000000018  0002444600000002 R_X86_64_PC32          00000000000678a0 __efi64_thunk + 0
  0000000000000020  0002659d00000001 R_X86_64_64            0000000000113160 __crash_kexec + 0
Quoted 8 lines

So that weird .rodata entry is optprobe_template_func.

It being in .rodata also means it's not validated and there is no ORC data generated, is that all intentional? The changelog for:

877b145f0f47 ("x86/kprobes: Move trampoline code into RODATA")

doesn't really say anything useful about any of that :/

This commit was introduced just for reducing attack surface (the trampoline code is NOT executed but just copied into trampoline buffers), but if the ORC unwinder doesn't work correctly, please revert it.
I think there is no functional change.

Thanks,

Signature · 1 line
Masami Hiramatsu <[email protected]>
#7 lore raw
In reply to Peter Zijlstra
Quoted 21 lines · Peter Zijlstra

STACK_FRAME_NON_STANDARD results in inconsistent relocation types depending on .c or .S usage:

Relocation section '.rela.discard.func_stack_frame_non_standard' at offset 0x3c01090 contains 5 entries:
Offset Info Type Symbol's Value Symbol's Name + Addend

  0000000000000000  00020c2200000002 R_X86_64_PC32          0000000000047b40 do_suspend_lowlevel + 0
  0000000000000008  0002461e00000001 R_X86_64_64            00000000000480a0 machine_real_restart + 0
  0000000000000010  0000001400000001 R_X86_64_64            0000000000000000 .rodata + b3d4
  0000000000000018  0002444600000002 R_X86_64_PC32          00000000000678a0 __efi64_thunk + 0
  0000000000000020  0002659d00000001 R_X86_64_64            0000000000113160 __crash_kexec + 0
---
 include/linux/objtool.h |    6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
include/linux/objtool.h +0 -0
@@ -137,7 +137,11 @@ struct unwind_hint {
 

include/linux/objtool.h struct unwind_hint { · line 146

Quoted 10 lines
include/linux/objtool.h +5 -1
 .macro STACK_FRAME_NON_STANDARD func:req
 	.pushsection .discard.func_stack_frame_non_standard, "aw"
-		.long \func - .
+#ifdef CONFIG_64BIT
+		.quad \func
+#else
+		.long \func
+#endif
 	.popsection
 .endm

Can use _ASM_PTR here, and objtool.h needs synced to tools.

Signature · 1 line
Josh
author maintainer
#8 lore raw

PATCH v2 1/1 objtool: Fix STACK_FRAME_NON_STANDARD reloc type

Quoted 1 line · Josh Poimboeuf

Can use _ASM_PTR here, and objtool.h needs synced to tools.

Here goes..

---
Subject: objtool: Fix STACK_FRAME_NON_STANDARD reloc type
From: Peter Zijlstra <[email protected]>
Date: Tue Apr 26 17:08:53 CEST 2022

STACK_FRAME_NON_STANDARD results in inconsistent relocation types depending on .c or .S usage:

Relocation section '.rela.discard.func_stack_frame_non_standard' at offset 0x3c01090 contains 5 entries:
Offset Info Type Symbol's Value Symbol's Name + Addend

  0000000000000000  00020c2200000002 R_X86_64_PC32          0000000000047b40 do_suspend_lowlevel + 0
  0000000000000008  0002461e00000001 R_X86_64_64            00000000000480a0 machine_real_restart + 0
  0000000000000010  0000001400000001 R_X86_64_64            0000000000000000 .rodata + b3d4
  0000000000000018  0002444600000002 R_X86_64_PC32          00000000000678a0 __efi64_thunk + 0
  0000000000000020  0002659d00000001 R_X86_64_64            0000000000113160 __crash_kexec + 0
---
 include/linux/objtool.h       |    2 +-
 tools/include/linux/objtool.h |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
include/linux/objtool.h +1 -1
@@ -137,7 +137,7 @@ struct unwind_hint {
 
 .macro STACK_FRAME_NON_STANDARD func:req
 	.pushsection .discard.func_stack_frame_non_standard, "aw"
-		.long \func - .
+	_ASM_PTR \func
 	.popsection
 .endm
 
tools/include/linux/objtool.h +1 -1
@@ -137,7 +137,7 @@ struct unwind_hint {
 
 .macro STACK_FRAME_NON_STANDARD func:req
 	.pushsection .discard.func_stack_frame_non_standard, "aw"
-		.long \func - .
+	_ASM_PTR \func
 	.popsection
 .endm
 
#9 lore raw
Quoted 30 lines · Peter Zijlstra
Quoted 1 line · Josh Poimboeuf

Can use _ASM_PTR here, and objtool.h needs synced to tools.

Here goes..

---
Subject: objtool: Fix STACK_FRAME_NON_STANDARD reloc type
From: Peter Zijlstra <[email protected]>
Date: Tue Apr 26 17:08:53 CEST 2022

STACK_FRAME_NON_STANDARD results in inconsistent relocation types depending on .c or .S usage:

Relocation section '.rela.discard.func_stack_frame_non_standard' at offset 0x3c01090 contains 5 entries:
Offset Info Type Symbol's Value Symbol's Name + Addend

  0000000000000000  00020c2200000002 R_X86_64_PC32          0000000000047b40 do_suspend_lowlevel + 0
  0000000000000008  0002461e00000001 R_X86_64_64            00000000000480a0 machine_real_restart + 0
  0000000000000010  0000001400000001 R_X86_64_64            0000000000000000 .rodata + b3d4
  0000000000000018  0002444600000002 R_X86_64_PC32          00000000000678a0 __efi64_thunk + 0
  0000000000000020  0002659d00000001 R_X86_64_64            0000000000113160 __crash_kexec + 0
---
 include/linux/objtool.h       |    2 +-
 tools/include/linux/objtool.h |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

include/linux/objtool.h struct unwind_hint { · line 142

Quoted 7 lines
include/linux/objtool.h +1 -1
@@ -137,7 +137,7 @@ struct unwind_hint {
 
 .macro STACK_FRAME_NON_STANDARD func:req
 	.pushsection .discard.func_stack_frame_non_standard, "aw"
-		.long \func - .
+	_ASM_PTR \func
 	.popsection
 .endm

This file needs to include <asm/asm.h>, otherwise you get:

arch/x86/kernel/acpi/wakeup_64.S: Assembler messages:
arch/x86/kernel/acpi/wakeup_64.S:132: Error: no such instruction: `_asm_ptr do_suspend_lowlevel'

Signature · 1 line
Josh