arm64 sigreturn unwinding fixes

For better or worse, GDB relies on the exact instruction sequence in the VDSO sigreturn trampoline in order to unwind from signals correctly. Commit c91db232da48 ("arm64: vdso: Convert to modern assembler annotations") unfortunately added a BTI C instruction to the start of __kernel_rt_sigreturn, which breaks this check. Thankfully, it's also not required, since the trampoline is called from a RET instruction when returning from the signal handler

For better or worse, GDB relies on the exact instruction sequence in the VDSO sigreturn trampoline in order to unwind from signals correctly. Commit c91db232da48 ("arm64: vdso: Convert to modern assembler annotations") unfortunately added a BTI C instruction to the start of __kernel_rt_sigreturn, which breaks this check. Thankfully, it's also not required, since the trampoline is called from a RET instruction when returning from the signal handler

Remove the unnecessary BTI C instruction from __kernel_rt_sigreturn, and do the same for the 32-bit VDSO as well for good measure.

Cc Dave Martin <[email protected]>

Cc Mark Brown <[email protected]>

Cc Daniel Kiss <[email protected]>

Cc Tamas Zsoldos <[email protected]>

Fixes c91db232da48 ("arm64: vdso: Convert to modern assembler annotations")

Signed-off-by Will Deacon <[email protected]>

---
 arch/arm64/kernel/vdso/sigreturn.S   | 11 +++++++++--
 arch/arm64/kernel/vdso32/sigreturn.S | 16 ++++++++--------
 2 files changed, 17 insertions(+), 10 deletions(-)

arch/arm64/kernel/vdso/sigreturn.S +0 -0

index 3fb13b81f780..0c921130002a 100644
@@ -15,7 +15,14 @@
 	.text

arch/arm64/kernel/vdso/sigreturn.S +2 -1

+SYM_CODE_START(__kernel_rt_sigreturn)
 	.cfi_startproc
 	.cfi_signal_frame
 	.cfi_def_cfa	x29, 0

@@ -24,6 +31,6 @@ SYM_FUNC_START(__kernel_rt_sigreturn)
 	mov	x8, #__NR_rt_sigreturn
 	svc	#0
 	.cfi_endproc
-SYM_FUNC_END(__kernel_rt_sigreturn)
+SYM_CODE_END(__kernel_rt_sigreturn)
 
 emit_aarch64_feature_1_and

arch/arm64/kernel/vdso32/sigreturn.S +0 -0

index 620524969696..b36d4e2267a3 100644

Quoted 29 lines · Will Deacon

For better or worse, GDB relies on the exact instruction sequence in the VDSO sigreturn trampoline in order to unwind from signals correctly. Commit c91db232da48 ("arm64: vdso: Convert to modern assembler annotations") unfortunately added a BTI C instruction to the start of __kernel_rt_sigreturn, which breaks this check. Thankfully, it's also not required, since the trampoline is called from a RET instruction when returning from the signal handler

Remove the unnecessary BTI C instruction from __kernel_rt_sigreturn, and do the same for the 32-bit VDSO as well for good measure.

Cc Dave Martin <[email protected]>

Cc Mark Brown <[email protected]>

Cc Daniel Kiss <[email protected]>

Cc Tamas Zsoldos <[email protected]>

Fixes c91db232da48 ("arm64: vdso: Convert to modern assembler annotations")

Signed-off-by Will Deacon <[email protected]>

---
 arch/arm64/kernel/vdso/sigreturn.S   | 11 +++++++++--
 arch/arm64/kernel/vdso32/sigreturn.S | 16 ++++++++--------
 2 files changed, 17 insertions(+), 10 deletions(-)

arch/arm64/kernel/vdso/sigreturn.S +0 -0

index 3fb13b81f780..0c921130002a 100644
@@ -15,7 +15,14 @@
 	.text
 

arch/arm64/kernel/vdso/sigreturn.S line 24

Quoted 9 lines

arch/arm64/kernel/vdso/sigreturn.S +7 -1

 	nop
-SYM_FUNC_START(__kernel_rt_sigreturn)
+/*
+ * GDB relies on being able to identify the sigreturn instruction sequence to
+ * unwind from signal handlers. We cannot, therefore, use SYM_FUNC_START()
+ * here, as it will emit a BTI C instruction and break the unwinder. Thankfully,
+ * this function is only ever called from a RET and so omitting the landing pad
+ * is perfectly fine.
+ */

Can we cross-reference or duplicate (perhaps abridged) this comment for vdso32?

Can we also fix the comment by the definition of SYM_FUNC_START()?

arch/arm64/kernel/vdso32/sigreturn.S line 20

Quoted 10 lines

arch/arm64/kernel/vdso32/sigreturn.S +1 -1

index 620524969696..b36d4e2267a3 100644
@@ -17,39 +17,39 @@
 	.save {r0-r15}
 	.pad #COMPAT_SIGFRAME_REGS_OFFSET
 	nop
-SYM_FUNC_START(__kernel_sigreturn_arm)
+SYM_CODE_START(__kernel_sigreturn_arm)

...although do we actually need this? 32-bit doesn't have BTI.

But for the reasons given above, this is not a "function" and so SYM_FUNC_START() is trap for future maintenance even if it makes no difference now.

Either way,

Reviewed-by Dave Martin <[email protected]>

Daniel reports that the .cfi_startproc is misplaced for the sigreturn trampoline, which causes LLVM's unwinder to misbehave:

| I run into this with LLVM’s unwinder.
| This combination was always broken.

This prompted Dave to question our use of CFI directives more generally, and I ended up going down a rabbit hole trying to figure out how this very poorly documented stuff gets used.

Move the CFI directives so that the "mysterious NOP" is included in the .cfi_{start,end}proc block and add a bunch of comments so that I can save myself another headache in future.

Cc Tamas Zsoldos <[email protected]>

Reported-by Dave Martin <[email protected]>

Reported-by Daniel Kiss <[email protected]>

Signed-off-by Will Deacon <[email protected]>

---
 arch/arm64/kernel/vdso/sigreturn.S | 40 ++++++++++++++++++++++++------
 1 file changed, 33 insertions(+), 7 deletions(-)

arch/arm64/kernel/vdso/sigreturn.S +14 -2

index 0c921130002a..cb47dfb3bd5a 100644
@@ -1,7 +1,11 @@
 /* SPDX-License-Identifier: GPL-2.0-only */
 /*
  * Sigreturn trampoline for returning from a signal when the SA_RESTORER
- * flag is not set.
+ * flag is not set. It serves primarily as a hall of shame for crappy
+ * unwinders and features an exciting but mysterious NOP instruction.
+ *
+ * It's also fragile as hell, so please think twice before changing anything
+ * in here.
  *
  * Copyright (C) 2012 ARM Limited
  *

@@ -14,7 +18,34 @@
 
 	.text
 
-	nop
+/* Ensure that the mysterious NOP can be associated with a function. */
+	.cfi_startproc
+
+/*
+ * .cfi_signal_frame causes the corresponding Frame Description Entry in the
+ * .eh_frame section to be annotated as a signal frame. This allows DWARF
+ * unwinders (e.g. libstdc++) to implement _Unwind_GetIPInfo(), which permits
+ * unwinding out of the signal trampoline without the need for the mysterious
+ * NOP.

Quoted 21 lines · Will Deacon

arch/arm64/kernel/vdso/sigreturn.S +16 -1

@@ -14,7 +18,34 @@
 
 	.text
 
-	nop
+/* Ensure that the mysterious NOP can be associated with a function. */
+	.cfi_startproc
+
+/*
+ * .cfi_signal_frame causes the corresponding Frame Description Entry in the
+ * .eh_frame section to be annotated as a signal frame. This allows DWARF
+ * unwinders (e.g. libstdc++) to implement _Unwind_GetIPInfo(), which permits
+ * unwinding out of the signal trampoline without the need for the mysterious
+ * NOP.
+ */
+	.cfi_signal_frame
+
+/*
+ * Tell the unwinder where to locate the frame record linking back to the
+ * interrupted context.
+ */

arch/arm64/kernel/vdso/sigreturn.S line 39

Quoted 3 lines

arch/arm64/kernel/vdso/sigreturn.S +3 -0

+	.cfi_def_cfa    x29, 0
+	.cfi_offset     x29, 0 * 8
+	.cfi_offset     x29, 1 * 8

We should also give rationale for why we don't describe how to recover other regs here. At a signal, every reg is potentially live with data essential to the backtrace, so custom unwind entries further up the stack may unwind badly after trying to unwind out of the signal handler.

Quoted 22 lines · Dave Martin

Quoted 21 lines · Will Deacon

arch/arm64/kernel/vdso/sigreturn.S +16 -1

@@ -14,7 +18,34 @@
 
 	.text
 
-	nop
+/* Ensure that the mysterious NOP can be associated with a function. */
+	.cfi_startproc
+
+/*
+ * .cfi_signal_frame causes the corresponding Frame Description Entry in the
+ * .eh_frame section to be annotated as a signal frame. This allows DWARF
+ * unwinders (e.g. libstdc++) to implement _Unwind_GetIPInfo(), which permits
+ * unwinding out of the signal trampoline without the need for the mysterious
+ * NOP.
+ */
+	.cfi_signal_frame
+
+/*
+ * Tell the unwinder where to locate the frame record linking back to the
+ * interrupted context.
+ */

Quoted 8 lines

arch/arm64/kernel/vdso/sigreturn.S line 39

Quoted 3 lines

arch/arm64/kernel/vdso/sigreturn.S +3 -0

+	.cfi_def_cfa    x29, 0
+	.cfi_offset     x29, 0 * 8
+	.cfi_offset     x29, 1 * 8

We should also give rationale for why we don't describe how to recover other regs here. At a signal, every reg is potentially live with data essential to the backtrace, so custom unwind entries further up the stack may unwind badly after trying to unwind out of the signal handler.

Hmm, I'm not sure I get what you're asking for. We can't recover the other registers even if we tried, can we? I think the only way to get a reliable backtrace here is not to clobber the framepointer.

Quoted 32 lines · Will Deacon

Quoted 22 lines · Dave Martin

Quoted 21 lines · Will Deacon

arch/arm64/kernel/vdso/sigreturn.S +16 -1

@@ -14,7 +18,34 @@
 
 	.text
 
-	nop
+/* Ensure that the mysterious NOP can be associated with a function. */
+	.cfi_startproc
+
+/*
+ * .cfi_signal_frame causes the corresponding Frame Description Entry in the
+ * .eh_frame section to be annotated as a signal frame. This allows DWARF
+ * unwinders (e.g. libstdc++) to implement _Unwind_GetIPInfo(), which permits
+ * unwinding out of the signal trampoline without the need for the mysterious
+ * NOP.
+ */
+	.cfi_signal_frame
+
+/*
+ * Tell the unwinder where to locate the frame record linking back to the
+ * interrupted context.
+ */

Quoted 8 lines

arch/arm64/kernel/vdso/sigreturn.S line 39

Quoted 3 lines

arch/arm64/kernel/vdso/sigreturn.S +3 -0

+	.cfi_def_cfa    x29, 0
+	.cfi_offset     x29, 0 * 8
+	.cfi_offset     x29, 1 * 8

We should also give rationale for why we don't describe how to recover other regs here. At a signal, every reg is potentially live with data essential to the backtrace, so custom unwind entries further up the stack may unwind badly after trying to unwind out of the signal handler.

Quoted 3 lines

Hmm, I'm not sure I get what you're asking for. We can't recover the other registers even if we tried, can we? I think the only way to get a reliable backtrace here is not to clobber the framepointer.

A caller somewhere up the stack could have stashed stuff in nonstandard places, with a custom unwind entry that doesn't use x29 in the usual way.

If x29 and x30 were stashed in x8 and x9, say, then the unwinder needs to restore x8 and x9 correctly before that frame is reached. Dwarf unwind tables are expressive enough to describe how to unwind such a
frames: the directives work on all the registers, not just x29, lr.

For this kind of unwinding scenario to wokr, the userspace environment would need to provide correct unwind info for _everything_ rather than relying on the frame chain on the stack alone, so this scenario isn't applicable to C.

I'm not saying we should try to support this, but a comment to indicate what we are and are not trying to do might be a good idea.

How about something along these lines:

/*
* Don't try to provide unwind into for the other regs of the
* interrupted context here. C/C++ based runtimes don't rely on
* this for unwinding in practice. Debuggers need more, but they
* already have baked-in knowledge about how to unwind out of
* signals.
*/

Quoted 37 lines · Dave Martin

Quoted 32 lines · Will Deacon

Quoted 22 lines · Dave Martin

Quoted 21 lines · Will Deacon

arch/arm64/kernel/vdso/sigreturn.S +16 -1

@@ -14,7 +18,34 @@
 
 	.text
 
-	nop
+/* Ensure that the mysterious NOP can be associated with a function. */
+	.cfi_startproc
+
+/*
+ * .cfi_signal_frame causes the corresponding Frame Description Entry in the
+ * .eh_frame section to be annotated as a signal frame. This allows DWARF
+ * unwinders (e.g. libstdc++) to implement _Unwind_GetIPInfo(), which permits
+ * unwinding out of the signal trampoline without the need for the mysterious
+ * NOP.
+ */
+	.cfi_signal_frame
+
+/*
+ * Tell the unwinder where to locate the frame record linking back to the
+ * interrupted context.
+ */

Quoted 8 lines

arch/arm64/kernel/vdso/sigreturn.S line 39

Quoted 3 lines

arch/arm64/kernel/vdso/sigreturn.S +3 -0

+	.cfi_def_cfa    x29, 0
+	.cfi_offset     x29, 0 * 8
+	.cfi_offset     x29, 1 * 8

We should also give rationale for why we don't describe how to recover other regs here. At a signal, every reg is potentially live with data essential to the backtrace, so custom unwind entries further up the stack may unwind badly after trying to unwind out of the signal handler.

Quoted 3 lines

Hmm, I'm not sure I get what you're asking for. We can't recover the other registers even if we tried, can we? I think the only way to get a reliable backtrace here is not to clobber the framepointer.

Quoted 7 lines

A caller somewhere up the stack could have stashed stuff in nonstandard places, with a custom unwind entry that doesn't use x29 in the usual way.

If x29 and x30 were stashed in x8 and x9, say, then the unwinder needs to restore x8 and x9 correctly before that frame is reached. Dwarf unwind tables are expressive enough to describe how to unwind such a
frames: the directives work on all the registers, not just x29, lr.

Understood, I just can't figure out how we could support that even if we wanted to. The only evidence we have of those registers is in the sigcontext, but that may have been modified by the time we end up in the return trampoline. Would we need to push the registers twice (i.e. expand the frame record to include the GPRs)? Not saying we should do this, just wondering what it would take.

Quoted 10 lines

For this kind of unwinding scenario to wokr, the userspace environment would need to provide correct unwind info for _everything_ rather than relying on the frame chain on the stack alone, so this scenario isn't applicable to C.

I'm not saying we should try to support this, but a comment to indicate what we are and are not trying to do might be a good idea.

How about something along these lines:

Quoted 7 lines

/*
* Don't try to provide unwind into for the other regs of the
* interrupted context here. C/C++ based runtimes don't rely on
* this for unwinding in practice. Debuggers need more, but they
* already have baked-in knowledge about how to unwind out of
* signals.
*/

I'll fold that in, thanks.

Daniel reports that the .cfi_startproc is misplaced for the sigreturn trampoline, which causes LLVM's unwinder to misbehave:

| I run into this with LLVM’s unwinder.
| This combination was always broken.

This prompted Dave to question our use of CFI directives more generally, and I ended up going down a rabbit hole trying to figure out how this very poorly documented stuff gets used.

Move the CFI directives so that the "mysterious NOP" is included in the .cfi_{start,end}proc block and add a bunch of comments so that I can save myself another headache in future.

Cc Tamas Zsoldos <[email protected]>

Reported-by Dave Martin <[email protected]>

Reported-by Daniel Kiss <[email protected]>

Signed-off-by Will Deacon <[email protected]>

---
 arch/arm64/kernel/vdso/sigreturn.S | 40 ++++++++++++++++++++++++------
 1 file changed, 33 insertions(+), 7 deletions(-)

arch/arm64/kernel/vdso/sigreturn.S +14 -2

index 0c921130002a..cb47dfb3bd5a 100644
@@ -1,7 +1,11 @@
 /* SPDX-License-Identifier: GPL-2.0-only */
 /*
  * Sigreturn trampoline for returning from a signal when the SA_RESTORER
- * flag is not set.
+ * flag is not set. It serves primarily as a hall of shame for crappy
+ * unwinders and features an exciting but mysterious NOP instruction.
+ *
+ * It's also fragile as hell, so please think twice before changing anything
+ * in here.
  *
  * Copyright (C) 2012 ARM Limited
  *

@@ -14,7 +18,34 @@
 
 	.text
 
-	nop
+/* Ensure that the mysterious NOP can be associated with a function. */
+	.cfi_startproc
+
+/*
+ * .cfi_signal_frame causes the corresponding Frame Description Entry in the
+ * .eh_frame section to be annotated as a signal frame. This allows DWARF
+ * unwinders (e.g. libstdc++) to implement _Unwind_GetIPInfo(), which permits
+ * unwinding out of the signal trampoline without the need for the mysterious
+ * NOP.

Quoted 57 lines · Will Deacon

Daniel reports that the .cfi_startproc is misplaced for the sigreturn trampoline, which causes LLVM's unwinder to misbehave:

| I run into this with LLVM’s unwinder.
| This combination was always broken.

This prompted Dave to question our use of CFI directives more generally, and I ended up going down a rabbit hole trying to figure out how this very poorly documented stuff gets used.

Move the CFI directives so that the "mysterious NOP" is included in the .cfi_{start,end}proc block and add a bunch of comments so that I can save myself another headache in future.

Cc Tamas Zsoldos <[email protected]>

Reported-by Dave Martin <[email protected]>

Reported-by Daniel Kiss <[email protected]>

Signed-off-by Will Deacon <[email protected]>

---
 arch/arm64/kernel/vdso/sigreturn.S | 40 ++++++++++++++++++++++++------
 1 file changed, 33 insertions(+), 7 deletions(-)

arch/arm64/kernel/vdso/sigreturn.S +18 -2

index 0c921130002a..cb47dfb3bd5a 100644
@@ -1,7 +1,11 @@
 /* SPDX-License-Identifier: GPL-2.0-only */
 /*
  * Sigreturn trampoline for returning from a signal when the SA_RESTORER
- * flag is not set.
+ * flag is not set. It serves primarily as a hall of shame for crappy
+ * unwinders and features an exciting but mysterious NOP instruction.
+ *
+ * It's also fragile as hell, so please think twice before changing anything
+ * in here.
  *
  * Copyright (C) 2012 ARM Limited
  *

@@ -14,7 +18,34 @@
 
 	.text
 
-	nop
+/* Ensure that the mysterious NOP can be associated with a function. */
+	.cfi_startproc
+
+/*
+ * .cfi_signal_frame causes the corresponding Frame Description Entry in the
+ * .eh_frame section to be annotated as a signal frame. This allows DWARF
+ * unwinders (e.g. libstdc++) to implement _Unwind_GetIPInfo(), which permits
+ * unwinding out of the signal trampoline without the need for the mysterious
+ * NOP.
+ */
+	.cfi_signal_frame
+
+/*

arch/arm64/kernel/vdso/sigreturn.S line 39

Quoted 6 lines

arch/arm64/kernel/vdso/sigreturn.S +6 -0

+ * Tell the unwinder where to locate the frame record linking back to the
+ * interrupted context.
+ */
+	.cfi_def_cfa    x29, 0
+	.cfi_offset     x29, 0 * 8
+	.cfi_offset     x29, 1 * 8

Oops, just spotted this bug: second entry should be x30.