Skip to contentSkip to conversation

x86/mm: annotate no_context with UNWIND_HINTS

PATCH In review Nick Desaulniers 5 participants 12 messages Oct 15, 2018

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 x86/mm: annotate no_context with UNWIND_HINTS

Fixes the objtool warning:
arch/x86/mm/fault.o: warning: objtool: no_context()+0x220: unreachable instruction

---
 arch/x86/mm/fault.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
arch/x86/mm/fault.c +4 -2
index 47bebfe6efa7..057d2178fa19 100644
@@ -760,9 +760,11 @@ no_context(struct pt_regs *regs, unsigned long error_code,
 		 * and then double-fault, though, because we're likely to
 		 * break the console driver and lose most of the stack dump.
 		 */
-		asm volatile ("movq %[stack], %%rsp\n\t"
+		asm volatile (UNWIND_HINT_SAVE
+			      "movq %[stack], %%rsp\n\t"
 			      "call handle_stack_overflow\n\t"
-			      "1: jmp 1b"
+			      "1: jmp 1b\n\t"
+			      UNWIND_HINT_RESTORE
 			      : ASM_CALL_CONSTRAINT
 			      : "D" ("kernel stack overflow (page fault)"),
 				"S" (regs), "d" (address),
Signature · 1 line
2.17.1
maintainer
#1 lore raw
Quoted 19 lines · Nick Desaulniers

Fixes the objtool warning:
arch/x86/mm/fault.o: warning: objtool: no_context()+0x220: unreachable instruction

---
 arch/x86/mm/fault.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
arch/x86/mm/fault.c +0 -0
index 47bebfe6efa7..057d2178fa19 100644
@@ -760,9 +760,11 @@ no_context(struct pt_regs *regs, unsigned long error_code,
                 * and then double-fault, though, because we're likely to
                 * break the console driver and lose most of the stack dump.
                 */

arch/x86/mm/fault.c no_context(struct pt_regs *regs, unsigned long error_code, · line 770

Quoted 10 lines
arch/x86/mm/fault.c +4 -2
-               asm volatile ("movq %[stack], %%rsp\n\t"
+               asm volatile (UNWIND_HINT_SAVE
+                             "movq %[stack], %%rsp\n\t"
                              "call handle_stack_overflow\n\t"
-                             "1: jmp 1b"
+                             "1: jmp 1b\n\t"
+                             UNWIND_HINT_RESTORE
                              : ASM_CALL_CONSTRAINT
                              : "D" ("kernel stack overflow (page fault)"),
                                "S" (regs), "d" (address),

NAK. Just below this snippet is unreachable();

Can you reply with objtool -dr output on a problematic fault.o? Josh, it *looks* like annotate_unreachable() should be doing the right thing, but something is clearly busted.

Also, shouldn't compiler-clang.h contain a reasonable definition of unreachable()?

--Andy

#2 lore raw
Quoted 34 lines · Andy Lutomirski
Quoted 19 lines · Nick Desaulniers

Fixes the objtool warning:
arch/x86/mm/fault.o: warning: objtool: no_context()+0x220: unreachable instruction

---
 arch/x86/mm/fault.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
arch/x86/mm/fault.c +0 -0
index 47bebfe6efa7..057d2178fa19 100644
@@ -760,9 +760,11 @@ no_context(struct pt_regs *regs, unsigned long error_code,
                 * and then double-fault, though, because we're likely to
                 * break the console driver and lose most of the stack dump.
                 */

arch/x86/mm/fault.c no_context(struct pt_regs *regs, unsigned long error_code, · line 770

Quoted 10 lines
arch/x86/mm/fault.c +4 -2
-               asm volatile ("movq %[stack], %%rsp\n\t"
+               asm volatile (UNWIND_HINT_SAVE
+                             "movq %[stack], %%rsp\n\t"
                              "call handle_stack_overflow\n\t"
-                             "1: jmp 1b"
+                             "1: jmp 1b\n\t"
+                             UNWIND_HINT_RESTORE
                              : ASM_CALL_CONSTRAINT
                              : "D" ("kernel stack overflow (page fault)"),
                                "S" (regs), "d" (address),

NAK. Just below this snippet is unreachable();

Quoted 8 lines

Can you reply with objtool -dr output on a problematic fault.o? Josh, it *looks* like annotate_unreachable() should be doing the right thing, but something is clearly busted.

Also, shouldn't compiler-clang.h contain a reasonable definition of unreachable()?

--Andy

Hi Andy,

Did you mean 'objdump -dr'? If so, here you go (rather long, sorry if I should have pasted it here instead):
https://gist.github.com/nathanchance/f038bb0a6653b975bb8a4e64fcd5503e

Nathan

#3 lore raw
Quoted 42 lines · Nathan Chancellor
Quoted 41 lines · Andy Lutomirski
Quoted 19 lines · Nick Desaulniers

Fixes the objtool warning:
arch/x86/mm/fault.o: warning: objtool: no_context()+0x220: unreachable instruction

---
 arch/x86/mm/fault.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
arch/x86/mm/fault.c +0 -0
index 47bebfe6efa7..057d2178fa19 100644
@@ -760,9 +760,11 @@ no_context(struct pt_regs *regs, unsigned long error_code,
                 * and then double-fault, though, because we're likely to
                 * break the console driver and lose most of the stack dump.
                 */

arch/x86/mm/fault.c no_context(struct pt_regs *regs, unsigned long error_code, · line 770

Quoted 10 lines
arch/x86/mm/fault.c +4 -2
-               asm volatile ("movq %[stack], %%rsp\n\t"
+               asm volatile (UNWIND_HINT_SAVE
+                             "movq %[stack], %%rsp\n\t"
                              "call handle_stack_overflow\n\t"
-                             "1: jmp 1b"
+                             "1: jmp 1b\n\t"
+                             UNWIND_HINT_RESTORE
                              : ASM_CALL_CONSTRAINT
                              : "D" ("kernel stack overflow (page fault)"),
                                "S" (regs), "d" (address),

NAK. Just below this snippet is unreachable();

Can you reply with objtool -dr output on a problematic fault.o? Josh, it *looks* like annotate_unreachable() should be doing the right thing, but something is clearly busted.

Also, shouldn't compiler-clang.h contain a reasonable definition of unreachable()?

Quoted 7 lines
Quoted 1 line

--Andy

Hi Andy,

Did you mean 'objdump -dr'? If so, here you go (rather long, sorry if I should have pasted it here instead):
https://gist.github.com/nathanchance/f038bb0a6653b975bb8a4e64fcd5503e

As Andy said, I think compiler-clang.h needs an unreachable() definition. Without that, I'm guessing you're getting this:

  #ifndef unreachable
  # define unreachable() do { annotate_reachable(); do { } while (1); } while (0)
  #endif

which converts "unreachable" to "reachable". That used to be needed for pre-4.5 versions of GCC when we supported them, but it's the wrong behavior for this piece of code (and I should remove all those old GCC hacks soon...)

Signature · 1 line
Josh
#4 lore raw
In reply to Nathan Chancellor
Quoted 46 lines · Nathan Chancellor
Quoted 43 lines · Andy Lutomirski
Quoted 19 lines · Nick Desaulniers

Fixes the objtool warning:
arch/x86/mm/fault.o: warning: objtool: no_context()+0x220: unreachable instruction

---
arch/x86/mm/fault.c | 6 ++++--

1 file changed, 4 insertions(+), 2 deletions(-)
arch/x86/mm/fault.c +0 -0
index 47bebfe6efa7..057d2178fa19 100644
@@ -760,9 +760,11 @@ no_context(struct pt_regs *regs, unsigned long error_code,
                * and then double-fault, though, because we're likely to
                * break the console driver and lose most of the stack dump.
                */

arch/x86/mm/fault.c no_context(struct pt_regs *regs, unsigned long error_code, · line 770

Quoted 10 lines
arch/x86/mm/fault.c +4 -2
-               asm volatile ("movq %[stack], %%rsp\n\t"
+               asm volatile (UNWIND_HINT_SAVE
+                             "movq %[stack], %%rsp\n\t"
                             "call handle_stack_overflow\n\t"
-                             "1: jmp 1b"
+                             "1: jmp 1b\n\t"
+                             UNWIND_HINT_RESTORE
                             : ASM_CALL_CONSTRAINT
                             : "D" ("kernel stack overflow (page fault)"),
                               "S" (regs), "d" (address),

NAK. Just below this snippet is unreachable();

Can you reply with objtool -dr output on a problematic fault.o? Josh, it *looks* like annotate_unreachable() should be doing the right thing, but something is clearly busted.

Also, shouldn't compiler-clang.h contain a reasonable definition of unreachable()?

Quoted 9 lines
Quoted 1 line

--Andy

Hi Andy,

Did you mean 'objdump -dr'? If so, here you go (rather long, sorry if I should have pasted it here instead):
https://gist.github.com/nathanchance/f038bb0a6653b975bb8a4e64fcd5503e

Hmm, -dr wasn’t quite enough to dump the .discard bits, assuming they’re there at all. Can you just put the whole .o file somewhere?

#5 lore raw
Quoted 50 lines · Andy Lutomirski
Quoted 50 lines · Nathan Chancellor
Quoted 36 lines · Andy Lutomirski
Quoted 19 lines · Nick Desaulniers

Fixes the objtool warning:
arch/x86/mm/fault.o: warning: objtool: no_context()+0x220: unreachable instruction

---
arch/x86/mm/fault.c | 6 ++++--

1 file changed, 4 insertions(+), 2 deletions(-)
arch/x86/mm/fault.c +0 -0
index 47bebfe6efa7..057d2178fa19 100644
@@ -760,9 +760,11 @@ no_context(struct pt_regs *regs, unsigned long error_code,
                * and then double-fault, though, because we're likely to
                * break the console driver and lose most of the stack dump.
                */

arch/x86/mm/fault.c no_context(struct pt_regs *regs, unsigned long error_code, · line 770

Quoted 10 lines
arch/x86/mm/fault.c +4 -2
-               asm volatile ("movq %[stack], %%rsp\n\t"
+               asm volatile (UNWIND_HINT_SAVE
+                             "movq %[stack], %%rsp\n\t"
                             "call handle_stack_overflow\n\t"
-                             "1: jmp 1b"
+                             "1: jmp 1b\n\t"
+                             UNWIND_HINT_RESTORE
                             : ASM_CALL_CONSTRAINT
                             : "D" ("kernel stack overflow (page fault)"),
                               "S" (regs), "d" (address),

NAK. Just below this snippet is unreachable();

Quoted 8 lines

Can you reply with objtool -dr output on a problematic fault.o? Josh, it *looks* like annotate_unreachable() should be doing the right thing, but something is clearly busted.

Also, shouldn't compiler-clang.h contain a reasonable definition of unreachable()?

--Andy

Hi Andy,

Quoted 7 lines
Quoted 5 lines

Did you mean 'objdump -dr'? If so, here you go (rather long, sorry if I should have pasted it here instead):
https://gist.github.com/nathanchance/f038bb0a6653b975bb8a4e64fcd5503e

Hmm, -dr wasn’t quite enough to dump the .discard bits, assuming they’re there at all. Can you just put the whole .o file somewhere?

Here you go: https://nathanchance.me/downloads/.tmp/fault.o

Thanks,
Nathan

#6 lore raw
Quoted 34 lines · Nathan Chancellor
Quoted 34 lines
Quoted 33 lines
Quoted 17 lines
Quoted 4 lines
arch/x86/mm/fault.c +0 -0
@@ -760,9 +760,11 @@ no_context(struct pt_regs *regs, unsigned long error_code,
                * and then double-fault, though, because we're likely to
                * break the console driver and lose most of the stack dump.
                */

arch/x86/mm/fault.c no_context(struct pt_regs *regs, unsigned long error_code, · line 770

Quoted 10 lines
arch/x86/mm/fault.c +4 -2
-               asm volatile ("movq %[stack], %%rsp\n\t"
+               asm volatile (UNWIND_HINT_SAVE
+                             "movq %[stack], %%rsp\n\t"
                             "call handle_stack_overflow\n\t"
-                             "1: jmp 1b"
+                             "1: jmp 1b\n\t"
+                             UNWIND_HINT_RESTORE
                             : ASM_CALL_CONSTRAINT
                             : "D" ("kernel stack overflow (page fault)"),
                               "S" (regs), "d" (address),

NAK. Just below this snippet is unreachable();

Quoted 8 lines

Can you reply with objtool -dr output on a problematic fault.o? Josh, it *looks* like annotate_unreachable() should be doing the right thing, but something is clearly busted.

Also, shouldn't compiler-clang.h contain a reasonable definition of unreachable()?

--Andy

Hi Andy,

Did you mean 'objdump -dr'? If so, here you go (rather long, sorry if I should have pasted it here instead):
https://gist.github.com/nathanchance/f038bb0a6653b975bb8a4e64fcd5503e

Quoted 3 lines
Quoted 1 line

Hmm, -dr wasn’t quite enough to dump the .discard bits, assuming they’re there at all. Can you just put the whole .o file somewhere?

Here you go: https://nathanchance.me/downloads/.tmp/fault.o

$ eu-readelf -S /tmp/fault.o  |grep reachable
[12] .discard.reachable   PROGBITS     0000000000000000 00002bc0 00000014  0        0   0  1
[13] .rela.discard.reachable RELA         0000000000000000 00002bd8 00000078 24 I     32  12  8

That confirms that you need a clang version of the unreachable() macro.

Signature · 1 line
Josh
#7 lore raw
Quoted 38 lines · Josh Poimboeuf
Quoted 36 lines · Nathan Chancellor
Quoted 35 lines
Quoted 24 lines
Quoted 24 lines
Quoted 4 lines
arch/x86/mm/fault.c +0 -0
@@ -760,9 +760,11 @@ no_context(struct pt_regs *regs, unsigned long error_code,
                * and then double-fault, though, because we're likely to
                * break the console driver and lose most of the stack dump.
                */

arch/x86/mm/fault.c no_context(struct pt_regs *regs, unsigned long error_code, · line 770

Quoted 10 lines
arch/x86/mm/fault.c +4 -2
-               asm volatile ("movq %[stack], %%rsp\n\t"
+               asm volatile (UNWIND_HINT_SAVE
+                             "movq %[stack], %%rsp\n\t"
                             "call handle_stack_overflow\n\t"
-                             "1: jmp 1b"
+                             "1: jmp 1b\n\t"
+                             UNWIND_HINT_RESTORE
                             : ASM_CALL_CONSTRAINT
                             : "D" ("kernel stack overflow (page fault)"),
                               "S" (regs), "d" (address),

NAK. Just below this snippet is unreachable();

Can you reply with objtool -dr output on a problematic fault.o? Josh, it *looks* like annotate_unreachable() should be doing the right thing, but something is clearly busted.

Also, shouldn't compiler-clang.h contain a reasonable definition of unreachable()?

Quoted 9 lines
Quoted 1 line

--Andy

Hi Andy,

Did you mean 'objdump -dr'? If so, here you go (rather long, sorry if I should have pasted it here instead):
https://gist.github.com/nathanchance/f038bb0a6653b975bb8a4e64fcd5503e

Hmm, -dr wasn’t quite enough to dump the .discard bits, assuming they’re there at all. Can you just put the whole .o file somewhere?

Quoted 8 lines
Quoted 1 line

Here you go: https://nathanchance.me/downloads/.tmp/fault.o

$ eu-readelf -S /tmp/fault.o  |grep reachable
[12] .discard.reachable   PROGBITS     0000000000000000 00002bc0 00000014  0        0   0  1
[13] .rela.discard.reachable RELA         0000000000000000 00002bd8 00000078 24 I     32  12  8

That confirms that you need a clang version of the unreachable() macro.

Duh.

That being said, the generic macro is:

# define unreachable() do { annotate_reachable(); do { } while (1); } while (0)

I'm probably missing some subtlety here, but shouldn't that be annotate_*un*reachable()?

Of course, there are any number of reasons why there should be a real definition. Nathan and Nick, does adding something like:

#define unreachable() \
        do {                                    \
                annotate_unreachable();         \
                __builtin_unreachable();        \
        } while (0)

to compiler-clang.h fix the problem?

--Andy

#8 lore raw
Quoted 66 lines · Andy Lutomirski
Quoted 38 lines · Josh Poimboeuf
Quoted 36 lines · Nathan Chancellor
Quoted 35 lines
Quoted 24 lines
Quoted 24 lines
Quoted 4 lines
(quoted diff) +0 -0
@@ -760,9 +760,11 @@ no_context(struct pt_regs *regs, unsigned long error_code,
                * and then double-fault, though, because we're likely to
                * break the console driver and lose most of the stack dump.
                */

Review no_context(struct pt_regs *regs, unsigned long error_code, · line 770

Quoted 10 lines
(quoted diff) +4 -2
-               asm volatile ("movq %[stack], %%rsp\n\t"
+               asm volatile (UNWIND_HINT_SAVE
+                             "movq %[stack], %%rsp\n\t"
                             "call handle_stack_overflow\n\t"
-                             "1: jmp 1b"
+                             "1: jmp 1b\n\t"
+                             UNWIND_HINT_RESTORE
                             : ASM_CALL_CONSTRAINT
                             : "D" ("kernel stack overflow (page fault)"),
                               "S" (regs), "d" (address),

NAK. Just below this snippet is unreachable();

Can you reply with objtool -dr output on a problematic fault.o? Josh, it *looks* like annotate_unreachable() should be doing the right thing, but something is clearly busted.

Also, shouldn't compiler-clang.h contain a reasonable definition of unreachable()?

Quoted 9 lines
Quoted 1 line

--Andy

Hi Andy,

Did you mean 'objdump -dr'? If so, here you go (rather long, sorry if I should have pasted it here instead):
https://gist.github.com/nathanchance/f038bb0a6653b975bb8a4e64fcd5503e

Hmm, -dr wasn’t quite enough to dump the .discard bits, assuming they’re there at all. Can you just put the whole .o file somewhere?

Quoted 8 lines
Quoted 1 line

Here you go: https://nathanchance.me/downloads/.tmp/fault.o

$ eu-readelf -S /tmp/fault.o  |grep reachable
[12] .discard.reachable   PROGBITS     0000000000000000 00002bc0 00000014  0        0   0  1
[13] .rela.discard.reachable RELA         0000000000000000 00002bd8 00000078 24 I     32  12  8

That confirms that you need a clang version of the unreachable() macro.

Duh.

That being said, the generic macro is:

# define unreachable() do { annotate_reachable(); do { } while (1); } while (0)

I'm probably missing some subtlety here, but shouldn't that be annotate_*un*reachable()?

Of course, there are any number of reasons why there should be a real definition. Nathan and Nick, does adding something like:

#define unreachable() \
        do {                                    \
                annotate_unreachable();         \
                __builtin_unreachable();        \
        } while (0)
Quoted 3 lines

to compiler-clang.h fix the problem?

--Andy

Ha, I was just typing out a message summarizing that that exact definition fixed this warning.

Nathan

#9 lore raw
In reply to Andy Lutomirski
Quoted 6 lines · Andy Lutomirski

That being said, the generic macro is:

# define unreachable() do { annotate_reachable(); do { } while (1); } while (0)

I'm probably missing some subtlety here, but shouldn't that be annotate_*un*reachable()?

That code should have had a comment, but that subtlety was intentional. As I mentioned earlier, that was a hack for old versions of GCC which didn't have __builtin_unreachable(). In those cases, GCC doesn't treat "ud2" as fatal, so this was a way of telling objtool that. Luckily we can get rid of this hack now that the minimum supported GCC version has gone up to 4.6.

Signature · 1 line
Josh
#10 lore raw
In reply to Andy Lutomirski
Quoted 61 lines · Andy Lutomirski
Quoted 38 lines · Josh Poimboeuf
Quoted 36 lines · Nathan Chancellor
Quoted 35 lines
Quoted 24 lines
Quoted 24 lines
Quoted 4 lines
(quoted diff) +0 -0
@@ -760,9 +760,11 @@ no_context(struct pt_regs *regs, unsigned long error_code,
                * and then double-fault, though, because we're likely to
                * break the console driver and lose most of the stack dump.
                */

Review no_context(struct pt_regs *regs, unsigned long error_code, · line 770

Quoted 10 lines
(quoted diff) +4 -2
-               asm volatile ("movq %[stack], %%rsp\n\t"
+               asm volatile (UNWIND_HINT_SAVE
+                             "movq %[stack], %%rsp\n\t"
                             "call handle_stack_overflow\n\t"
-                             "1: jmp 1b"
+                             "1: jmp 1b\n\t"
+                             UNWIND_HINT_RESTORE
                             : ASM_CALL_CONSTRAINT
                             : "D" ("kernel stack overflow (page fault)"),
                               "S" (regs), "d" (address),

NAK. Just below this snippet is unreachable();

Can you reply with objtool -dr output on a problematic fault.o? Josh, it *looks* like annotate_unreachable() should be doing the right thing, but something is clearly busted.

Also, shouldn't compiler-clang.h contain a reasonable definition of unreachable()?

Quoted 9 lines
Quoted 1 line

--Andy

Hi Andy,

Did you mean 'objdump -dr'? If so, here you go (rather long, sorry if I should have pasted it here instead):
https://gist.github.com/nathanchance/f038bb0a6653b975bb8a4e64fcd5503e

Hmm, -dr wasn’t quite enough to dump the .discard bits, assuming they’re there at all. Can you just put the whole .o file somewhere?

Quoted 8 lines
Quoted 1 line

Here you go: https://nathanchance.me/downloads/.tmp/fault.o

$ eu-readelf -S /tmp/fault.o  |grep reachable
[12] .discard.reachable   PROGBITS     0000000000000000 00002bc0 00000014  0        0   0  1
[13] .rela.discard.reachable RELA         0000000000000000 00002bd8 00000078 24 I     32  12  8

That confirms that you need a clang version of the unreachable() macro.

Duh.

That being said, the generic macro is:

# define unreachable() do { annotate_reachable(); do { } while (1); } while (0)

I'm probably missing some subtlety here, but shouldn't that be annotate_*un*reachable()?

Of course, there are any number of reasons why there should be a real definition. Nathan and Nick, does adding something like:

Quoted 7 lines
#define unreachable() \
        do {                                    \
                annotate_unreachable();         \
                __builtin_unreachable();        \
        } while (0)

to compiler-clang.h fix the problem?

I broke this myself in commit 815f0ddb346c ("include/linux/compiler*.h: make compiler-*.h mutually exclusive"). Thanks for the suggestion, will verify then send a patch with your suggested by tag. Thanks everyone for helping us sort this out!

#11 lore raw
Quoted 4 lines · Nick Desaulniers

I broke this myself in commit 815f0ddb346c ("include/linux/compiler*.h: make compiler-*.h mutually exclusive"). Thanks for the suggestion, will verify then send a patch with your suggested by tag. Thanks everyone for helping us sort this out!

https://lkml.org/lkml/2018/10/15/626