Skip to contentSkip to conversation

objtool: vmlinux.o and moinstr validation

PATCH Superseded Peter Zijlstra 5 participants 63 messages Mar 25, 2020 – Apr 2, 2020

Subsystems (6)
x86 Kbuild Library code ftrace +2 more subsystems

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

Peter Zijlstra

author maintainer
#0 lore raw

v4 cover objtool: vmlinux.o and moinstr validation

Hi all,

This lacks the first 17 patches of the v3 posting which are en-route to tip/core/objtool and should show up there somewhere later this evening.

As should be familiar by now; these patches implement the noinstr (no-instrument) validation in objtool as requested by Thomas, to ensure critical code (entry for now, idle later) run no unexpected code.

Functions are marked with: noinstr, which implies notrace, noinline and sticks things in the .noinstr.text section. Such functions can then use instr_begin() and instr_end() to allow calls to code outside of this section in sanctioned areas.

author maintainer
#1 lore raw

PATCH v4 1/13 objtool: Remove CFI save/restore special case

There is a special case in the UNWIND_HINT_RESTORE code. When, upon looking for the UNWIND_HINT_SAVE instruction to restore from, it finds the instruction hasn't been visited yet, it normally issues a WARN, except when this HINT_SAVE instruction is the first instruction of this branch.

This special case is of dubious correctness and is certainly unused (verified with an allmodconfig build), the two sites that employ UNWIND_HINT_SAVE/RESTORE (sync_core() and ftrace_regs_caller()) have the SAVE on unconditional instructions at the start of the function. It is therefore impossible for the save_insn not to have been visited when we do hit the RESTORE.

Remove this.

---
 tools/objtool/check.c |   15 ++-------------
 1 file changed, 2 insertions(+), 13 deletions(-)
tools/objtool/check.c +2 -13
@@ -2040,15 +2040,14 @@ static int validate_return(struct symbol
  * tools/objtool/Documentation/stack-validation.txt.
  */
 static int validate_branch(struct objtool_file *file, struct symbol *func,
-			   struct instruction *first, struct insn_state state)
+			   struct instruction *insn, struct insn_state state)
 {
+	struct instruction *next_insn;
 	struct alternative *alt;
-	struct instruction *insn, *next_insn;
 	struct section *sec;
 	u8 visited;
 	int ret;
 
-	insn = first;
 	sec = insn->sec;
 
 	if (insn->alt_group && list_empty(&insn->alts)) {
@@ -2101,16 +2100,6 @@ static int validate_branch(struct objtoo
 				}
 
 				if (!save_insn->visited) {
-					/*
-					 * Oops, no state to copy yet.
-					 * Hopefully we can reach this
-					 * instruction from another branch
-					 * after the save insn has been
-					 * visited.
-					 */
-					if (insn == first)
-						return 0;
-
 					WARN_FUNC("objtool isn't smart enough to handle this CFI save/restore combo",
 						  sec, insn->offset);
 					return 1;
author maintainer
#2 lore raw

Re: [PATCH v4 01/13] objtool: Remove CFI save/restore special case

Quoted 6 lines · Peter Zijlstra

There is a special case in the UNWIND_HINT_RESTORE code. When, upon looking for the UNWIND_HINT_SAVE instruction to restore from, it finds the instruction hasn't been visited yet, it normally issues a WARN, except when this HINT_SAVE instruction is the first instruction of this branch.

Quoted 6 lines

This special case is of dubious correctness and is certainly unused (verified with an allmodconfig build), the two sites that employ UNWIND_HINT_SAVE/RESTORE (sync_core() and ftrace_regs_caller()) have the SAVE on unconditional instructions at the start of the function. It is therefore impossible for the save_insn not to have been visited when we do hit the RESTORE.

Clearly I was too tired when I did that allmodconfig build, because it very much does generate a warning :-/.

Thank you 0day:

kernel/sched/core.o: warning: objtool: finish_task_switch()+0x1c0: objtool isn't smart enough to handle this CFI save/restore combo

At least this gives clue as to what it was trying to do.

---
Subject: objtool: Remove CFI save/restore special case
From: Peter Zijlstra <[email protected]>
Date: Wed Mar 25 12:58:16 CET 2020

There is a special case in the UNWIND_HINT_RESTORE code. When, upon looking for the UNWIND_HINT_SAVE instruction to restore from, it finds the instruction hasn't been visited yet, it normally issues a WARN, except when this HINT_SAVE instruction is the first instruction of this branch.

The reason for this special case comes apparent when we remove it; code like:

	if (cond) {
		UNWIND_HINT_SAVE
		// do stuff
		UNWIND_HINT_RESTORE
	}
	// more stuff

will now trigger the warning. This is because UNWIND_HINT_RESTORE is just a label, and there is nothing keeping it inside the (extended) basic block covered by @cond. It will attach itself to the first instruction of 'more stuff' and we'll hit it outside of the @cond, confusing things.

I don't much like this special case, it confuses things and will come apart horribly if/when the annotation needs to support nesting. Instead extend the affected code to at least form an extended basic block.

In particular, of the 2 users of this annotation: ftrace_regs_caller() and sync_core(), only the latter suffers this problem. Extend it's code sequence with a NOP to make it an extended basic block.

This isn't ideal either; stuffing code with NOPs just to make annotations work is certainly sub-optimal, but given that sync_core() is stupid expensive in any case, one extra nop isn't going to be a problem here.

---
 arch/x86/include/asm/processor.h |    9 ++++++++-
 tools/objtool/check.c            |   15 ++-------------
 2 files changed, 10 insertions(+), 14 deletions(-)
arch/x86/include/asm/processor.h +8 -1
@@ -727,6 +727,13 @@ static inline void sync_core(void)
 #else
 	unsigned int tmp;
 
+	/*
+	 * The trailing NOP is required to make this an extended basic block,
+	 * such that we can argue about it locally. Specifically this is
+	 * important for the UNWIND_HINTs, without this the UNWIND_HINT_RESTORE
+	 * can fall outside our extended basic block and objtool gets
+	 * (rightfully) confused.
+	 */
 	asm volatile (
 		UNWIND_HINT_SAVE
 		"mov %%ss, %0\n\t"
@@ -739,7 +746,7 @@ static inline void sync_core(void)
 		"pushq $1f\n\t"
 		"iretq\n\t"
 		UNWIND_HINT_RESTORE
-		"1:"
+		"1: nop\n\t"
 		: "=&r" (tmp), ASM_CALL_CONSTRAINT : : "cc", "memory");
 #endif
 }
tools/objtool/check.c +2 -13
@@ -2000,15 +2000,14 @@ static int validate_sibling_call(struct
  * tools/objtool/Documentation/stack-validation.txt.
  */
 static int validate_branch(struct objtool_file *file, struct symbol *func,
-			   struct instruction *first, struct insn_state state)
+			   struct instruction *insn, struct insn_state state)
 {
+	struct instruction *next_insn;
 	struct alternative *alt;
-	struct instruction *insn, *next_insn;
 	struct section *sec;
 	u8 visited;
 	int ret;
 
-	insn = first;
 	sec = insn->sec;
 
 	if (insn->alt_group && list_empty(&insn->alts)) {
@@ -2061,16 +2060,6 @@ static int validate_branch(struct objtoo
 				}
 
 				if (!save_insn->visited) {
-					/*
-					 * Oops, no state to copy yet.
-					 * Hopefully we can reach this
-					 * instruction from another branch
-					 * after the save insn has been
-					 * visited.
-					 */
-					if (insn == first)
-						return 0;
-
 					WARN_FUNC("objtool isn't smart enough to handle this CFI save/restore combo",
 						  sec, insn->offset);
 					return 1;
author maintainer
#3 lore raw

Re: [PATCH v4 01/13] objtool: Remove CFI save/restore special case

Quoted 27 lines · Peter Zijlstra

There is a special case in the UNWIND_HINT_RESTORE code. When, upon looking for the UNWIND_HINT_SAVE instruction to restore from, it finds the instruction hasn't been visited yet, it normally issues a WARN, except when this HINT_SAVE instruction is the first instruction of this branch.

The reason for this special case comes apparent when we remove it; code like:

	if (cond) {
		UNWIND_HINT_SAVE
		// do stuff
		UNWIND_HINT_RESTORE
	}
	// more stuff

will now trigger the warning. This is because UNWIND_HINT_RESTORE is just a label, and there is nothing keeping it inside the (extended) basic block covered by @cond. It will attach itself to the first instruction of 'more stuff' and we'll hit it outside of the @cond, confusing things.

I don't much like this special case, it confuses things and will come apart horribly if/when the annotation needs to support nesting. Instead extend the affected code to at least form an extended basic block.

Quoted 8 lines

In particular, of the 2 users of this annotation: ftrace_regs_caller() and sync_core(), only the latter suffers this problem. Extend it's code sequence with a NOP to make it an extended basic block.

This isn't ideal either; stuffing code with NOPs just to make annotations work is certainly sub-optimal, but given that sync_core() is stupid expensive in any case, one extra nop isn't going to be a problem here.

So instr_begin() / instr_end() have this exact problem, but worse. Those actually do nest and I've ran into the following situation:

	if (cond1) {
		instr_begin();
		// code1
		instr_end();
	}
	// code

	if (cond2) {
		instr_begin();
		// code2
		instr_end();
	}
	// tail

Where objtool then finds the path: !cond1, cond2, which ends up at code2 with 0, instead of 1.

I've also seen:

	if (cond) {
		instr_begin();
		// code1
		instr_end();
	}
	instr_begin();
	// code2
	instr_end();

Where instr_end() and instr_begin() merge onto the same instruction of code2 as a 0, and again code2 will issue a false warning.

You can also not make objtool lift the end marker to the previous instruction, because then:

	if (cond1) {
		instr_begin();
		if (cond2) {
			// code2
		}
		instr_end();
	}

Suffers the reverse problem, instr_end() becomes part of the @cond2 block and cond1 grows a path that misses it entirely.

So far I've not had any actual solution except adding a NOP to anchor the annotation on.

#4 lore raw

Re: [PATCH v4 01/13] objtool: Remove CFI save/restore special case

Quoted 17 lines · Peter Zijlstra

So instr_begin() / instr_end() have this exact problem, but worse. Those actually do nest and I've ran into the following situation:

	if (cond1) {
		instr_begin();
		// code1
		instr_end();
	}
	// code

	if (cond2) {
		instr_begin();
		// code2
		instr_end();
	}
	// tail
Quoted 2 lines

Where objtool then finds the path: !cond1, cond2, which ends up at code2 with 0, instead of 1.

Hm, I don't see the nesting in this example, can you clarify?

Quoted 25 lines

I've also seen:

	if (cond) {
		instr_begin();
		// code1
		instr_end();
	}
	instr_begin();
	// code2
	instr_end();

Where instr_end() and instr_begin() merge onto the same instruction of code2 as a 0, and again code2 will issue a false warning.

You can also not make objtool lift the end marker to the previous instruction, because then:

	if (cond1) {
		instr_begin();
		if (cond2) {
			// code2
		}
		instr_end();
	}
Quoted 5 lines

Suffers the reverse problem, instr_end() becomes part of the @cond2 block and cond1 grows a path that misses it entirely.

So far I've not had any actual solution except adding a NOP to anchor the annotation on.

Are you adding the NOP to the instr_end() annotation itself? Seems like that would be the cleanest/easiest.

Though it is sad that we have to change the code to make objtool happy -- would be nice if we could come up with something less intrusive.

Signature · 1 line
Josh
author maintainer
#5 lore raw

Re: [PATCH v4 01/13] objtool: Remove CFI save/restore special case

Quoted 18 lines · Josh Poimboeuf
Quoted 17 lines · Peter Zijlstra

So instr_begin() / instr_end() have this exact problem, but worse. Those actually do nest and I've ran into the following situation:

	if (cond1) {
		instr_begin();
		// code1
		instr_end();
	}
	// code

	if (cond2) {
		instr_begin();
		// code2
		instr_end();
	}
	// tail
Quoted 4 lines
Quoted 2 lines

Where objtool then finds the path: !cond1, cond2, which ends up at code2 with 0, instead of 1.

Hm, I don't see the nesting in this example, can you clarify?

Indeed no nesting here, but because they can nest we have that begin as +1, end as -1 and then we sum it over the code flow.

Then given that, the above, ends up as -1 + 1 in the !cond1,cond2 case, because that -1 escapes the cond1 block.

Quoted 25 lines
Quoted 25 lines

I've also seen:

	if (cond) {
		instr_begin();
		// code1
		instr_end();
	}
	instr_begin();
	// code2
	instr_end();

Where instr_end() and instr_begin() merge onto the same instruction of code2 as a 0, and again code2 will issue a false warning.

You can also not make objtool lift the end marker to the previous instruction, because then:

	if (cond1) {
		instr_begin();
		if (cond2) {
			// code2
		}
		instr_end();
	}
Quoted 8 lines
Quoted 5 lines

Suffers the reverse problem, instr_end() becomes part of the @cond2 block and cond1 grows a path that misses it entirely.

So far I've not had any actual solution except adding a NOP to anchor the annotation on.

Are you adding the NOP to the instr_end() annotation itself? Seems like that would be the cleanest/easiest.

That actually generates a whole bunch of 'stupid' unreachable warnings. Also, in the hope of still coming up with something saner, we've been carrying a minimal set of explicit nop()s.

Quoted 2 lines

Though it is sad that we have to change the code to make objtool happy -- would be nice if we could come up with something less intrusive.

Very much yes, but so far that's been eluding me.

#6 lore raw

Re: [PATCH v4 01/13] objtool: Remove CFI save/restore special case

Quoted 22 lines · Peter Zijlstra
Quoted 21 lines · Josh Poimboeuf
Quoted 19 lines · Peter Zijlstra

So instr_begin() / instr_end() have this exact problem, but worse. Those actually do nest and I've ran into the following situation:

	if (cond1) {
		instr_begin();
		// code1
		instr_end();
	}
	// code

	if (cond2) {
		instr_begin();
		// code2
		instr_end();
	}
	// tail

Where objtool then finds the path: !cond1, cond2, which ends up at code2 with 0, instead of 1.

Quoted 7 lines
Quoted 1 line

Hm, I don't see the nesting in this example, can you clarify?

Indeed no nesting here, but because they can nest we have that begin as +1, end as -1 and then we sum it over the code flow.

Then given that, the above, ends up as -1 + 1 in the !cond1,cond2 case, because that -1 escapes the cond1 block.

Ok, I see now. I wonder if you can take a similar approach to the save/restore patch I sent out. Though I guess the nesting adds an additional wrinkle I'm too tired to think about.

Signature · 1 line
Josh
#7 lore raw

Re: [PATCH v4 01/13] objtool: Remove CFI save/restore special case

In reply to Peter Zijlstra
Quoted 83 lines · Peter Zijlstra
Quoted 27 lines · Peter Zijlstra

There is a special case in the UNWIND_HINT_RESTORE code. When, upon looking for the UNWIND_HINT_SAVE instruction to restore from, it finds the instruction hasn't been visited yet, it normally issues a WARN, except when this HINT_SAVE instruction is the first instruction of this branch.

The reason for this special case comes apparent when we remove it; code like:

	if (cond) {
		UNWIND_HINT_SAVE
		// do stuff
		UNWIND_HINT_RESTORE
	}
	// more stuff

will now trigger the warning. This is because UNWIND_HINT_RESTORE is just a label, and there is nothing keeping it inside the (extended) basic block covered by @cond. It will attach itself to the first instruction of 'more stuff' and we'll hit it outside of the @cond, confusing things.

I don't much like this special case, it confuses things and will come apart horribly if/when the annotation needs to support nesting. Instead extend the affected code to at least form an extended basic block.

Quoted 8 lines

In particular, of the 2 users of this annotation: ftrace_regs_caller() and sync_core(), only the latter suffers this problem. Extend it's code sequence with a NOP to make it an extended basic block.

This isn't ideal either; stuffing code with NOPs just to make annotations work is certainly sub-optimal, but given that sync_core() is stupid expensive in any case, one extra nop isn't going to be a problem here.

So instr_begin() / instr_end() have this exact problem, but worse. Those actually do nest and I've ran into the following situation:

	if (cond1) {
		instr_begin();
		// code1
		instr_end();
	}
	// code

	if (cond2) {
		instr_begin();
		// code2
		instr_end();
	}
	// tail

Where objtool then finds the path: !cond1, cond2, which ends up at code2 with 0, instead of 1.

I've also seen:

	if (cond) {
		instr_begin();
		// code1
		instr_end();
	}
	instr_begin();
	// code2
	instr_end();

Where instr_end() and instr_begin() merge onto the same instruction of code2 as a 0, and again code2 will issue a false warning.

You can also not make objtool lift the end marker to the previous instruction, because then:

	if (cond1) {
		instr_begin();
		if (cond2) {
			// code2
		}
		instr_end();
	}
Quoted 2 lines

Suffers the reverse problem, instr_end() becomes part of the @cond2 block and cond1 grows a path that misses it entirely.

One could argue that this is really nasty and the correct way should be

	if (cond1) {
		if (cond2) {
			instr_begin();
			// code2
			instr_end();
		}
	}

Then it should work if instr_begin() marks the next instruction and instr_end() marks the previous one, no? There is a corner case when code2 is exactly one instruction, so instr counting would have to be updated.

Hopefully there is a better way.

Miroslav

#8 lore raw

Re: [PATCH v4 01/13] objtool: Remove CFI save/restore special case

Quoted 98 lines · Miroslav Benes
Quoted 83 lines · Peter Zijlstra
Quoted 27 lines · Peter Zijlstra

There is a special case in the UNWIND_HINT_RESTORE code. When, upon looking for the UNWIND_HINT_SAVE instruction to restore from, it finds the instruction hasn't been visited yet, it normally issues a WARN, except when this HINT_SAVE instruction is the first instruction of this branch.

The reason for this special case comes apparent when we remove it; code like:

	if (cond) {
		UNWIND_HINT_SAVE
		// do stuff
		UNWIND_HINT_RESTORE
	}
	// more stuff

will now trigger the warning. This is because UNWIND_HINT_RESTORE is just a label, and there is nothing keeping it inside the (extended) basic block covered by @cond. It will attach itself to the first instruction of 'more stuff' and we'll hit it outside of the @cond, confusing things.

I don't much like this special case, it confuses things and will come apart horribly if/when the annotation needs to support nesting. Instead extend the affected code to at least form an extended basic block.

Quoted 8 lines

In particular, of the 2 users of this annotation: ftrace_regs_caller() and sync_core(), only the latter suffers this problem. Extend it's code sequence with a NOP to make it an extended basic block.

This isn't ideal either; stuffing code with NOPs just to make annotations work is certainly sub-optimal, but given that sync_core() is stupid expensive in any case, one extra nop isn't going to be a problem here.

So instr_begin() / instr_end() have this exact problem, but worse. Those actually do nest and I've ran into the following situation:

	if (cond1) {
		instr_begin();
		// code1
		instr_end();
	}
	// code

	if (cond2) {
		instr_begin();
		// code2
		instr_end();
	}
	// tail

Where objtool then finds the path: !cond1, cond2, which ends up at code2 with 0, instead of 1.

I've also seen:

	if (cond) {
		instr_begin();
		// code1
		instr_end();
	}
	instr_begin();
	// code2
	instr_end();

Where instr_end() and instr_begin() merge onto the same instruction of code2 as a 0, and again code2 will issue a false warning.

You can also not make objtool lift the end marker to the previous instruction, because then:

	if (cond1) {
		instr_begin();
		if (cond2) {
			// code2
		}
		instr_end();
	}
Quoted 2 lines

Suffers the reverse problem, instr_end() becomes part of the @cond2 block and cond1 grows a path that misses it entirely.

One could argue that this is really nasty and the correct way should be

	if (cond1) {
		if (cond2) {
			instr_begin();
			// code2
			instr_end();
		}
	}
Quoted 3 lines

Then it should work if instr_begin() marks the next instruction and instr_end() marks the previous one, no? There is a corner case when code2 is exactly one instruction, so instr counting would have to be updated.

	if (cond1) {
		instr_begin()
		if (cond2) {
			// code2
		}
		// code1
		instr_end();
	}

is a counter example though, so I take it back.

M

author maintainer
#9 lore raw

Re: [PATCH v4 01/13] objtool: Remove CFI save/restore special case

In reply to Peter Zijlstra
Quoted 46 lines · Peter Zijlstra

Subject: objtool: Remove CFI save/restore special case
From: Peter Zijlstra <[email protected]>
Date: Wed Mar 25 12:58:16 CET 2020

There is a special case in the UNWIND_HINT_RESTORE code. When, upon looking for the UNWIND_HINT_SAVE instruction to restore from, it finds the instruction hasn't been visited yet, it normally issues a WARN, except when this HINT_SAVE instruction is the first instruction of this branch.

The reason for this special case comes apparent when we remove it; code like:

	if (cond) {
		UNWIND_HINT_SAVE
		// do stuff
		UNWIND_HINT_RESTORE
	}
	// more stuff

will now trigger the warning. This is because UNWIND_HINT_RESTORE is just a label, and there is nothing keeping it inside the (extended) basic block covered by @cond. It will attach itself to the first instruction of 'more stuff' and we'll hit it outside of the @cond, confusing things.

I don't much like this special case, it confuses things and will come apart horribly if/when the annotation needs to support nesting. Instead extend the affected code to at least form an extended basic block.

arch/x86/include/asm/processor.h +7 -0
@@ -727,6 +727,13 @@ static inline void sync_core(void)
 #else
 	unsigned int tmp;
 
+	/*
+	 * The trailing NOP is required to make this an extended basic block,
+	 * such that we can argue about it locally. Specifically this is
+	 * important for the UNWIND_HINTs, without this the UNWIND_HINT_RESTORE
+	 * can fall outside our extended basic block and objtool gets
+	 * (rightfully) confused.
+	 */
 	asm volatile (
 		UNWIND_HINT_SAVE
 		"mov %%ss, %0\n\t"

arch/x86/include/asm/processor.h static inline void sync_core(void) · line 750

Quoted 6 lines
arch/x86/include/asm/processor.h +1 -1
@@ -739,7 +746,7 @@ static inline void sync_core(void)
 		"pushq $1f\n\t"
 		"iretq\n\t"
 		UNWIND_HINT_RESTORE
-		"1:"
+		"1: nop\n\t"
 		: "=&r" (tmp), ASM_CALL_CONSTRAINT : : "cc", "memory");

Note that the special case very much relies on the HINT_SAVE being the first instruction of the (extended) basic block, which is only true in this one usage anyway.

#10 lore raw

Re: [PATCH v4 01/13] objtool: Remove CFI save/restore special case

In reply to Peter Zijlstra
Quoted 4 lines · Peter Zijlstra

This isn't ideal either; stuffing code with NOPs just to make annotations work is certainly sub-optimal, but given that sync_core() is stupid expensive in any case, one extra nop isn't going to be a problem here.

/me puts his hardened objtool maintainer's glasses on...

The problem is, when you do this kind of change to somebody else's code -- like adding a NOP to driver code -- there's a 90% chance they'll NACK it and tell you to fix your shit. Because they'll be happy to tell you the code itself should never be changed just to "make objtool happy".

And honestly, they'd be right, and there's not much you can say in reply. And then we end up having to fix it in objtool again anyway.

The 'insn == first' check isn't ideal, but at least it works (I think?).

Maybe there's some cleaner approach. I'll need to think about it later after I make some progress on the massive oil well fire I call my INBOX.

Signature · 1 line
Josh
author maintainer
#11 lore raw

Re: [PATCH v4 01/13] objtool: Remove CFI save/restore special case

Quoted 6 lines · Josh Poimboeuf
Quoted 4 lines · Peter Zijlstra

This isn't ideal either; stuffing code with NOPs just to make annotations work is certainly sub-optimal, but given that sync_core() is stupid expensive in any case, one extra nop isn't going to be a problem here.

Quoted 6 lines

/me puts his hardened objtool maintainer's glasses on...

The problem is, when you do this kind of change to somebody else's code -- like adding a NOP to driver code -- there's a 90% chance they'll NACK it and tell you to fix your shit. Because they'll be happy to tell you the code itself should never be changed just to "make objtool happy".

So the only objtool annotation drivers tend to run into is that uaccess crud. Drivers really had better not need the CFI annotations, otherwise they doing massively dodgy things.

And the nice thing with the uaccess crud is that they're anchored to actual instructions, as opposed to this.

Quoted 2 lines

And honestly, they'd be right, and there's not much you can say in reply. And then we end up having to fix it in objtool again anyway.

Well, I agree. I just haven't managed to come up with anything sensible.

Quoted 1 line

The 'insn == first' check isn't ideal, but at least it works (I think?).

It works, yes, for exactly this one case.

author maintainer
#12 lore raw

Re: [PATCH v4 01/13] objtool: Remove CFI save/restore special case

Quoted 3 lines · Peter Zijlstra
Quoted 1 line

The 'insn == first' check isn't ideal, but at least it works (I think?).

It works, yes, for exactly this one case.

How's this? Ignore the ignore_cfi bits, that's a 'failed' experiment.

---
 arch/x86/include/asm/orc_types.h       |   2 +
 arch/x86/include/asm/processor.h       |   6 +-
 arch/x86/include/asm/unwind_hints.h    |   4 ++
 tools/arch/x86/include/asm/orc_types.h |   2 +
 tools/objtool/check.c                  | 109 +++++++++++++++++++--------------
 tools/objtool/check.h                  |   3 +-
 6 files changed, 75 insertions(+), 51 deletions(-)
arch/x86/include/asm/orc_types.h +2 -0
index 6e060907c163..82b5c685341a 100644
@@ -60,6 +60,8 @@
 #define ORC_TYPE_REGS_IRET		2
 #define UNWIND_HINT_TYPE_SAVE		3
 #define UNWIND_HINT_TYPE_RESTORE	4
+#define UNWIND_HINT_TYPE_IGNORE		5
+#define UNWIND_HINT_TYPE_IRET_CONT	6
 
 #ifndef __ASSEMBLY__
 /*
arch/x86/include/asm/processor.h +3 -3
index 94789db550df..45c74cbc0a83 100644
@@ -728,8 +728,8 @@ static inline void sync_core(void)
 	unsigned int tmp;
 
 	asm volatile (
-		UNWIND_HINT_SAVE
 		"mov %%ss, %0\n\t"
+		UNWIND_HINT_SAVE
 		"pushq %q0\n\t"
 		"pushq %%rsp\n\t"
 		"addq $8, (%%rsp)\n\t"
@@ -737,9 +737,9 @@ static inline void sync_core(void)
 		"mov %%cs, %0\n\t"
 		"pushq %q0\n\t"
 		"pushq $1f\n\t"
+		UNWIND_HINT_IRET_CONT
 		"iretq\n\t"
-		UNWIND_HINT_RESTORE
-		"1:"
+		"1:\n\t"
 		: "=&r" (tmp), ASM_CALL_CONSTRAINT : : "cc", "memory");
 #endif
 }
arch/x86/include/asm/unwind_hints.h +4 -0
index f5e2eb12cb71..d8a07749c323 100644
@@ -112,6 +112,10 @@
 
 #define UNWIND_HINT_RESTORE UNWIND_HINT(0, 0, UNWIND_HINT_TYPE_RESTORE, 0)
 
+#define UNWIND_HINT_IGNORE UNWIND_HINT(0, 0, UNWIND_HINT_TYPE_IGNORE, 0)
+
+#define UNWIND_HINT_IRET_CONT UNWIND_HINT(0, 0, UNWIND_HINT_TYPE_IRET_CONT, 0)
+
 #endif /* __ASSEMBLY__ */
 
 #endif /* _ASM_X86_UNWIND_HINTS_H */
tools/arch/x86/include/asm/orc_types.h +2 -0
index 6e060907c163..82b5c685341a 100644
@@ -60,6 +60,8 @@
 #define ORC_TYPE_REGS_IRET		2
 #define UNWIND_HINT_TYPE_SAVE		3
 #define UNWIND_HINT_TYPE_RESTORE	4
+#define UNWIND_HINT_TYPE_IGNORE		5
+#define UNWIND_HINT_TYPE_IRET_CONT	6
 
 #ifndef __ASSEMBLY__
 /*
tools/objtool/check.c +62 -47

168 lines not on this page · raw message

tools/objtool/check.h +2 -1
index 6d875ca6fce0..f2b6172e119b 100644
@@ -33,7 +33,8 @@ struct instruction {
 	unsigned int len;
 	enum insn_type type;
 	unsigned long immediate;
-	bool alt_group, dead_end, ignore, hint, save, restore, ignore_alts;
+	bool alt_group, dead_end, ignore, ignore_alts;
+	bool hint, save, restore, ignore_cfi, iret_cont;
 	bool retpoline_safe;
 	u8 visited;
 	struct symbol *call_dest;
#13 lore raw

Re: [PATCH v4 01/13] objtool: Remove CFI save/restore special case

Quoted 6 lines · Peter Zijlstra
Quoted 3 lines · Peter Zijlstra
Quoted 1 line

The 'insn == first' check isn't ideal, but at least it works (I think?).

It works, yes, for exactly this one case.

How's this? Ignore the ignore_cfi bits, that's a 'failed' experiment.

It still seems complex to me.

What do you think about this? If we store save_insn in the state when we see insn->save, the restore logic becomes a lot easier. Then if we get a restore without a save, we can just ignore the restore hint in that path. Later, when we see the restore insn again from the save path, we can then compare the insn state with the saved state to make sure they match.

This assumes no crazy save/restore scenarios. It also means that the restore path has to be reachable from the save path, for which I had to make a change to make IRETQ *not* a dead end if there's a restore hint immediately after it.

tools/objtool/check.c +29 -47
index e637a4a38d2a..e9becd50f148 100644
@@ -1265,7 +1265,6 @@ static int read_unwind_hints(struct objtool_file *file)
 
 		} else if (hint->type == UNWIND_HINT_TYPE_RESTORE) {
 			insn->restore = true;
-			insn->hint = true;
 			continue;
 		}
 
@@ -2003,7 +2002,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
 			   struct instruction *first, struct insn_state state)
 {
 	struct alternative *alt;
-	struct instruction *insn, *next_insn;
+	struct instruction *insn, *next_insn, *save_insn = NULL;
 	struct section *sec;
 	u8 visited;
 	int ret;
@@ -2034,54 +2033,32 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
 
 		visited = 1 << state.uaccess;
 		if (insn->visited) {
-			if (!insn->hint && !insn_state_match(insn, &state))
+			if (!insn->hint && !insn->restore &&
+			    !insn_state_match(insn, &state)) {
 				return 1;
+			}
+
+			if (insn->restore && save_insn) {
+				if (!insn_state_match(insn, &save_insn->state))
+					return 1;
+				save_insn = NULL;
+			}
 
 			if (insn->visited & visited)
 				return 0;
 		}
 
-		if (insn->hint) {
-			if (insn->restore) {
-				struct instruction *save_insn, *i;
-
-				i = insn;
-				save_insn = NULL;
-				func_for_each_insn_continue_reverse(file, func, i) {
-					if (i->save) {
-						save_insn = i;
-						break;
-					}
-				}
-
-				if (!save_insn) {
-					WARN_FUNC("no corresponding CFI save for CFI restore",
-						  sec, insn->offset);
-					return 1;
-				}
-
-				if (!save_insn->visited) {
-					/*
-					 * Oops, no state to copy yet.
-					 * Hopefully we can reach this
-					 * instruction from another branch
-					 * after the save insn has been
-					 * visited.
-					 */
-					if (insn == first)
-						return 0;
-
-					WARN_FUNC("objtool isn't smart enough to handle this CFI save/restore combo",
-						  sec, insn->offset);
-					return 1;
-				}
+		if (insn->save)
+			save_insn = insn;
 
-				insn->state = save_insn->state;
-			}
+		if (insn->restore && save_insn) {
+			insn->state = save_insn->state;
+			save_insn = NULL;
+		}
 
+		if (insn->hint)
 			state = insn->state;
-
-		} else
+		else
 			insn->state = state;
 
 		insn->visited |= visited;
@@ -2191,12 +2168,17 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
 			break;
 
 		case INSN_CONTEXT_SWITCH:
-			if (func && (!next_insn || !next_insn->hint)) {
-				WARN_FUNC("unsupported instruction in callable function",
-					  sec, insn->offset);
-				return 1;
+			if (!next_insn || !next_insn->restore) {
+				if (func) {
+					WARN_FUNC("unsupported instruction in callable function",
+							sec, insn->offset);
+					return 1;
+				}
+
+				return 0;
 			}
-			return 0;
+
+			break;
 
 		case INSN_STACK:
 			if (update_insn_state(insn, &state))
@@ -2293,7 +2275,7 @@ static int validate_unwind_hints(struct objtool_file *file)
 	clear_insn_state(&state);
 
 	for_each_insn(file, insn) {
-		if (insn->hint && !insn->visited) {
+		if ((insn->hint || insn->save || insn->restore) && !insn->visited) {
 			ret = validate_branch(file, insn->func, insn, state);
 			if (ret && backtrace)
 				BT_FUNC("<=== (hint)", insn);
author maintainer
#14 lore raw

Re: [PATCH v4 01/13] objtool: Remove CFI save/restore special case

Quoted 11 lines · Josh Poimboeuf
Quoted 6 lines · Peter Zijlstra
Quoted 3 lines · Peter Zijlstra
Quoted 1 line

The 'insn == first' check isn't ideal, but at least it works (I think?).

It works, yes, for exactly this one case.

How's this? Ignore the ignore_cfi bits, that's a 'failed' experiment.

It still seems complex to me.

What do you think about this?

Perhaps, but I still don't really like it. After much thinking I have yet another proposal, see below.

Quoted 2 lines

If we store save_insn in the state when
we see insn->save, the restore logic becomes a lot easier.

Yes, that does help.

Quoted 5 lines

Then if we
get a restore without a save, we can just ignore the restore hint in that path. Later, when we see the restore insn again from the save path, we can then compare the insn state with the saved state to make sure they match.

That basically removes the error.

Quoted 1 line

This assumes no crazy save/restore scenarios.

This.

Quoted 4 lines

It also means that the
restore path has to be reachable from the save path, for which I had to make a change to make IRETQ *not* a dead end if there's a restore hint immediately after it.

Right, I did that too. But that then makes me wonder why you needed that change to validate_unwind_hints(), because if the restore hint makes it continue, you'll not have unreachable code after it.

---
Subject: objtool: Implement RET_TAIL hint

This replaces the SAVE/RESTORE hints with a RET_TAIL hint that applies to:

- regular RETURN and sibling calls (which are also function exists) it allows the stack-frame to be off by one word, ie. it allows a return-tail-call.

- EXCEPTION_RETURN (a new INSN_type that splits IRET out of CONTEXT_SWITCH) and here it denotes a return to self by having it consume arch_exception_frame_size bytes off the stack and continuing.

Apply this hint to ftrace_64.S and sync_core(), the two existing users of the SAVE/RESTORE hints.

For ftrace_64.S we split the return path and make sure the ftrace_epilogue call is seen as a sibling/tail-call turning it into it's own function.

By splitting the return path every instruction has a unique stack setup and ORC can generate correct unwinds (XXX check if/how the ftrace trampolines map into the ORC). Then employ the RET_TAIL hint to the tail-call exit that has the direct-call (orig_eax) return-tail-call on.

For sync_core() annotate the IRET with RET_TAIL to mark it as a control-flow NOP that consumes the exception frame.

XXX this adds a tail-call to ftrace_caller()
XXX compile tested only

---
 arch/x86/include/asm/orc_types.h       |  3 +-
 arch/x86/include/asm/processor.h       |  3 +-
 arch/x86/include/asm/unwind_hints.h    | 12 ++----
 arch/x86/kernel/ftrace.c               | 12 +++++-
 arch/x86/kernel/ftrace_64.S            | 27 +++++--------
 tools/arch/x86/include/asm/orc_types.h |  3 +-
 tools/objtool/Makefile                 |  2 +-
 tools/objtool/arch.h                   |  3 ++
 tools/objtool/arch/x86/decode.c        |  5 ++-
 tools/objtool/check.c                  | 74 ++++++++++------------------------
 tools/objtool/check.h                  |  3 +-
 11 files changed, 59 insertions(+), 88 deletions(-)
arch/x86/include/asm/orc_types.h +1 -2
index 6e060907c163..49536fdd29ce 100644
@@ -58,8 +58,7 @@
 #define ORC_TYPE_CALL			0
 #define ORC_TYPE_REGS			1
 #define ORC_TYPE_REGS_IRET		2
-#define UNWIND_HINT_TYPE_SAVE		3
-#define UNWIND_HINT_TYPE_RESTORE	4
+#define UNWIND_HINT_TYPE_RET_TAIL	3
 
 #ifndef __ASSEMBLY__
 /*
arch/x86/include/asm/processor.h +1 -2
index 94789db550df..cb09ee18cbab 100644
@@ -728,7 +728,6 @@ static inline void sync_core(void)
 	unsigned int tmp;
 
 	asm volatile (
-		UNWIND_HINT_SAVE
 		"mov %%ss, %0\n\t"
 		"pushq %q0\n\t"
 		"pushq %%rsp\n\t"
@@ -737,8 +736,8 @@ static inline void sync_core(void)
 		"mov %%cs, %0\n\t"
 		"pushq %q0\n\t"
 		"pushq $1f\n\t"
+		UNWIND_HINT_RET_TAIL
 		"iretq\n\t"
-		UNWIND_HINT_RESTORE
 		"1:"
 		: "=&r" (tmp), ASM_CALL_CONSTRAINT : : "cc", "memory");
 #endif
arch/x86/include/asm/unwind_hints.h +3 -9
index f5e2eb12cb71..64ec9c9a6674 100644
@@ -86,12 +86,8 @@
 	UNWIND_HINT sp_offset=\sp_offset
 .endm
 
-.macro UNWIND_HINT_SAVE
-	UNWIND_HINT type=UNWIND_HINT_TYPE_SAVE
-.endm
-
-.macro UNWIND_HINT_RESTORE
-	UNWIND_HINT type=UNWIND_HINT_TYPE_RESTORE
+.macro UNWIND_HINT_RET_TAIL
+	UNWIND_HINT type=UNWIND_HINT_TYPE_RET_TAIL
 .endm
 
 #else /* !__ASSEMBLY__ */
@@ -108,9 +104,7 @@
 	".balign 4 \n\t"					\
 	".popsection\n\t"
 
-#define UNWIND_HINT_SAVE UNWIND_HINT(0, 0, UNWIND_HINT_TYPE_SAVE, 0)
-
-#define UNWIND_HINT_RESTORE UNWIND_HINT(0, 0, UNWIND_HINT_TYPE_RESTORE, 0)
+#define UNWIND_HINT_RET_TAIL UNWIND_HINT(0, 0, UNWIND_HINT_TYPE_RET_TAIL, 0)
 
 #endif /* __ASSEMBLY__ */
 
arch/x86/kernel/ftrace.c +10 -2
index 37a0aeaf89e7..3fda2ee7bf71 100644
@@ -282,7 +282,8 @@ static inline void tramp_free(void *tramp) { }
 
 /* Defined as markers to the end of the ftrace default trampolines */
 extern void ftrace_regs_caller_end(void);
-extern void ftrace_epilogue(void);
+extern void ftrace_regs_caller_ret(void);
+extern void ftrace_caller_end(void);
 extern void ftrace_caller_op_ptr(void);
 extern void ftrace_regs_caller_op_ptr(void);
 
@@ -334,7 +335,7 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size)
 		call_offset = (unsigned long)ftrace_regs_call;
 	} else {
 		start_offset = (unsigned long)ftrace_caller;
-		end_offset = (unsigned long)ftrace_epilogue;
+		end_offset = (unsigned long)ftrace_caller_end;
 		op_offset = (unsigned long)ftrace_caller_op_ptr;
 		call_offset = (unsigned long)ftrace_call;
 	}
@@ -366,6 +367,13 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size)
 	if (WARN_ON(ret < 0))
 		goto fail;
 
+	if (ops->flags & FTRACE_OPS_FL_SAVE_REGS) {
+		ip = ftrace_regs_caller_ret;
+		ret = probe_kernel_read(ip, (void *)retq, RET_SIZE);
+		if (WARN_ON(ret < 0))
+			goto fail;
+	}
+
 	/*
 	 * The address of the ftrace_ops that is used for this trampoline
 	 * is stored at the end of the trampoline. This will be used to
arch/x86/kernel/ftrace_64.S +11 -16
index 369e61faacfe..28834d715817 100644
@@ -157,8 +157,12 @@ SYM_INNER_LABEL(ftrace_call, SYM_L_GLOBAL)
 	 * think twice before adding any new code or changing the
 	 * layout here.
 	 */
-SYM_INNER_LABEL(ftrace_epilogue, SYM_L_GLOBAL)
+SYM_INNER_LABEL(ftrace_caller_end, SYM_L_GLOBAL)
 
+	jmp ftrace_epilogue
+SYM_FUNC_END(ftrace_caller);
+
+SYM_FUNC_START(ftrace_epilogue)
 #ifdef CONFIG_FUNCTION_GRAPH_TRACER
 SYM_INNER_LABEL(ftrace_graph_call, SYM_L_GLOBAL)
 	jmp ftrace_stub
@@ -170,14 +174,12 @@ SYM_INNER_LABEL(ftrace_graph_call, SYM_L_GLOBAL)
  */
 SYM_INNER_LABEL_ALIGN(ftrace_stub, SYM_L_WEAK)
 	retq
-SYM_FUNC_END(ftrace_caller)
+SYM_FUNC_END(ftrace_epilogue)
 
 SYM_FUNC_START(ftrace_regs_caller)
 	/* Save the current flags before any operations that can change them */
 	pushfq
 
-	UNWIND_HINT_SAVE
-
 	/* added 8 bytes to save flags */
 	save_mcount_regs 8
 	/* save_mcount_regs fills in first two parameters */
@@ -244,20 +246,14 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_GLOBAL)
 	movq %rax, MCOUNT_REG_SIZE(%rsp)
 
 	restore_mcount_regs 8
+	/* Restore flags */
+	popfq
 
-	jmp	2f
+SYM_INNER_LABEL(ftrace_regs_caller_ret, SYM_L_GLOBAL);
+	UNWIND_HINT_RET_TAIL
+	jmp	ftrace_epilogue
 
 1:	restore_mcount_regs
-
-
-2:
-	/*
-	 * The stack layout is nondetermistic here, depending on which path was
-	 * taken.  This confuses objtool and ORC, rightfully so.  For now,
-	 * pretend the stack always looks like the non-direct case.
-	 */
-	UNWIND_HINT_RESTORE
-
 	/* Restore flags */
 	popfq
 
@@ -268,7 +264,6 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_GLOBAL)
 	 * to the return.
 	 */
 SYM_INNER_LABEL(ftrace_regs_caller_end, SYM_L_GLOBAL)
-
 	jmp ftrace_epilogue
 
 SYM_FUNC_END(ftrace_regs_caller)
tools/arch/x86/include/asm/orc_types.h +1 -2
index 6e060907c163..49536fdd29ce 100644
@@ -58,8 +58,7 @@
 #define ORC_TYPE_CALL			0
 #define ORC_TYPE_REGS			1
 #define ORC_TYPE_REGS_IRET		2
-#define UNWIND_HINT_TYPE_SAVE		3
-#define UNWIND_HINT_TYPE_RESTORE	4
+#define UNWIND_HINT_TYPE_RET_TAIL	3
 
 #ifndef __ASSEMBLY__
 /*
tools/objtool/Makefile +1 -1
index ee08aeff30a1..ebbf92a8836f 100644
@@ -31,7 +31,7 @@ INCLUDES := -I$(srctree)/tools/include \
 	    -I$(srctree)/tools/arch/$(HOSTARCH)/include/uapi \
 	    -I$(srctree)/tools/arch/$(SRCARCH)/include
 WARNINGS := $(EXTRA_WARNINGS) -Wno-switch-default -Wno-switch-enum -Wno-packed
-CFLAGS   := -Werror $(WARNINGS) $(KBUILD_HOSTCFLAGS) -g $(INCLUDES) $(LIBELF_FLAGS)
+CFLAGS   := -Werror $(WARNINGS) $(KBUILD_HOSTCFLAGS) -ggdb3 $(INCLUDES) $(LIBELF_FLAGS)
 LDFLAGS  += $(LIBELF_LIBS) $(LIBSUBCMD) $(KBUILD_HOSTLDFLAGS)
 
 # Allow old libelf to be used:
tools/objtool/arch.h +3 -0
index ced3765c4f44..6d04a683f8cc 100644
@@ -19,6 +19,7 @@ enum insn_type {
 	INSN_CALL,
 	INSN_CALL_DYNAMIC,
 	INSN_RETURN,
+	INSN_EXCEPTION_RETURN,
 	INSN_CONTEXT_SWITCH,
 	INSN_STACK,
 	INSN_BUG,
@@ -73,6 +74,8 @@ int arch_decode_instruction(struct elf *elf, struct section *sec,
 			    unsigned int *len, enum insn_type *type,
 			    unsigned long *immediate, struct stack_op *op);
 
+static const int arch_exception_frame_size = 5*8;
+
 bool arch_callee_saved_reg(unsigned char reg);
 
 #endif /* _ARCH_H */
tools/objtool/arch/x86/decode.c +4 -1
index a62e032863a8..1f9dd97e4a50 100644
@@ -431,10 +431,13 @@ int arch_decode_instruction(struct elf *elf, struct section *sec,
 
 	case 0xca: /* retf */
 	case 0xcb: /* retf */
-	case 0xcf: /* iret */
 		*type = INSN_CONTEXT_SWITCH;
 		break;
 
+	case 0xcf: /* iret */
+		*type = INSN_EXCEPTION_RETURN;
+		break;
+
 	case 0xe8:
 		*type = INSN_CALL;
 		break;
tools/objtool/check.c +22 -52

118 lines not on this page · raw message

tools/objtool/check.h +2 -1
index 6d875ca6fce0..138533ac23e2 100644
@@ -33,7 +33,8 @@ struct instruction {
 	unsigned int len;
 	enum insn_type type;
 	unsigned long immediate;
-	bool alt_group, dead_end, ignore, hint, save, restore, ignore_alts;
+	bool alt_group, dead_end, ignore, ignore_alts;
+	bool hint, ret_tail;
 	bool retpoline_safe;
 	u8 visited;
 	struct symbol *call_dest;
#15 lore raw

Re: [PATCH v4 01/13] objtool: Remove CFI save/restore special case

Quoted 19 lines · Peter Zijlstra

Subject: objtool: Implement RET_TAIL hint

This replaces the SAVE/RESTORE hints with a RET_TAIL hint that applies to:

- regular RETURN and sibling calls (which are also function exists) it allows the stack-frame to be off by one word, ie. it allows a return-tail-call.

- EXCEPTION_RETURN (a new INSN_type that splits IRET out of CONTEXT_SWITCH) and here it denotes a return to self by having it consume arch_exception_frame_size bytes off the stack and continuing.

Apply this hint to ftrace_64.S and sync_core(), the two existing users of the SAVE/RESTORE hints.

For ftrace_64.S we split the return path and make sure the ftrace_epilogue call is seen as a sibling/tail-call turning it into it's own function.

Quoted 7 lines

By splitting the return path every instruction has a unique stack setup and ORC can generate correct unwinds (XXX check if/how the ftrace trampolines map into the ORC). Then employ the RET_TAIL hint to the tail-call exit that has the direct-call (orig_eax) return-tail-call on.

For sync_core() annotate the IRET with RET_TAIL to mark it as a control-flow NOP that consumes the exception frame.

I do like the idea to get rid of SAVE/RESTORE altogether. And it's nice to make that ftrace code unwinder-deterministic.

However sync_core() and ftrace_regs_caller() are very different from each other and I find the RET_TAIL hint usage to be extremely confusing.

For example, IRETQ isn't even a tail cail.

And the need for the hint to come *before* the insn which changes the state is different from the other hints.

And now objtool has to know the arch exception stack size because of a single code site.

And for a proper tail call, the stack should be empty. I don't understand the +8 thing in has_modified_stack_frame(). It seems hard-coded for the weird ftrace case, rather than for tail calls in general (which should already work as designed).

How about a more general hint like UNWIND_HINT_ADJUST?

For sync_core(), after the IRETQ:

UNWIND_HINT_ADJUST sp_add=40

And ftrace_regs_caller_ret could have:

UNWIND_HINT_ADJUST sp_add=8

Signature · 1 line
Josh
author maintainer
#16 lore raw

Re: [PATCH v4 01/13] objtool: Remove CFI save/restore special case

Quoted 25 lines · Josh Poimboeuf
Quoted 24 lines · Peter Zijlstra

Subject: objtool: Implement RET_TAIL hint

This replaces the SAVE/RESTORE hints with a RET_TAIL hint that applies to:

- regular RETURN and sibling calls (which are also function exists) it allows the stack-frame to be off by one word, ie. it allows a return-tail-call.

- EXCEPTION_RETURN (a new INSN_type that splits IRET out of CONTEXT_SWITCH) and here it denotes a return to self by having it consume arch_exception_frame_size bytes off the stack and continuing.

Apply this hint to ftrace_64.S and sync_core(), the two existing users of the SAVE/RESTORE hints.

For ftrace_64.S we split the return path and make sure the ftrace_epilogue call is seen as a sibling/tail-call turning it into it's own function.

By splitting the return path every instruction has a unique stack setup and ORC can generate correct unwinds (XXX check if/how the ftrace trampolines map into the ORC). Then employ the RET_TAIL hint to the tail-call exit that has the direct-call (orig_eax) return-tail-call on.

Quoted 8 lines
Quoted 2 lines

For sync_core() annotate the IRET with RET_TAIL to mark it as a control-flow NOP that consumes the exception frame.

I do like the idea to get rid of SAVE/RESTORE altogether. And it's nice to make that ftrace code unwinder-deterministic.

However sync_core() and ftrace_regs_caller() are very different from each other and I find the RET_TAIL hint usage to be extremely confusing.

I was going with the pattern:

push target
ret

which is an indirect tail-call that doesn't need a register. We use it in various places. We use it here exactly because it preserves all registers, but we use it in function-graph tracer and retprobes to insert the return handler. But also in retpoline, because it uses the return stack predictor, which by happy accident isn't the indirect branch predictor.

Quoted 1 line

For example, IRETQ isn't even a tail cail.

It's the same indirect call, except with a bigger frame ;-)

push # ss
push # rsp
push # flags
push # cs
push # ip
iret

Quoted 2 lines

And the need for the hint to come *before* the insn which changes the state is different from the other hints.

makes sense to me... but yah.

Quoted 2 lines

And now objtool has to know the arch exception stack size because of a single code site.

Agreed.

Quoted 1 line

And for a proper tail call, the stack should be empty.

All depends what you call proper :-)

Quoted 2 lines

I don't
understand the +8 thing in has_modified_stack_frame().

push target
ret

means we hit ret with one extra word on the stack.

Quoted 3 lines

It seems
hard-coded for the weird ftrace case, rather than for tail calls in general (which should already work as designed).

Like I said, we have it all over the place, but I suspect they're all mostly hidden from objtool.

Quoted 9 lines

How about a more general hint like UNWIND_HINT_ADJUST?

For sync_core(), after the IRETQ:

UNWIND_HINT_ADJUST sp_add=40

And ftrace_regs_caller_ret could have:

UNWIND_HINT_ADJUST sp_add=8

I like, I'll make it happen in the morning.

author maintainer
#17 lore raw

Re: [PATCH v4 01/13] objtool: Remove CFI save/restore special case

Quoted 27 lines · Peter Zijlstra
Quoted 2 lines · Josh Poimboeuf

However sync_core() and ftrace_regs_caller() are very different from each other and I find the RET_TAIL hint usage to be extremely confusing.

I was going with the pattern:

push target
ret

which is an indirect tail-call that doesn't need a register. We use it in various places. We use it here exactly because it preserves all registers, but we use it in function-graph tracer and retprobes to insert the return handler. But also in retpoline, because it uses the return stack predictor, which by happy accident isn't the indirect branch predictor.

Quoted 1 line

For example, IRETQ isn't even a tail cail.

It's the same indirect call, except with a bigger frame ;-)

push # ss
push # rsp
push # flags
push # cs
push # ip
iret

Quoted 4 lines
Quoted 2 lines

And the need for the hint to come *before* the insn which changes the state is different from the other hints.

makes sense to me... but yah.

Also, naturally, there are no instructions after RET to stick the annotation to.

author maintainer
#18 lore raw

RFC PATCH 1/1 objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

In reply to Peter Zijlstra
Quoted 6 lines · Peter Zijlstra
Quoted 4 lines · Josh Poimboeuf

How about a more general hint like UNWIND_HINT_ADJUST?

For sync_core(), after the IRETQ:

Quoted 7 lines
Quoted 5 lines

UNWIND_HINT_ADJUST sp_add=40

And ftrace_regs_caller_ret could have:

UNWIND_HINT_ADJUST sp_add=8

I like, I'll make it happen in the morning.

Still compile tested only. But I did find orc_ftrace_find() which should indeed allow ORC unwinding of the ftrace trampolines.

---
Subject: objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

This replaces the SAVE/RESTORE hints with a RET_OFFSET hint that applies to the following instructions:

- any instruction that terminates a function, like: RETURN and sibling calls. It allows the stack-frame to be off by @sp_offset, ie. it allows stuffing the return stack.

- EXCEPTION_RETURN (a new INSN_type that splits IRET out of CONTEXT_SWITCH) and here it denotes a @sp_offset sized POP and makes the instruction continue.

Apply this hint to ftrace_64.S and sync_core(), the two existing users of the SAVE/RESTORE hints.

For ftrace_64.S we split the return path and make sure the ftrace_epilogue call is seen as a sibling/tail-call turning it into it's own function.

By splitting the return path every instruction has a unique stack setup and ORC can generate correct unwinds. Then employ the RET_OFFSET hint to the tail-call exit that has the direct-call (orig_eax) stuffed on the return stack.

For sync_core() annotate the IRET with RET_OFFSET to mark it as a control-flow NOP that consumes the exception frame.

---
 arch/x86/include/asm/orc_types.h       | 11 ++++-
 arch/x86/include/asm/processor.h       |  3 +-
 arch/x86/include/asm/unwind_hints.h    | 12 ++----
 arch/x86/kernel/ftrace.c               | 12 +++++-
 arch/x86/kernel/ftrace_64.S            | 31 ++++++--------
 tools/arch/x86/include/asm/orc_types.h | 11 ++++-
 tools/objtool/Makefile                 |  2 +-
 tools/objtool/arch.h                   |  1 +
 tools/objtool/arch/x86/decode.c        |  5 ++-
 tools/objtool/check.c                  | 75 +++++++++++-----------------------
 tools/objtool/check.h                  |  4 +-
 11 files changed, 77 insertions(+), 90 deletions(-)
arch/x86/include/asm/orc_types.h +9 -2
index 6e060907c163..0133ec8d19b6 100644
@@ -58,8 +58,15 @@
 #define ORC_TYPE_CALL			0
 #define ORC_TYPE_REGS			1
 #define ORC_TYPE_REGS_IRET		2
-#define UNWIND_HINT_TYPE_SAVE		3
-#define UNWIND_HINT_TYPE_RESTORE	4
+
+/*
+ * RET_OFFSET: Used on instructions that terminate a function; mostly RETURN
+ * and sibling calls. On these, sp_offset denotes the expected offset from
+ * initial_func_cfi. It can also be used on EXCEPTION_RETURN instructions
+ * where sp_offset will denote the exception frame size consumed and will
+ * make objtool assume the instruction is a fall-through.
+ */
+#define UNWIND_HINT_TYPE_RET_OFFSET	3
 
 #ifndef __ASSEMBLY__
 /*
arch/x86/include/asm/processor.h +1 -2
index 94789db550df..c8e0496e25dd 100644
@@ -728,7 +728,6 @@ static inline void sync_core(void)
 	unsigned int tmp;
 
 	asm volatile (
-		UNWIND_HINT_SAVE
 		"mov %%ss, %0\n\t"
 		"pushq %q0\n\t"
 		"pushq %%rsp\n\t"
@@ -737,8 +736,8 @@ static inline void sync_core(void)
 		"mov %%cs, %0\n\t"
 		"pushq %q0\n\t"
 		"pushq $1f\n\t"
+		UNWIND_HINT_RET_OFFSET(5*8)
 		"iretq\n\t"
-		UNWIND_HINT_RESTORE
 		"1:"
 		: "=&r" (tmp), ASM_CALL_CONSTRAINT : : "cc", "memory");
 #endif
arch/x86/include/asm/unwind_hints.h +3 -9
index f5e2eb12cb71..c4acd99716c1 100644
@@ -86,12 +86,8 @@
 	UNWIND_HINT sp_offset=\sp_offset
 .endm
 
-.macro UNWIND_HINT_SAVE
-	UNWIND_HINT type=UNWIND_HINT_TYPE_SAVE
-.endm
-
-.macro UNWIND_HINT_RESTORE
-	UNWIND_HINT type=UNWIND_HINT_TYPE_RESTORE
+.macro UNWIND_HINT_RET_OFFSET sp_offset=8
+	UNWIND_HINT type=UNWIND_HINT_TYPE_RET_OFFSET sp_offset=\sp_offset
 .endm
 
 #else /* !__ASSEMBLY__ */
@@ -108,9 +104,7 @@
 	".balign 4 \n\t"					\
 	".popsection\n\t"
 
-#define UNWIND_HINT_SAVE UNWIND_HINT(0, 0, UNWIND_HINT_TYPE_SAVE, 0)
-
-#define UNWIND_HINT_RESTORE UNWIND_HINT(0, 0, UNWIND_HINT_TYPE_RESTORE, 0)
+#define UNWIND_HINT_RET_OFFSET(offset) UNWIND_HINT(0, (offset), UNWIND_HINT_TYPE_RET_OFFSET, 0)
 
 #endif /* __ASSEMBLY__ */
 
arch/x86/kernel/ftrace.c +10 -2
index 37a0aeaf89e7..3fda2ee7bf71 100644
@@ -282,7 +282,8 @@ static inline void tramp_free(void *tramp) { }
 
 /* Defined as markers to the end of the ftrace default trampolines */
 extern void ftrace_regs_caller_end(void);
-extern void ftrace_epilogue(void);
+extern void ftrace_regs_caller_ret(void);
+extern void ftrace_caller_end(void);
 extern void ftrace_caller_op_ptr(void);
 extern void ftrace_regs_caller_op_ptr(void);
 
@@ -334,7 +335,7 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size)
 		call_offset = (unsigned long)ftrace_regs_call;
 	} else {
 		start_offset = (unsigned long)ftrace_caller;
-		end_offset = (unsigned long)ftrace_epilogue;
+		end_offset = (unsigned long)ftrace_caller_end;
 		op_offset = (unsigned long)ftrace_caller_op_ptr;
 		call_offset = (unsigned long)ftrace_call;
 	}
@@ -366,6 +367,13 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size)
 	if (WARN_ON(ret < 0))
 		goto fail;
 
+	if (ops->flags & FTRACE_OPS_FL_SAVE_REGS) {
+		ip = ftrace_regs_caller_ret;
+		ret = probe_kernel_read(ip, (void *)retq, RET_SIZE);
+		if (WARN_ON(ret < 0))
+			goto fail;
+	}
+
 	/*
 	 * The address of the ftrace_ops that is used for this trampoline
 	 * is stored at the end of the trampoline. This will be used to
arch/x86/kernel/ftrace_64.S +13 -18
index 369e61faacfe..1e4a82ff97ea 100644
@@ -157,8 +157,12 @@ SYM_INNER_LABEL(ftrace_call, SYM_L_GLOBAL)
 	 * think twice before adding any new code or changing the
 	 * layout here.
 	 */
-SYM_INNER_LABEL(ftrace_epilogue, SYM_L_GLOBAL)
+SYM_INNER_LABEL(ftrace_caller_end, SYM_L_GLOBAL)
 
+	jmp ftrace_epilogue
+SYM_FUNC_END(ftrace_caller);
+
+SYM_FUNC_START(ftrace_epilogue)
 #ifdef CONFIG_FUNCTION_GRAPH_TRACER
 SYM_INNER_LABEL(ftrace_graph_call, SYM_L_GLOBAL)
 	jmp ftrace_stub
@@ -170,14 +174,12 @@ SYM_INNER_LABEL(ftrace_graph_call, SYM_L_GLOBAL)
  */
 SYM_INNER_LABEL_ALIGN(ftrace_stub, SYM_L_WEAK)
 	retq
-SYM_FUNC_END(ftrace_caller)
+SYM_FUNC_END(ftrace_epilogue)
 
 SYM_FUNC_START(ftrace_regs_caller)
 	/* Save the current flags before any operations that can change them */
 	pushfq
 
-	UNWIND_HINT_SAVE
-
 	/* added 8 bytes to save flags */
 	save_mcount_regs 8
 	/* save_mcount_regs fills in first two parameters */
@@ -235,8 +237,8 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_GLOBAL)
 
 	/* If ORIG_RAX is anything but zero, make this a call to that */
 	movq ORIG_RAX(%rsp), %rax
-	cmpq	$0, %rax
-	je	1f
+	testq	%rax, %rax
+	jz	1f
 
 	/* Swap the flags with orig_rax */
 	movq MCOUNT_REG_SIZE(%rsp), %rdi
@@ -244,20 +246,14 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_GLOBAL)
 	movq %rax, MCOUNT_REG_SIZE(%rsp)
 
 	restore_mcount_regs 8
+	/* Restore flags */
+	popfq
 
-	jmp	2f
+SYM_INNER_LABEL(ftrace_regs_caller_ret, SYM_L_GLOBAL);
+	UNWIND_HINT_RET_OFFSET
+	jmp	ftrace_epilogue
 
 1:	restore_mcount_regs
-
-
-2:
-	/*
-	 * The stack layout is nondetermistic here, depending on which path was
-	 * taken.  This confuses objtool and ORC, rightfully so.  For now,
-	 * pretend the stack always looks like the non-direct case.
-	 */
-	UNWIND_HINT_RESTORE
-
 	/* Restore flags */
 	popfq
 
@@ -268,7 +264,6 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_GLOBAL)
 	 * to the return.
 	 */
 SYM_INNER_LABEL(ftrace_regs_caller_end, SYM_L_GLOBAL)
-
 	jmp ftrace_epilogue
 
 SYM_FUNC_END(ftrace_regs_caller)
tools/arch/x86/include/asm/orc_types.h +9 -2
index 6e060907c163..0133ec8d19b6 100644
@@ -58,8 +58,15 @@
 #define ORC_TYPE_CALL			0
 #define ORC_TYPE_REGS			1
 #define ORC_TYPE_REGS_IRET		2
-#define UNWIND_HINT_TYPE_SAVE		3
-#define UNWIND_HINT_TYPE_RESTORE	4
+
+/*
+ * RET_OFFSET: Used on instructions that terminate a function; mostly RETURN
+ * and sibling calls. On these, sp_offset denotes the expected offset from
+ * initial_func_cfi. It can also be used on EXCEPTION_RETURN instructions
+ * where sp_offset will denote the exception frame size consumed and will
+ * make objtool assume the instruction is a fall-through.
+ */
+#define UNWIND_HINT_TYPE_RET_OFFSET	3
 
 #ifndef __ASSEMBLY__
 /*
tools/objtool/Makefile +1 -1
index ee08aeff30a1..ebbf92a8836f 100644
@@ -31,7 +31,7 @@ INCLUDES := -I$(srctree)/tools/include \
 	    -I$(srctree)/tools/arch/$(HOSTARCH)/include/uapi \
 	    -I$(srctree)/tools/arch/$(SRCARCH)/include
 WARNINGS := $(EXTRA_WARNINGS) -Wno-switch-default -Wno-switch-enum -Wno-packed
-CFLAGS   := -Werror $(WARNINGS) $(KBUILD_HOSTCFLAGS) -g $(INCLUDES) $(LIBELF_FLAGS)
+CFLAGS   := -Werror $(WARNINGS) $(KBUILD_HOSTCFLAGS) -ggdb3 $(INCLUDES) $(LIBELF_FLAGS)
 LDFLAGS  += $(LIBELF_LIBS) $(LIBSUBCMD) $(KBUILD_HOSTLDFLAGS)
 
 # Allow old libelf to be used:
tools/objtool/arch.h +1 -0
index ced3765c4f44..cba72e1c47ce 100644
@@ -19,6 +19,7 @@ enum insn_type {
 	INSN_CALL,
 	INSN_CALL_DYNAMIC,
 	INSN_RETURN,
+	INSN_EXCEPTION_RETURN,
 	INSN_CONTEXT_SWITCH,
 	INSN_STACK,
 	INSN_BUG,
tools/objtool/arch/x86/decode.c +4 -1
index a62e032863a8..1f9dd97e4a50 100644
@@ -431,10 +431,13 @@ int arch_decode_instruction(struct elf *elf, struct section *sec,
 
 	case 0xca: /* retf */
 	case 0xcb: /* retf */
-	case 0xcf: /* iret */
 		*type = INSN_CONTEXT_SWITCH;
 		break;
 
+	case 0xcf: /* iret */
+		*type = INSN_EXCEPTION_RETURN;
+		break;
+
 	case 0xe8:
 		*type = INSN_CALL;
 		break;
tools/objtool/check.c +23 -52

119 lines not on this page · raw message

tools/objtool/check.h +3 -1
index 6d875ca6fce0..aaccaef8f074 100644
@@ -33,9 +33,11 @@ struct instruction {
 	unsigned int len;
 	enum insn_type type;
 	unsigned long immediate;
-	bool alt_group, dead_end, ignore, hint, save, restore, ignore_alts;
+	bool alt_group, dead_end, ignore, ignore_alts;
+	bool hint;
 	bool retpoline_safe;
 	u8 visited;
+	u8 ret_offset;
 	struct symbol *call_dest;
 	struct instruction *jump_dest;
 	struct instruction *first_jump_src;
maintainer
#19 lore raw

Re: [RFC][PATCH] objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

arch/x86/kernel/ftrace_64.S SYM_INNER_LABEL(ftrace_regs_call, SYM_L_GLOBAL) · line 244

Quoted 11 lines · On Tue, 31 Mar 2020 13:16:52 +0200 Peter Zijlstra <[email protected]> wrote:
arch/x86/kernel/ftrace_64.S +2 -2
@@ -235,8 +237,8 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_GLOBAL)
 
 	/* If ORIG_RAX is anything but zero, make this a call to that */
 	movq ORIG_RAX(%rsp), %rax
-	cmpq	$0, %rax
-	je	1f
+	testq	%rax, %rax
+	jz	1f
 
 	/* Swap the flags with orig_rax */
 	movq MCOUNT_REG_SIZE(%rsp), %rdi

Hi Peter,

Can you send this change as a separate patch as it has nothing to do with this current change, and is a clean up patch that stands on its own.

Thanks,

-- Steve

author maintainer
#20 lore raw

RFC PATCH 1/1 x86,ftrace: Shrink ftrace_regs_caller() by one byte

Quoted 12 lines · Steven Rostedt
Quoted 9 lines · On Tue, 31 Mar 2020 13:16:52 +0200 Peter Zijlstra <[email protected]> wrote:
(quoted diff) +2 -2
@@ -235,8 +237,8 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_GLOBAL)
 
 	/* If ORIG_RAX is anything but zero, make this a call to that */
 	movq ORIG_RAX(%rsp), %rax
-	cmpq	$0, %rax
-	je	1f
+	testq	%rax, %rax
+	jz	1f
 
Quoted 7 lines

Review SYM_INNER_LABEL(ftrace_regs_call, SYM_L_GLOBAL) · line 244

Quoted 2 lines
(quoted diff) +0 -0
 	/* Swap the flags with orig_rax */
 	movq MCOUNT_REG_SIZE(%rsp), %rdi

Hi Peter,

Can you send this change as a separate patch as it has nothing to do with this current change, and is a clean up patch that stands on its own.

Sure. But then I have to like write a Changelog for it... :/

---
Subject: x86,ftrace: Shrink ftrace_regs_caller() by one byte

'Optimize' ftrace_regs_caller. Instead of comparing against an immediate, the more natural way to test for zero on x86 is: 'test %r,%r'.

  48 83 f8 00             cmp    $0x0,%rax
  74 49                   je     226 <ftrace_regs_call+0xa3>
  48 85 c0                test   %rax,%rax
  74 49                   je     225 <ftrace_regs_call+0xa2>
---
 arch/x86/kernel/ftrace_64.S | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/kernel/ftrace_64.S +2 -2
index 369e61faacfe..8e71c492d623 100644
@@ -235,8 +235,8 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_GLOBAL)
 
 	/* If ORIG_RAX is anything but zero, make this a call to that */
 	movq ORIG_RAX(%rsp), %rax
-	cmpq	$0, %rax
-	je	1f
+	testq	%rax, %rax
+	jz	1f
 
 	/* Swap the flags with orig_rax */
 	movq MCOUNT_REG_SIZE(%rsp), %rdi
author maintainer
#21 lore raw

Re: [RFC][PATCH] objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

In reply to Steven Rostedt
Quoted 2 lines · Steven Rostedt

Can you send this change as a separate patch as it has nothing to do with this current change, and is a clean up patch that stands on its own.

I also found this.. should I write it up?

---

arch/x86/kernel/ftrace_64.S +2 -2
index 369e61faacfe..0f108096f278 100644
@@ -23,7 +23,7 @@
 #endif /* CONFIG_FRAME_POINTER */
 
 /* Size of stack used to save mcount regs in save_mcount_regs */
-#define MCOUNT_REG_SIZE		(SS+8 + MCOUNT_FRAME_SIZE)
+#define MCOUNT_REG_SIZE		(SIZEOF_PTREGS + MCOUNT_FRAME_SIZE)
 
 /*
  * gcc -pg option adds a call to 'mcount' in most functions.
@@ -77,7 +77,7 @@
 	/*
 	 * We add enough stack to save all regs.
 	 */
-	subq $(MCOUNT_REG_SIZE - MCOUNT_FRAME_SIZE), %rsp
+	subq $(SIZEOF_PTREGS), %rsp
 	movq %rax, RAX(%rsp)
 	movq %rcx, RCX(%rsp)
 	movq %rdx, RDX(%rsp)
#22 lore raw

Re: [RFC][PATCH] objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

Quoted 22 lines · Peter Zijlstra
Quoted 2 lines · Steven Rostedt

Can you send this change as a separate patch as it has nothing to do with this current change, and is a clean up patch that stands on its own.

I also found this.. should I write it up?

---

arch/x86/kernel/ftrace_64.S +1 -1
index 369e61faacfe..0f108096f278 100644
@@ -23,7 +23,7 @@
 #endif /* CONFIG_FRAME_POINTER */
 
 /* Size of stack used to save mcount regs in save_mcount_regs */
-#define MCOUNT_REG_SIZE		(SS+8 + MCOUNT_FRAME_SIZE)
+#define MCOUNT_REG_SIZE		(SIZEOF_PTREGS + MCOUNT_FRAME_SIZE)
 
 /*
  * gcc -pg option adds a call to 'mcount' in most functions.

arch/x86/kernel/ftrace_64.S line 83

Quoted 8 lines
arch/x86/kernel/ftrace_64.S +1 -1
@@ -77,7 +77,7 @@
 	/*
 	 * We add enough stack to save all regs.
 	 */
-	subq $(MCOUNT_REG_SIZE - MCOUNT_FRAME_SIZE), %rsp
+	subq $(SIZEOF_PTREGS), %rsp
 	movq %rax, RAX(%rsp)
 	movq %rcx, RCX(%rsp)
 	movq %rdx, RDX(%rsp)

I was going to suggest the same thing :-)

Signature · 1 line
Josh
#23 lore raw

Re: [RFC][PATCH] objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

In reply to Peter Zijlstra
Quoted 5 lines · Peter Zijlstra

Subject: objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

This replaces the SAVE/RESTORE hints with a RET_OFFSET hint that applies to the following instructions:

Quoted 7 lines

- any instruction that terminates a function, like: RETURN and sibling calls. It allows the stack-frame to be off by @sp_offset, ie. it allows stuffing the return stack.

- EXCEPTION_RETURN (a new INSN_type that splits IRET out of CONTEXT_SWITCH) and here it denotes a @sp_offset sized POP and makes the instruction continue.

Looking closer, I see how my UNWIND_HINT_ADJUST idea doesn't work for the ftrace_regs_caller() case. The ORC data is actually correct there. So basically we need a way to tell objtool to be quiet.

I now understand what you're trying to do with the RET_TAIL thing, and I guess it's ok for the ftrace case. But I'd rather an UNWIND_HINT_IGNORE before the tail cail, which would tell objtool to just silence the tail call warning. It's simpler for the user to understand, it's simpler logic in objtool, and I think an "ignore warnings for the next insn" hint would be more generally applicable anyway.

But also... the RET_OFFSET usage for sync_core() *really* bugs me.

I know you said it's like an indirect tail call with a bigger frame, but that's kind of stretching it because the function frame is still there.

And objtool doesn't treat it like a tail call at all. In fact, it handles it *completely* differently from the normal ret-tail-call case. Instead of silencing a tail call warning, it adjusts the stack offset and continues the code path.

This basically adds *two* new hint types, while trying to call them the same thing. There's no overlapping functionality between them in objtool, other than the use of the same insn->ret_offset variable. But it's two distinct functionalities, depending on the context (return/tail vs IRETQ).

I'll try to work up some patches with a different approach in a bit.

Signature · 1 line
Josh
author maintainer
#24 lore raw

Re: [RFC][PATCH] objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

Quoted 10 lines · Josh Poimboeuf
Quoted 9 lines · Peter Zijlstra

Subject: objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

This replaces the SAVE/RESTORE hints with a RET_OFFSET hint that applies to the following instructions:

- any instruction that terminates a function, like: RETURN and sibling calls. It allows the stack-frame to be off by @sp_offset, ie. it allows stuffing the return stack.

Quoted 7 lines
Quoted 3 lines

- EXCEPTION_RETURN (a new INSN_type that splits IRET out of CONTEXT_SWITCH) and here it denotes a @sp_offset sized POP and makes the instruction continue.

Looking closer, I see how my UNWIND_HINT_ADJUST idea doesn't work for the ftrace_regs_caller() case. The ORC data is actually correct there. So basically we need a way to tell objtool to be quiet.

Right.

Quoted 6 lines

I now understand what you're trying to do with the RET_TAIL thing, and I guess it's ok for the ftrace case. But I'd rather an UNWIND_HINT_IGNORE before the tail cail, which would tell objtool to just silence the tail call warning. It's simpler for the user to understand, it's simpler logic in objtool, and I think an "ignore warnings for the next insn" hint would be more generally applicable anyway.

I like how this is specific on how far the stack can be off, as opposed so say 'ignore any warning on this instruction'.

Because by saying this RET should be +8, we'll still get a warning when this is not the case (and in fact I should strengthen the patch to implement that).

Also, you don't want to suppress any other valid warning at that instruction.

Furthermore, I really don't think we ought to worry about ease-of-use here, there's really not that many people writing x86 assembly.

Quoted 1 line

But also... the RET_OFFSET usage for sync_core() *really* bugs me.

Fair enough.

Quoted 8 lines

I know you said it's like an indirect tail call with a bigger frame, but that's kind of stretching it because the function frame is still there.

And objtool doesn't treat it like a tail call at all. In fact, it handles it *completely* differently from the normal ret-tail-call case. Instead of silencing a tail call warning, it adjusts the stack offset and continues the code path.

Quoted 5 lines

This basically adds *two* new hint types, while trying to call them the same thing. There's no overlapping functionality between them in objtool, other than the use of the same insn->ret_offset variable. But it's two distinct functionalities, depending on the context (return/tail vs IRETQ).

I'm not against adding a second/separate hint for this. In fact, I almost considered teaching objtool how to interpret the whole IRET frame so that we can do it without hints. It's just that that's too much code for this one case.

HINT_IRET_SELF ?

author maintainer
#25 lore raw

Re: [RFC][PATCH] objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

Quoted 9 lines · Peter Zijlstra
Quoted 6 lines · Josh Poimboeuf

I now understand what you're trying to do with the RET_TAIL thing, and I guess it's ok for the ftrace case. But I'd rather an UNWIND_HINT_IGNORE before the tail cail, which would tell objtool to just silence the tail call warning. It's simpler for the user to understand, it's simpler logic in objtool, and I think an "ignore warnings for the next insn" hint would be more generally applicable anyway.

Quoted 6 lines

I like how this is specific on how far the stack can be off, as opposed so say 'ignore any warning on this instruction'.

Because by saying this RET should be +8, we'll still get a warning when this is not the case (and in fact I should strengthen the patch to implement that).

Like this; I'm confused on what cfa.offset is vs stack_size though.

But this way we're strict and always warn when the unexpected happens.

tools/objtool/check.c +1 -2
@@ -1423,8 +1423,7 @@ static bool has_modified_stack_frame(str
 	    !(ret_offset && state->cfa.offset == initial_func_cfi.cfa.offset + ret_offset))
 		return true;
 
-	if (state->stack_size != initial_func_cfi.cfa.offset &&
-	    !(ret_offset && state->stack_size == initial_func_cfi.cfa.offset + ret_offset))
+	if (state->stack_size != initial_func_cfi.cfa.offset + ret_offset)
 		return true;
 
 	for (i = 0; i < CFI_NUM_REGS; i++) {
#26 lore raw

Re: [RFC][PATCH] objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

In reply to Peter Zijlstra
Quoted 10 lines · Peter Zijlstra
Quoted 6 lines

I now understand what you're trying to do with the RET_TAIL thing, and I guess it's ok for the ftrace case. But I'd rather an UNWIND_HINT_IGNORE before the tail cail, which would tell objtool to just silence the tail call warning. It's simpler for the user to understand, it's simpler logic in objtool, and I think an "ignore warnings for the next insn" hint would be more generally applicable anyway.

I like how this is specific on how far the stack can be off, as opposed so say 'ignore any warning on this instruction'.

Quoted 6 lines

Because by saying this RET should be +8, we'll still get a warning when this is not the case (and in fact I should strengthen the patch to implement that).

Also, you don't want to suppress any other valid warning at that instruction.

Ok, I guess I'm convinced :-) As we continue to add new warnings to objtool, it is true that "ignore all warnings at this insn" is probably too broad.

/me stops writing patch

BTW, if we're in agreement that this hint doesn't belong for sync_core(), will sp_offset always be +8? Just wondering if we can hard-code that assumption.

Quoted 14 lines
Quoted 8 lines

I know you said it's like an indirect tail call with a bigger frame, but that's kind of stretching it because the function frame is still there.

And objtool doesn't treat it like a tail call at all. In fact, it handles it *completely* differently from the normal ret-tail-call case. Instead of silencing a tail call warning, it adjusts the stack offset and continues the code path.

Quoted 5 lines

This basically adds *two* new hint types, while trying to call them the same thing. There's no overlapping functionality between them in objtool, other than the use of the same insn->ret_offset variable. But it's two distinct functionalities, depending on the context (return/tail vs IRETQ).

Quoted 6 lines

I'm not against adding a second/separate hint for this. In fact, I almost considered teaching objtool how to interpret the whole IRET frame so that we can do it without hints. It's just that that's too much code for this one case.

HINT_IRET_SELF ?

Despite my earlier complaint about stack size knowledge, we could just forget the hint and make "iretq in C code" equivalent to "reduce stack size by arch_exception_stack_size()" and keep going. There's file->c_file which tells you it's a C file.

Signature · 1 line
Josh
#27 lore raw

Re: [RFC][PATCH] objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

Quoted 11 lines · Josh Poimboeuf
Quoted 6 lines

I'm not against adding a second/separate hint for this. In fact, I almost considered teaching objtool how to interpret the whole IRET frame so that we can do it without hints. It's just that that's too much code for this one case.

HINT_IRET_SELF ?

Despite my earlier complaint about stack size knowledge, we could just forget the hint and make "iretq in C code" equivalent to "reduce stack size by arch_exception_stack_size()" and keep going. There's file->c_file which tells you it's a C file.

Or maybe "iretq in an STT_FUNC" is better since this pattern could presumably happen in a callable asm function.

Signature · 1 line
Josh
author maintainer
#28 lore raw

PATCH v2 1/1 objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

Quoted 8 lines · Josh Poimboeuf
Quoted 7 lines · Josh Poimboeuf
Quoted 6 lines

I'm not against adding a second/separate hint for this. In fact, I almost considered teaching objtool how to interpret the whole IRET frame so that we can do it without hints. It's just that that's too much code for this one case.

HINT_IRET_SELF ?

Quoted 7 lines
Quoted 4 lines

Despite my earlier complaint about stack size knowledge, we could just forget the hint and make "iretq in C code" equivalent to "reduce stack size by arch_exception_stack_size()" and keep going. There's file->c_file which tells you it's a C file.

Or maybe "iretq in an STT_FUNC" is better since this pattern could presumably happen in a callable asm function.

Like so then?

---
Subject: objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET
From: Peter Zijlstra <[email protected]>
Date: Tue, 31 Mar 2020 13:16:52 +0200

This replaces the SAVE/RESTORE hints with a RET_OFFSET hint that applies to any instruction that terminates a function, like: RETURN and sibling calls. It allows the stack-frame to be off by @sp_offset, ie. it allows stuffing the return stack.

For ftrace_64.S we split the return path and make sure the ftrace_epilogue call is seen as a sibling/tail-call turning it into it's own function.

By splitting the return path every instruction has a unique stack setup and ORC can generate correct unwinds. Then employ the RET_OFFSET hint to the tail-call exit that has the direct-call (orig_eax) stuffed on the return stack.

For sync_core() we teach objtool that an IRET inside an STT_FUNC simply consumes the exception stack and continues.

---
 arch/x86/include/asm/orc_types.h       |    9 ++-
 arch/x86/include/asm/processor.h       |    2
 arch/x86/include/asm/unwind_hints.h    |   12 +---
 arch/x86/kernel/ftrace.c               |   12 ++++
 arch/x86/kernel/ftrace_64.S            |   27 ++++-------
 tools/arch/x86/include/asm/orc_types.h |    9 ++-
 tools/objtool/Makefile                 |    2
 tools/objtool/arch.h                   |    3 +
 tools/objtool/arch/x86/decode.c        |    5 +-
 tools/objtool/check.c                  |   80 ++++++++++-----------------------
 tools/objtool/check.h                  |    4 +
 11 files changed, 74 insertions(+), 91 deletions(-)
arch/x86/include/asm/orc_types.h +7 -2
@@ -58,8 +58,13 @@
 #define ORC_TYPE_CALL			0
 #define ORC_TYPE_REGS			1
 #define ORC_TYPE_REGS_IRET		2
-#define UNWIND_HINT_TYPE_SAVE		3
-#define UNWIND_HINT_TYPE_RESTORE	4
+
+/*
+ * RET_OFFSET: Used on instructions that terminate a function; mostly RETURN
+ * and sibling calls. On these, sp_offset denotes the expected offset from
+ * initial_func_cfi.
+ */
+#define UNWIND_HINT_TYPE_RET_OFFSET	3
 
 #ifndef __ASSEMBLY__
 /*
arch/x86/include/asm/processor.h +0 -2
@@ -738,7 +738,6 @@ static inline void sync_core(void)
 	unsigned int tmp;
 
 	asm volatile (
-		UNWIND_HINT_SAVE
 		"mov %%ss, %0\n\t"
 		"pushq %q0\n\t"
 		"pushq %%rsp\n\t"
@@ -748,7 +747,6 @@ static inline void sync_core(void)
 		"pushq %q0\n\t"
 		"pushq $1f\n\t"
 		"iretq\n\t"
-		UNWIND_HINT_RESTORE
 		"1:"
 		: "=&r" (tmp), ASM_CALL_CONSTRAINT : : "cc", "memory");
 #endif
arch/x86/include/asm/unwind_hints.h +3 -9
@@ -86,12 +86,8 @@
 	UNWIND_HINT sp_offset=\sp_offset
 .endm
 
-.macro UNWIND_HINT_SAVE
-	UNWIND_HINT type=UNWIND_HINT_TYPE_SAVE
-.endm
-
-.macro UNWIND_HINT_RESTORE
-	UNWIND_HINT type=UNWIND_HINT_TYPE_RESTORE
+.macro UNWIND_HINT_RET_OFFSET sp_offset=8
+	UNWIND_HINT type=UNWIND_HINT_TYPE_RET_OFFSET sp_offset=\sp_offset
 .endm
 
 #else /* !__ASSEMBLY__ */
@@ -108,9 +104,7 @@
 	".balign 4 \n\t"					\
 	".popsection\n\t"
 
-#define UNWIND_HINT_SAVE UNWIND_HINT(0, 0, UNWIND_HINT_TYPE_SAVE, 0)
-
-#define UNWIND_HINT_RESTORE UNWIND_HINT(0, 0, UNWIND_HINT_TYPE_RESTORE, 0)
+#define UNWIND_HINT_RET_OFFSET(offset) UNWIND_HINT(0, (offset), UNWIND_HINT_TYPE_RET_OFFSET, 0)
 
 #endif /* __ASSEMBLY__ */
 
arch/x86/kernel/ftrace.c +10 -2
@@ -282,7 +282,8 @@ static inline void tramp_free(void *tram
 
 /* Defined as markers to the end of the ftrace default trampolines */
 extern void ftrace_regs_caller_end(void);
-extern void ftrace_epilogue(void);
+extern void ftrace_regs_caller_ret(void);
+extern void ftrace_caller_end(void);
 extern void ftrace_caller_op_ptr(void);
 extern void ftrace_regs_caller_op_ptr(void);
 
@@ -334,7 +335,7 @@ create_trampoline(struct ftrace_ops *ops
 		call_offset = (unsigned long)ftrace_regs_call;
 	} else {
 		start_offset = (unsigned long)ftrace_caller;
-		end_offset = (unsigned long)ftrace_epilogue;
+		end_offset = (unsigned long)ftrace_caller_end;
 		op_offset = (unsigned long)ftrace_caller_op_ptr;
 		call_offset = (unsigned long)ftrace_call;
 	}
@@ -366,6 +367,13 @@ create_trampoline(struct ftrace_ops *ops
 	if (WARN_ON(ret < 0))
 		goto fail;
 
+	if (ops->flags & FTRACE_OPS_FL_SAVE_REGS) {
+		ip = ftrace_regs_caller_ret;
+		ret = probe_kernel_read(ip, (void *)retq, RET_SIZE);
+		if (WARN_ON(ret < 0))
+			goto fail;
+	}
+
 	/*
 	 * The address of the ftrace_ops that is used for this trampoline
 	 * is stored at the end of the trampoline. This will be used to
arch/x86/kernel/ftrace_64.S +11 -16
@@ -157,8 +157,12 @@ SYM_INNER_LABEL(ftrace_call, SYM_L_GLOBA
 	 * think twice before adding any new code or changing the
 	 * layout here.
 	 */
-SYM_INNER_LABEL(ftrace_epilogue, SYM_L_GLOBAL)
+SYM_INNER_LABEL(ftrace_caller_end, SYM_L_GLOBAL)
 
+	jmp ftrace_epilogue
+SYM_FUNC_END(ftrace_caller);
+
+SYM_FUNC_START(ftrace_epilogue)
 #ifdef CONFIG_FUNCTION_GRAPH_TRACER
 SYM_INNER_LABEL(ftrace_graph_call, SYM_L_GLOBAL)
 	jmp ftrace_stub
@@ -170,14 +174,12 @@ SYM_INNER_LABEL(ftrace_graph_call, SYM_L
  */
 SYM_INNER_LABEL_ALIGN(ftrace_stub, SYM_L_WEAK)
 	retq
-SYM_FUNC_END(ftrace_caller)
+SYM_FUNC_END(ftrace_epilogue)
 
 SYM_FUNC_START(ftrace_regs_caller)
 	/* Save the current flags before any operations that can change them */
 	pushfq
 
-	UNWIND_HINT_SAVE
-
 	/* added 8 bytes to save flags */
 	save_mcount_regs 8
 	/* save_mcount_regs fills in first two parameters */
@@ -244,20 +246,14 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_
 	movq %rax, MCOUNT_REG_SIZE(%rsp)
 
 	restore_mcount_regs 8
+	/* Restore flags */
+	popfq
 
-	jmp	2f
+SYM_INNER_LABEL(ftrace_regs_caller_ret, SYM_L_GLOBAL);
+	UNWIND_HINT_RET_OFFSET
+	jmp	ftrace_epilogue
 
 1:	restore_mcount_regs
-
-
-2:
-	/*
-	 * The stack layout is nondetermistic here, depending on which path was
-	 * taken.  This confuses objtool and ORC, rightfully so.  For now,
-	 * pretend the stack always looks like the non-direct case.
-	 */
-	UNWIND_HINT_RESTORE
-
 	/* Restore flags */
 	popfq
 
@@ -268,7 +264,6 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_
 	 * to the return.
 	 */
 SYM_INNER_LABEL(ftrace_regs_caller_end, SYM_L_GLOBAL)
-
 	jmp ftrace_epilogue
 
 SYM_FUNC_END(ftrace_regs_caller)
tools/arch/x86/include/asm/orc_types.h +7 -2
@@ -58,8 +58,13 @@
 #define ORC_TYPE_CALL			0
 #define ORC_TYPE_REGS			1
 #define ORC_TYPE_REGS_IRET		2
-#define UNWIND_HINT_TYPE_SAVE		3
-#define UNWIND_HINT_TYPE_RESTORE	4
+
+/*
+ * RET_OFFSET: Used on instructions that terminate a function; mostly RETURN
+ * and sibling calls. On these, sp_offset denotes the expected offset from
+ * initial_func_cfi.
+ */
+#define UNWIND_HINT_TYPE_RET_OFFSET	3
 
 #ifndef __ASSEMBLY__
 /*
tools/objtool/Makefile +1 -1
@@ -31,7 +31,7 @@ INCLUDES := -I$(srctree)/tools/include \
 	    -I$(srctree)/tools/arch/$(HOSTARCH)/include/uapi \
 	    -I$(srctree)/tools/arch/$(SRCARCH)/include
 WARNINGS := $(EXTRA_WARNINGS) -Wno-switch-default -Wno-switch-enum -Wno-packed
-CFLAGS   := -Werror $(WARNINGS) $(KBUILD_HOSTCFLAGS) -g $(INCLUDES) $(LIBELF_FLAGS)
+CFLAGS   := -Werror $(WARNINGS) $(KBUILD_HOSTCFLAGS) -ggdb3 $(INCLUDES) $(LIBELF_FLAGS)
 LDFLAGS  += $(LIBELF_LIBS) $(LIBSUBCMD) $(KBUILD_HOSTLDFLAGS)
 
 # Allow old libelf to be used:
tools/objtool/arch.h +3 -0
@@ -19,6 +19,7 @@ enum insn_type {
 	INSN_CALL,
 	INSN_CALL_DYNAMIC,
 	INSN_RETURN,
+	INSN_EXCEPTION_RETURN,
 	INSN_CONTEXT_SWITCH,
 	INSN_STACK,
 	INSN_BUG,
@@ -75,4 +76,6 @@ int arch_decode_instruction(struct elf *
 
 bool arch_callee_saved_reg(unsigned char reg);
 
+static const int arch_exception_frame_size = 5*8;
+
 #endif /* _ARCH_H */
tools/objtool/arch/x86/decode.c +4 -1
@@ -431,10 +431,13 @@ int arch_decode_instruction(struct elf *
 
 	case 0xca: /* retf */
 	case 0xcb: /* retf */
-	case 0xcf: /* iret */
 		*type = INSN_CONTEXT_SWITCH;
 		break;
 
+	case 0xcf: /* iret */
+		*type = INSN_EXCEPTION_RETURN;
+		break;
+
 	case 0xe8:
 		*type = INSN_CALL;
 		break;
tools/objtool/check.c +25 -55

125 lines not on this page · raw message

tools/objtool/check.h +3 -1
@@ -33,9 +33,11 @@ struct instruction {
 	unsigned int len;
 	enum insn_type type;
 	unsigned long immediate;
-	bool alt_group, dead_end, ignore, hint, save, restore, ignore_alts;
+	bool alt_group, dead_end, ignore, ignore_alts;
+	bool hint;
 	bool retpoline_safe;
 	u8 visited;
+	u8 ret_offset;
 	struct symbol *call_dest;
 	struct instruction *jump_dest;
 	struct instruction *first_jump_src;
#29 lore raw

Re: [PATCH v2] objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

Quoted 14 lines · Peter Zijlstra
Quoted 13 lines · Josh Poimboeuf
Quoted 11 lines · Josh Poimboeuf
Quoted 6 lines

I'm not against adding a second/separate hint for this. In fact, I almost considered teaching objtool how to interpret the whole IRET frame so that we can do it without hints. It's just that that's too much code for this one case.

HINT_IRET_SELF ?

Despite my earlier complaint about stack size knowledge, we could just forget the hint and make "iretq in C code" equivalent to "reduce stack size by arch_exception_stack_size()" and keep going. There's file->c_file which tells you it's a C file.

Quoted 4 lines
Quoted 2 lines

Or maybe "iretq in an STT_FUNC" is better since this pattern could presumably happen in a callable asm function.

Like so then?

I'd suggest a patch split like:

1) objtool: automagic IRET-in-func
2) objtool: add RET_OFFSET
3) ftrace: re-organize asm (and use RET_OFFSET hint)
4) objtool: remove now-unused SAVE/RESTORE

Quoted 5 lines
arch/x86/include/asm/orc_types.h +0 -0
@@ -58,8 +58,13 @@
 #define ORC_TYPE_CALL			0
 #define ORC_TYPE_REGS			1

arch/x86/include/asm/orc_types.h line 67

Quoted 10 lines
arch/x86/include/asm/orc_types.h +7 -2
 #define ORC_TYPE_REGS_IRET		2
-#define UNWIND_HINT_TYPE_SAVE		3
-#define UNWIND_HINT_TYPE_RESTORE	4
+
+/*
+ * RET_OFFSET: Used on instructions that terminate a function; mostly RETURN
+ * and sibling calls. On these, sp_offset denotes the expected offset from
+ * initial_func_cfi.
+ */
+#define UNWIND_HINT_TYPE_RET_OFFSET	3

I think this comment belongs at the UNWIND_HINT_RET_OFFSET macro definition.

Quoted 22 lines
arch/x86/kernel/ftrace.c +3 -2
@@ -282,7 +282,8 @@ static inline void tramp_free(void *tram
 
 /* Defined as markers to the end of the ftrace default trampolines */
 extern void ftrace_regs_caller_end(void);
-extern void ftrace_epilogue(void);
+extern void ftrace_regs_caller_ret(void);
+extern void ftrace_caller_end(void);
 extern void ftrace_caller_op_ptr(void);
 extern void ftrace_regs_caller_op_ptr(void);
 
@@ -334,7 +335,7 @@ create_trampoline(struct ftrace_ops *ops
 		call_offset = (unsigned long)ftrace_regs_call;
 	} else {
 		start_offset = (unsigned long)ftrace_caller;
-		end_offset = (unsigned long)ftrace_epilogue;
+		end_offset = (unsigned long)ftrace_caller_end;
 		op_offset = (unsigned long)ftrace_caller_op_ptr;
 		call_offset = (unsigned long)ftrace_call;
 	}

arch/x86/kernel/ftrace.c create_trampoline(struct ftrace_ops *ops · line 376

Quoted 10 lines
arch/x86/kernel/ftrace.c +7 -0
@@ -366,6 +367,13 @@ create_trampoline(struct ftrace_ops *ops
 	if (WARN_ON(ret < 0))
 		goto fail;
 
+	if (ops->flags & FTRACE_OPS_FL_SAVE_REGS) {
+		ip = ftrace_regs_caller_ret;
+		ret = probe_kernel_read(ip, (void *)retq, RET_SIZE);
+		if (WARN_ON(ret < 0))
+			goto fail;
+	}
+

Hm? This function creates a trampoline but it looks like this change is overwriting the original ftrace_64 code itself?

tools/objtool/Makefile INCLUDES := -I$(srctree)/tools/include \ · line 35

Quoted 9 lines
tools/objtool/Makefile +1 -1
@@ -31,7 +31,7 @@ INCLUDES := -I$(srctree)/tools/include \
 	    -I$(srctree)/tools/arch/$(HOSTARCH)/include/uapi \
 	    -I$(srctree)/tools/arch/$(SRCARCH)/include
 WARNINGS := $(EXTRA_WARNINGS) -Wno-switch-default -Wno-switch-enum -Wno-packed
-CFLAGS   := -Werror $(WARNINGS) $(KBUILD_HOSTCFLAGS) -g $(INCLUDES) $(LIBELF_FLAGS)
+CFLAGS   := -Werror $(WARNINGS) $(KBUILD_HOSTCFLAGS) -ggdb3 $(INCLUDES) $(LIBELF_FLAGS)
 LDFLAGS  += $(LIBELF_LIBS) $(LIBSUBCMD) $(KBUILD_HOSTLDFLAGS)

Why? Smells like a separate patch at least.

Signature · 1 line
Josh
author maintainer
#30 lore raw

Re: [PATCH v2] objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

Quoted 18 lines · Josh Poimboeuf
Quoted 17 lines · Peter Zijlstra
Quoted 15 lines · Josh Poimboeuf
Quoted 11 lines · Josh Poimboeuf
Quoted 6 lines

I'm not against adding a second/separate hint for this. In fact, I almost considered teaching objtool how to interpret the whole IRET frame so that we can do it without hints. It's just that that's too much code for this one case.

HINT_IRET_SELF ?

Despite my earlier complaint about stack size knowledge, we could just forget the hint and make "iretq in C code" equivalent to "reduce stack size by arch_exception_stack_size()" and keep going. There's file->c_file which tells you it's a C file.

Or maybe "iretq in an STT_FUNC" is better since this pattern could presumably happen in a callable asm function.

Quoted 8 lines
Quoted 1 line

Like so then?

I'd suggest a patch split like:

1) objtool: automagic IRET-in-func
2) objtool: add RET_OFFSET
3) ftrace: re-organize asm (and use RET_OFFSET hint)
4) objtool: remove now-unused SAVE/RESTORE

Sure.

Quoted 10 lines
Quoted 10 lines
arch/x86/include/asm/orc_types.h +2 -2
@@ -58,8 +58,13 @@
 #define ORC_TYPE_CALL			0
 #define ORC_TYPE_REGS			1
 #define ORC_TYPE_REGS_IRET		2
-#define UNWIND_HINT_TYPE_SAVE		3
-#define UNWIND_HINT_TYPE_RESTORE	4
+
+/*
Quoted 8 lines

arch/x86/include/asm/orc_types.h line 67

Quoted 5 lines
arch/x86/include/asm/orc_types.h +5 -0
+ * RET_OFFSET: Used on instructions that terminate a function; mostly RETURN
+ * and sibling calls. On these, sp_offset denotes the expected offset from
+ * initial_func_cfi.
+ */
+#define UNWIND_HINT_TYPE_RET_OFFSET	3

I think this comment belongs at the UNWIND_HINT_RET_OFFSET macro definition.

Humph, ok, but there's two of those :/

Quoted 25 lines
Quoted 25 lines
arch/x86/kernel/ftrace.c +3 -2
@@ -282,7 +282,8 @@ static inline void tramp_free(void *tram
 
 /* Defined as markers to the end of the ftrace default trampolines */
 extern void ftrace_regs_caller_end(void);
-extern void ftrace_epilogue(void);
+extern void ftrace_regs_caller_ret(void);
+extern void ftrace_caller_end(void);
 extern void ftrace_caller_op_ptr(void);
 extern void ftrace_regs_caller_op_ptr(void);
 
@@ -334,7 +335,7 @@ create_trampoline(struct ftrace_ops *ops
 		call_offset = (unsigned long)ftrace_regs_call;
 	} else {
 		start_offset = (unsigned long)ftrace_caller;
-		end_offset = (unsigned long)ftrace_epilogue;
+		end_offset = (unsigned long)ftrace_caller_end;
 		op_offset = (unsigned long)ftrace_caller_op_ptr;
 		call_offset = (unsigned long)ftrace_call;
 	}
@@ -366,6 +367,13 @@ create_trampoline(struct ftrace_ops *ops
 	if (WARN_ON(ret < 0))
 		goto fail;
 
Quoted 10 lines

arch/x86/kernel/ftrace.c create_trampoline(struct ftrace_ops *ops · line 376

Quoted 7 lines
arch/x86/kernel/ftrace.c +7 -0
+	if (ops->flags & FTRACE_OPS_FL_SAVE_REGS) {
+		ip = ftrace_regs_caller_ret;
+		ret = probe_kernel_read(ip, (void *)retq, RET_SIZE);
+		if (WARN_ON(ret < 0))
+			goto fail;
+	}
+

Hm? This function creates a trampoline but it looks like this change is overwriting the original ftrace_64 code itself?

Ahh. So if you look at what the trampoline copies, you'll note we'll copy until -- but *NOT* including -- the jmp ftrace_epilogue. Instead we'll write a RET at the end.

However, due to splitting the return path, such that each instruction has a unique stack offset, we now have a second jmp ftrace_epilogue in the middle of the function. That too needs to be overwritten by a RET.

Quoted 11 lines

tools/objtool/Makefile INCLUDES := -I$(srctree)/tools/include \ · line 35

Quoted 9 lines
tools/objtool/Makefile +1 -1
@@ -31,7 +31,7 @@ INCLUDES := -I$(srctree)/tools/include \
 	    -I$(srctree)/tools/arch/$(HOSTARCH)/include/uapi \
 	    -I$(srctree)/tools/arch/$(SRCARCH)/include
 WARNINGS := $(EXTRA_WARNINGS) -Wno-switch-default -Wno-switch-enum -Wno-packed
-CFLAGS   := -Werror $(WARNINGS) $(KBUILD_HOSTCFLAGS) -g $(INCLUDES) $(LIBELF_FLAGS)
+CFLAGS   := -Werror $(WARNINGS) $(KBUILD_HOSTCFLAGS) -ggdb3 $(INCLUDES) $(LIBELF_FLAGS)
 LDFLAGS  += $(LIBELF_LIBS) $(LIBSUBCMD) $(KBUILD_HOSTLDFLAGS)

Why? Smells like a separate patch at least.

Oh, whoops :-) I keep doing this every time I need to run gdb on it. I'll make it go away.

#31 lore raw

Re: [PATCH v2] objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

Quoted 19 lines · Peter Zijlstra
Quoted 10 lines
Quoted 10 lines
arch/x86/include/asm/orc_types.h +2 -2
@@ -58,8 +58,13 @@
 #define ORC_TYPE_CALL			0
 #define ORC_TYPE_REGS			1
 #define ORC_TYPE_REGS_IRET		2
-#define UNWIND_HINT_TYPE_SAVE		3
-#define UNWIND_HINT_TYPE_RESTORE	4
+
+/*
Quoted 8 lines

arch/x86/include/asm/orc_types.h line 67

Quoted 5 lines
arch/x86/include/asm/orc_types.h +5 -0
+ * RET_OFFSET: Used on instructions that terminate a function; mostly RETURN
+ * and sibling calls. On these, sp_offset denotes the expected offset from
+ * initial_func_cfi.
+ */
+#define UNWIND_HINT_TYPE_RET_OFFSET	3

I think this comment belongs at the UNWIND_HINT_RET_OFFSET macro definition.

Quoted 1 line

Humph, ok, but there's two of those :/

Just commenting the asm one should be fine I think.

That way a reader of code using the macro is more likely to understand its purpose.

Quoted 36 lines
Quoted 25 lines
Quoted 25 lines
arch/x86/kernel/ftrace.c +3 -2
@@ -282,7 +282,8 @@ static inline void tramp_free(void *tram
 
 /* Defined as markers to the end of the ftrace default trampolines */
 extern void ftrace_regs_caller_end(void);
-extern void ftrace_epilogue(void);
+extern void ftrace_regs_caller_ret(void);
+extern void ftrace_caller_end(void);
 extern void ftrace_caller_op_ptr(void);
 extern void ftrace_regs_caller_op_ptr(void);
 
@@ -334,7 +335,7 @@ create_trampoline(struct ftrace_ops *ops
 		call_offset = (unsigned long)ftrace_regs_call;
 	} else {
 		start_offset = (unsigned long)ftrace_caller;
-		end_offset = (unsigned long)ftrace_epilogue;
+		end_offset = (unsigned long)ftrace_caller_end;
 		op_offset = (unsigned long)ftrace_caller_op_ptr;
 		call_offset = (unsigned long)ftrace_call;
 	}
@@ -366,6 +367,13 @@ create_trampoline(struct ftrace_ops *ops
 	if (WARN_ON(ret < 0))
 		goto fail;
 
Quoted 10 lines

arch/x86/kernel/ftrace.c create_trampoline(struct ftrace_ops *ops · line 376

Quoted 7 lines
arch/x86/kernel/ftrace.c +7 -0
+	if (ops->flags & FTRACE_OPS_FL_SAVE_REGS) {
+		ip = ftrace_regs_caller_ret;
+		ret = probe_kernel_read(ip, (void *)retq, RET_SIZE);
+		if (WARN_ON(ret < 0))
+			goto fail;
+	}
+

Hm? This function creates a trampoline but it looks like this change is overwriting the original ftrace_64 code itself?

Quoted 7 lines

Ahh. So if you look at what the trampoline copies, you'll note we'll copy until -- but *NOT* including -- the jmp ftrace_epilogue. Instead we'll write a RET at the end.

However, due to splitting the return path, such that each instruction has a unique stack offset, we now have a second jmp ftrace_epilogue in the middle of the function. That too needs to be overwritten by a RET.

Right, but 'ip' needs to point to the trampoline's version of 'ftrace_regs_caller_ret', not the original ftrace_64 version.

Quoted 12 lines
Quoted 11 lines

tools/objtool/Makefile INCLUDES := -I$(srctree)/tools/include \ · line 35

Quoted 9 lines
tools/objtool/Makefile +1 -1
@@ -31,7 +31,7 @@ INCLUDES := -I$(srctree)/tools/include \
 	    -I$(srctree)/tools/arch/$(HOSTARCH)/include/uapi \
 	    -I$(srctree)/tools/arch/$(SRCARCH)/include
 WARNINGS := $(EXTRA_WARNINGS) -Wno-switch-default -Wno-switch-enum -Wno-packed
-CFLAGS   := -Werror $(WARNINGS) $(KBUILD_HOSTCFLAGS) -g $(INCLUDES) $(LIBELF_FLAGS)
+CFLAGS   := -Werror $(WARNINGS) $(KBUILD_HOSTCFLAGS) -ggdb3 $(INCLUDES) $(LIBELF_FLAGS)
 LDFLAGS  += $(LIBELF_LIBS) $(LIBSUBCMD) $(KBUILD_HOSTLDFLAGS)

Why? Smells like a separate patch at least.

Quoted 2 lines

Oh, whoops :-) I keep doing this every time I need to run gdb on it. I'll make it go away.

Ha, I do something similar, I just add -O0 to CFLAGS.

Signature · 1 line
Josh
author maintainer
#32 lore raw

Re: [PATCH v2] objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

Quoted 6 lines · Josh Poimboeuf
Quoted 4 lines · Peter Zijlstra
Quoted 4 lines
Quoted 4 lines
arch/x86/kernel/ftrace.c +0 -0
@@ -366,6 +367,13 @@ create_trampoline(struct ftrace_ops *ops
 	if (WARN_ON(ret < 0))
 		goto fail;
 
Quoted 10 lines
Quoted 7 lines
Quoted 7 lines

arch/x86/kernel/ftrace.c create_trampoline(struct ftrace_ops *ops · line 376

Quoted 7 lines
arch/x86/kernel/ftrace.c +7 -0
+	if (ops->flags & FTRACE_OPS_FL_SAVE_REGS) {
+		ip = ftrace_regs_caller_ret;
+		ret = probe_kernel_read(ip, (void *)retq, RET_SIZE);
+		if (WARN_ON(ret < 0))
+			goto fail;
+	}
+

Right, but 'ip' needs to point to the trampoline's version of 'ftrace_regs_caller_ret', not the original ftrace_64 version.

Duh,

		ip = trampline + (ftrace_regs_caller_ret - ftrace_regs_caller);

it is.

Luckily this would've triggerd a splat the first time I'd have actually booted this, because the regular text is RO, while the trampoline is still RW here. Still, good spotting.

maintainer
#33 lore raw

Re: [PATCH v2] objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

In reply to Josh Poimboeuf
Quoted 15 lines · On Wed, 1 Apr 2020 09:39:16 -0500 Josh Poimboeuf <[email protected]> wrote:
Quoted 15 lines
Quoted 10 lines

arch/x86/kernel/ftrace.c create_trampoline(struct ftrace_ops *ops · line 376

Quoted 7 lines
arch/x86/kernel/ftrace.c +7 -0
+	if (ops->flags & FTRACE_OPS_FL_SAVE_REGS) {
+		ip = ftrace_regs_caller_ret;
+		ret = probe_kernel_read(ip, (void *)retq, RET_SIZE);
+		if (WARN_ON(ret < 0))
+			goto fail;
+	}
+  

Hm? This function creates a trampoline but it looks like this change is overwriting the original ftrace_64 code itself?

Ahh. So if you look at what the trampoline copies, you'll note we'll copy until -- but *NOT* including -- the jmp ftrace_epilogue. Instead we'll write a RET at the end.

Quoted 6 lines
Quoted 3 lines

However, due to splitting the return path, such that each instruction has a unique stack offset, we now have a second jmp ftrace_epilogue in the middle of the function. That too needs to be overwritten by a RET.

Right, but 'ip' needs to point to the trampoline's version of 'ftrace_regs_caller_ret', not the original ftrace_64 version.

I noticed the same thing. And after applying just this patch (with tweaking the obtool mark ups to make it compile) it crashed.

-- Steve

#34 lore raw

Re: [PATCH v2] objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

In reply to Peter Zijlstra

Hi Peter,

Quoted 56 lines · Peter Zijlstra
Quoted 8 lines · Josh Poimboeuf
Quoted 7 lines · Josh Poimboeuf
Quoted 6 lines

I'm not against adding a second/separate hint for this. In fact, I almost considered teaching objtool how to interpret the whole IRET frame so that we can do it without hints. It's just that that's too much code for this one case.

HINT_IRET_SELF ?

Quoted 7 lines
Quoted 4 lines

Despite my earlier complaint about stack size knowledge, we could just forget the hint and make "iretq in C code" equivalent to "reduce stack size by arch_exception_stack_size()" and keep going. There's file->c_file which tells you it's a C file.

Or maybe "iretq in an STT_FUNC" is better since this pattern could presumably happen in a callable asm function.

Like so then?

---
Subject: objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET
From: Peter Zijlstra <[email protected]>
Date: Tue, 31 Mar 2020 13:16:52 +0200

This replaces the SAVE/RESTORE hints with a RET_OFFSET hint that applies to any instruction that terminates a function, like: RETURN and sibling calls. It allows the stack-frame to be off by @sp_offset, ie. it allows stuffing the return stack.

For ftrace_64.S we split the return path and make sure the ftrace_epilogue call is seen as a sibling/tail-call turning it into it's own function.

By splitting the return path every instruction has a unique stack setup and ORC can generate correct unwinds. Then employ the RET_OFFSET hint to the tail-call exit that has the direct-call (orig_eax) stuffed on the return stack.

For sync_core() we teach objtool that an IRET inside an STT_FUNC simply consumes the exception stack and continues.

---
  arch/x86/include/asm/orc_types.h       |    9 ++-
  arch/x86/include/asm/processor.h       |    2
  arch/x86/include/asm/unwind_hints.h    |   12 +---
  arch/x86/kernel/ftrace.c               |   12 ++++
  arch/x86/kernel/ftrace_64.S            |   27 ++++-------
  tools/arch/x86/include/asm/orc_types.h |    9 ++-
  tools/objtool/Makefile                 |    2
  tools/objtool/arch.h                   |    3 +
  tools/objtool/arch/x86/decode.c        |    5 +-
  tools/objtool/check.c                  |   80 ++++++++++-----------------------
  tools/objtool/check.h                  |    4 +
  11 files changed, 74 insertions(+), 91 deletions(-)

[snip]

Quoted 27 lines
tools/objtool/check.c +4 -8
@@ -1246,13 +1246,8 @@ static int read_unwind_hints(struct objt
 
  		cfa = &insn->state.cfa;
 
-		if (hint->type == UNWIND_HINT_TYPE_SAVE) {
-			insn->save = true;
-			continue;
-
-		} else if (hint->type == UNWIND_HINT_TYPE_RESTORE) {
-			insn->restore = true;
-			insn->hint = true;
+		if (hint->type == UNWIND_HINT_TYPE_RET_OFFSET) {
+			insn->ret_offset = hint->sp_offset;
  			continue;
  		}
 
@@ -1416,20 +1411,26 @@ static bool is_fentry_call(struct instru
  	return false;
  }
 
-static bool has_modified_stack_frame(struct insn_state *state)
+static bool has_modified_stack_frame(struct instruction *insn, struct insn_state *state)
  {
+	u8 ret_offset = insn->ret_offset;
  	int i;

tools/objtool/check.c static bool is_fentry_call(struct instru · line 1423

Quoted 10 lines
tools/objtool/check.c +5 -4
 
-	if (state->cfa.base != initial_func_cfi.cfa.base ||
-	    state->cfa.offset != initial_func_cfi.cfa.offset ||
-	    state->stack_size != initial_func_cfi.cfa.offset ||
-	    state->drap)
+	if (state->cfa.base != initial_func_cfi.cfa.base || state->drap)
+		return true;
+
+	if (state->cfa.offset != initial_func_cfi.cfa.offset &&
+	    !(ret_offset && state->cfa.offset == initial_func_cfi.cfa.offset + ret_offset))

Isn't that the same thing as "state->cfa.offset != initial_func_cfi.cfa.offset + ret_offset" ?

Quoted 83 lines
tools/objtool/check.c +9 -43
+		return true;
+
+	if (state->stack_size != initial_func_cfi.cfa.offset + ret_offset)
  		return true;
 
-	for (i = 0; i < CFI_NUM_REGS; i++)
+	for (i = 0; i < CFI_NUM_REGS; i++) {
  		if (state->regs[i].base != initial_func_cfi.regs[i].base ||
  		    state->regs[i].offset != initial_func_cfi.regs[i].offset)
  			return true;
+	}
 
  	return false;
  }
@@ -1971,7 +1972,7 @@ static int validate_call(struct instruct
 
  static int validate_sibling_call(struct instruction *insn, struct insn_state *state)
  {
-	if (has_modified_stack_frame(state)) {
+	if (has_modified_stack_frame(insn, state)) {
  		WARN_FUNC("sibling call from callable instruction with modified stack frame",
  				insn->sec, insn->offset);
  		return 1;
@@ -2000,7 +2001,7 @@ static int validate_return(struct symbol
  		return 1;
  	}
 
-	if (func && has_modified_stack_frame(state)) {
+	if (func && has_modified_stack_frame(insn, state)) {
  		WARN_FUNC("return with modified stack frame",
  			  insn->sec, insn->offset);
  		return 1;
@@ -2063,47 +2064,9 @@ static int validate_branch(struct objtoo
  				return 0;
  		}
 
-		if (insn->hint) {
-			if (insn->restore) {
-				struct instruction *save_insn, *i;
-
-				i = insn;
-				save_insn = NULL;
-				sym_for_each_insn_continue_reverse(file, func, i) {
-					if (i->save) {
-						save_insn = i;
-						break;
-					}
-				}
-
-				if (!save_insn) {
-					WARN_FUNC("no corresponding CFI save for CFI restore",
-						  sec, insn->offset);
-					return 1;
-				}
-
-				if (!save_insn->visited) {
-					/*
-					 * Oops, no state to copy yet.
-					 * Hopefully we can reach this
-					 * instruction from another branch
-					 * after the save insn has been
-					 * visited.
-					 */
-					if (insn == first)
-						return 0;
-
-					WARN_FUNC("objtool isn't smart enough to handle this CFI save/restore combo",
-						  sec, insn->offset);
-					return 1;
-				}
-
-				insn->state = save_insn->state;
-			}
-
+		if (insn->hint)
  			state = insn->state;
-
-		} else
+		else
  			insn->state = state;
 
  		insn->visited |= visited;

tools/objtool/check.c static int validate_branch(struct objtoo · line 2154

Quoted 7 lines
tools/objtool/check.c +4 -0
@@ -2185,6 +2148,13 @@ static int validate_branch(struct objtoo
 
  			break;
 
+		case INSN_EXCEPTION_RETURN:
+			if (func) {
+				state.stack_size -= arch_exception_frame_size;
+				break;

Why break instead of returning? Shouldn't an exception return mark the end of a branch (whether inside or outside a function) ?

Here it seems it will continue to the next instruction which might have been unreachable.

tools/objtool/check.c static int validate_branch(struct objtoo · line 2157

Quoted 3 lines
tools/objtool/check.c +3 -0
+			}
+
+			/* fallthrough */

What is the purpose of the fallthrough here? If the exception return was in a function, it carried on to the next instruction, so it won't use the WARN_FUNC(). So, if I'm looking at the right version of the code only the "return 0;" will be used. And, unless my previous comment is wrong, I'd argue that we should return both for func and !func.

Quoted 10 lines
tools/objtool/check.c +0 -0
  		case INSN_CONTEXT_SWITCH:
  			if (func && (!next_insn || !next_insn->hint)) {
  				WARN_FUNC("unsupported instruction in callable function",
tools/objtool/check.h +0 -0
@@ -33,9 +33,11 @@ struct instruction {
  	unsigned int len;
  	enum insn_type type;

tools/objtool/check.h struct instruction { · line 43

Quoted 10 lines
tools/objtool/check.h +3 -1
  	unsigned long immediate;
-	bool alt_group, dead_end, ignore, hint, save, restore, ignore_alts;
+	bool alt_group, dead_end, ignore, ignore_alts;
+	bool hint;
  	bool retpoline_safe;
  	u8 visited;
+	u8 ret_offset;
  	struct symbol *call_dest;
  	struct instruction *jump_dest;
  	struct instruction *first_jump_src;

Cheers,

Signature · 1 line
Julien Thierry
author maintainer
#35 lore raw

Re: [PATCH v2] objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

Quoted 5 lines · Julien Thierry
Quoted 5 lines
tools/objtool/check.c +2 -0
+static bool has_modified_stack_frame(struct instruction *insn, struct insn_state *state)
  {
+	u8 ret_offset = insn->ret_offset;
  	int i;
Quoted 12 lines

tools/objtool/check.c static bool is_fentry_call(struct instru · line 1423

Quoted 9 lines
tools/objtool/check.c +5 -4
-	if (state->cfa.base != initial_func_cfi.cfa.base ||
-	    state->cfa.offset != initial_func_cfi.cfa.offset ||
-	    state->stack_size != initial_func_cfi.cfa.offset ||
-	    state->drap)
+	if (state->cfa.base != initial_func_cfi.cfa.base || state->drap)
+		return true;
+
+	if (state->cfa.offset != initial_func_cfi.cfa.offset &&
+	    !(ret_offset && state->cfa.offset == initial_func_cfi.cfa.offset + ret_offset))

Isn't that the same thing as "state->cfa.offset != initial_func_cfi.cfa.offset + ret_offset" ?

I'm confused on what cfa.offset is, sometimes it increase with stack_size, sometimes it doesn't.

ISTR that for the ftrace case it was indeed cfa.offset + 8, but for the IRET case below (where it is now not used anymore) it was cfa.offset (not cfa.offset + 40, which I was expecting).

Quoted 27 lines
Quoted 16 lines
tools/objtool/check.c +5 -1
+		return true;
+
+	if (state->stack_size != initial_func_cfi.cfa.offset + ret_offset)
  		return true;
 
-	for (i = 0; i < CFI_NUM_REGS; i++)
+	for (i = 0; i < CFI_NUM_REGS; i++) {
  		if (state->regs[i].base != initial_func_cfi.regs[i].base ||
  		    state->regs[i].offset != initial_func_cfi.regs[i].offset)
  			return true;
+	}
 
  	return false;
  }

tools/objtool/check.c static int validate_branch(struct objtoo · line 2154

Quoted 7 lines
tools/objtool/check.c +4 -0
@@ -2185,6 +2148,13 @@ static int validate_branch(struct objtoo
 
  			break;
 
+		case INSN_EXCEPTION_RETURN:
+			if (func) {
+				state.stack_size -= arch_exception_frame_size;
+				break;

Why break instead of returning? Shouldn't an exception return mark the end of a branch (whether inside or outside a function) ?

Quoted 2 lines

Here it seems it will continue to the next instruction which might have been unreachable.

The code in question (x86's sync_core()), is an exception return to self. It pushes an exception frame that points to right after the exception return instruction.

This is the only usage of IRET in STT_FUNC symbols.

So rather than teaching objtool how to interpret the whole push;push;push;push;push;iret sequence, teach it how big the frame is (arch_exception_frame_size) and let it continue.

All the other (real) IRETs are in STT_NOTYPE in the entry assembly.

Quoted 9 lines

tools/objtool/check.c static int validate_branch(struct objtoo · line 2157

Quoted 3 lines
tools/objtool/check.c +3 -0
+			}
+
+			/* fallthrough */

What is the purpose of the fallthrough here? If the exception return was in a function, it carried on to the next instruction, so it won't use the WARN_FUNC(). So, if I'm looking at the right version of the code only the "return 0;" will be used. And, unless my previous comment is wrong, I'd argue that we should return both for func and !func.

That came from the fact that we split it out of INSN_CONTEXT_SWITCH. You're right that it has now reduced to just return 0.

Quoted 3 lines
Quoted 3 lines
tools/objtool/check.c +0 -0
  		case INSN_CONTEXT_SWITCH:
  			if (func && (!next_insn || !next_insn->hint)) {
  				WARN_FUNC("unsupported instruction in callable function",
maintainer
#36 lore raw

Re: [PATCH v2] objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

Quoted 8 lines · On Wed, 1 Apr 2020 19:09:10 +0200 Peter Zijlstra <[email protected]> wrote:
Quoted 5 lines

tools/objtool/check.c static bool is_fentry_call(struct instru · line 1423

Quoted 2 lines
tools/objtool/check.c +2 -0
+	if (state->cfa.offset != initial_func_cfi.cfa.offset &&
+	    !(ret_offset && state->cfa.offset == initial_func_cfi.cfa.offset + ret_offset))  

Isn't that the same thing as "state->cfa.offset != initial_func_cfi.cfa.offset + ret_offset" ?

I'm confused on what cfa.offset is, sometimes it increase with stack_size, sometimes it doesn't.

I believe what Julien is saying is the above logic is equivalent:

	if (x != y &&
	    !(z && x == y + z))

is the same as:

	if (x != y + z)

-- Steve

author maintainer
#37 lore raw

Re: [PATCH v2] objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

Quoted 12 lines · Steven Rostedt
Quoted 8 lines · On Wed, 1 Apr 2020 19:09:10 +0200 Peter Zijlstra <[email protected]> wrote:
Quoted 5 lines

tools/objtool/check.c static bool is_fentry_call(struct instru · line 1423

Quoted 2 lines
tools/objtool/check.c +2 -0
+	if (state->cfa.offset != initial_func_cfi.cfa.offset &&
+	    !(ret_offset && state->cfa.offset == initial_func_cfi.cfa.offset + ret_offset))  

Isn't that the same thing as "state->cfa.offset != initial_func_cfi.cfa.offset + ret_offset" ?

I'm confused on what cfa.offset is, sometimes it increase with stack_size, sometimes it doesn't.

Quoted 8 lines

I believe what Julien is saying is the above logic is equivalent:

	if (x != y &&
	    !(z && x == y + z))

is the same as:

	if (x != y + z)

It is not, the former will accept either x==y || x==y+z, while the latter will only accept x==y+z.

For stack_size, I'm confident in just x==y+z, for offset I saw conflicting things.

Since the annotation is now used in only a single place, maybe x==y+z will just work, I'll go try once I've managed to unconfuse myself on the whole fp-unwind situation.

maintainer
#38 lore raw

Re: [PATCH v2] objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

Quoted 11 lines · On Wed, 1 Apr 2020 19:45:44 +0200 Peter Zijlstra <[email protected]> wrote:
Quoted 8 lines

I believe what Julien is saying is the above logic is equivalent:

	if (x != y &&
	    !(z && x == y + z))

is the same as:

	if (x != y + z)  

It is not, the former will accept either x==y || x==y+z, while the latter will only accept x==y+z.

No, the former accepts:

	x==y || (z && x == y + z)

Which is the same as: x == y + z

As the second condition is only tested if z != 0, and x == y is the same as x == y + 0

-- Steve

author maintainer
#39 lore raw

Re: [PATCH v2] objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

Quoted 15 lines · Steven Rostedt
Quoted 11 lines · On Wed, 1 Apr 2020 19:45:44 +0200 Peter Zijlstra <[email protected]> wrote:
Quoted 8 lines

I believe what Julien is saying is the above logic is equivalent:

	if (x != y &&
	    !(z && x == y + z))

is the same as:

	if (x != y + z)  

It is not, the former will accept either x==y || x==y+z, while the latter will only accept x==y+z.

Quoted 8 lines

No, the former accepts:

	x==y || (z && x == y + z)

Which is the same as: x == y + z

As the second condition is only tested if z != 0, and x == y is the same as x == y + 0

Right, so it accepts both +0 and +z, while the latter will only accept +z.

( in the iret case I had offset at +0 and stack_size at +40, while with
the ftrace case I had both at +8; which is why I wrote the form that accepts +0 and +z )

Anyway, I tested it, and for the ftrace case (the only current user of the hint) +z is correct for both offset and stack_size. I build both FP and ORC variants.

#40 lore raw

Re: [PATCH v2] objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

In reply to Peter Zijlstra
Quoted 20 lines · Peter Zijlstra
Quoted 5 lines · Julien Thierry
Quoted 5 lines
tools/objtool/check.c +2 -0
+static bool has_modified_stack_frame(struct instruction *insn, struct insn_state *state)
  {
+	u8 ret_offset = insn->ret_offset;
  	int i;
Quoted 12 lines

tools/objtool/check.c static bool is_fentry_call(struct instru · line 1423

Quoted 9 lines
tools/objtool/check.c +5 -4
-	if (state->cfa.base != initial_func_cfi.cfa.base ||
-	    state->cfa.offset != initial_func_cfi.cfa.offset ||
-	    state->stack_size != initial_func_cfi.cfa.offset ||
-	    state->drap)
+	if (state->cfa.base != initial_func_cfi.cfa.base || state->drap)
+		return true;
+
+	if (state->cfa.offset != initial_func_cfi.cfa.offset &&
+	    !(ret_offset && state->cfa.offset == initial_func_cfi.cfa.offset + ret_offset))

Isn't that the same thing as "state->cfa.offset != initial_func_cfi.cfa.offset + ret_offset" ?

Quoted 6 lines

I'm confused on what cfa.offset is, sometimes it increase with stack_size, sometimes it doesn't.

ISTR that for the ftrace case it was indeed cfa.offset + 8, but for the IRET case below (where it is now not used anymore) it was cfa.offset (not cfa.offset + 40, which I was expecting).

It depends on the value of cfa.base. If cfa.base is CFI_SP, then cfa.offset changes with stack_size. If cfa.base is CFI_BP (i.e. if the function uses a frame pointer), then cfa.offset is constant (the distance between RBP on the stack and the previous frame).

Signature · 1 line
Josh
#41 lore raw

Re: [PATCH v2] objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

In reply to Peter Zijlstra
Quoted 20 lines · Peter Zijlstra
Quoted 5 lines · Julien Thierry
Quoted 5 lines
tools/objtool/check.c +2 -0
+static bool has_modified_stack_frame(struct instruction *insn, struct insn_state *state)
   {
+	u8 ret_offset = insn->ret_offset;
   	int i;
Quoted 12 lines

tools/objtool/check.c static bool is_fentry_call(struct instru · line 1423

Quoted 9 lines
tools/objtool/check.c +5 -4
-	if (state->cfa.base != initial_func_cfi.cfa.base ||
-	    state->cfa.offset != initial_func_cfi.cfa.offset ||
-	    state->stack_size != initial_func_cfi.cfa.offset ||
-	    state->drap)
+	if (state->cfa.base != initial_func_cfi.cfa.base || state->drap)
+		return true;
+
+	if (state->cfa.offset != initial_func_cfi.cfa.offset &&
+	    !(ret_offset && state->cfa.offset == initial_func_cfi.cfa.offset + ret_offset))

Isn't that the same thing as "state->cfa.offset != initial_func_cfi.cfa.offset + ret_offset" ?

Quoted 3 lines

I'm confused on what cfa.offset is, sometimes it increase with stack_size, sometimes it doesn't.

Steven already replied for me about that :) .

Quoted 38 lines

ISTR that for the ftrace case it was indeed cfa.offset + 8, but for the IRET case below (where it is now not used anymore) it was cfa.offset (not cfa.offset + 40, which I was expecting).

Quoted 27 lines
Quoted 16 lines
tools/objtool/check.c +5 -1
+		return true;
+
+	if (state->stack_size != initial_func_cfi.cfa.offset + ret_offset)
   		return true;
 
-	for (i = 0; i < CFI_NUM_REGS; i++)
+	for (i = 0; i < CFI_NUM_REGS; i++) {
   		if (state->regs[i].base != initial_func_cfi.regs[i].base ||
   		    state->regs[i].offset != initial_func_cfi.regs[i].offset)
   			return true;
+	}
 
   	return false;
   }

tools/objtool/check.c static int validate_branch(struct objtoo · line 2154

Quoted 7 lines
tools/objtool/check.c +4 -0
@@ -2185,6 +2148,13 @@ static int validate_branch(struct objtoo
 
   			break;
 
+		case INSN_EXCEPTION_RETURN:
+			if (func) {
+				state.stack_size -= arch_exception_frame_size;
+				break;

Why break instead of returning? Shouldn't an exception return mark the end of a branch (whether inside or outside a function) ?

Quoted 2 lines

Here it seems it will continue to the next instruction which might have been unreachable.

The code in question (x86's sync_core()), is an exception return to self. It pushes an exception frame that points to right after the exception return instruction.

Quoted 8 lines

This is the only usage of IRET in STT_FUNC symbols.

So rather than teaching objtool how to interpret the whole push;push;push;push;push;iret sequence, teach it how big the frame is (arch_exception_frame_size) and let it continue.

All the other (real) IRETs are in STT_NOTYPE in the entry assembly.

Right, I see.. However I'm not completely convinced by this. I must admit I haven't followed the whole conversation, but what was the issue with the HINT_IRET_SELF? It seemed more elegant, but I might be missing some context.

Otherwise, it might be worth having a comment in the code to point that this only handles the sync_core() case.

Also, instead of adding a special "arch_exception_frame_size", I could suggest:
- Picking this patch [1] from a completely arbitrary source
- Getting rid of INSN_STACK type, any instruction could then include stack ops on top of their existing semantics, they can just have an empty list if they don't touch SP/BP
- x86 decoder adds a stack_op to the iret to modify the stack pointer by the right amount

[1] https://www.spinics.net/lists/kernel/msg3453725.html

Thanks,

Signature · 1 line
Julien Thierry
#42 lore raw

Re: [PATCH v2] objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

Quoted 52 lines · Julien Thierry
Quoted 35 lines · Peter Zijlstra
Quoted 27 lines · Julien Thierry
Quoted 16 lines

+        return true;
+
+    if (state->stack_size != initial_func_cfi.cfa.offset + ret_offset) return true;

-    for (i = 0; i < CFI_NUM_REGS; i++)
+    for (i = 0; i < CFI_NUM_REGS; i++) {
           if (state->regs[i].base != initial_func_cfi.regs[i].base || state->regs[i].offset != initial_func_cfi.regs[i].offset) return true;
+    }

       return false;
   }

tools/objtool/check.c static int validate_branch(struct objtoo · line 2154

Quoted 7 lines
tools/objtool/check.c +4 -0
@@ -2185,6 +2148,13 @@ static int validate_branch(struct objtoo
 
                break;
 
+        case INSN_EXCEPTION_RETURN:
+            if (func) {
+                state.stack_size -= arch_exception_frame_size;
+                break;

Why break instead of returning? Shouldn't an exception return mark the end of a branch (whether inside or outside a function) ?

Quoted 2 lines

Here it seems it will continue to the next instruction which might have been unreachable.

The code in question (x86's sync_core()), is an exception return to self. It pushes an exception frame that points to right after the exception return instruction.

Quoted 8 lines

This is the only usage of IRET in STT_FUNC symbols.

So rather than teaching objtool how to interpret the whole push;push;push;push;push;iret sequence, teach it how big the frame is (arch_exception_frame_size) and let it continue.

All the other (real) IRETs are in STT_NOTYPE in the entry assembly.

Right, I see.. However I'm not completely convinced by this. I must admit I haven't followed the whole conversation, but what was the issue with the HINT_IRET_SELF? It seemed more elegant, but I might be missing some context.

Otherwise, it might be worth having a comment in the code to point that this only handles the sync_core() case.

Quoted 7 lines

Also, instead of adding a special "arch_exception_frame_size", I could suggest:
- Picking this patch [1] from a completely arbitrary source
- Getting rid of INSN_STACK type, any instruction could then include
stack ops on top of their existing semantics, they can just have an empty list if they don't touch SP/BP
- x86 decoder adds a stack_op to the iret to modify the stack pointer by
the right amount

And the x86 decode could also lookup the symbol containing an IRET and chose whether its type should be INSN_CONTEXT_SWITCH or INSN_OTHER depending on whether the symbol is a function or not.

This would avoid having the arch specific pattern detected the generic stack validation part of objtool.

Signature · 1 line
Julien Thierry
author maintainer
#43 lore raw

Re: [PATCH v2] objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

In reply to Julien Thierry
Quoted 12 lines · Julien Thierry
Quoted 10 lines · Peter Zijlstra

The code in question (x86's sync_core()), is an exception return to self. It pushes an exception frame that points to right after the exception return instruction.

This is the only usage of IRET in STT_FUNC symbols.

So rather than teaching objtool how to interpret the whole push;push;push;push;push;iret sequence, teach it how big the frame is (arch_exception_frame_size) and let it continue.

Quoted 6 lines
Quoted 2 lines

All the other (real) IRETs are in STT_NOTYPE in the entry assembly.

Right, I see.. However I'm not completely convinced by this. I must admit I haven't followed the whole conversation, but what was the issue with the HINT_IRET_SELF? It seemed more elegant, but I might be missing some context.

https://lkml.kernel.org/r/20200331211755.pb7f3wa6oxzjnswc@treble

Josh didn't think it was worth it, I think.

Quoted 2 lines

Otherwise, it might be worth having a comment in the code to point that this only handles the sync_core() case.

I can certainly do that. Does ARM have any ERETs sprinkled around in places it should not have? That is, is this going to be a problem for you?

Quoted 8 lines

Also, instead of adding a special "arch_exception_frame_size", I could suggest:
- Picking this patch [1] from a completely arbitrary source
- Getting rid of INSN_STACK type, any instruction could then include stack ops on top of their existing semantics, they can just have an empty list if they don't touch SP/BP
- x86 decoder adds a stack_op to the iret to modify the stack pointer by the right amount

That's not the worst idea, lemme try that.

#44 lore raw

Re: [PATCH v2] objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

Quoted 22 lines · Peter Zijlstra
Quoted 12 lines · Julien Thierry
Quoted 10 lines · Peter Zijlstra

The code in question (x86's sync_core()), is an exception return to self. It pushes an exception frame that points to right after the exception return instruction.

This is the only usage of IRET in STT_FUNC symbols.

So rather than teaching objtool how to interpret the whole push;push;push;push;push;iret sequence, teach it how big the frame is (arch_exception_frame_size) and let it continue.

Quoted 6 lines
Quoted 2 lines

All the other (real) IRETs are in STT_NOTYPE in the entry assembly.

Right, I see.. However I'm not completely convinced by this. I must admit I haven't followed the whole conversation, but what was the issue with the HINT_IRET_SELF? It seemed more elegant, but I might be missing some context.

https://lkml.kernel.org/r/20200331211755.pb7f3wa6oxzjnswc@treble

Quoted 9 lines

Josh didn't think it was worth it, I think.

Quoted 2 lines

Otherwise, it might be worth having a comment in the code to point that this only handles the sync_core() case.

I can certainly do that. Does ARM have any ERETs sprinkled around in places it should not have? That is, is this going to be a problem for you?

I had a quick look and I don't think there are ERETS in function symbols. And, worst case scenario, I could also just keep the arm64 decoder making ERETS as INSN_CONTEXT_SWITCH as I didn't need more semantics so far (and arm64 ERET don't affect the stack anyway).

After you pointed out this only affect this very specific pattern, I admit that my concerns are more about "not having weird stuff in the generic part". If it's too much of a hassle I can understand if you prefer to just put a comment. But if most of this can be kept to the arch specific decoder I think it'd be nicer :) .

Quoted 11 lines
Quoted 8 lines

Also, instead of adding a special "arch_exception_frame_size", I could suggest:
- Picking this patch [1] from a completely arbitrary source
- Getting rid of INSN_STACK type, any instruction could then include stack ops on top of their existing semantics, they can just have an empty list if they don't touch SP/BP
- x86 decoder adds a stack_op to the iret to modify the stack pointer by the right amount

That's not the worst idea, lemme try that.

Thanks, keep me in Cc if you post a new version using that!

Cheers,

Signature · 1 line
Julien Thierry
author maintainer
#45 lore raw

Re: [PATCH v2] objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

In reply to Peter Zijlstra
Quoted 11 lines · Peter Zijlstra
Quoted 8 lines · Julien Thierry

Also, instead of adding a special "arch_exception_frame_size", I could suggest:
- Picking this patch [1] from a completely arbitrary source
- Getting rid of INSN_STACK type, any instruction could then include stack ops on top of their existing semantics, they can just have an empty list if they don't touch SP/BP
- x86 decoder adds a stack_op to the iret to modify the stack pointer by the right amount

Quoted 1 line

That's not the worst idea, lemme try that.

Something like so then?

arch/x86/include/asm/processor.h +0 -2
@@ -738,7 +738,6 @@ static inline void sync_core(void)
 	unsigned int tmp;
 
 	asm volatile (
-		UNWIND_HINT_SAVE
 		"mov %%ss, %0\n\t"
 		"pushq %q0\n\t"
 		"pushq %%rsp\n\t"
@@ -748,7 +747,6 @@ static inline void sync_core(void)
 		"pushq %q0\n\t"
 		"pushq $1f\n\t"
 		"iretq\n\t"
-		UNWIND_HINT_RESTORE
 		"1:"
 		: "=&r" (tmp), ASM_CALL_CONSTRAINT : : "cc", "memory");
 #endif
tools/objtool/arch.h +1 -0
@@ -19,6 +19,7 @@ enum insn_type {
 	INSN_CALL,
 	INSN_CALL_DYNAMIC,
 	INSN_RETURN,
+	INSN_EXCEPTION_RETURN,
 	INSN_CONTEXT_SWITCH,
 	INSN_STACK,
 	INSN_BUG,
tools/objtool/arch/x86/decode.c +12 -2
@@ -435,9 +435,19 @@ int arch_decode_instruction(struct elf *
 		*type = INSN_RETURN;
 		break;
 
+	case 0xcf: /* iret */
+		*type = INSN_EXCEPTION_RETURN;
+
+		/* add $40, %rsp */
+		op->src.type = OP_SRC_ADD;
+		op->src.reg = CFI_SP;
+		op->src.offset = 5*8;
+		op->dest.type = OP_DEST_REG;
+		op->dest.reg = CFI_SP;
+		break;
+
 	case 0xca: /* retf */
 	case 0xcb: /* retf */
-	case 0xcf: /* iret */
 		*type = INSN_CONTEXT_SWITCH;
 		break;
 
@@ -483,7 +493,7 @@ int arch_decode_instruction(struct elf *
 
 	*immediate = insn.immediate.nbytes ? insn.immediate.value : 0;
 
-	if (*type == INSN_STACK)
+	if (*type == INSN_STACK || *type == INSN_EXCEPTION_RETURN)
 		list_add_tail(&op->list, ops_list);
 	else
 		free(op);
tools/objtool/check.c +14 -0
@@ -2224,6 +2224,20 @@ static int validate_branch(struct objtoo
 
 			break;
 
+		case INSN_EXCEPTION_RETURN:
+			if (handle_insn_ops(insn, &state))
+				return 1;
+
+			/*
+			 * This handles x86's sync_core() case, where we use an
+			 * IRET to self. All 'normal' IRET instructions are in
+			 * STT_NOTYPE entry symbols.
+			 */
+			if (func)
+				break;
+
+			return 0;
+
 		case INSN_CONTEXT_SWITCH:
 			if (func && (!next_insn || !next_insn->hint)) {
 				WARN_FUNC("unsupported instruction in callable function",
#46 lore raw

Re: [PATCH v2] objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

Quoted 12 lines · Peter Zijlstra
Quoted 11 lines · Peter Zijlstra
Quoted 8 lines · Julien Thierry

Also, instead of adding a special "arch_exception_frame_size", I could suggest:
- Picking this patch [1] from a completely arbitrary source
- Getting rid of INSN_STACK type, any instruction could then include stack ops on top of their existing semantics, they can just have an empty list if they don't touch SP/BP
- x86 decoder adds a stack_op to the iret to modify the stack pointer by the right amount

Quoted 4 lines
Quoted 1 line

That's not the worst idea, lemme try that.

Something like so then?

Yes, you could even remove INSN_STACK from insn_type and just always call handle_insn_ops() before the switch statement on insn->type. If the list is empty it does nothing. This way you wouldn't need to call it for the INSN_EXCEPTION_RETURN case, and any type of instructions could use stack_ops.

And the other suggestion is my other email was that you don't even need to add INSN_EXCEPTION_RETURN. You can keep IRET as INSN_CONTEXT_SWITCH by default and x86 decoder lookups the symbol conaining an iret. If it's a function symbol, it can just set the type to INSN_OTHER so that it caries on to the next instruction after having handled the stack_op.

And everything fits under tools/objtool/arch/x86 :) .

Or is it too far-fetch'd?

Quoted 74 lines
arch/x86/include/asm/processor.h +0 -2
@@ -738,7 +738,6 @@ static inline void sync_core(void)
  	unsigned int tmp;
 
  	asm volatile (
-		UNWIND_HINT_SAVE
  		"mov %%ss, %0\n\t"
  		"pushq %q0\n\t"
  		"pushq %%rsp\n\t"
@@ -748,7 +747,6 @@ static inline void sync_core(void)
  		"pushq %q0\n\t"
  		"pushq $1f\n\t"
  		"iretq\n\t"
-		UNWIND_HINT_RESTORE
  		"1:"
  		: "=&r" (tmp), ASM_CALL_CONSTRAINT : : "cc", "memory");
  #endif
tools/objtool/arch.h +1 -0
@@ -19,6 +19,7 @@ enum insn_type {
  	INSN_CALL,
  	INSN_CALL_DYNAMIC,
  	INSN_RETURN,
+	INSN_EXCEPTION_RETURN,
  	INSN_CONTEXT_SWITCH,
  	INSN_STACK,
  	INSN_BUG,
tools/objtool/arch/x86/decode.c +12 -2
@@ -435,9 +435,19 @@ int arch_decode_instruction(struct elf *
  		*type = INSN_RETURN;
  		break;
 
+	case 0xcf: /* iret */
+		*type = INSN_EXCEPTION_RETURN;
+
+		/* add $40, %rsp */
+		op->src.type = OP_SRC_ADD;
+		op->src.reg = CFI_SP;
+		op->src.offset = 5*8;
+		op->dest.type = OP_DEST_REG;
+		op->dest.reg = CFI_SP;
+		break;
+
  	case 0xca: /* retf */
  	case 0xcb: /* retf */
-	case 0xcf: /* iret */
  		*type = INSN_CONTEXT_SWITCH;
  		break;
 
@@ -483,7 +493,7 @@ int arch_decode_instruction(struct elf *
 
  	*immediate = insn.immediate.nbytes ? insn.immediate.value : 0;
 
-	if (*type == INSN_STACK)
+	if (*type == INSN_STACK || *type == INSN_EXCEPTION_RETURN)
  		list_add_tail(&op->list, ops_list);
  	else
  		free(op);
tools/objtool/check.c +7 -0
@@ -2224,6 +2224,20 @@ static int validate_branch(struct objtoo
 
  			break;
 
+		case INSN_EXCEPTION_RETURN:
+			if (handle_insn_ops(insn, &state))
+				return 1;
+
+			/*
+			 * This handles x86's sync_core() case, where we use an
+			 * IRET to self. All 'normal' IRET instructions are in

tools/objtool/check.c static int validate_branch(struct objtoo · line 2243

Quoted 10 lines
tools/objtool/check.c +7 -0
+			 * STT_NOTYPE entry symbols.
+			 */
+			if (func)
+				break;
+
+			return 0;
+
  		case INSN_CONTEXT_SWITCH:
  			if (func && (!next_insn || !next_insn->hint)) {
  				WARN_FUNC("unsupported instruction in callable function",

Cheers,

Signature · 1 line
Julien Thierry
#47 lore raw

Re: [PATCH v2] objtool,ftrace: Implement UNWIND_HINT_RET_OFFSET

Quoted 35 lines · Julien Thierry
Quoted 14 lines · Peter Zijlstra
Quoted 13 lines · Peter Zijlstra
Quoted 10 lines · Julien Thierry

Also, instead of adding a special "arch_exception_frame_size", I could suggest:
- Picking this patch [1] from a completely arbitrary source
- Getting rid of INSN_STACK type, any instruction could then include stack ops on top of their existing semantics, they can just have an empty list if
they don't touch SP/BP
- x86 decoder adds a stack_op to the iret to modify the stack pointer by the
right amount

Quoted 4 lines
Quoted 1 line

That's not the worst idea, lemme try that.

Something like so then?

Yes, you could even remove INSN_STACK from insn_type and just always call handle_insn_ops() before the switch statement on insn->type. If the list is empty it does nothing.
This way you wouldn't need to call it for the INSN_EXCEPTION_RETURN case, and any type of instructions could use stack_ops.

And the other suggestion is my other email was that you don't even need to add INSN_EXCEPTION_RETURN. You can keep IRET as INSN_CONTEXT_SWITCH by default and x86 decoder lookups the symbol conaining an iret. If it's a function symbol, it can just set the type to INSN_OTHER so that it caries on to the next instruction after having handled the stack_op.

Quoted 3 lines

And everything fits under tools/objtool/arch/x86 :) .

Or is it too far-fetch'd?

Imho no. Well, it depends. I can see benefits of both approach. PeterZ's patch is quite minimal and it demonstrates itself as a one-off hack quite well. On the other hand, it is in a generic code, which is not nice especially when other archs do not have such thing. So your proposal would indeed make sense to hide it in arch-specific code. Especially for the future. And INSN_STACK is not used much in the code, so it can be removed easily as you proposed.

And going more in direction of supporting more archs in the future, I'd say it would make sense to allow more generic things such as "forget about INSN_STACK. If an instruction has non-empty stack_ops list, just process it". It is definitely more flexible.

So yes, I think it make sense unless I am missing something.

Miroslav

author maintainer
#48 lore raw

PATCH v4 2/13 objtool: Factor out CFI hints

In reply to Peter Zijlstra

Move the application of CFI hints into it's own function. No functional changes intended.

---
 tools/objtool/check.c |   67 ++++++++++++++++++++++++++++----------------------
 1 file changed, 38 insertions(+), 29 deletions(-)
tools/objtool/check.c +38 -29
@@ -2033,6 +2033,41 @@ static int validate_return(struct symbol
 	return 0;
 }
 
+static int apply_insn_hint(struct objtool_file *file, struct section *sec,
+			   struct symbol *func, struct instruction *insn,
+			   struct insn_state *state)
+{
+	if (insn->restore) {
+		struct instruction *save_insn, *i;
+
+		i = insn;
+		save_insn = NULL;
+		sym_for_each_insn_continue_reverse(file, func, i) {
+			if (i->save) {
+				save_insn = i;
+				break;
+			}
+		}
+
+		if (!save_insn) {
+			WARN_FUNC("no corresponding CFI save for CFI restore",
+				  sec, insn->offset);
+			return 1;
+		}
+
+		if (!save_insn->visited) {
+			WARN_FUNC("objtool isn't smart enough to handle this CFI save/restore combo",
+				  sec, insn->offset);
+			return 1;
+		}
+
+		insn->state = save_insn->state;
+	}
+
+	state = insn->state;
+	return 0;
+}
+
 /*
  * Follow the branch starting at the given instruction, and recursively follow
  * any other branches (jumps).  Meanwhile, track the frame pointer state at
@@ -2081,35 +2116,9 @@ static int validate_branch(struct objtoo
 		}
 
 		if (insn->hint) {
-			if (insn->restore) {
-				struct instruction *save_insn, *i;
-
-				i = insn;
-				save_insn = NULL;
-				sym_for_each_insn_continue_reverse(file, func, i) {
-					if (i->save) {
-						save_insn = i;
-						break;
-					}
-				}
-
-				if (!save_insn) {
-					WARN_FUNC("no corresponding CFI save for CFI restore",
-						  sec, insn->offset);
-					return 1;
-				}
-
-				if (!save_insn->visited) {
-					WARN_FUNC("objtool isn't smart enough to handle this CFI save/restore combo",
-						  sec, insn->offset);
-					return 1;
-				}
-
-				insn->state = save_insn->state;
-			}
-
-			state = insn->state;
-
+			ret = apply_insn_hint(file, sec, func, insn, &state);
+			if (ret)
+				return ret;
 		} else
 			insn->state = state;
 
#49 lore raw

Re: [PATCH v4 02/13] objtool: Factor out CFI hints

Quoted 37 lines · Peter Zijlstra

Move the application of CFI hints into it's own function. No functional changes intended.

---
 tools/objtool/check.c |   67 ++++++++++++++++++++++++++++----------------------
 1 file changed, 38 insertions(+), 29 deletions(-)
tools/objtool/check.c +22 -0
@@ -2033,6 +2033,41 @@ static int validate_return(struct symbol
 	return 0;
 }
 
+static int apply_insn_hint(struct objtool_file *file, struct section *sec,
+			   struct symbol *func, struct instruction *insn,
+			   struct insn_state *state)
+{
+	if (insn->restore) {
+		struct instruction *save_insn, *i;
+
+		i = insn;
+		save_insn = NULL;
+		sym_for_each_insn_continue_reverse(file, func, i) {
+			if (i->save) {
+				save_insn = i;
+				break;
+			}
+		}
+
+		if (!save_insn) {
+			WARN_FUNC("no corresponding CFI save for CFI restore",
+				  sec, insn->offset);
+			return 1;
+		}
+

tools/objtool/check.c static int validate_return(struct symbol · line 2067

Quoted 10 lines
tools/objtool/check.c +10 -0
+		if (!save_insn->visited) {
+			WARN_FUNC("objtool isn't smart enough to handle this CFI save/restore combo",
+				  sec, insn->offset);
+			return 1;
+		}
+
+		insn->state = save_insn->state;
+	}
+
+	state = insn->state;

It does not matter, because it will change later again, but there should be

*state = insn->state;

here, right?

Quoted 47 lines
tools/objtool/check.c +6 -29
+	return 0;
+}
+
 /*
  * Follow the branch starting at the given instruction, and recursively follow
  * any other branches (jumps).  Meanwhile, track the frame pointer state at
@@ -2081,35 +2116,9 @@ static int validate_branch(struct objtoo
 		}
 
 		if (insn->hint) {
-			if (insn->restore) {
-				struct instruction *save_insn, *i;
-
-				i = insn;
-				save_insn = NULL;
-				sym_for_each_insn_continue_reverse(file, func, i) {
-					if (i->save) {
-						save_insn = i;
-						break;
-					}
-				}
-
-				if (!save_insn) {
-					WARN_FUNC("no corresponding CFI save for CFI restore",
-						  sec, insn->offset);
-					return 1;
-				}
-
-				if (!save_insn->visited) {
-					WARN_FUNC("objtool isn't smart enough to handle this CFI save/restore combo",
-						  sec, insn->offset);
-					return 1;
-				}
-
-				insn->state = save_insn->state;
-			}
-
-			state = insn->state;
-
+			ret = apply_insn_hint(file, sec, func, insn, &state);
+			if (ret)
+				return ret;
 		} else
 			insn->state = state;
 
author maintainer
#50 lore raw

Re: [PATCH v4 02/13] objtool: Factor out CFI hints

Quoted 53 lines · Miroslav Benes
Quoted 37 lines · Peter Zijlstra

Move the application of CFI hints into it's own function. No functional changes intended.

---
 tools/objtool/check.c |   67 ++++++++++++++++++++++++++++----------------------
 1 file changed, 38 insertions(+), 29 deletions(-)
tools/objtool/check.c +22 -0
@@ -2033,6 +2033,41 @@ static int validate_return(struct symbol
 	return 0;
 }
 
+static int apply_insn_hint(struct objtool_file *file, struct section *sec,
+			   struct symbol *func, struct instruction *insn,
+			   struct insn_state *state)
+{
+	if (insn->restore) {
+		struct instruction *save_insn, *i;
+
+		i = insn;
+		save_insn = NULL;
+		sym_for_each_insn_continue_reverse(file, func, i) {
+			if (i->save) {
+				save_insn = i;
+				break;
+			}
+		}
+
+		if (!save_insn) {
+			WARN_FUNC("no corresponding CFI save for CFI restore",
+				  sec, insn->offset);
+			return 1;
+		}
+

tools/objtool/check.c static int validate_return(struct symbol · line 2067

Quoted 10 lines
tools/objtool/check.c +10 -0
+		if (!save_insn->visited) {
+			WARN_FUNC("objtool isn't smart enough to handle this CFI save/restore combo",
+				  sec, insn->offset);
+			return 1;
+		}
+
+		insn->state = save_insn->state;
+	}
+
+	state = insn->state;

It does not matter, because it will change later again, but there should be

Quoted 3 lines

*state = insn->state;

here, right?

Argh, yes. Let me go edit the patches.

author maintainer
#51 lore raw

PATCH v4 3/13 objtool: Rename struct cfi_state

In reply to Peter Zijlstra

There's going to be a new struct cfi_state, rename this one to make place.

---
 tools/objtool/arch.h            |    2 +-
 tools/objtool/arch/x86/decode.c |    2 +-
 tools/objtool/cfi.h             |    2 +-
 tools/objtool/check.c           |    2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)
tools/objtool/arch.h +1 -1
@@ -66,7 +66,7 @@ struct stack_op {
 	struct op_src src;
 };
 
-void arch_initial_func_cfi_state(struct cfi_state *state);
+void arch_initial_func_cfi_state(struct cfi_init_state *state);
 
 int arch_decode_instruction(struct elf *elf, struct section *sec,
 			    unsigned long offset, unsigned int maxlen,
tools/objtool/arch/x86/decode.c +1 -1
@@ -480,7 +480,7 @@ int arch_decode_instruction(struct elf *
 	return 0;
 }
 
-void arch_initial_func_cfi_state(struct cfi_state *state)
+void arch_initial_func_cfi_state(struct cfi_init_state *state)
 {
 	int i;
 
tools/objtool/cfi.h +1 -1
@@ -35,7 +35,7 @@ struct cfi_reg {
 	int offset;
 };
 
-struct cfi_state {
+struct cfi_init_state {
 	struct cfi_reg cfa;
 	struct cfi_reg regs[CFI_NUM_REGS];
 };
tools/objtool/check.c +1 -1
@@ -27,7 +27,7 @@ struct alternative {
 };
 
 const char *objname;
-struct cfi_state initial_func_cfi;
+struct cfi_init_state initial_func_cfi;
 
 struct instruction *find_insn(struct objtool_file *file,
 			      struct section *sec, unsigned long offset)
author maintainer
#52 lore raw

PATCH v4 4/13 objtool: Fix !CFI insn_state propagation

In reply to Peter Zijlstra

Objtool keeps per instruction CFI state in struct insn_state and will save/restore this where required. However, insn_state has grown some !CFI state, and this must not be saved/restored (that would loose/destroy state).

Fix this by moving the CFI specific parts of insn_state into struct cfi_state.

---
 tools/objtool/cfi.h     |   12 ++
 tools/objtool/check.c   |  270 ++++++++++++++++++++++++------------------------
 tools/objtool/check.h   |   13 --
 tools/objtool/orc_gen.c |    8 -
 4 files changed, 160 insertions(+), 143 deletions(-)
tools/objtool/cfi.h +12 -0
@@ -36,8 +36,20 @@ struct cfi_reg {
 };
 
 struct cfi_init_state {
+	struct cfi_reg regs[CFI_NUM_REGS];
 	struct cfi_reg cfa;
+};
+
+struct cfi_state {
 	struct cfi_reg regs[CFI_NUM_REGS];
+	struct cfi_reg vals[CFI_NUM_REGS];
+	struct cfi_reg cfa;
+	int stack_size;
+	int drap_reg, drap_offset;
+	unsigned char type;
+	bool bp_scratch;
+	bool drap;
+	bool end;
 };
 
 #endif /* _OBJTOOL_CFI_H */
tools/objtool/check.c +140 -130

631 lines not on this page · raw message

tools/objtool/check.h +4 -9
@@ -14,15 +14,10 @@
 #include <linux/hashtable.h>
 
 struct insn_state {
-	struct cfi_reg cfa;
-	struct cfi_reg regs[CFI_NUM_REGS];
-	int stack_size;
-	unsigned char type;
-	bool bp_scratch;
-	bool drap, end, uaccess, df;
+	struct cfi_state cfi;
 	unsigned int uaccess_stack;
-	int drap_reg, drap_offset;
-	struct cfi_reg vals[CFI_NUM_REGS];
+	bool uaccess;
+	bool df;
 };
 
 struct instruction {
@@ -43,7 +38,7 @@ struct instruction {
 	struct list_head alts;
 	struct symbol *func;
 	struct stack_op stack_op;
-	struct insn_state state;
+	struct cfi_state cfi;
 	struct orc_entry orc;
 };
 
tools/objtool/orc_gen.c +4 -4
@@ -16,10 +16,10 @@ int create_orc(struct objtool_file *file
 
 	for_each_insn(file, insn) {
 		struct orc_entry *orc = &insn->orc;
-		struct cfi_reg *cfa = &insn->state.cfa;
-		struct cfi_reg *bp = &insn->state.regs[CFI_BP];
+		struct cfi_reg *cfa = &insn->cfi.cfa;
+		struct cfi_reg *bp = &insn->cfi.regs[CFI_BP];
 
-		orc->end = insn->state.end;
+		orc->end = insn->cfi.end;
 
 		if (cfa->base == CFI_UNDEFINED) {
 			orc->sp_reg = ORC_REG_UNDEFINED;
@@ -75,7 +75,7 @@ int create_orc(struct objtool_file *file
 
 		orc->sp_offset = cfa->offset;
 		orc->bp_offset = bp->offset;
-		orc->type = insn->state.type;
+		orc->type = insn->cfi.type;
 	}
 
 	return 0;
author maintainer
#53 lore raw

PATCH v4 5/13 objtool: Implement noinstr validation

In reply to Peter Zijlstra

Validate that any call out of .noinstr.text is in between instr_begin() and instr_end() annotations.

This annotation is useful to ensure correct behaviour wrt tracing sensitive code like entry/exit and idle code. When we run code in a sensitive context we want a guarantee no unknown code is ran.

Since this validation relies on knowing the section of call destination symbols, we must run it on vmlinux.o instead of on individual object files.

Add two options:

-d/--duplicate "duplicate validation for vmlinux" -l/--vmlinux "vmlinux.o validation"

Where the latter auto-detects when objname ends with "vmlinux.o" and the former will force all validations, also those already done on !vmlinux object files.

---
 tools/objtool/builtin-check.c |   11 +++-
 tools/objtool/builtin.h       |    2 
 tools/objtool/check.c         |   98 ++++++++++++++++++++++++++++++++++++++++++
 tools/objtool/check.h         |    3 +
 tools/objtool/elf.h           |    2 
 5 files changed, 112 insertions(+), 4 deletions(-)
tools/objtool/builtin-check.c +9 -2
@@ -14,10 +14,11 @@
  */
 
 #include <subcmd/parse-options.h>
+#include <string.h>
 #include "builtin.h"
 #include "check.h"
 
-bool no_fp, no_unreachable, retpoline, module, backtrace, uaccess, stats;
+bool no_fp, no_unreachable, retpoline, module, backtrace, uaccess, stats, validate_dup, vmlinux;
 
 static const char * const check_usage[] = {
 	"objtool check [<options>] file.o",
@@ -32,12 +33,14 @@ const struct option check_options[] = {
 	OPT_BOOLEAN('b', "backtrace", &backtrace, "unwind on error"),
 	OPT_BOOLEAN('a', "uaccess", &uaccess, "enable uaccess checking"),
 	OPT_BOOLEAN('s', "stats", &stats, "print statistics"),
+	OPT_BOOLEAN('d', "duplicate", &validate_dup, "duplicate validation for vmlinux.o"),
+	OPT_BOOLEAN('l', "vmlinux", &vmlinux, "vmlinux.o validation"),
 	OPT_END(),
 };
 
 int cmd_check(int argc, const char **argv)
 {
-	const char *objname;
+	const char *objname, *s;
 
 	argc = parse_options(argc, argv, check_options, check_usage, 0);
 
@@ -46,5 +49,9 @@ int cmd_check(int argc, const char **arg
 
 	objname = argv[0];
 
+	s = strstr(objname, "vmlinux.o");
+	if (s && !s[9])
+		vmlinux = true;
+
 	return check(objname, false);
 }
tools/objtool/builtin.h +1 -1
@@ -8,7 +8,7 @@
 #include <subcmd/parse-options.h>
 
 extern const struct option check_options[];
-extern bool no_fp, no_unreachable, retpoline, module, backtrace, uaccess, stats;
+extern bool no_fp, no_unreachable, retpoline, module, backtrace, uaccess, stats, validate_dup, vmlinux;
 
 extern int cmd_check(int argc, const char **argv);
 extern int cmd_orc(int argc, const char **argv);
tools/objtool/check.c +98 -0

152 lines not on this page · raw message

tools/objtool/check.h +3 -0
@@ -18,6 +18,8 @@ struct insn_state {
 	unsigned int uaccess_stack;
 	bool uaccess;
 	bool df;
+	bool noinstr;
+	s8 instr;
 };
 
 struct instruction {
@@ -30,6 +32,7 @@ struct instruction {
 	unsigned long immediate;
 	bool alt_group, dead_end, ignore, hint, save, restore, ignore_alts;
 	bool retpoline_safe;
+	s8 instr;
 	u8 visited;
 	struct symbol *call_dest;
 	struct instruction *jump_dest;
tools/objtool/elf.h +1 -1
@@ -39,7 +39,7 @@ struct section {
 	char *name;
 	int idx;
 	unsigned int len;
-	bool changed, text, rodata;
+	bool changed, text, rodata, noinstr;
 };
 
 struct symbol {
author maintainer
#54 lore raw

PATCH v4 6/13 objtool: Optimize !vmlinux.o again

In reply to Peter Zijlstra

When doing kbuild tests to see if the objtool changes affected those I found that there was a measurable regression:

pre post

  real    1m13.594        1m16.488s
  user    34m58.246s      35m23.947s
  sys     4m0.393s        4m27.312s

Perf showed that for small files the increased hash-table sizes were a measurable difference. Since we already have -l "vmlinux" to distinguish between the modes, make it also use a smaller portion of the hash-tables.

This flips it into a small win:

  real    1m14.143s
  user    34m49.292s
  sys     3m44.746s
---
 tools/objtool/elf.c     |   62 +++++++++++++++++++++++++++++++++---------------
 tools/objtool/elf.h     |   13 ++++++----
 tools/objtool/orc_gen.c |    3 --
 3 files changed, 52 insertions(+), 26 deletions(-)
tools/objtool/elf.c +43 -19
@@ -27,6 +27,22 @@ static inline u32 str_hash(const char *s
 	return jhash(str, strlen(str), 0);
 }
 
+static inline int elf_hash_bits(void)
+{
+	return vmlinux ? ELF_HASH_BITS : 16;
+}
+
+#define elf_hash_add(hashtable, node, key) \
+	hlist_add_head(node, &hashtable[hash_min(key, elf_hash_bits())])
+
+static void elf_hash_init(struct hlist_head *table)
+{
+	__hash_init(table, 1U << elf_hash_bits());
+}
+
+#define elf_hash_for_each_possible(name, obj, member, key)			\
+	hlist_for_each_entry(obj, &name[hash_min(key, elf_hash_bits())], member)
+
 static void rb_add(struct rb_root *tree, struct rb_node *node,
 		   int (*cmp)(struct rb_node *, const struct rb_node *))
 {
@@ -115,7 +131,7 @@ struct section *find_section_by_name(str
 {
 	struct section *sec;
 
-	hash_for_each_possible(elf->section_name_hash, sec, name_hash, str_hash(name))
+	elf_hash_for_each_possible(elf->section_name_hash, sec, name_hash, str_hash(name))
 		if (!strcmp(sec->name, name))
 			return sec;
 
@@ -127,7 +143,7 @@ static struct section *find_section_by_i
 {
 	struct section *sec;
 
-	hash_for_each_possible(elf->section_hash, sec, hash, idx)
+	elf_hash_for_each_possible(elf->section_hash, sec, hash, idx)
 		if (sec->idx == idx)
 			return sec;
 
@@ -138,7 +154,7 @@ static struct symbol *find_symbol_by_ind
 {
 	struct symbol *sym;
 
-	hash_for_each_possible(elf->symbol_hash, sym, hash, idx)
+	elf_hash_for_each_possible(elf->symbol_hash, sym, hash, idx)
 		if (sym->idx == idx)
 			return sym;
 
@@ -205,7 +221,7 @@ struct symbol *find_symbol_by_name(struc
 {
 	struct symbol *sym;
 
-	hash_for_each_possible(elf->symbol_name_hash, sym, name_hash, str_hash(name))
+	elf_hash_for_each_possible(elf->symbol_name_hash, sym, name_hash, str_hash(name))
 		if (!strcmp(sym->name, name))
 			return sym;
 
@@ -224,7 +240,7 @@ struct rela *find_rela_by_dest_range(str
 	sec = sec->rela;
 
 	for_offset_range(o, offset, offset + len) {
-		hash_for_each_possible(elf->rela_hash, rela, hash,
+		elf_hash_for_each_possible(elf->rela_hash, rela, hash,
 				       sec_offset_hash(sec, o)) {
 			if (rela->sec != sec)
 				continue;
@@ -309,8 +325,8 @@ static int read_sections(struct elf *elf
 		sec->len = sec->sh.sh_size;
 
 		list_add_tail(&sec->list, &elf->sections);
-		hash_add(elf->section_hash, &sec->hash, sec->idx);
-		hash_add(elf->section_name_hash, &sec->name_hash, str_hash(sec->name));
+		elf_hash_add(elf->section_hash, &sec->hash, sec->idx);
+		elf_hash_add(elf->section_name_hash, &sec->name_hash, str_hash(sec->name));
 	}
 
 	if (stats)
@@ -394,8 +410,8 @@ static int read_symbols(struct elf *elf)
 		else
 			entry = &sym->sec->symbol_list;
 		list_add(&sym->list, entry);
-		hash_add(elf->symbol_hash, &sym->hash, sym->idx);
-		hash_add(elf->symbol_name_hash, &sym->name_hash, str_hash(sym->name));
+		elf_hash_add(elf->symbol_hash, &sym->hash, sym->idx);
+		elf_hash_add(elf->symbol_name_hash, &sym->name_hash, str_hash(sym->name));
 	}
 
 	if (stats)
@@ -456,6 +472,14 @@ static int read_symbols(struct elf *elf)
 	return -1;
 }
 
+void elf_add_rela(struct elf *elf, struct rela *rela)
+{
+	struct section *sec = rela->sec;
+
+	list_add_tail(&rela->list, &sec->rela_list);
+	elf_hash_add(elf->rela_hash, &rela->hash, rela_hash(rela));
+}
+
 static int read_relas(struct elf *elf)
 {
 	struct section *sec;
@@ -503,8 +527,7 @@ static int read_relas(struct elf *elf)
 				return -1;
 			}
 
-			list_add_tail(&rela->list, &sec->rela_list);
-			hash_add(elf->rela_hash, &rela->hash, rela_hash(rela));
+			elf_add_rela(elf, rela);
 			nr_rela++;
 		}
 		max_rela = max(max_rela, nr_rela);
@@ -531,15 +554,16 @@ struct elf *elf_read(const char *name, i
 		perror("malloc");
 		return NULL;
 	}
-	memset(elf, 0, sizeof(*elf));
+	memset(elf, 0, offsetof(struct elf, sections));
 
-	hash_init(elf->symbol_hash);
-	hash_init(elf->symbol_name_hash);
-	hash_init(elf->section_hash);
-	hash_init(elf->section_name_hash);
-	hash_init(elf->rela_hash);
 	INIT_LIST_HEAD(&elf->sections);
 
+	elf_hash_init(elf->symbol_hash);
+	elf_hash_init(elf->symbol_name_hash);
+	elf_hash_init(elf->section_hash);
+	elf_hash_init(elf->section_name_hash);
+	elf_hash_init(elf->rela_hash);
+
 	elf->fd = open(name, flags);
 	if (elf->fd == -1) {
 		fprintf(stderr, "objtool: Can't open '%s': %s\n",
@@ -676,8 +700,8 @@ struct section *elf_create_section(struc
 	shstrtab->changed = true;
 
 	list_add_tail(&sec->list, &elf->sections);
-	hash_add(elf->section_hash, &sec->hash, sec->idx);
-	hash_add(elf->section_name_hash, &sec->name_hash, str_hash(sec->name));
+	elf_hash_add(elf->section_hash, &sec->hash, sec->idx);
+	elf_hash_add(elf->section_name_hash, &sec->name_hash, str_hash(sec->name));
 
 	return sec;
 }
tools/objtool/elf.h +8 -5
@@ -70,17 +70,19 @@ struct rela {
 	bool jump_table_start;
 };
 
+#define ELF_HASH_BITS	20
+
 struct elf {
 	Elf *elf;
 	GElf_Ehdr ehdr;
 	int fd;
 	char *name;
 	struct list_head sections;
-	DECLARE_HASHTABLE(symbol_hash, 20);
-	DECLARE_HASHTABLE(symbol_name_hash, 20);
-	DECLARE_HASHTABLE(section_hash, 16);
-	DECLARE_HASHTABLE(section_name_hash, 16);
-	DECLARE_HASHTABLE(rela_hash, 20);
+	DECLARE_HASHTABLE(symbol_hash, ELF_HASH_BITS);
+	DECLARE_HASHTABLE(symbol_name_hash, ELF_HASH_BITS);
+	DECLARE_HASHTABLE(section_hash, ELF_HASH_BITS);
+	DECLARE_HASHTABLE(section_name_hash, ELF_HASH_BITS);
+	DECLARE_HASHTABLE(rela_hash, ELF_HASH_BITS);
 };
 
 #define OFFSET_STRIDE_BITS	4
@@ -127,6 +129,7 @@ struct section *elf_create_rela_section(
 int elf_rebuild_rela_section(struct section *sec);
 int elf_write(struct elf *elf);
 void elf_close(struct elf *elf);
+void elf_add_rela(struct elf *elf, struct rela *rela);
 
 #define for_each_sec(file, sec)						\
 	list_for_each_entry(sec, &file->elf->sections, list)
tools/objtool/orc_gen.c +1 -2
@@ -111,8 +111,7 @@ static int create_orc_entry(struct elf *
 	rela->offset = idx * sizeof(int);
 	rela->sec = ip_relasec;
 
-	list_add_tail(&rela->list, &ip_relasec->rela_list);
-	hash_add(elf->rela_hash, &rela->hash, rela_hash(rela));
+	elf_add_rela(elf, rela);
 
 	return 0;
 }
author maintainer
#55 lore raw

PATCH v4 7/13 objtool: Use sec_offset_hash() for insn_hash

In reply to Peter Zijlstra

In preparation for find_insn_containing(), change insn_hash to use sec_offset_hash().

This actually reduces runtime; probably because mixing in the section index reduces the collisions due to text sections all starting their instructions at offset 0.

Runtime on vmlinux.o from 3.1 to 2.5 seconds.

---
 tools/objtool/check.c |    5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
tools/objtool/check.c +3 -2
@@ -34,9 +34,10 @@ struct instruction *find_insn(struct obj
 {
 	struct instruction *insn;
 
-	hash_for_each_possible(file->insn_hash, insn, hash, offset)
+	hash_for_each_possible(file->insn_hash, insn, hash, sec_offset_hash(sec, offset)) {
 		if (insn->sec == sec && insn->offset == offset)
 			return insn;
+	}
 
 	return NULL;
 }
@@ -276,7 +277,7 @@ static int decode_instructions(struct ob
 			if (ret)
 				goto err;
 
-			hash_add(file->insn_hash, &insn->hash, insn->offset);
+			hash_add(file->insn_hash, &insn->hash, sec_offset_hash(sec, insn->offset));
 			list_add_tail(&insn->list, &file->insn_list);
 			nr_insns++;
 		}
author maintainer
#56 lore raw

PATCH v4 8/13 objtool: Detect loading function pointers across noinstr

In reply to Peter Zijlstra

Detect if noinstr text loads functions pointers from regular text, doing so is a definite sign that indirect function calls are unsafe.

---
 tools/objtool/arch.h  |    2 +
 tools/objtool/check.c |   71 ++++++++++++++++++++++++++++++++++++++++++++++++++
 tools/objtool/check.h |    2 -
 3 files changed, 74 insertions(+), 1 deletion(-)
tools/objtool/arch.h +2 -0
@@ -75,4 +75,6 @@ int arch_decode_instruction(struct elf *
 
 bool arch_callee_saved_reg(unsigned char reg);
 
+#define MAX_INSN_SIZE 15
+
 #endif /* _ARCH_H */
tools/objtool/check.c +71 -0

101 lines not on this page · raw message

tools/objtool/check.h +1 -1
@@ -31,7 +31,7 @@ struct instruction {
 	enum insn_type type;
 	unsigned long immediate;
 	bool alt_group, dead_end, ignore, hint, save, restore, ignore_alts;
-	bool retpoline_safe;
+	bool retpoline_safe, has_text_rela;
 	s8 instr;
 	u8 visited;
 	struct symbol *call_dest;
author maintainer
#57 lore raw

PATCH v4 9/13 kbuild/objtool: Add objtool-vmlinux.o pass

In reply to Peter Zijlstra

Now that objtool is capable of processing vmlinux.o and actually has something useful to do there, (conditionally) add it to the final link pass.

This will increase build time by a few seconds.

---
 lib/Kconfig.debug       |    5 +++++
 scripts/link-vmlinux.sh |   24 ++++++++++++++++++++++++
 2 files changed, 29 insertions(+)
lib/Kconfig.debug +5 -0
@@ -379,6 +379,11 @@ config STACK_VALIDATION
 	  For more information, see
 	  tools/objtool/Documentation/stack-validation.txt.
 
+config VMLINUX_VALIDATION
+	bool
+	depends on STACK_VALIDATION && DEBUG_ENTRY && !PARAVIRT
+	default y
+
 config DEBUG_FORCE_WEAK_PER_CPU
 	bool "Force weak per-cpu definitions"
 	depends on DEBUG_KERNEL
scripts/link-vmlinux.sh +24 -0
@@ -55,6 +55,29 @@ modpost_link()
 	${LD} ${KBUILD_LDFLAGS} -r -o ${1} ${objects}
 }
 
+objtool_link()
+{
+	local objtoolopt;
+
+	if [ -n "${CONFIG_VMLINUX_VALIDATION}" ]; then
+		objtoolopt="check"
+		if [ -z "${CONFIG_FRAME_POINTER}" ]; then
+			objtoolopt="${objtoolopt} --no-fp"
+		fi
+		if [ -n "${CONFIG_GCOV_KERNEL}" ]; then
+			objtoolopt="${objtoolopt} --no-unreachable"
+		fi
+		if [ -n "${CONFIG_RETPOLINE}" ]; then
+			objtoolopt="${objtoolopt} --retpoline"
+		fi
+		if [ -n "${CONFIG_X86_SMAP}" ]; then
+			objtoolopt="${objtoolopt} --uaccess"
+		fi
+		info OBJTOOL ${1}
+		tools/objtool/objtool ${objtoolopt} ${1}
+	fi
+}
+
 # Link of vmlinux
 # ${1} - output file
 # ${2}, ${3}, ... - optional extra .o files
@@ -244,6 +267,7 @@ ${MAKE} -f "${srctree}/scripts/Makefile.
 #link vmlinux.o
 info LD vmlinux.o
 modpost_link vmlinux.o
+objtool_link vmlinux.o
 
 # modpost vmlinux.o to check for section mismatches
 ${MAKE} -f "${srctree}/scripts/Makefile.modpost" MODPOST_VMLINUX=1
author maintainer
#58 lore raw

PATCH v4 10/13 objtool: Avoid iterating !text section symbols

In reply to Peter Zijlstra

validate_functions() iterates all sections their symbols; this is pointless to do for !text sections as they won't have instructions anyway.

---
 tools/objtool/check.c |    6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
tools/objtool/check.c +5 -1
@@ -2621,8 +2621,12 @@ static int validate_functions(struct obj
 	struct section *sec;
 	int warnings = 0;
 
-	for_each_sec(file, sec)
+	for_each_sec(file, sec) {
+		if (!(sec->sh.sh_flags & SHF_EXECINSTR))
+			continue;
+
 		warnings += validate_section(file, sec);
+	}
 
 	return warnings;
 }
author maintainer
#59 lore raw

PATCH v4 11/13 objtool: Rearrange validate_section()

In reply to Peter Zijlstra

In preparation of further changes, once again break out the loop body. No functional changes intended.

---
 tools/objtool/check.c |   51 ++++++++++++++++++++++++++++----------------------
 1 file changed, 29 insertions(+), 22 deletions(-)
tools/objtool/check.c +29 -22
@@ -2559,12 +2559,37 @@ static void prepare_insn_rela(struct obj
 	}
 }
 
-static int validate_section(struct objtool_file *file, struct section *sec)
+static int validate_symbol(struct objtool_file *file, struct section *sec,
+			   struct symbol *sym, struct insn_state *state)
 {
-	struct symbol *func;
 	struct instruction *insn;
+	int ret;
+
+	if (!sym->len) {
+		WARN("%s() is missing an ELF size annotation", sym->name);
+		return 1;
+	}
+
+	if (sym->pfunc != sym || sym->alias != sym)
+		return 0;
+
+	insn = find_insn(file, sec, sym->offset);
+	if (!insn || insn->ignore || insn->visited)
+		return 0;
+
+	state->uaccess = sym->uaccess_safe;
+
+	ret = validate_branch(file, insn->func, insn, *state);
+	if (ret && backtrace)
+		BT_FUNC("<=== (sym)", insn);
+	return ret;
+}
+
+static int validate_section(struct objtool_file *file, struct section *sec)
+{
 	struct insn_state state;
-	int ret, warnings = 0;
+	struct symbol *func;
+	int warnings = 0;
 
 	clear_insn_state(&state);
 
@@ -2588,25 +2613,7 @@ static int validate_section(struct objto
 		if (func->type != STT_FUNC)
 			continue;
 
-		if (!func->len) {
-			WARN("%s() is missing an ELF size annotation",
-			     func->name);
-			warnings++;
-		}
-
-		if (func->pfunc != func || func->alias != func)
-			continue;
-
-		insn = find_insn(file, sec, func->offset);
-		if (!insn || insn->ignore || insn->visited)
-			continue;
-
-		state.uaccess = func->uaccess_safe;
-
-		ret = validate_branch(file, func, insn, state);
-		if (ret && backtrace)
-			BT_FUNC("<=== (func)", insn);
-		warnings += ret;
+		warnings += validate_symbol(file, sec, func, &state);
 	}
 
 	return warnings;
author maintainer
#60 lore raw

PATCH v4 12/13 objtool: Add STT_NOTYPE noinstr validation

In reply to Peter Zijlstra

Make sure to also check STT_NOTYPE symbols for noinstr violations.

---
 tools/objtool/check.c |   46 ++++++++++++++++++++++++++++++----------------
 1 file changed, 30 insertions(+), 16 deletions(-)
tools/objtool/check.c +30 -16
@@ -248,10 +248,18 @@ static void init_cfi_state(struct cfi_st
 	cfi->drap_offset = -1;
 }
 
-static void clear_insn_state(struct insn_state *state)
+static void init_insn_state(struct insn_state *state, struct section *sec)
 {
 	memset(state, 0, sizeof(*state));
 	init_cfi_state(&state->cfi);
+
+	/*
+	 * We need the full vmlinux for noinstr validation, otherwise we can
+	 * not correctly determine insn->call_dest->sec (external symbols do
+	 * not have a section).
+	 */
+	if (vmlinux && sec)
+		state->noinstr = sec->noinstr;
 }
 
 /*
@@ -2422,24 +2430,34 @@ static int validate_branch(struct objtoo
 	return 0;
 }
 
-static int validate_unwind_hints(struct objtool_file *file)
+static int validate_unwind_hints(struct objtool_file *file, struct section *sec)
 {
 	struct instruction *insn;
-	int ret, warnings = 0;
 	struct insn_state state;
+	int ret, warnings = 0;
 
 	if (!file->hints)
 		return 0;
 
-	clear_insn_state(&state);
+	init_insn_state(&state, sec);
 
-	for_each_insn(file, insn) {
+	if (sec) {
+		insn = find_insn(file, sec, 0);
+		if (!insn)
+			return 0;
+	} else {
+		insn = list_first_entry(&file->insn_list, typeof(*insn), list);
+	}
+
+	while (&insn->list != &file->insn_list && (!sec || insn->sec == sec)) {
 		if (insn->hint && !insn->visited) {
 			ret = validate_branch(file, insn->func, insn, state);
 			if (ret && backtrace)
 				BT_FUNC("<=== (hint)", insn);
 			warnings += ret;
 		}
+
+		insn = list_next_entry(insn, list);
 	}
 
 	return warnings;
@@ -2592,21 +2610,13 @@ static int validate_section(struct objto
 	struct symbol *func;
 	int warnings = 0;
 
-	clear_insn_state(&state);
+	init_insn_state(&state, sec);
 
 	state.cfi.cfa = initial_func_cfi.cfa;
 	memcpy(&state.cfi.regs, &initial_func_cfi.regs,
 	       CFI_NUM_REGS * sizeof(struct cfi_reg));
 	state.cfi.stack_size = initial_func_cfi.cfa.offset;
 
-	/*
-	 * We need the full vmlinux for noinstr validation, otherwise we can
-	 * not correctly determine insn->call_dest->sec (external symbols do
-	 * not have a section).
-	 */
-	if (vmlinux)
-		state.noinstr = sec->noinstr;
-
 	if (state.noinstr)
 		prepare_insn_rela(file, sec);
 
@@ -2623,12 +2633,16 @@ static int validate_section(struct objto
 static int validate_vmlinux_functions(struct objtool_file *file)
 {
 	struct section *sec;
+	int warnings = 0;
 
 	sec = find_section_by_name(file->elf, ".noinstr.text");
 	if (!sec)
 		return 0;
 
-	return validate_section(file, sec);
+	warnings += validate_section(file, sec);
+	warnings += validate_unwind_hints(file, sec);
+
+	return warnings;
 }
 
 static int validate_functions(struct objtool_file *file)
@@ -2713,7 +2727,7 @@ int check(const char *_objname, bool orc
 		goto out;
 	warnings += ret;
 
-	ret = validate_unwind_hints(&file);
+	ret = validate_unwind_hints(&file, NULL);
 	if (ret < 0)
 		goto out;
 	warnings += ret;
author maintainer
#61 lore raw

PATCH v4 13/13 objtool: Also consider .entry.text as noinstr

In reply to Peter Zijlstra

From: Thomas Gleixner <[email protected]>

Consider all of .entry.text as noinstr. This gets us coverage across the PTI boundary. While we could add everything .noinstr.text into .entry.text that would bloat the amount of code in the user mapping.

---
 tools/objtool/check.c |   18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)
tools/objtool/check.c +12 -6
@@ -285,7 +285,8 @@ static int decode_instructions(struct ob
 		    strncmp(sec->name, ".discard.", 9))
 			sec->text = true;
 
-		if (!strcmp(sec->name, ".noinstr.text"))
+		if (!strcmp(sec->name, ".noinstr.text") ||
+		    !strcmp(sec->name, ".entry.text"))
 			sec->noinstr = true;
 
 		for (offset = 0; offset < sec->len; offset += insn->len) {
@@ -2065,7 +2066,7 @@ static inline const char *call_dest_name
 static int validate_call(struct instruction *insn, struct insn_state *state)
 {
 	if (state->noinstr && state->instr <= 0 &&
-	    (!insn->call_dest || insn->call_dest->sec != insn->sec)) {
+	    (!insn->call_dest || !insn->call_dest->sec->noinstr)) {
 		WARN_FUNC("call to %s() leaves .noinstr.text section",
 				insn->sec, insn->offset, call_dest_name(insn));
 		return 1;
@@ -2636,11 +2637,16 @@ static int validate_vmlinux_functions(st
 	int warnings = 0;
 
 	sec = find_section_by_name(file->elf, ".noinstr.text");
-	if (!sec)
-		return 0;
+	if (sec) {
+		warnings += validate_section(file, sec);
+		warnings += validate_unwind_hints(file, sec);
+	}
 
-	warnings += validate_section(file, sec);
-	warnings += validate_unwind_hints(file, sec);
+	sec = find_section_by_name(file->elf, ".entry.text");
+	if (sec) {
+		warnings += validate_section(file, sec);
+		warnings += validate_unwind_hints(file, sec);
+	}
 
 	return warnings;
 }
Reviewed-by on the v4 series
#62 lore raw
In reply to Peter Zijlstra
Quoted 5 lines · Peter Zijlstra

Hi all,

This lacks the first 17 patches of the v3 posting which are en-route to tip/core/objtool and should show up there somewhere later this evening.

Quoted 8 lines

As should be familiar by now; these patches implement the noinstr (no-instrument) validation in objtool as requested by Thomas, to ensure critical code (entry for now, idle later) run no unexpected code.

Functions are marked with: noinstr, which implies notrace, noinline and sticks things in the .noinstr.text section. Such functions can then use instr_begin() and instr_end() to allow calls to code outside of this section in sanctioned areas.

M