Skip to contentSkip to conversation

objtool: vmlinux.o and noinstr validation

PATCH Applied Peter Zijlstra 7 participants 64 messages Apr 16, 2020 – Apr 23, 2020

Subsystems (7)
x86 Kbuild Library code Tracing +3 more subsystems
lore
Share & tools
lore

Peter Zijlstra

author maintainer
#0 lore raw

v5 cover objtool: vmlinux.o and noinstr validation

As should be familiar by now; these patches implement the noinstr (no-instrument) validation in objtool as requested by Thomas, to ensure critical code (entry for now, idle later) run no unexpected code.

Functions are marked with: noinstr, which implies notrace, noinline and sticks things in the .noinstr.text section. Such functions can then use instr_begin() and instr_end() to allow calls to code outside of this section in sanctioned areas.

On the way of getting there, it also fixes the x86/ftrace trampiline ORC unwind.

author maintainer
#1 lore raw

PATCH v5 1/17 objtool: Support multiple stack_op per instruction

From: Julien Thierry <[email protected]>

Instruction sets can include more or less complex operations which might not fit the currently defined set of stack_ops.

Combining more than one stack_op provides more flexibility to describe the behaviour of an instruction. This also reduces the need to define new stack_ops specific to a single instruction set.

Allow instruction decoders to generate multiple stack_op per instruction.

---
 tools/objtool/arch.h            |    4 +-
 tools/objtool/arch/x86/decode.c |   13 ++++++-
 tools/objtool/check.c           |   74 ++++++++++++++++++++++++----------------
 tools/objtool/check.h           |    2 -
 4 files changed, 62 insertions(+), 31 deletions(-)
tools/objtool/arch.h +3 -1
@@ -64,6 +64,7 @@ struct op_src {
 struct stack_op {
 	struct op_dest dest;
 	struct op_src src;
+	struct list_head list;
 };
 
 void arch_initial_func_cfi_state(struct cfi_state *state);
@@ -71,7 +72,8 @@ void arch_initial_func_cfi_state(struct
 int arch_decode_instruction(struct elf *elf, struct section *sec,
 			    unsigned long offset, unsigned int maxlen,
 			    unsigned int *len, enum insn_type *type,
-			    unsigned long *immediate, struct stack_op *op);
+			    unsigned long *immediate,
+			    struct list_head *ops_list);
 
 bool arch_callee_saved_reg(unsigned char reg);
 
tools/objtool/arch/x86/decode.c +12 -1
@@ -69,13 +69,15 @@ bool arch_callee_saved_reg(unsigned char
 int arch_decode_instruction(struct elf *elf, struct section *sec,
 			    unsigned long offset, unsigned int maxlen,
 			    unsigned int *len, enum insn_type *type,
-			    unsigned long *immediate, struct stack_op *op)
+			    unsigned long *immediate,
+			    struct list_head *ops_list)
 {
 	struct insn insn;
 	int x86_64, sign;
 	unsigned char op1, op2, rex = 0, rex_b = 0, rex_r = 0, rex_w = 0,
 		      rex_x = 0, modrm = 0, modrm_mod = 0, modrm_rm = 0,
 		      modrm_reg = 0, sib = 0;
+	struct stack_op *op;
 
 	x86_64 = is_x86_64(elf);
 	if (x86_64 == -1)
@@ -116,6 +118,10 @@ int arch_decode_instruction(struct elf *
 	if (insn.sib.nbytes)
 		sib = insn.sib.bytes[0];
 
+	op = calloc(1, sizeof(*op));
+	if (!op)
+		return -1;
+
 	switch (op1) {
 
 	case 0x1:
@@ -477,6 +483,11 @@ int arch_decode_instruction(struct elf *
 
 	*immediate = insn.immediate.nbytes ? insn.immediate.value : 0;
 
+	if (*type == INSN_STACK)
+		list_add_tail(&op->list, ops_list);
+	else
+		free(op);
+
 	return 0;
 }
 
tools/objtool/check.c +46 -28

126 lines not on this page · raw message

tools/objtool/check.h +1 -1
@@ -42,7 +42,7 @@ struct instruction {
 	struct rela *jump_table;
 	struct list_head alts;
 	struct symbol *func;
-	struct stack_op stack_op;
+	struct list_head stack_ops;
 	struct insn_state state;
 	struct orc_entry orc;
 };
author maintainer
#2 lore raw

PATCH v5 2/17 objtool: Better handle IRET

In reply to Peter Zijlstra

Teach objtool a little more about IRET so that we can avoid using the SAVE/RESTORE annotation. In particular, make the weird corner case in insn->restore go away.

The purpose of that corner case is to deal with the fact that UNWIND_HINT_RESTORE lands on the instruction after IRET, but that instruction can end up being outside the basic block, consider:

	if (cond)
		sync_core()
	foo();

Then the hint will land on foo(), and we'll encounter the restore hint without ever having seen the save hint.

By teaching objtool about the arch specific exception frame size, and assuming that any IRET in an STT_FUNC symbol is an exception frame sized POP, we can remove the use of save/restore hints for this code.

---
 arch/x86/include/asm/processor.h |    2 --
 tools/objtool/arch.h             |    1 +
 tools/objtool/arch/x86/decode.c  |   14 ++++++++++++--
 tools/objtool/check.c            |   29 ++++++++++++++++-------------
 4 files changed, 29 insertions(+), 17 deletions(-)
arch/x86/include/asm/processor.h +0 -2
@@ -727,7 +727,6 @@ static inline void sync_core(void)
 	unsigned int tmp;
 
 	asm volatile (
-		UNWIND_HINT_SAVE
 		"mov %%ss, %0\n\t"
 		"pushq %q0\n\t"
 		"pushq %%rsp\n\t"
@@ -737,7 +736,6 @@ static inline void sync_core(void)
 		"pushq %q0\n\t"
 		"pushq $1f\n\t"
 		"iretq\n\t"
-		UNWIND_HINT_RESTORE
 		"1:"
 		: "=&r" (tmp), ASM_CALL_CONSTRAINT : : "cc", "memory");
 #endif
tools/objtool/arch.h +1 -0
@@ -19,6 +19,7 @@ enum insn_type {
 	INSN_CALL,
 	INSN_CALL_DYNAMIC,
 	INSN_RETURN,
+	INSN_EXCEPTION_RETURN,
 	INSN_CONTEXT_SWITCH,
 	INSN_STACK,
 	INSN_BUG,
tools/objtool/arch/x86/decode.c +12 -2
@@ -435,9 +435,19 @@ int arch_decode_instruction(struct elf *
 		*type = INSN_RETURN;
 		break;
 
+	case 0xcf: /* iret */
+		*type = INSN_EXCEPTION_RETURN;
+
+		/* add $40, %rsp */
+		op->src.type = OP_SRC_ADD;
+		op->src.reg = CFI_SP;
+		op->src.offset = 5*8;
+		op->dest.type = OP_DEST_REG;
+		op->dest.reg = CFI_SP;
+		break;
+
 	case 0xca: /* retf */
 	case 0xcb: /* retf */
-	case 0xcf: /* iret */
 		*type = INSN_CONTEXT_SWITCH;
 		break;
 
@@ -483,7 +493,7 @@ int arch_decode_instruction(struct elf *
 
 	*immediate = insn.immediate.nbytes ? insn.immediate.value : 0;
 
-	if (*type == INSN_STACK)
+	if (*type == INSN_STACK || *type == INSN_EXCEPTION_RETURN)
 		list_add_tail(&op->list, ops_list);
 	else
 		free(op);
tools/objtool/check.c +16 -13
@@ -2080,15 +2080,14 @@ static int validate_return(struct symbol
  * tools/objtool/Documentation/stack-validation.txt.
  */
 static int validate_branch(struct objtool_file *file, struct symbol *func,
-			   struct instruction *first, struct insn_state state)
+			   struct instruction *insn, struct insn_state state)
 {
 	struct alternative *alt;
-	struct instruction *insn, *next_insn;
+	struct instruction *next_insn;
 	struct section *sec;
 	u8 visited;
 	int ret;
 
-	insn = first;
 	sec = insn->sec;
 
 	if (insn->alt_group && list_empty(&insn->alts)) {
@@ -2141,16 +2140,6 @@ static int validate_branch(struct objtoo
 				}
 
 				if (!save_insn->visited) {
-					/*
-					 * Oops, no state to copy yet.
-					 * Hopefully we can reach this
-					 * instruction from another branch
-					 * after the save insn has been
-					 * visited.
-					 */
-					if (insn == first)
-						return 0;
-
 					WARN_FUNC("objtool isn't smart enough to handle this CFI save/restore combo",
 						  sec, insn->offset);
 					return 1;
@@ -2243,6 +2232,20 @@ static int validate_branch(struct objtoo
 
 			break;
 
+		case INSN_EXCEPTION_RETURN:
+			if (handle_insn_ops(insn, &state))
+				return 1;
+
+			/*
+			 * This handles x86's sync_core() case, where we use an
+			 * IRET to self. All 'normal' IRET instructions are in
+			 * STT_NOTYPE entry symbols.
+			 */
+			if (func)
+				break;
+
+			return 0;
+
 		case INSN_CONTEXT_SWITCH:
 			if (func && (!next_insn || !next_insn->hint)) {
 				WARN_FUNC("unsupported instruction in callable function",
#3 lore raw

Re: [PATCH v5 02/17] objtool: Better handle IRET

Quoted 135 lines · Peter Zijlstra

Teach objtool a little more about IRET so that we can avoid using the SAVE/RESTORE annotation. In particular, make the weird corner case in insn->restore go away.

The purpose of that corner case is to deal with the fact that UNWIND_HINT_RESTORE lands on the instruction after IRET, but that instruction can end up being outside the basic block, consider:

	if (cond)
		sync_core()
	foo();

Then the hint will land on foo(), and we'll encounter the restore hint without ever having seen the save hint.

By teaching objtool about the arch specific exception frame size, and assuming that any IRET in an STT_FUNC symbol is an exception frame sized POP, we can remove the use of save/restore hints for this code.

---
 arch/x86/include/asm/processor.h |    2 --
 tools/objtool/arch.h             |    1 +
 tools/objtool/arch/x86/decode.c  |   14 ++++++++++++--
 tools/objtool/check.c            |   29 ++++++++++++++++-------------
 4 files changed, 29 insertions(+), 17 deletions(-)
arch/x86/include/asm/processor.h +0 -2
@@ -727,7 +727,6 @@ static inline void sync_core(void)
 	unsigned int tmp;
 
 	asm volatile (
-		UNWIND_HINT_SAVE
 		"mov %%ss, %0\n\t"
 		"pushq %q0\n\t"
 		"pushq %%rsp\n\t"
@@ -737,7 +736,6 @@ static inline void sync_core(void)
 		"pushq %q0\n\t"
 		"pushq $1f\n\t"
 		"iretq\n\t"
-		UNWIND_HINT_RESTORE
 		"1:"
 		: "=&r" (tmp), ASM_CALL_CONSTRAINT : : "cc", "memory");
 #endif
tools/objtool/arch.h +1 -0
@@ -19,6 +19,7 @@ enum insn_type {
 	INSN_CALL,
 	INSN_CALL_DYNAMIC,
 	INSN_RETURN,
+	INSN_EXCEPTION_RETURN,
 	INSN_CONTEXT_SWITCH,
 	INSN_STACK,
 	INSN_BUG,
tools/objtool/arch/x86/decode.c +12 -2
@@ -435,9 +435,19 @@ int arch_decode_instruction(struct elf *
 		*type = INSN_RETURN;
 		break;
 
+	case 0xcf: /* iret */
+		*type = INSN_EXCEPTION_RETURN;
+
+		/* add $40, %rsp */
+		op->src.type = OP_SRC_ADD;
+		op->src.reg = CFI_SP;
+		op->src.offset = 5*8;
+		op->dest.type = OP_DEST_REG;
+		op->dest.reg = CFI_SP;
+		break;
+
 	case 0xca: /* retf */
 	case 0xcb: /* retf */
-	case 0xcf: /* iret */
 		*type = INSN_CONTEXT_SWITCH;
 		break;
 
@@ -483,7 +493,7 @@ int arch_decode_instruction(struct elf *
 
 	*immediate = insn.immediate.nbytes ? insn.immediate.value : 0;
 
-	if (*type == INSN_STACK)
+	if (*type == INSN_STACK || *type == INSN_EXCEPTION_RETURN)
 		list_add_tail(&op->list, ops_list);
 	else
 		free(op);
tools/objtool/check.c +9 -13
@@ -2080,15 +2080,14 @@ static int validate_return(struct symbol
  * tools/objtool/Documentation/stack-validation.txt.
  */
 static int validate_branch(struct objtool_file *file, struct symbol *func,
-			   struct instruction *first, struct insn_state state)
+			   struct instruction *insn, struct insn_state state)
 {
 	struct alternative *alt;
-	struct instruction *insn, *next_insn;
+	struct instruction *next_insn;
 	struct section *sec;
 	u8 visited;
 	int ret;
 
-	insn = first;
 	sec = insn->sec;
 
 	if (insn->alt_group && list_empty(&insn->alts)) {
@@ -2141,16 +2140,6 @@ static int validate_branch(struct objtoo
 				}
 
 				if (!save_insn->visited) {
-					/*
-					 * Oops, no state to copy yet.
-					 * Hopefully we can reach this
-					 * instruction from another branch
-					 * after the save insn has been
-					 * visited.
-					 */
-					if (insn == first)
-						return 0;
-
 					WARN_FUNC("objtool isn't smart enough to handle this CFI save/restore combo",
 						  sec, insn->offset);
 					return 1;
@@ -2243,6 +2232,20 @@ static int validate_branch(struct objtoo
 
 			break;
 
+		case INSN_EXCEPTION_RETURN:
+			if (handle_insn_ops(insn, &state))
+				return 1;
+
+			/*
+			 * This handles x86's sync_core() case, where we use an
+			 * IRET to self. All 'normal' IRET instructions are in

tools/objtool/check.c static int validate_branch(struct objtoo · line 2251

Quoted 10 lines
tools/objtool/check.c +7 -0
+			 * STT_NOTYPE entry symbols.
+			 */
+			if (func)
+				break;
+
+			return 0;
+
 		case INSN_CONTEXT_SWITCH:
 			if (func && (!next_insn || !next_insn->hint)) {
 				WARN_FUNC("unsupported instruction in callable function",

It looks really simple.

Have you tried Julien's proposal about removing INSN_STACK altogether, move the x86 to arch/x86/ and call handle_insn_ops() unconditionally, or have you just postponed it? As I said, I think it could be better in the long term, but the above looks good for now as well.

Miroslav

author maintainer
#4 lore raw

Re: [PATCH v5 02/17] objtool: Better handle IRET

Quoted 22 lines · Miroslav Benes
Quoted 7 lines · Peter Zijlstra
tools/objtool/check.c +7 -0
+		case INSN_EXCEPTION_RETURN:
+			if (handle_insn_ops(insn, &state))
+				return 1;
+
+			/*
+			 * This handles x86's sync_core() case, where we use an
+			 * IRET to self. All 'normal' IRET instructions are in

tools/objtool/check.c static int validate_branch(struct objtoo · line 2251

Quoted 10 lines
tools/objtool/check.c +7 -0
+			 * STT_NOTYPE entry symbols.
+			 */
+			if (func)
+				break;
+
+			return 0;
+
 		case INSN_CONTEXT_SWITCH:
 			if (func && (!next_insn || !next_insn->hint)) {
 				WARN_FUNC("unsupported instruction in callable function",

It looks really simple.

Quoted 4 lines

Have you tried Julien's proposal about removing INSN_STACK altogether, move the x86 to arch/x86/ and call handle_insn_ops() unconditionally, or have you just postponed it? As I said, I think it could be better in the long term, but the above looks good for now as well.

If you look at this other set I send yesterday:

https://lkml.kernel.org/r/[email protected]

(also, sorry for not adding you to the Cc; also best look at the gitweb version, the patches I send out are missing a hunk and lacking some back-merges.. clearly I wasn't having a good day yesterday).

it has this intra_function_calls crud that needs explicit conditional handle_insn_ops().

#5 lore raw

Re: [PATCH v5 02/17] objtool: Better handle IRET

Quoted 30 lines · Peter Zijlstra
Quoted 22 lines · Miroslav Benes
Quoted 7 lines · Peter Zijlstra
tools/objtool/check.c +7 -0
+		case INSN_EXCEPTION_RETURN:
+			if (handle_insn_ops(insn, &state))
+				return 1;
+
+			/*
+			 * This handles x86's sync_core() case, where we use an
+			 * IRET to self. All 'normal' IRET instructions are in

tools/objtool/check.c static int validate_branch(struct objtoo · line 2251

Quoted 10 lines
tools/objtool/check.c +7 -0
+			 * STT_NOTYPE entry symbols.
+			 */
+			if (func)
+				break;
+
+			return 0;
+
 		case INSN_CONTEXT_SWITCH:
 			if (func && (!next_insn || !next_insn->hint)) {
 				WARN_FUNC("unsupported instruction in callable function",

It looks really simple.

Quoted 4 lines

Have you tried Julien's proposal about removing INSN_STACK altogether, move the x86 to arch/x86/ and call handle_insn_ops() unconditionally, or have you just postponed it? As I said, I think it could be better in the long term, but the above looks good for now as well.

If you look at this other set I send yesterday:

Quoted 8 lines

https://lkml.kernel.org/r/[email protected]

(also, sorry for not adding you to the Cc; also best look at the gitweb version, the patches I send out are missing a hunk and lacking some back-merges.. clearly I wasn't having a good day yesterday).

it has this intra_function_calls crud that needs explicit conditional handle_insn_ops().

Ah, ok. Thanks for letting me know. There are so many patches for objtool flying around now that it is easy to miss something.

Miroslav

#6 lore raw

Re: [PATCH v5 02/17] objtool: Better handle IRET

In reply to Peter Zijlstra
Quoted 125 lines · Peter Zijlstra

Teach objtool a little more about IRET so that we can avoid using the SAVE/RESTORE annotation. In particular, make the weird corner case in insn->restore go away.

The purpose of that corner case is to deal with the fact that UNWIND_HINT_RESTORE lands on the instruction after IRET, but that instruction can end up being outside the basic block, consider:

	if (cond)
		sync_core()
	foo();

Then the hint will land on foo(), and we'll encounter the restore hint without ever having seen the save hint.

By teaching objtool about the arch specific exception frame size, and assuming that any IRET in an STT_FUNC symbol is an exception frame sized POP, we can remove the use of save/restore hints for this code.

---
  arch/x86/include/asm/processor.h |    2 --
  tools/objtool/arch.h             |    1 +
  tools/objtool/arch/x86/decode.c  |   14 ++++++++++++--
  tools/objtool/check.c            |   29 ++++++++++++++++-------------
  4 files changed, 29 insertions(+), 17 deletions(-)
arch/x86/include/asm/processor.h +0 -2
@@ -727,7 +727,6 @@ static inline void sync_core(void)
  	unsigned int tmp;
 
  	asm volatile (
-		UNWIND_HINT_SAVE
  		"mov %%ss, %0\n\t"
  		"pushq %q0\n\t"
  		"pushq %%rsp\n\t"
@@ -737,7 +736,6 @@ static inline void sync_core(void)
  		"pushq %q0\n\t"
  		"pushq $1f\n\t"
  		"iretq\n\t"
-		UNWIND_HINT_RESTORE
  		"1:"
  		: "=&r" (tmp), ASM_CALL_CONSTRAINT : : "cc", "memory");
  #endif
tools/objtool/arch.h +1 -0
@@ -19,6 +19,7 @@ enum insn_type {
  	INSN_CALL,
  	INSN_CALL_DYNAMIC,
  	INSN_RETURN,
+	INSN_EXCEPTION_RETURN,
  	INSN_CONTEXT_SWITCH,
  	INSN_STACK,
  	INSN_BUG,
tools/objtool/arch/x86/decode.c +12 -2
@@ -435,9 +435,19 @@ int arch_decode_instruction(struct elf *
  		*type = INSN_RETURN;
  		break;
 
+	case 0xcf: /* iret */
+		*type = INSN_EXCEPTION_RETURN;
+
+		/* add $40, %rsp */
+		op->src.type = OP_SRC_ADD;
+		op->src.reg = CFI_SP;
+		op->src.offset = 5*8;
+		op->dest.type = OP_DEST_REG;
+		op->dest.reg = CFI_SP;
+		break;
+
  	case 0xca: /* retf */
  	case 0xcb: /* retf */
-	case 0xcf: /* iret */
  		*type = INSN_CONTEXT_SWITCH;
  		break;
 
@@ -483,7 +493,7 @@ int arch_decode_instruction(struct elf *
 
  	*immediate = insn.immediate.nbytes ? insn.immediate.value : 0;
 
-	if (*type == INSN_STACK)
+	if (*type == INSN_STACK || *type == INSN_EXCEPTION_RETURN)
  		list_add_tail(&op->list, ops_list);
  	else
  		free(op);
tools/objtool/check.c +2 -13
@@ -2080,15 +2080,14 @@ static int validate_return(struct symbol
   * tools/objtool/Documentation/stack-validation.txt.
   */
  static int validate_branch(struct objtool_file *file, struct symbol *func,
-			   struct instruction *first, struct insn_state state)
+			   struct instruction *insn, struct insn_state state)
  {
  	struct alternative *alt;
-	struct instruction *insn, *next_insn;
+	struct instruction *next_insn;
  	struct section *sec;
  	u8 visited;
  	int ret;
 
-	insn = first;
  	sec = insn->sec;
 
  	if (insn->alt_group && list_empty(&insn->alts)) {
@@ -2141,16 +2140,6 @@ static int validate_branch(struct objtoo
  				}
 
  				if (!save_insn->visited) {
-					/*
-					 * Oops, no state to copy yet.
-					 * Hopefully we can reach this
-					 * instruction from another branch
-					 * after the save insn has been
-					 * visited.
-					 */
-					if (insn == first)
-						return 0;
-
  					WARN_FUNC("objtool isn't smart enough to handle this CFI save/restore combo",
  						  sec, insn->offset);
  					return 1;

tools/objtool/check.c static int validate_branch(struct objtoo · line 2237

Quoted 6 lines
tools/objtool/check.c +3 -0
@@ -2243,6 +2232,20 @@ static int validate_branch(struct objtoo
 
  			break;
 
+		case INSN_EXCEPTION_RETURN:
+			if (handle_insn_ops(insn, &state))
+				return 1;

Do we need to update the stack state for normal IRET? This wasn't done before. So shouldn't this better be:

               case INSN_EXCEPTION_RETURN:
                        if (!func)
                                return 0;

                        if (handle_insn_ops(insn, &state))
                                return 1;

                        break;

tools/objtool/check.c static int validate_branch(struct objtoo · line 2245

Quoted 8 lines
tools/objtool/check.c +8 -0
+
+			/*
+			 * This handles x86's sync_core() case, where we use an
+			 * IRET to self. All 'normal' IRET instructions are in
+			 * STT_NOTYPE entry symbols.
+			 */
+			if (func)
+				break;

Is it worth checking that func->name is effectively "sync_core"?

alex.

Quoted 8 lines
tools/objtool/check.c +3 -0
+
+			return 0;
+
  		case INSN_CONTEXT_SWITCH:
  			if (func && (!next_insn || !next_insn->hint)) {
  				WARN_FUNC("unsupported instruction in callable function",
author maintainer
#7 lore raw

Re: [PATCH v5 02/17] objtool: Better handle IRET

Quoted 9 lines · Alexandre Chartre

tools/objtool/check.c static int validate_branch(struct objtoo · line 2235

Quoted 5 lines
tools/objtool/check.c +3 -0
@@ -2243,6 +2232,20 @@ static int validate_branch(struct objtoo
  			break;
+		case INSN_EXCEPTION_RETURN:
+			if (handle_insn_ops(insn, &state))
+				return 1;

Do we need to update the stack state for normal IRET? This wasn't done before. So shouldn't this better be:

Quoted 8 lines
               case INSN_EXCEPTION_RETURN:
                        if (!func)
                                return 0;

                        if (handle_insn_ops(insn, &state))
                                return 1;

                        break;

Well, I was going to do the unconditional handle_insn_ops(), like mentioned, but then that intra_function_call thing spoiled it.

It doesn't matter though; STT_NOTYPE doesn't care.

Quoted 10 lines

tools/objtool/check.c static int validate_branch(struct objtoo · line 2245

Quoted 8 lines
tools/objtool/check.c +8 -0
+
+			/*
+			 * This handles x86's sync_core() case, where we use an
+			 * IRET to self. All 'normal' IRET instructions are in
+			 * STT_NOTYPE entry symbols.
+			 */
+			if (func)
+				break;

Is it worth checking that func->name is effectively "sync_core"?

It's an inline..

#8 lore raw

Re: [PATCH v5 02/17] objtool: Better handle IRET

Quoted 34 lines · Peter Zijlstra
Quoted 9 lines · Alexandre Chartre

tools/objtool/check.c static int validate_branch(struct objtoo · line 2235

Quoted 5 lines
tools/objtool/check.c +3 -0
@@ -2243,6 +2232,20 @@ static int validate_branch(struct objtoo
                    break;
+           case INSN_EXCEPTION_RETURN:
+                   if (handle_insn_ops(insn, &state))
+                           return 1;

Do we need to update the stack state for normal IRET? This wasn't done before. So shouldn't this better be:

Quoted 8 lines
               case INSN_EXCEPTION_RETURN:
                        if (!func)
                                return 0;

                        if (handle_insn_ops(insn, &state))
                                return 1;

                        break;

Well, I was going to do the unconditional handle_insn_ops(), like mentioned, but then that intra_function_call thing spoiled it.

It doesn't matter though; STT_NOTYPE doesn't care.

Quoted 9 lines
Quoted 8 lines
tools/objtool/check.c +8 -0
+
+                   /*
+                    * This handles x86's sync_core() case, where we use an
+                    * IRET to self. All 'normal' IRET instructions are in
+                    * STT_NOTYPE entry symbols.
+                    */
+                   if (func)
+                           break;
Quoted 3 lines
Quoted 1 line

Is it worth checking that func->name is effectively "sync_core"?

It's an inline..

I'm wondering if this would be easier if we just moved the guts of sync_core() into asm.

In the near future, I think we want to rework it a tiny bit. In particular, I think we're going to want to make sync_core() do:

if (static_cpu_has(X86_FEATURE_SERIALIZE))
asm volatile ("serialize");
else
iret_to_self();

where iret_to_self() is the meat of the IRET hack. And then we're going to add a new thingy unmask_nmi() that does iret_to_self() on everything except SEV-ES. The near-term motivation is that I think we have some genuine bugs in a couple of corner cases:

1. On AMD chips, if NMI hits user code with invalid CS or SS, we will enter on the NMI stack, switch to the normal stack, and return with IRET, and the IRET will fail. And then we end up in a nasty state in which NMIs are masked but the code path we run doesn't expect that. So we should unmask_nmi() in fixup_bad_iret() or similar. Intel CPUs are unaffected because Intel is differently quirky.

2. do_nmi() does this:

    if (user_mode(regs))
                mds_user_clear_cpu_buffers();

because it can't safely call prepare_exit_to_usermode(). This is a gross wart and I'd like to fix it. Fixing it involves teaching the relevant code paths to unmask_nmis() if they're going to so IRQs-on exit work.

None of this is really relevant to the current patch, but it wouldn't be unreasonable to turn the IRET thing from an inline asm into a real asm function if it makes objtool's life easier.

#9 lore raw

Re: [PATCH v5 02/17] objtool: Better handle IRET

Quoted 28 lines · Andy Lutomirski

I'm wondering if this would be easier if we just moved the guts of sync_core() into asm.

In the near future, I think we want to rework it a tiny bit. In particular, I think we're going to want to make sync_core() do:

if (static_cpu_has(X86_FEATURE_SERIALIZE))
asm volatile ("serialize");
else
iret_to_self();

where iret_to_self() is the meat of the IRET hack. And then we're going to add a new thingy unmask_nmi() that does iret_to_self() on everything except SEV-ES. The near-term motivation is that I think we have some genuine bugs in a couple of corner cases:

1. On AMD chips, if NMI hits user code with invalid CS or SS, we will enter on the NMI stack, switch to the normal stack, and return with IRET, and the IRET will fail. And then we end up in a nasty state in which NMIs are masked but the code path we run doesn't expect that. So we should unmask_nmi() in fixup_bad_iret() or similar. Intel CPUs are unaffected because Intel is differently quirky.

2. do_nmi() does this:

    if (user_mode(regs))
                mds_user_clear_cpu_buffers();
Quoted 8 lines

because it can't safely call prepare_exit_to_usermode(). This is a gross wart and I'd like to fix it. Fixing it involves teaching the relevant code paths to unmask_nmis() if they're going to so IRQs-on exit work.

None of this is really relevant to the current patch, but it wouldn't be unreasonable to turn the IRET thing from an inline asm into a real asm function if it makes objtool's life easier.

I don't think that would make objtool's life any easier -- it still has to understand the stack impact of the IRET-to-self thing regardless.

Signature · 1 line
Josh
merge bot Applied
#10 lore raw

[tip: objtool/core] objtool: Better handle IRET

In reply to Peter Zijlstra
The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     016db2d9c63e3ef0e7c3776efb38f352053cdd1e
Gitweb:        https://git.kernel.org/tip/016db2d9c63e3ef0e7c3776efb38f352053cdd1e
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Thu, 02 Apr 2020 10:15:51 +02:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Wed, 22 Apr 2020 23:10:05 +02:00

objtool: Better handle IRET

Teach objtool a little more about IRET so that we can avoid using the SAVE/RESTORE annotation. In particular, make the weird corner case in insn->restore go away.

The purpose of that corner case is to deal with the fact that UNWIND_HINT_RESTORE lands on the instruction after IRET, but that instruction can end up being outside the basic block, consider:

	if (cond)
		sync_core()
	foo();

Then the hint will land on foo(), and we'll encounter the restore hint without ever having seen the save hint.

By teaching objtool about the arch specific exception frame size, and assuming that any IRET in an STT_FUNC symbol is an exception frame sized POP, we can remove the use of save/restore hints for this code.

---
 arch/x86/include/asm/processor.h |  2 --
 tools/objtool/arch.h             |  1 +
 tools/objtool/arch/x86/decode.c  | 14 ++++++++++++--
 tools/objtool/check.c            | 29 ++++++++++++++++-------------
 4 files changed, 29 insertions(+), 17 deletions(-)
arch/x86/include/asm/processor.h +0 -2
index 3bcf27c..3eeaaeb 100644
@@ -727,7 +727,6 @@ static inline void sync_core(void)
 	unsigned int tmp;
 
 	asm volatile (
-		UNWIND_HINT_SAVE
 		"mov %%ss, %0\n\t"
 		"pushq %q0\n\t"
 		"pushq %%rsp\n\t"
@@ -737,7 +736,6 @@ static inline void sync_core(void)
 		"pushq %q0\n\t"
 		"pushq $1f\n\t"
 		"iretq\n\t"
-		UNWIND_HINT_RESTORE
 		"1:"
 		: "=&r" (tmp), ASM_CALL_CONSTRAINT : : "cc", "memory");
 #endif
tools/objtool/arch.h +1 -0
index f9883c4..55396df 100644
@@ -19,6 +19,7 @@ enum insn_type {
 	INSN_CALL,
 	INSN_CALL_DYNAMIC,
 	INSN_RETURN,
+	INSN_EXCEPTION_RETURN,
 	INSN_CONTEXT_SWITCH,
 	INSN_STACK,
 	INSN_BUG,
tools/objtool/arch/x86/decode.c +12 -2
index 199b408..3273638 100644
@@ -446,9 +446,19 @@ int arch_decode_instruction(struct elf *elf, struct section *sec,
 		*type = INSN_RETURN;
 		break;
 
+	case 0xcf: /* iret */
+		*type = INSN_EXCEPTION_RETURN;
+
+		/* add $40, %rsp */
+		op->src.type = OP_SRC_ADD;
+		op->src.reg = CFI_SP;
+		op->src.offset = 5*8;
+		op->dest.type = OP_DEST_REG;
+		op->dest.reg = CFI_SP;
+		break;
+
 	case 0xca: /* retf */
 	case 0xcb: /* retf */
-	case 0xcf: /* iret */
 		*type = INSN_CONTEXT_SWITCH;
 		break;
 
@@ -494,7 +504,7 @@ int arch_decode_instruction(struct elf *elf, struct section *sec,
 
 	*immediate = insn.immediate.nbytes ? insn.immediate.value : 0;
 
-	if (*type == INSN_STACK)
+	if (*type == INSN_STACK || *type == INSN_EXCEPTION_RETURN)
 		list_add_tail(&op->list, ops_list);
 	else
 		free(op);
tools/objtool/check.c +16 -13
index 819de0d..72bf5cc 100644
@@ -2081,15 +2081,14 @@ static int validate_return(struct symbol *func, struct instruction *insn, struct
  * tools/objtool/Documentation/stack-validation.txt.
  */
 static int validate_branch(struct objtool_file *file, struct symbol *func,
-			   struct instruction *first, struct insn_state state)
+			   struct instruction *insn, struct insn_state state)
 {
 	struct alternative *alt;
-	struct instruction *insn, *next_insn;
+	struct instruction *next_insn;
 	struct section *sec;
 	u8 visited;
 	int ret;
 
-	insn = first;
 	sec = insn->sec;
 
 	if (insn->alt_group && list_empty(&insn->alts)) {
@@ -2142,16 +2141,6 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
 				}
 
 				if (!save_insn->visited) {
-					/*
-					 * Oops, no state to copy yet.
-					 * Hopefully we can reach this
-					 * instruction from another branch
-					 * after the save insn has been
-					 * visited.
-					 */
-					if (insn == first)
-						return 0;
-
 					WARN_FUNC("objtool isn't smart enough to handle this CFI save/restore combo",
 						  sec, insn->offset);
 					return 1;
@@ -2244,6 +2233,20 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
 
 			break;
 
+		case INSN_EXCEPTION_RETURN:
+			if (handle_insn_ops(insn, &state))
+				return 1;
+
+			/*
+			 * This handles x86's sync_core() case, where we use an
+			 * IRET to self. All 'normal' IRET instructions are in
+			 * STT_NOTYPE entry symbols.
+			 */
+			if (func)
+				break;
+
+			return 0;
+
 		case INSN_CONTEXT_SWITCH:
 			if (func && (!next_insn || !next_insn->hint)) {
 				WARN_FUNC("unsupported instruction in callable function",
merge bot Applied
#11 lore raw

[tip: objtool/core] objtool: Better handle IRET

In reply to Peter Zijlstra
The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     b746046238bb99b8f703c79f6d95357428fb6476
Gitweb:        https://git.kernel.org/tip/b746046238bb99b8f703c79f6d95357428fb6476
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Thu, 02 Apr 2020 10:15:51 +02:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Wed, 22 Apr 2020 10:53:50 +02:00

objtool: Better handle IRET

Teach objtool a little more about IRET so that we can avoid using the SAVE/RESTORE annotation. In particular, make the weird corner case in insn->restore go away.

The purpose of that corner case is to deal with the fact that UNWIND_HINT_RESTORE lands on the instruction after IRET, but that instruction can end up being outside the basic block, consider:

	if (cond)
		sync_core()
	foo();

Then the hint will land on foo(), and we'll encounter the restore hint without ever having seen the save hint.

By teaching objtool about the arch specific exception frame size, and assuming that any IRET in an STT_FUNC symbol is an exception frame sized POP, we can remove the use of save/restore hints for this code.

---
 arch/x86/include/asm/processor.h |  2 --
 tools/objtool/arch.h             |  1 +
 tools/objtool/arch/x86/decode.c  | 14 ++++++++++++--
 tools/objtool/check.c            | 29 ++++++++++++++++-------------
 4 files changed, 29 insertions(+), 17 deletions(-)
arch/x86/include/asm/processor.h +0 -2
index 3bcf27c..3eeaaeb 100644
@@ -727,7 +727,6 @@ static inline void sync_core(void)
 	unsigned int tmp;
 
 	asm volatile (
-		UNWIND_HINT_SAVE
 		"mov %%ss, %0\n\t"
 		"pushq %q0\n\t"
 		"pushq %%rsp\n\t"
@@ -737,7 +736,6 @@ static inline void sync_core(void)
 		"pushq %q0\n\t"
 		"pushq $1f\n\t"
 		"iretq\n\t"
-		UNWIND_HINT_RESTORE
 		"1:"
 		: "=&r" (tmp), ASM_CALL_CONSTRAINT : : "cc", "memory");
 #endif
tools/objtool/arch.h +1 -0
index f9883c4..55396df 100644
@@ -19,6 +19,7 @@ enum insn_type {
 	INSN_CALL,
 	INSN_CALL_DYNAMIC,
 	INSN_RETURN,
+	INSN_EXCEPTION_RETURN,
 	INSN_CONTEXT_SWITCH,
 	INSN_STACK,
 	INSN_BUG,
tools/objtool/arch/x86/decode.c +12 -2
index 199b408..3273638 100644
@@ -446,9 +446,19 @@ int arch_decode_instruction(struct elf *elf, struct section *sec,
 		*type = INSN_RETURN;
 		break;
 
+	case 0xcf: /* iret */
+		*type = INSN_EXCEPTION_RETURN;
+
+		/* add $40, %rsp */
+		op->src.type = OP_SRC_ADD;
+		op->src.reg = CFI_SP;
+		op->src.offset = 5*8;
+		op->dest.type = OP_DEST_REG;
+		op->dest.reg = CFI_SP;
+		break;
+
 	case 0xca: /* retf */
 	case 0xcb: /* retf */
-	case 0xcf: /* iret */
 		*type = INSN_CONTEXT_SWITCH;
 		break;
 
@@ -494,7 +504,7 @@ int arch_decode_instruction(struct elf *elf, struct section *sec,
 
 	*immediate = insn.immediate.nbytes ? insn.immediate.value : 0;
 
-	if (*type == INSN_STACK)
+	if (*type == INSN_STACK || *type == INSN_EXCEPTION_RETURN)
 		list_add_tail(&op->list, ops_list);
 	else
 		free(op);
tools/objtool/check.c +16 -13
index 9e854fd..781b3a3 100644
@@ -2065,15 +2065,14 @@ static int validate_return(struct symbol *func, struct instruction *insn, struct
  * tools/objtool/Documentation/stack-validation.txt.
  */
 static int validate_branch(struct objtool_file *file, struct symbol *func,
-			   struct instruction *first, struct insn_state state)
+			   struct instruction *insn, struct insn_state state)
 {
 	struct alternative *alt;
-	struct instruction *insn, *next_insn;
+	struct instruction *next_insn;
 	struct section *sec;
 	u8 visited;
 	int ret;
 
-	insn = first;
 	sec = insn->sec;
 
 	if (insn->alt_group && list_empty(&insn->alts)) {
@@ -2126,16 +2125,6 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
 				}
 
 				if (!save_insn->visited) {
-					/*
-					 * Oops, no state to copy yet.
-					 * Hopefully we can reach this
-					 * instruction from another branch
-					 * after the save insn has been
-					 * visited.
-					 */
-					if (insn == first)
-						return 0;
-
 					WARN_FUNC("objtool isn't smart enough to handle this CFI save/restore combo",
 						  sec, insn->offset);
 					return 1;
@@ -2228,6 +2217,20 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
 
 			break;
 
+		case INSN_EXCEPTION_RETURN:
+			if (handle_insn_ops(insn, &state))
+				return 1;
+
+			/*
+			 * This handles x86's sync_core() case, where we use an
+			 * IRET to self. All 'normal' IRET instructions are in
+			 * STT_NOTYPE entry symbols.
+			 */
+			if (func)
+				break;
+
+			return 0;
+
 		case INSN_CONTEXT_SWITCH:
 			if (func && (!next_insn || !next_insn->hint)) {
 				WARN_FUNC("unsupported instruction in callable function",
author maintainer
#12 lore raw

PATCH v5 3/17 objtool: Introduce HINT_RET_OFFSET

In reply to Peter Zijlstra

Normally objtool ensures a function keeps the stack layout invariant. But there is a useful exception, it is possible to stuff the return stack in order to 'inject' a 'call':

push $fun
ret

In this case the invariant mentioned above is violated.

Add an objtool HINT to annotate this and allow a function exit with a modified stack frame.

---
 arch/x86/include/asm/orc_types.h       |    1 +
 arch/x86/include/asm/unwind_hints.h    |   10 ++++++++++
 tools/arch/x86/include/asm/orc_types.h |    1 +
 tools/objtool/check.c                  |   24 ++++++++++++++++--------
 tools/objtool/check.h                  |    4 +++-
 5 files changed, 31 insertions(+), 9 deletions(-)
arch/x86/include/asm/orc_types.h +1 -0
@@ -60,6 +60,7 @@
 #define ORC_TYPE_REGS_IRET		2
 #define UNWIND_HINT_TYPE_SAVE		3
 #define UNWIND_HINT_TYPE_RESTORE	4
+#define UNWIND_HINT_TYPE_RET_OFFSET	5
 
 #ifndef __ASSEMBLY__
 /*
arch/x86/include/asm/unwind_hints.h +10 -0
@@ -94,6 +94,16 @@
 	UNWIND_HINT type=UNWIND_HINT_TYPE_RESTORE
 .endm
 
+
+/*
+ * RET_OFFSET: Used on instructions that terminate a function; mostly RETURN
+ * and sibling calls. On these, sp_offset denotes the expected offset from
+ * initial_func_cfi.
+ */
+.macro UNWIND_HINT_RET_OFFSET sp_offset=8
+	UNWIND_HINT type=UNWIND_HINT_TYPE_RET_OFFSET sp_offset=\sp_offset
+.endm
+
 #else /* !__ASSEMBLY__ */
 
 #define UNWIND_HINT(sp_reg, sp_offset, type, end)		\
tools/arch/x86/include/asm/orc_types.h +1 -0
@@ -60,6 +60,7 @@
 #define ORC_TYPE_REGS_IRET		2
 #define UNWIND_HINT_TYPE_SAVE		3
 #define UNWIND_HINT_TYPE_RESTORE	4
+#define UNWIND_HINT_TYPE_RET_OFFSET	5
 
 #ifndef __ASSEMBLY__
 /*
tools/objtool/check.c +16 -8
@@ -1253,6 +1253,9 @@ static int read_unwind_hints(struct objt
 		} else if (hint->type == UNWIND_HINT_TYPE_RESTORE) {
 			insn->restore = true;
 			insn->hint = true;
+
+		} else if (hint->type == UNWIND_HINT_TYPE_RET_OFFSET) {
+			insn->ret_offset = hint->sp_offset;
 			continue;
 		}
 
@@ -1416,20 +1419,25 @@ static bool is_fentry_call(struct instru
 	return false;
 }
 
-static bool has_modified_stack_frame(struct insn_state *state)
+static bool has_modified_stack_frame(struct instruction *insn, struct insn_state *state)
 {
+	u8 ret_offset = insn->ret_offset;
 	int i;
 
-	if (state->cfa.base != initial_func_cfi.cfa.base ||
-	    state->cfa.offset != initial_func_cfi.cfa.offset ||
-	    state->stack_size != initial_func_cfi.cfa.offset ||
-	    state->drap)
+	if (state->cfa.base != initial_func_cfi.cfa.base || state->drap)
+		return true;
+
+	if (state->cfa.offset != initial_func_cfi.cfa.offset + ret_offset)
 		return true;
 
-	for (i = 0; i < CFI_NUM_REGS; i++)
+	if (state->stack_size != initial_func_cfi.cfa.offset + ret_offset)
+		return true;
+
+	for (i = 0; i < CFI_NUM_REGS; i++) {
 		if (state->regs[i].base != initial_func_cfi.regs[i].base ||
 		    state->regs[i].offset != initial_func_cfi.regs[i].offset)
 			return true;
+	}
 
 	return false;
 }
@@ -1971,7 +1979,7 @@ static int validate_call(struct instruct
 
 static int validate_sibling_call(struct instruction *insn, struct insn_state *state)
 {
-	if (has_modified_stack_frame(state)) {
+	if (has_modified_stack_frame(insn, state)) {
 		WARN_FUNC("sibling call from callable instruction with modified stack frame",
 				insn->sec, insn->offset);
 		return 1;
@@ -2000,7 +2008,7 @@ static int validate_return(struct symbol
 		return 1;
 	}
 
-	if (func && has_modified_stack_frame(state)) {
+	if (func && has_modified_stack_frame(insn, state)) {
 		WARN_FUNC("return with modified stack frame",
 			  insn->sec, insn->offset);
 		return 1;
tools/objtool/check.h +3 -1
@@ -33,9 +33,11 @@ struct instruction {
 	unsigned int len;
 	enum insn_type type;
 	unsigned long immediate;
-	bool alt_group, dead_end, ignore, hint, save, restore, ignore_alts;
+	bool alt_group, dead_end, ignore, ignore_alts;
+	bool hint, save, restore;
 	bool retpoline_safe;
 	u8 visited;
+	u8 ret_offset;
 	struct symbol *call_dest;
 	struct instruction *jump_dest;
 	struct instruction *first_jump_src;
merge bot Applied
#13 lore raw

[tip: objtool/core] objtool: Introduce HINT_RET_OFFSET

The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     9949d38e5ee16c7ff20247b6b36cc3538c84195d
Gitweb:        https://git.kernel.org/tip/9949d38e5ee16c7ff20247b6b36cc3538c84195d
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Wed, 01 Apr 2020 16:38:19 +02:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Wed, 22 Apr 2020 23:10:05 +02:00

objtool: Introduce HINT_RET_OFFSET

Normally objtool ensures a function keeps the stack layout invariant. But there is a useful exception, it is possible to stuff the return stack in order to 'inject' a 'call':

push $fun
ret

In this case the invariant mentioned above is violated.

Add an objtool HINT to annotate this and allow a function exit with a modified stack frame.

---
 arch/x86/include/asm/orc_types.h       |  1 +
 arch/x86/include/asm/unwind_hints.h    | 10 ++++++++++
 tools/arch/x86/include/asm/orc_types.h |  1 +
 tools/objtool/check.c                  | 24 ++++++++++++++++--------
 tools/objtool/check.h                  |  4 +++-
 5 files changed, 31 insertions(+), 9 deletions(-)
arch/x86/include/asm/orc_types.h +1 -0
index 6e06090..5f18ca7 100644
@@ -60,6 +60,7 @@
 #define ORC_TYPE_REGS_IRET		2
 #define UNWIND_HINT_TYPE_SAVE		3
 #define UNWIND_HINT_TYPE_RESTORE	4
+#define UNWIND_HINT_TYPE_RET_OFFSET	5
 
 #ifndef __ASSEMBLY__
 /*
arch/x86/include/asm/unwind_hints.h +10 -0
index f5e2eb1..aabf7ac 100644
@@ -94,6 +94,16 @@
 	UNWIND_HINT type=UNWIND_HINT_TYPE_RESTORE
 .endm
 
+
+/*
+ * RET_OFFSET: Used on instructions that terminate a function; mostly RETURN
+ * and sibling calls. On these, sp_offset denotes the expected offset from
+ * initial_func_cfi.
+ */
+.macro UNWIND_HINT_RET_OFFSET sp_offset=8
+	UNWIND_HINT type=UNWIND_HINT_TYPE_RET_OFFSET sp_offset=\sp_offset
+.endm
+
 #else /* !__ASSEMBLY__ */
 
 #define UNWIND_HINT(sp_reg, sp_offset, type, end)		\
tools/arch/x86/include/asm/orc_types.h +1 -0
index 6e06090..5f18ca7 100644
@@ -60,6 +60,7 @@
 #define ORC_TYPE_REGS_IRET		2
 #define UNWIND_HINT_TYPE_SAVE		3
 #define UNWIND_HINT_TYPE_RESTORE	4
+#define UNWIND_HINT_TYPE_RET_OFFSET	5
 
 #ifndef __ASSEMBLY__
 /*
tools/objtool/check.c +16 -8
index 72bf5cc..fe6ae45 100644
@@ -1277,6 +1277,9 @@ static int read_unwind_hints(struct objtool_file *file)
 		} else if (hint->type == UNWIND_HINT_TYPE_RESTORE) {
 			insn->restore = true;
 			insn->hint = true;
+
+		} else if (hint->type == UNWIND_HINT_TYPE_RET_OFFSET) {
+			insn->ret_offset = hint->sp_offset;
 			continue;
 		}
 
@@ -1440,20 +1443,25 @@ static bool is_fentry_call(struct instruction *insn)
 	return false;
 }
 
-static bool has_modified_stack_frame(struct insn_state *state)
+static bool has_modified_stack_frame(struct instruction *insn, struct insn_state *state)
 {
+	u8 ret_offset = insn->ret_offset;
 	int i;
 
-	if (state->cfa.base != initial_func_cfi.cfa.base ||
-	    state->cfa.offset != initial_func_cfi.cfa.offset ||
-	    state->stack_size != initial_func_cfi.cfa.offset ||
-	    state->drap)
+	if (state->cfa.base != initial_func_cfi.cfa.base || state->drap)
+		return true;
+
+	if (state->cfa.offset != initial_func_cfi.cfa.offset + ret_offset)
 		return true;
 
-	for (i = 0; i < CFI_NUM_REGS; i++)
+	if (state->stack_size != initial_func_cfi.cfa.offset + ret_offset)
+		return true;
+
+	for (i = 0; i < CFI_NUM_REGS; i++) {
 		if (state->regs[i].base != initial_func_cfi.regs[i].base ||
 		    state->regs[i].offset != initial_func_cfi.regs[i].offset)
 			return true;
+	}
 
 	return false;
 }
@@ -2030,7 +2038,7 @@ static int validate_call(struct instruction *insn, struct insn_state *state)
 
 static int validate_sibling_call(struct instruction *insn, struct insn_state *state)
 {
-	if (has_modified_stack_frame(state)) {
+	if (has_modified_stack_frame(insn, state)) {
 		WARN_FUNC("sibling call from callable instruction with modified stack frame",
 				insn->sec, insn->offset);
 		return 1;
@@ -2059,7 +2067,7 @@ static int validate_return(struct symbol *func, struct instruction *insn, struct
 		return 1;
 	}
 
-	if (func && has_modified_stack_frame(state)) {
+	if (func && has_modified_stack_frame(insn, state)) {
 		WARN_FUNC("return with modified stack frame",
 			  insn->sec, insn->offset);
 		return 1;
tools/objtool/check.h +3 -1
index 2c55f75..81ce27e 100644
@@ -33,9 +33,11 @@ struct instruction {
 	unsigned int len;
 	enum insn_type type;
 	unsigned long immediate;
-	bool alt_group, dead_end, ignore, hint, save, restore, ignore_alts;
+	bool alt_group, dead_end, ignore, ignore_alts;
+	bool hint, save, restore;
 	bool retpoline_safe;
 	u8 visited;
+	u8 ret_offset;
 	struct symbol *call_dest;
 	struct instruction *jump_dest;
 	struct instruction *first_jump_src;
merge bot Applied
#14 lore raw

[tip: objtool/core] objtool: Introduce HINT_RET_OFFSET

In reply to Peter Zijlstra
The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     e25eea89bb8853763a22fa2547199cf96b571ba1
Gitweb:        https://git.kernel.org/tip/e25eea89bb8853763a22fa2547199cf96b571ba1
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Wed, 01 Apr 2020 16:38:19 +02:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Wed, 22 Apr 2020 10:53:50 +02:00

objtool: Introduce HINT_RET_OFFSET

Normally objtool ensures a function keeps the stack layout invariant. But there is a useful exception, it is possible to stuff the return stack in order to 'inject' a 'call':

push $fun
ret

In this case the invariant mentioned above is violated.

Add an objtool HINT to annotate this and allow a function exit with a modified stack frame.

---
 arch/x86/include/asm/orc_types.h       |  1 +
 arch/x86/include/asm/unwind_hints.h    | 10 ++++++++++
 tools/arch/x86/include/asm/orc_types.h |  1 +
 tools/objtool/check.c                  | 24 ++++++++++++++++--------
 tools/objtool/check.h                  |  4 +++-
 5 files changed, 31 insertions(+), 9 deletions(-)
arch/x86/include/asm/orc_types.h +1 -0
index 6e06090..5f18ca7 100644
@@ -60,6 +60,7 @@
 #define ORC_TYPE_REGS_IRET		2
 #define UNWIND_HINT_TYPE_SAVE		3
 #define UNWIND_HINT_TYPE_RESTORE	4
+#define UNWIND_HINT_TYPE_RET_OFFSET	5
 
 #ifndef __ASSEMBLY__
 /*
arch/x86/include/asm/unwind_hints.h +10 -0
index f5e2eb1..aabf7ac 100644
@@ -94,6 +94,16 @@
 	UNWIND_HINT type=UNWIND_HINT_TYPE_RESTORE
 .endm
 
+
+/*
+ * RET_OFFSET: Used on instructions that terminate a function; mostly RETURN
+ * and sibling calls. On these, sp_offset denotes the expected offset from
+ * initial_func_cfi.
+ */
+.macro UNWIND_HINT_RET_OFFSET sp_offset=8
+	UNWIND_HINT type=UNWIND_HINT_TYPE_RET_OFFSET sp_offset=\sp_offset
+.endm
+
 #else /* !__ASSEMBLY__ */
 
 #define UNWIND_HINT(sp_reg, sp_offset, type, end)		\
tools/arch/x86/include/asm/orc_types.h +1 -0
index 6e06090..5f18ca7 100644
@@ -60,6 +60,7 @@
 #define ORC_TYPE_REGS_IRET		2
 #define UNWIND_HINT_TYPE_SAVE		3
 #define UNWIND_HINT_TYPE_RESTORE	4
+#define UNWIND_HINT_TYPE_RET_OFFSET	5
 
 #ifndef __ASSEMBLY__
 /*
tools/objtool/check.c +16 -8
index 781b3a3..93c88ac 100644
@@ -1261,6 +1261,9 @@ static int read_unwind_hints(struct objtool_file *file)
 		} else if (hint->type == UNWIND_HINT_TYPE_RESTORE) {
 			insn->restore = true;
 			insn->hint = true;
+
+		} else if (hint->type == UNWIND_HINT_TYPE_RET_OFFSET) {
+			insn->ret_offset = hint->sp_offset;
 			continue;
 		}
 
@@ -1424,20 +1427,25 @@ static bool is_fentry_call(struct instruction *insn)
 	return false;
 }
 
-static bool has_modified_stack_frame(struct insn_state *state)
+static bool has_modified_stack_frame(struct instruction *insn, struct insn_state *state)
 {
+	u8 ret_offset = insn->ret_offset;
 	int i;
 
-	if (state->cfa.base != initial_func_cfi.cfa.base ||
-	    state->cfa.offset != initial_func_cfi.cfa.offset ||
-	    state->stack_size != initial_func_cfi.cfa.offset ||
-	    state->drap)
+	if (state->cfa.base != initial_func_cfi.cfa.base || state->drap)
+		return true;
+
+	if (state->cfa.offset != initial_func_cfi.cfa.offset + ret_offset)
 		return true;
 
-	for (i = 0; i < CFI_NUM_REGS; i++)
+	if (state->stack_size != initial_func_cfi.cfa.offset + ret_offset)
+		return true;
+
+	for (i = 0; i < CFI_NUM_REGS; i++) {
 		if (state->regs[i].base != initial_func_cfi.regs[i].base ||
 		    state->regs[i].offset != initial_func_cfi.regs[i].offset)
 			return true;
+	}
 
 	return false;
 }
@@ -2014,7 +2022,7 @@ static int validate_call(struct instruction *insn, struct insn_state *state)
 
 static int validate_sibling_call(struct instruction *insn, struct insn_state *state)
 {
-	if (has_modified_stack_frame(state)) {
+	if (has_modified_stack_frame(insn, state)) {
 		WARN_FUNC("sibling call from callable instruction with modified stack frame",
 				insn->sec, insn->offset);
 		return 1;
@@ -2043,7 +2051,7 @@ static int validate_return(struct symbol *func, struct instruction *insn, struct
 		return 1;
 	}
 
-	if (func && has_modified_stack_frame(state)) {
+	if (func && has_modified_stack_frame(insn, state)) {
 		WARN_FUNC("return with modified stack frame",
 			  insn->sec, insn->offset);
 		return 1;
tools/objtool/check.h +3 -1
index 2c55f75..81ce27e 100644
@@ -33,9 +33,11 @@ struct instruction {
 	unsigned int len;
 	enum insn_type type;
 	unsigned long immediate;
-	bool alt_group, dead_end, ignore, hint, save, restore, ignore_alts;
+	bool alt_group, dead_end, ignore, ignore_alts;
+	bool hint, save, restore;
 	bool retpoline_safe;
 	u8 visited;
+	u8 ret_offset;
 	struct symbol *call_dest;
 	struct instruction *jump_dest;
 	struct instruction *first_jump_src;
author maintainer
#15 lore raw

PATCH v5 4/17 x86,ftrace: Fix ftrace_regs_caller() unwind

In reply to Peter Zijlstra

The ftrace_regs_caller() trampoline does something 'funny' when there is a direct-caller present. In that case it stuffs the 'direct-caller' address on the return stack and then exits the function. This then results in 'returning' to the direct-caller with the exact registers we came in with -- an indirect tail-call without using a register.

This however (rightfully) confuses objtool because the function shares a few instruction in order to have a single exit path, but the stack layout is different for them, depending through which path we came there.

This is currently cludged by forcing the stack state to the non-direct case, but this generates actively wrong (ORC) unwind information for the direct case, leading to potential broken unwinds.

Fix this issue by fully separating the exit paths. This results in having to poke a second RET into the trampoline copy, see ftrace_regs_caller_ret.

This brings us to a second objtool problem, in order for it to perceive the 'jmp ftrace_epilogue' as a function exit, it needs to be recognised as a tail call. In order to make that happen, ftrace_epilogue needs to be the start of an STT_FUNC, so re-arrange code to make this so.

Finally, a third issue is that objtool requires functions to exit with the same stack layout they started with, which is obviously violated in the direct case, employ the new HINT_RET_OFFSET to tell objtool this is an expected exception.

Together, this results in generating correct ORC unwind information for the ftrace_regs_caller() function and it's trampoline copies.

---
 arch/x86/kernel/ftrace.c    |   12 ++++++++++--
 arch/x86/kernel/ftrace_64.S |   32 +++++++++++++++-----------------
 2 files changed, 25 insertions(+), 19 deletions(-)
arch/x86/kernel/ftrace.c +10 -2
@@ -282,7 +282,8 @@ static inline void tramp_free(void *tram
 
 /* Defined as markers to the end of the ftrace default trampolines */
 extern void ftrace_regs_caller_end(void);
-extern void ftrace_epilogue(void);
+extern void ftrace_regs_caller_ret(void);
+extern void ftrace_caller_end(void);
 extern void ftrace_caller_op_ptr(void);
 extern void ftrace_regs_caller_op_ptr(void);
 
@@ -334,7 +335,7 @@ create_trampoline(struct ftrace_ops *ops
 		call_offset = (unsigned long)ftrace_regs_call;
 	} else {
 		start_offset = (unsigned long)ftrace_caller;
-		end_offset = (unsigned long)ftrace_epilogue;
+		end_offset = (unsigned long)ftrace_caller_end;
 		op_offset = (unsigned long)ftrace_caller_op_ptr;
 		call_offset = (unsigned long)ftrace_call;
 	}
@@ -366,6 +367,13 @@ create_trampoline(struct ftrace_ops *ops
 	if (WARN_ON(ret < 0))
 		goto fail;
 
+	if (ops->flags & FTRACE_OPS_FL_SAVE_REGS) {
+		ip = trampoline + (ftrace_regs_caller_ret - ftrace_regs_caller);
+		ret = probe_kernel_read(ip, (void *)retq, RET_SIZE);
+		if (WARN_ON(ret < 0))
+			goto fail;
+	}
+
 	/*
 	 * The address of the ftrace_ops that is used for this trampoline
 	 * is stored at the end of the trampoline. This will be used to
arch/x86/kernel/ftrace_64.S +15 -17
@@ -157,8 +157,12 @@ SYM_INNER_LABEL(ftrace_call, SYM_L_GLOBA
 	 * think twice before adding any new code or changing the
 	 * layout here.
 	 */
-SYM_INNER_LABEL(ftrace_epilogue, SYM_L_GLOBAL)
+SYM_INNER_LABEL(ftrace_caller_end, SYM_L_GLOBAL)
 
+	jmp ftrace_epilogue
+SYM_FUNC_END(ftrace_caller);
+
+SYM_FUNC_START(ftrace_epilogue)
 #ifdef CONFIG_FUNCTION_GRAPH_TRACER
 SYM_INNER_LABEL(ftrace_graph_call, SYM_L_GLOBAL)
 	jmp ftrace_stub
@@ -170,14 +174,12 @@ SYM_INNER_LABEL(ftrace_graph_call, SYM_L
  */
 SYM_INNER_LABEL_ALIGN(ftrace_stub, SYM_L_WEAK)
 	retq
-SYM_FUNC_END(ftrace_caller)
+SYM_FUNC_END(ftrace_epilogue)
 
 SYM_FUNC_START(ftrace_regs_caller)
 	/* Save the current flags before any operations that can change them */
 	pushfq
 
-	UNWIND_HINT_SAVE
-
 	/* added 8 bytes to save flags */
 	save_mcount_regs 8
 	/* save_mcount_regs fills in first two parameters */
@@ -233,7 +235,10 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_
 	movq ORIG_RAX(%rsp), %rax
 	movq %rax, MCOUNT_REG_SIZE-8(%rsp)
 
-	/* If ORIG_RAX is anything but zero, make this a call to that */
+	/*
+	 * If ORIG_RAX is anything but zero, make this a call to that.
+	 * See arch_ftrace_set_direct_caller().
+	 */
 	movq ORIG_RAX(%rsp), %rax
 	cmpq	$0, %rax
 	je	1f
@@ -244,20 +249,14 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_
 	movq %rax, MCOUNT_REG_SIZE(%rsp)
 
 	restore_mcount_regs 8
+	/* Restore flags */
+	popfq
 
-	jmp	2f
+SYM_INNER_LABEL(ftrace_regs_caller_ret, SYM_L_GLOBAL);
+	UNWIND_HINT_RET_OFFSET
+	jmp	ftrace_epilogue
 
 1:	restore_mcount_regs
-
-
-2:
-	/*
-	 * The stack layout is nondetermistic here, depending on which path was
-	 * taken.  This confuses objtool and ORC, rightfully so.  For now,
-	 * pretend the stack always looks like the non-direct case.
-	 */
-	UNWIND_HINT_RESTORE
-
 	/* Restore flags */
 	popfq
 
@@ -268,7 +267,6 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_
 	 * to the return.
 	 */
 SYM_INNER_LABEL(ftrace_regs_caller_end, SYM_L_GLOBAL)
-
 	jmp ftrace_epilogue
 
 SYM_FUNC_END(ftrace_regs_caller)
#16 lore raw

Re: [PATCH v5 04/17] x86,ftrace: Fix ftrace_regs_caller() unwind

Quoted 61 lines · Peter Zijlstra

The ftrace_regs_caller() trampoline does something 'funny' when there is a direct-caller present. In that case it stuffs the 'direct-caller' address on the return stack and then exits the function. This then results in 'returning' to the direct-caller with the exact registers we came in with -- an indirect tail-call without using a register.

This however (rightfully) confuses objtool because the function shares a few instruction in order to have a single exit path, but the stack layout is different for them, depending through which path we came there.

This is currently cludged by forcing the stack state to the non-direct case, but this generates actively wrong (ORC) unwind information for the direct case, leading to potential broken unwinds.

Fix this issue by fully separating the exit paths. This results in having to poke a second RET into the trampoline copy, see ftrace_regs_caller_ret.

This brings us to a second objtool problem, in order for it to perceive the 'jmp ftrace_epilogue' as a function exit, it needs to be recognised as a tail call. In order to make that happen, ftrace_epilogue needs to be the start of an STT_FUNC, so re-arrange code to make this so.

Finally, a third issue is that objtool requires functions to exit with the same stack layout they started with, which is obviously violated in the direct case, employ the new HINT_RET_OFFSET to tell objtool this is an expected exception.

Together, this results in generating correct ORC unwind information for the ftrace_regs_caller() function and it's trampoline copies.

---
  arch/x86/kernel/ftrace.c    |   12 ++++++++++--
  arch/x86/kernel/ftrace_64.S |   32 +++++++++++++++-----------------
  2 files changed, 25 insertions(+), 19 deletions(-)
arch/x86/kernel/ftrace.c +3 -2
@@ -282,7 +282,8 @@ static inline void tramp_free(void *tram
 
  /* Defined as markers to the end of the ftrace default trampolines */
  extern void ftrace_regs_caller_end(void);
-extern void ftrace_epilogue(void);
+extern void ftrace_regs_caller_ret(void);
+extern void ftrace_caller_end(void);
  extern void ftrace_caller_op_ptr(void);
  extern void ftrace_regs_caller_op_ptr(void);
 
@@ -334,7 +335,7 @@ create_trampoline(struct ftrace_ops *ops
  		call_offset = (unsigned long)ftrace_regs_call;
  	} else {
  		start_offset = (unsigned long)ftrace_caller;
-		end_offset = (unsigned long)ftrace_epilogue;
+		end_offset = (unsigned long)ftrace_caller_end;
  		op_offset = (unsigned long)ftrace_caller_op_ptr;
  		call_offset = (unsigned long)ftrace_call;
  	}

arch/x86/kernel/ftrace.c create_trampoline(struct ftrace_ops *ops · line 371

Quoted 5 lines
arch/x86/kernel/ftrace.c +2 -0
@@ -366,6 +367,13 @@ create_trampoline(struct ftrace_ops *ops
  	if (WARN_ON(ret < 0))
  		goto fail;
 
+	if (ops->flags & FTRACE_OPS_FL_SAVE_REGS) {
+		ip = trampoline + (ftrace_regs_caller_ret - ftrace_regs_caller);

It might be safer to use start_offset instead of ftrace_regs_caller (in case start_offset is ever changed to something different from ftrace_regs_caller in the future):

         ip = trampoline + (ftrace_regs_caller_ret - start_offset);

alex.

Quoted 88 lines
arch/x86/kernel/ftrace.c +5 -0
+		ret = probe_kernel_read(ip, (void *)retq, RET_SIZE);
+		if (WARN_ON(ret < 0))
+			goto fail;
+	}
+
  	/*
  	 * The address of the ftrace_ops that is used for this trampoline
  	 * is stored at the end of the trampoline. This will be used to
arch/x86/kernel/ftrace_64.S +15 -17
@@ -157,8 +157,12 @@ SYM_INNER_LABEL(ftrace_call, SYM_L_GLOBA
  	 * think twice before adding any new code or changing the
  	 * layout here.
  	 */
-SYM_INNER_LABEL(ftrace_epilogue, SYM_L_GLOBAL)
+SYM_INNER_LABEL(ftrace_caller_end, SYM_L_GLOBAL)
 
+	jmp ftrace_epilogue
+SYM_FUNC_END(ftrace_caller);
+
+SYM_FUNC_START(ftrace_epilogue)
  #ifdef CONFIG_FUNCTION_GRAPH_TRACER
  SYM_INNER_LABEL(ftrace_graph_call, SYM_L_GLOBAL)
  	jmp ftrace_stub
@@ -170,14 +174,12 @@ SYM_INNER_LABEL(ftrace_graph_call, SYM_L
   */
  SYM_INNER_LABEL_ALIGN(ftrace_stub, SYM_L_WEAK)
  	retq
-SYM_FUNC_END(ftrace_caller)
+SYM_FUNC_END(ftrace_epilogue)
 
  SYM_FUNC_START(ftrace_regs_caller)
  	/* Save the current flags before any operations that can change them */
  	pushfq
 
-	UNWIND_HINT_SAVE
-
  	/* added 8 bytes to save flags */
  	save_mcount_regs 8
  	/* save_mcount_regs fills in first two parameters */
@@ -233,7 +235,10 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_
  	movq ORIG_RAX(%rsp), %rax
  	movq %rax, MCOUNT_REG_SIZE-8(%rsp)
 
-	/* If ORIG_RAX is anything but zero, make this a call to that */
+	/*
+	 * If ORIG_RAX is anything but zero, make this a call to that.
+	 * See arch_ftrace_set_direct_caller().
+	 */
  	movq ORIG_RAX(%rsp), %rax
  	cmpq	$0, %rax
  	je	1f
@@ -244,20 +249,14 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_
  	movq %rax, MCOUNT_REG_SIZE(%rsp)
 
  	restore_mcount_regs 8
+	/* Restore flags */
+	popfq
 
-	jmp	2f
+SYM_INNER_LABEL(ftrace_regs_caller_ret, SYM_L_GLOBAL);
+	UNWIND_HINT_RET_OFFSET
+	jmp	ftrace_epilogue
 
  1:	restore_mcount_regs
-
-
-2:
-	/*
-	 * The stack layout is nondetermistic here, depending on which path was
-	 * taken.  This confuses objtool and ORC, rightfully so.  For now,
-	 * pretend the stack always looks like the non-direct case.
-	 */
-	UNWIND_HINT_RESTORE
-
  	/* Restore flags */
  	popfq
 
@@ -268,7 +267,6 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_
  	 * to the return.
  	 */
  SYM_INNER_LABEL(ftrace_regs_caller_end, SYM_L_GLOBAL)
-
  	jmp ftrace_epilogue
 
  SYM_FUNC_END(ftrace_regs_caller)
maintainer
#17 lore raw

Re: [PATCH v5 04/17] x86,ftrace: Fix ftrace_regs_caller() unwind

In reply to Peter Zijlstra

Hi Peter,

After looking at the code, I realized that the only possible way to do the "direct call" part, is if the direct function helper is executed there. All other ftrace_ops, will not call that path.

I added a trampoline that points to ftrace_regs_caller to the direct ftrace_ops, to force it never to allocate its own trampoline, and thus no created trampoline should ever do the jump for a direct caller.

By doing this, I was able to move the code around to be a bit simpler, and not need the double modifications (the double ret).

Of course, if any created trampoline were to touch the ORIG_RAX, then it would crash. We could always nop that compare in the created trampoline if that is a concern.

Anyway, I added the below patch on top of your patches and it appears to work. Thoughts?

-- Steve

arch/x86/kernel/ftrace.c +0 -7
index 867c126ddabe..2e11250d5647 100644
@@ -367,13 +367,6 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size)
 	if (WARN_ON(ret < 0))
 		goto fail;
 
-	if (ops->flags & FTRACE_OPS_FL_SAVE_REGS) {
-		ip = trampoline + (ftrace_regs_caller_ret - ftrace_regs_caller);
-		ret = probe_kernel_read(ip, (void *)retq, RET_SIZE);
-		if (WARN_ON(ret < 0))
-			goto fail;
-	}
-
 	/*
 	 * The address of the ftrace_ops that is used for this trampoline
 	 * is stored at the end of the trampoline. This will be used to
arch/x86/kernel/ftrace_64.S +13 -15
index 9738ed23964e..1f2afaa8f71f 100644
@@ -241,22 +241,9 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_GLOBAL)
 	 */
 	movq ORIG_RAX(%rsp), %rax
 	testq	%rax, %rax
-	jz	1f
+	jnz	1f
 
-	/* Swap the flags with orig_rax */
-	movq MCOUNT_REG_SIZE(%rsp), %rdi
-	movq %rdi, MCOUNT_REG_SIZE-8(%rsp)
-	movq %rax, MCOUNT_REG_SIZE(%rsp)
-
-	restore_mcount_regs 8
-	/* Restore flags */
-	popfq
-
-SYM_INNER_LABEL(ftrace_regs_caller_ret, SYM_L_GLOBAL);
-	UNWIND_HINT_RET_OFFSET
-	jmp	ftrace_epilogue
-
-1:	restore_mcount_regs
+	restore_mcount_regs
 	/* Restore flags */
 	popfq
 
@@ -267,8 +254,19 @@ SYM_INNER_LABEL(ftrace_regs_caller_ret, SYM_L_GLOBAL);
 	 * to the return.
 	 */
 SYM_INNER_LABEL(ftrace_regs_caller_end, SYM_L_GLOBAL)
+	UNWIND_HINT_RET_OFFSET
 	jmp ftrace_epilogue
 
+	/* Swap the flags with orig_rax */
+1:	movq MCOUNT_REG_SIZE(%rsp), %rdi
+	movq %rdi, MCOUNT_REG_SIZE-8(%rsp)
+	movq %rax, MCOUNT_REG_SIZE(%rsp)
+
+	restore_mcount_regs 8
+	/* Restore flags */
+	popfq
+	jmp	ftrace_epilogue
+
 SYM_FUNC_END(ftrace_regs_caller)
 
 
kernel/trace/ftrace.c +7 -0
index 041694a1eb74..8f540eef7511 100644
@@ -2399,6 +2399,13 @@ struct ftrace_ops direct_ops = {
 	.flags		= FTRACE_OPS_FL_IPMODIFY | FTRACE_OPS_FL_RECURSION_SAFE
 			  | FTRACE_OPS_FL_DIRECT | FTRACE_OPS_FL_SAVE_REGS
 			  | FTRACE_OPS_FL_PERMANENT,
+	/*
+	 * By declaring the main trampoline as this trampoline
+	 * it will never have one allocated for it. This is fine
+	 * as this should only be used if we are in the
+	 * ftrace_ops_list function.
+	 */
+	.trampoline	= FTRACE_REGS_ADDR,
 };
 #endif /* CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS */
 
author maintainer
#18 lore raw

Re: [PATCH v5 04/17] x86,ftrace: Fix ftrace_regs_caller() unwind

Quoted 13 lines · Steven Rostedt

Hi Peter,

After looking at the code, I realized that the only possible way to do the "direct call" part, is if the direct function helper is executed there. All other ftrace_ops, will not call that path.

I added a trampoline that points to ftrace_regs_caller to the direct ftrace_ops, to force it never to allocate its own trampoline, and thus no created trampoline should ever do the jump for a direct caller.

By doing this, I was able to move the code around to be a bit simpler, and not need the double modifications (the double ret).

Quoted 6 lines

Of course, if any created trampoline were to touch the ORIG_RAX, then it would crash. We could always nop that compare in the created trampoline if that is a concern.

Anyway, I added the below patch on top of your patches and it appears to work. Thoughts?

So can I push out those patches as is? :-) We can do this on top I suppose.

Firstly; your patch isn't objtool clean:

arch/x86/kernel/ftrace_64.o: warning: objtool: ftrace_regs_caller()+0x199: sibling call from callable instruction with modified stack frame

So 10 points deduction for that :-). You got the RET_OFFSET hint on the wrong sibling call.

Secondly, yes, that ORIG_RAX issue for copies, that _might_ crash and burn, but who knows, you're jumping into uninitialized memory afaict. So that definitely needs fixing. I'm also not sure of stubbing out the test, that's actually more work than poking in the 1 extra ret and also gives unexpected behaviour. If anything we should poke an UD2 at the 1:
location in the copy.

Over all, I'm thinking we should keep it as is. If you want to simplify the poking we can do something like the below; reading a known retq is daft.

---

arch/x86/kernel/ftrace.c +6 -15
index 867c126ddabe..b334adbacb0e 100644
@@ -307,8 +307,6 @@ union ftrace_op_code_union {
 	} __attribute__((packed));
 };
 
-#define RET_SIZE		1
-
 static unsigned long
 create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size)
 {
@@ -319,7 +317,6 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size)
 	unsigned long offset;
 	unsigned long npages;
 	unsigned long size;
-	unsigned long retq;
 	unsigned long *ptr;
 	void *trampoline;
 	void *ip;
@@ -347,11 +344,11 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size)
 	 * the iret , as well as the address of the ftrace_ops this
 	 * trampoline is used for.
 	 */
-	trampoline = alloc_tramp(size + RET_SIZE + sizeof(void *));
+	trampoline = alloc_tramp(size + RET_INSN_SIZE + sizeof(void *));
 	if (!trampoline)
 		return 0;
 
-	*tramp_size = size + RET_SIZE + sizeof(void *);
+	*tramp_size = size + RET_INSN_SIZE + sizeof(void *);
 	npages = DIV_ROUND_UP(*tramp_size, PAGE_SIZE);
 
 	/* Copy ftrace_caller onto the trampoline memory */
@@ -359,19 +356,13 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size)
 	if (WARN_ON(ret < 0))
 		goto fail;
 
-	ip = trampoline + size;
-
 	/* The trampoline ends with ret(q) */
-	retq = (unsigned long)ftrace_stub;
-	ret = probe_kernel_read(ip, (void *)retq, RET_SIZE);
-	if (WARN_ON(ret < 0))
-		goto fail;
+	ip = trampoline + size;
+	*(u8 *)ip = RET_INSN_OPCODE;
 
 	if (ops->flags & FTRACE_OPS_FL_SAVE_REGS) {
 		ip = trampoline + (ftrace_regs_caller_ret - ftrace_regs_caller);
-		ret = probe_kernel_read(ip, (void *)retq, RET_SIZE);
-		if (WARN_ON(ret < 0))
-			goto fail;
+		*(u8 *)ip = RET_INSN_OPCODE;
 	}
 
 	/*
@@ -382,7 +373,7 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size)
 	 * the global function_trace_op variable.
 	 */
 
-	ptr = (unsigned long *)(trampoline + size + RET_SIZE);
+	ptr = (unsigned long *)(trampoline + size + RET_INSN_SIZE);
 	*ptr = (unsigned long)ops;
 
 	op_offset -= start_offset;
maintainer
#19 lore raw

Re: [PATCH v5 04/17] x86,ftrace: Fix ftrace_regs_caller() unwind

Quoted 5 lines · On Wed, 22 Apr 2020 11:44:46 +0200 Peter Zijlstra <[email protected]> wrote:
Quoted 2 lines

Anyway, I added the below patch on top of your patches and it appears to work. Thoughts?

So can I push out those patches as is? :-) We can do this on top I suppose.

My tests are still running on your series. I want to make sure that nothing breaks between them. If the tests succeed, then sure, I'm OK building on top of this series.

Quoted 7 lines

Firstly; your patch isn't objtool clean:

arch/x86/kernel/ftrace_64.o: warning: objtool: ftrace_regs_caller()+0x199: sibling call from callable instruction with modified stack frame

So 10 points deduction for that :-). You got the RET_OFFSET hint on the wrong sibling call.

Ah, I just did his patch quickly and made sure that it booted and ran function tracing and direct calls. I didn't bother caring much about the objtool annotations.

Quoted 7 lines

Secondly, yes, that ORIG_RAX issue for copies, that _might_ crash and burn, but who knows, you're jumping into uninitialized memory afaict. So that definitely needs fixing. I'm also not sure of stubbing out the test, that's actually more work than poking in the 1 extra ret and also gives unexpected behaviour. If anything we should poke an UD2 at the 1:
location in the copy.

Well, that would crash the system just as well.

We could also just make the jnz into a nop (which would make the trampoline slightly faster as well).

Quoted 4 lines

Over all, I'm thinking we should keep it as is. If you want to simplify the poking we can do something like the below; reading a known retq is daft.

I'm actually more concerned about the performance of the created trampoline, as that is a fast path. I rather get rid of that jump.

I'll play a little more with it while the tests continue.

-- Steve

Quoted 69 lines

---

arch/x86/kernel/ftrace.c +6 -15
index 867c126ddabe..b334adbacb0e 100644
@@ -307,8 +307,6 @@ union ftrace_op_code_union {
 	} __attribute__((packed));
 };
 
-#define RET_SIZE		1
-
 static unsigned long
 create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size)
 {
@@ -319,7 +317,6 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size)
 	unsigned long offset;
 	unsigned long npages;
 	unsigned long size;
-	unsigned long retq;
 	unsigned long *ptr;
 	void *trampoline;
 	void *ip;
@@ -347,11 +344,11 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size)
 	 * the iret , as well as the address of the ftrace_ops this
 	 * trampoline is used for.
 	 */
-	trampoline = alloc_tramp(size + RET_SIZE + sizeof(void *));
+	trampoline = alloc_tramp(size + RET_INSN_SIZE + sizeof(void *));
 	if (!trampoline)
 		return 0;
 
-	*tramp_size = size + RET_SIZE + sizeof(void *);
+	*tramp_size = size + RET_INSN_SIZE + sizeof(void *);
 	npages = DIV_ROUND_UP(*tramp_size, PAGE_SIZE);
 
 	/* Copy ftrace_caller onto the trampoline memory */
@@ -359,19 +356,13 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size)
 	if (WARN_ON(ret < 0))
 		goto fail;
 
-	ip = trampoline + size;
-
 	/* The trampoline ends with ret(q) */
-	retq = (unsigned long)ftrace_stub;
-	ret = probe_kernel_read(ip, (void *)retq, RET_SIZE);
-	if (WARN_ON(ret < 0))
-		goto fail;
+	ip = trampoline + size;
+	*(u8 *)ip = RET_INSN_OPCODE;
 
 	if (ops->flags & FTRACE_OPS_FL_SAVE_REGS) {
 		ip = trampoline + (ftrace_regs_caller_ret - ftrace_regs_caller);
-		ret = probe_kernel_read(ip, (void *)retq, RET_SIZE);
-		if (WARN_ON(ret < 0))
-			goto fail;
+		*(u8 *)ip = RET_INSN_OPCODE;
 	}
 
 	/*
@@ -382,7 +373,7 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size)
 	 * the global function_trace_op variable.
 	 */
 
-	ptr = (unsigned long *)(trampoline + size + RET_SIZE);
+	ptr = (unsigned long *)(trampoline + size + RET_INSN_SIZE);
 	*ptr = (unsigned long)ops;
 
 	op_offset -= start_offset;
maintainer Reviewed-by
#20 lore raw

Re: [PATCH v5 04/17] x86,ftrace: Fix ftrace_regs_caller() unwind

In reply to Peter Zijlstra
Quoted 2 lines · On Wed, 22 Apr 2020 11:44:46 +0200 Peter Zijlstra <[email protected]> wrote:

So can I push out those patches as is? :-) We can do this on top I suppose.

My tests just finished. You can add my:

to the three patches that touch the ftrace code in this series.

I'll continue to tinker with my patches that modify that code as well.

-- Steve

merge bot Applied
#21 lore raw

[tip: objtool/core] x86,ftrace: Fix ftrace_regs_caller() unwind

In reply to Peter Zijlstra
The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     5d6cd2a33032b6283b43d9787807f21c1298fd6d
Gitweb:        https://git.kernel.org/tip/5d6cd2a33032b6283b43d9787807f21c1298fd6d
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Wed, 01 Apr 2020 16:53:19 +02:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Wed, 22 Apr 2020 23:10:05 +02:00

x86,ftrace: Fix ftrace_regs_caller() unwind

The ftrace_regs_caller() trampoline does something 'funny' when there is a direct-caller present. In that case it stuffs the 'direct-caller' address on the return stack and then exits the function. This then results in 'returning' to the direct-caller with the exact registers we came in with -- an indirect tail-call without using a register.

This however (rightfully) confuses objtool because the function shares a few instruction in order to have a single exit path, but the stack layout is different for them, depending through which path we came there.

This is currently cludged by forcing the stack state to the non-direct case, but this generates actively wrong (ORC) unwind information for the direct case, leading to potential broken unwinds.

Fix this issue by fully separating the exit paths. This results in having to poke a second RET into the trampoline copy, see ftrace_regs_caller_ret.

This brings us to a second objtool problem, in order for it to perceive the 'jmp ftrace_epilogue' as a function exit, it needs to be recognised as a tail call. In order to make that happen, ftrace_epilogue needs to be the start of an STT_FUNC, so re-arrange code to make this so.

Finally, a third issue is that objtool requires functions to exit with the same stack layout they started with, which is obviously violated in the direct case, employ the new HINT_RET_OFFSET to tell objtool this is an expected exception.

Together, this results in generating correct ORC unwind information for the ftrace_regs_caller() function and it's trampoline copies.

---
 arch/x86/kernel/ftrace.c    | 12 ++++++++++--
 arch/x86/kernel/ftrace_64.S | 32 +++++++++++++++-----------------
 2 files changed, 25 insertions(+), 19 deletions(-)
arch/x86/kernel/ftrace.c +10 -2
index 37a0aea..867c126 100644
@@ -282,7 +282,8 @@ static inline void tramp_free(void *tramp) { }
 
 /* Defined as markers to the end of the ftrace default trampolines */
 extern void ftrace_regs_caller_end(void);
-extern void ftrace_epilogue(void);
+extern void ftrace_regs_caller_ret(void);
+extern void ftrace_caller_end(void);
 extern void ftrace_caller_op_ptr(void);
 extern void ftrace_regs_caller_op_ptr(void);
 
@@ -334,7 +335,7 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size)
 		call_offset = (unsigned long)ftrace_regs_call;
 	} else {
 		start_offset = (unsigned long)ftrace_caller;
-		end_offset = (unsigned long)ftrace_epilogue;
+		end_offset = (unsigned long)ftrace_caller_end;
 		op_offset = (unsigned long)ftrace_caller_op_ptr;
 		call_offset = (unsigned long)ftrace_call;
 	}
@@ -366,6 +367,13 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size)
 	if (WARN_ON(ret < 0))
 		goto fail;
 
+	if (ops->flags & FTRACE_OPS_FL_SAVE_REGS) {
+		ip = trampoline + (ftrace_regs_caller_ret - ftrace_regs_caller);
+		ret = probe_kernel_read(ip, (void *)retq, RET_SIZE);
+		if (WARN_ON(ret < 0))
+			goto fail;
+	}
+
 	/*
 	 * The address of the ftrace_ops that is used for this trampoline
 	 * is stored at the end of the trampoline. This will be used to
arch/x86/kernel/ftrace_64.S +15 -17
index 369e61f..7657dc7 100644
@@ -157,8 +157,12 @@ SYM_INNER_LABEL(ftrace_call, SYM_L_GLOBAL)
 	 * think twice before adding any new code or changing the
 	 * layout here.
 	 */
-SYM_INNER_LABEL(ftrace_epilogue, SYM_L_GLOBAL)
+SYM_INNER_LABEL(ftrace_caller_end, SYM_L_GLOBAL)
 
+	jmp ftrace_epilogue
+SYM_FUNC_END(ftrace_caller);
+
+SYM_FUNC_START(ftrace_epilogue)
 #ifdef CONFIG_FUNCTION_GRAPH_TRACER
 SYM_INNER_LABEL(ftrace_graph_call, SYM_L_GLOBAL)
 	jmp ftrace_stub
@@ -170,14 +174,12 @@ SYM_INNER_LABEL(ftrace_graph_call, SYM_L_GLOBAL)
  */
 SYM_INNER_LABEL_ALIGN(ftrace_stub, SYM_L_WEAK)
 	retq
-SYM_FUNC_END(ftrace_caller)
+SYM_FUNC_END(ftrace_epilogue)
 
 SYM_FUNC_START(ftrace_regs_caller)
 	/* Save the current flags before any operations that can change them */
 	pushfq
 
-	UNWIND_HINT_SAVE
-
 	/* added 8 bytes to save flags */
 	save_mcount_regs 8
 	/* save_mcount_regs fills in first two parameters */
@@ -233,7 +235,10 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_GLOBAL)
 	movq ORIG_RAX(%rsp), %rax
 	movq %rax, MCOUNT_REG_SIZE-8(%rsp)
 
-	/* If ORIG_RAX is anything but zero, make this a call to that */
+	/*
+	 * If ORIG_RAX is anything but zero, make this a call to that.
+	 * See arch_ftrace_set_direct_caller().
+	 */
 	movq ORIG_RAX(%rsp), %rax
 	cmpq	$0, %rax
 	je	1f
@@ -244,20 +249,14 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_GLOBAL)
 	movq %rax, MCOUNT_REG_SIZE(%rsp)
 
 	restore_mcount_regs 8
+	/* Restore flags */
+	popfq
 
-	jmp	2f
+SYM_INNER_LABEL(ftrace_regs_caller_ret, SYM_L_GLOBAL);
+	UNWIND_HINT_RET_OFFSET
+	jmp	ftrace_epilogue
 
 1:	restore_mcount_regs
-
-
-2:
-	/*
-	 * The stack layout is nondetermistic here, depending on which path was
-	 * taken.  This confuses objtool and ORC, rightfully so.  For now,
-	 * pretend the stack always looks like the non-direct case.
-	 */
-	UNWIND_HINT_RESTORE
-
 	/* Restore flags */
 	popfq
 
@@ -268,7 +267,6 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_GLOBAL)
 	 * to the return.
 	 */
 SYM_INNER_LABEL(ftrace_regs_caller_end, SYM_L_GLOBAL)
-
 	jmp ftrace_epilogue
 
 SYM_FUNC_END(ftrace_regs_caller)
merge bot Applied
#22 lore raw

[tip: objtool/core] x86,ftrace: Fix ftrace_regs_caller() unwind

In reply to Peter Zijlstra
The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     0298739b7983cf9bf4fcfb4bfb815c539bdb87ca
Gitweb:        https://git.kernel.org/tip/0298739b7983cf9bf4fcfb4bfb815c539bdb87ca
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Wed, 01 Apr 2020 16:53:19 +02:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Wed, 22 Apr 2020 10:53:50 +02:00

x86,ftrace: Fix ftrace_regs_caller() unwind

The ftrace_regs_caller() trampoline does something 'funny' when there is a direct-caller present. In that case it stuffs the 'direct-caller' address on the return stack and then exits the function. This then results in 'returning' to the direct-caller with the exact registers we came in with -- an indirect tail-call without using a register.

This however (rightfully) confuses objtool because the function shares a few instruction in order to have a single exit path, but the stack layout is different for them, depending through which path we came there.

This is currently cludged by forcing the stack state to the non-direct case, but this generates actively wrong (ORC) unwind information for the direct case, leading to potential broken unwinds.

Fix this issue by fully separating the exit paths. This results in having to poke a second RET into the trampoline copy, see ftrace_regs_caller_ret.

This brings us to a second objtool problem, in order for it to perceive the 'jmp ftrace_epilogue' as a function exit, it needs to be recognised as a tail call. In order to make that happen, ftrace_epilogue needs to be the start of an STT_FUNC, so re-arrange code to make this so.

Finally, a third issue is that objtool requires functions to exit with the same stack layout they started with, which is obviously violated in the direct case, employ the new HINT_RET_OFFSET to tell objtool this is an expected exception.

Together, this results in generating correct ORC unwind information for the ftrace_regs_caller() function and it's trampoline copies.

---
 arch/x86/kernel/ftrace.c    | 12 ++++++++++--
 arch/x86/kernel/ftrace_64.S | 32 +++++++++++++++-----------------
 2 files changed, 25 insertions(+), 19 deletions(-)
arch/x86/kernel/ftrace.c +10 -2
index 37a0aea..867c126 100644
@@ -282,7 +282,8 @@ static inline void tramp_free(void *tramp) { }
 
 /* Defined as markers to the end of the ftrace default trampolines */
 extern void ftrace_regs_caller_end(void);
-extern void ftrace_epilogue(void);
+extern void ftrace_regs_caller_ret(void);
+extern void ftrace_caller_end(void);
 extern void ftrace_caller_op_ptr(void);
 extern void ftrace_regs_caller_op_ptr(void);
 
@@ -334,7 +335,7 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size)
 		call_offset = (unsigned long)ftrace_regs_call;
 	} else {
 		start_offset = (unsigned long)ftrace_caller;
-		end_offset = (unsigned long)ftrace_epilogue;
+		end_offset = (unsigned long)ftrace_caller_end;
 		op_offset = (unsigned long)ftrace_caller_op_ptr;
 		call_offset = (unsigned long)ftrace_call;
 	}
@@ -366,6 +367,13 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size)
 	if (WARN_ON(ret < 0))
 		goto fail;
 
+	if (ops->flags & FTRACE_OPS_FL_SAVE_REGS) {
+		ip = trampoline + (ftrace_regs_caller_ret - ftrace_regs_caller);
+		ret = probe_kernel_read(ip, (void *)retq, RET_SIZE);
+		if (WARN_ON(ret < 0))
+			goto fail;
+	}
+
 	/*
 	 * The address of the ftrace_ops that is used for this trampoline
 	 * is stored at the end of the trampoline. This will be used to
arch/x86/kernel/ftrace_64.S +15 -17
index 369e61f..7657dc7 100644
@@ -157,8 +157,12 @@ SYM_INNER_LABEL(ftrace_call, SYM_L_GLOBAL)
 	 * think twice before adding any new code or changing the
 	 * layout here.
 	 */
-SYM_INNER_LABEL(ftrace_epilogue, SYM_L_GLOBAL)
+SYM_INNER_LABEL(ftrace_caller_end, SYM_L_GLOBAL)
 
+	jmp ftrace_epilogue
+SYM_FUNC_END(ftrace_caller);
+
+SYM_FUNC_START(ftrace_epilogue)
 #ifdef CONFIG_FUNCTION_GRAPH_TRACER
 SYM_INNER_LABEL(ftrace_graph_call, SYM_L_GLOBAL)
 	jmp ftrace_stub
@@ -170,14 +174,12 @@ SYM_INNER_LABEL(ftrace_graph_call, SYM_L_GLOBAL)
  */
 SYM_INNER_LABEL_ALIGN(ftrace_stub, SYM_L_WEAK)
 	retq
-SYM_FUNC_END(ftrace_caller)
+SYM_FUNC_END(ftrace_epilogue)
 
 SYM_FUNC_START(ftrace_regs_caller)
 	/* Save the current flags before any operations that can change them */
 	pushfq
 
-	UNWIND_HINT_SAVE
-
 	/* added 8 bytes to save flags */
 	save_mcount_regs 8
 	/* save_mcount_regs fills in first two parameters */
@@ -233,7 +235,10 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_GLOBAL)
 	movq ORIG_RAX(%rsp), %rax
 	movq %rax, MCOUNT_REG_SIZE-8(%rsp)
 
-	/* If ORIG_RAX is anything but zero, make this a call to that */
+	/*
+	 * If ORIG_RAX is anything but zero, make this a call to that.
+	 * See arch_ftrace_set_direct_caller().
+	 */
 	movq ORIG_RAX(%rsp), %rax
 	cmpq	$0, %rax
 	je	1f
@@ -244,20 +249,14 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_GLOBAL)
 	movq %rax, MCOUNT_REG_SIZE(%rsp)
 
 	restore_mcount_regs 8
+	/* Restore flags */
+	popfq
 
-	jmp	2f
+SYM_INNER_LABEL(ftrace_regs_caller_ret, SYM_L_GLOBAL);
+	UNWIND_HINT_RET_OFFSET
+	jmp	ftrace_epilogue
 
 1:	restore_mcount_regs
-
-
-2:
-	/*
-	 * The stack layout is nondetermistic here, depending on which path was
-	 * taken.  This confuses objtool and ORC, rightfully so.  For now,
-	 * pretend the stack always looks like the non-direct case.
-	 */
-	UNWIND_HINT_RESTORE
-
 	/* Restore flags */
 	popfq
 
@@ -268,7 +267,6 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_GLOBAL)
 	 * to the return.
 	 */
 SYM_INNER_LABEL(ftrace_regs_caller_end, SYM_L_GLOBAL)
-
 	jmp ftrace_epilogue
 
 SYM_FUNC_END(ftrace_regs_caller)
author maintainer
#23 lore raw

PATCH v5 5/17 x86,ftrace: Use SIZEOF_PTREGS

In reply to Peter Zijlstra

There's a convenient macro for 'SS+8' called FRAME_SIZE. Use it to clarify things.

(entry/calling.h calls this SIZEOF_PTREGS but we're using asm/ptrace-abi.h)

---
 arch/x86/kernel/ftrace_64.S |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/kernel/ftrace_64.S +2 -2
@@ -23,7 +23,7 @@
 #endif /* CONFIG_FRAME_POINTER */
 
 /* Size of stack used to save mcount regs in save_mcount_regs */
-#define MCOUNT_REG_SIZE		(SS+8 + MCOUNT_FRAME_SIZE)
+#define MCOUNT_REG_SIZE		(FRAME_SIZE + MCOUNT_FRAME_SIZE)
 
 /*
  * gcc -pg option adds a call to 'mcount' in most functions.
@@ -77,7 +77,7 @@
 	/*
 	 * We add enough stack to save all regs.
 	 */
-	subq $(MCOUNT_REG_SIZE - MCOUNT_FRAME_SIZE), %rsp
+	subq $(FRAME_SIZE), %rsp
 	movq %rax, RAX(%rsp)
 	movq %rcx, RCX(%rsp)
 	movq %rdx, RDX(%rsp)
merge bot Applied
#24 lore raw

[tip: objtool/core] x86,ftrace: Use SIZEOF_PTREGS

The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     fecf897dd09bdba49cdbe69f3defaab0f5f688fb
Gitweb:        https://git.kernel.org/tip/fecf897dd09bdba49cdbe69f3defaab0f5f688fb
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Wed, 01 Apr 2020 16:50:40 +02:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Wed, 22 Apr 2020 23:10:06 +02:00

x86,ftrace: Use SIZEOF_PTREGS

There's a convenient macro for 'SS+8' called FRAME_SIZE. Use it to clarify things.

(entry/calling.h calls this SIZEOF_PTREGS but we're using asm/ptrace-abi.h)

---
 arch/x86/kernel/ftrace_64.S | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/kernel/ftrace_64.S +2 -2
index 7657dc7..be9aff2 100644
@@ -23,7 +23,7 @@
 #endif /* CONFIG_FRAME_POINTER */
 
 /* Size of stack used to save mcount regs in save_mcount_regs */
-#define MCOUNT_REG_SIZE		(SS+8 + MCOUNT_FRAME_SIZE)
+#define MCOUNT_REG_SIZE		(FRAME_SIZE + MCOUNT_FRAME_SIZE)
 
 /*
  * gcc -pg option adds a call to 'mcount' in most functions.
@@ -77,7 +77,7 @@
 	/*
 	 * We add enough stack to save all regs.
 	 */
-	subq $(MCOUNT_REG_SIZE - MCOUNT_FRAME_SIZE), %rsp
+	subq $(FRAME_SIZE), %rsp
 	movq %rax, RAX(%rsp)
 	movq %rcx, RCX(%rsp)
 	movq %rdx, RDX(%rsp)
merge bot Applied
#25 lore raw

[tip: objtool/core] x86,ftrace: Use SIZEOF_PTREGS

In reply to Peter Zijlstra
The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     dc2745b61907cf6faeb72cc25f2cc4b38d4a3cac
Gitweb:        https://git.kernel.org/tip/dc2745b61907cf6faeb72cc25f2cc4b38d4a3cac
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Wed, 01 Apr 2020 16:50:40 +02:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Wed, 22 Apr 2020 10:53:50 +02:00

x86,ftrace: Use SIZEOF_PTREGS

There's a convenient macro for 'SS+8' called FRAME_SIZE. Use it to clarify things.

(entry/calling.h calls this SIZEOF_PTREGS but we're using asm/ptrace-abi.h)

---
 arch/x86/kernel/ftrace_64.S | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/kernel/ftrace_64.S +2 -2
index 7657dc7..be9aff2 100644
@@ -23,7 +23,7 @@
 #endif /* CONFIG_FRAME_POINTER */
 
 /* Size of stack used to save mcount regs in save_mcount_regs */
-#define MCOUNT_REG_SIZE		(SS+8 + MCOUNT_FRAME_SIZE)
+#define MCOUNT_REG_SIZE		(FRAME_SIZE + MCOUNT_FRAME_SIZE)
 
 /*
  * gcc -pg option adds a call to 'mcount' in most functions.
@@ -77,7 +77,7 @@
 	/*
 	 * We add enough stack to save all regs.
 	 */
-	subq $(MCOUNT_REG_SIZE - MCOUNT_FRAME_SIZE), %rsp
+	subq $(FRAME_SIZE), %rsp
 	movq %rax, RAX(%rsp)
 	movq %rcx, RCX(%rsp)
 	movq %rdx, RDX(%rsp)
author maintainer
#26 lore raw

PATCH v5 6/17 x86,ftrace: Shrink ftrace_regs_caller() by one byte

In reply to Peter Zijlstra

'Optimize' ftrace_regs_caller. Instead of comparing against an immediate, the more natural way to test for zero on x86 is: 'test %r,%r'.

  48 83 f8 00             cmp    $0x0,%rax
  74 49                   je     226 <ftrace_regs_call+0xa3>
  48 85 c0                test   %rax,%rax
  74 49                   je     225 <ftrace_regs_call+0xa2>
---
 arch/x86/kernel/ftrace_64.S |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/kernel/ftrace_64.S +2 -2
@@ -240,8 +240,8 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_
 	 * See arch_ftrace_set_direct_caller().
 	 */
 	movq ORIG_RAX(%rsp), %rax
-	cmpq	$0, %rax
-	je	1f
+	testq	%rax, %rax
+	jz	1f
 
 	/* Swap the flags with orig_rax */
 	movq MCOUNT_REG_SIZE(%rsp), %rdi
merge bot Applied
#27 lore raw

[tip: objtool/core] x86,ftrace: Shrink ftrace_regs_caller() by one byte

The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     fdf6101cab968605564f211ab03f396dde67785b
Gitweb:        https://git.kernel.org/tip/fdf6101cab968605564f211ab03f396dde67785b
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Wed, 01 Apr 2020 16:51:11 +02:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Wed, 22 Apr 2020 23:10:06 +02:00

x86,ftrace: Shrink ftrace_regs_caller() by one byte

'Optimize' ftrace_regs_caller. Instead of comparing against an immediate, the more natural way to test for zero on x86 is: 'test %r,%r'.

  48 83 f8 00             cmp    $0x0,%rax
  74 49                   je     226 <ftrace_regs_call+0xa3>
  48 85 c0                test   %rax,%rax
  74 49                   je     225 <ftrace_regs_call+0xa2>
---
 arch/x86/kernel/ftrace_64.S | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/kernel/ftrace_64.S +2 -2
index be9aff2..9738ed2 100644
@@ -240,8 +240,8 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_GLOBAL)
 	 * See arch_ftrace_set_direct_caller().
 	 */
 	movq ORIG_RAX(%rsp), %rax
-	cmpq	$0, %rax
-	je	1f
+	testq	%rax, %rax
+	jz	1f
 
 	/* Swap the flags with orig_rax */
 	movq MCOUNT_REG_SIZE(%rsp), %rdi
merge bot Applied
#28 lore raw

[tip: objtool/core] x86,ftrace: Shrink ftrace_regs_caller() by one byte

In reply to Peter Zijlstra
The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     9f2dfd61dd022d4559d42a832fb03e76aad36c5f
Gitweb:        https://git.kernel.org/tip/9f2dfd61dd022d4559d42a832fb03e76aad36c5f
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Wed, 01 Apr 2020 16:51:11 +02:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Wed, 22 Apr 2020 10:53:50 +02:00

x86,ftrace: Shrink ftrace_regs_caller() by one byte

'Optimize' ftrace_regs_caller. Instead of comparing against an immediate, the more natural way to test for zero on x86 is: 'test %r,%r'.

  48 83 f8 00             cmp    $0x0,%rax
  74 49                   je     226 <ftrace_regs_call+0xa3>
  48 85 c0                test   %rax,%rax
  74 49                   je     225 <ftrace_regs_call+0xa2>
---
 arch/x86/kernel/ftrace_64.S | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/kernel/ftrace_64.S +2 -2
index be9aff2..9738ed2 100644
@@ -240,8 +240,8 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_GLOBAL)
 	 * See arch_ftrace_set_direct_caller().
 	 */
 	movq ORIG_RAX(%rsp), %rax
-	cmpq	$0, %rax
-	je	1f
+	testq	%rax, %rax
+	jz	1f
 
 	/* Swap the flags with orig_rax */
 	movq MCOUNT_REG_SIZE(%rsp), %rdi
author maintainer
#29 lore raw

PATCH v5 7/17 objtool: Remove SAVE/RESTORE hints

In reply to Peter Zijlstra

The SAVE/RESTORE hints are now unused; remove them.

---
 arch/x86/include/asm/orc_types.h       |    4 ---
 arch/x86/include/asm/unwind_hints.h    |   13 ----------
 tools/arch/x86/include/asm/orc_types.h |    4 ---
 tools/objtool/check.c                  |   42 ++-------------------------------
 tools/objtool/check.h                  |    2 -
 5 files changed, 7 insertions(+), 58 deletions(-)
arch/x86/include/asm/orc_types.h +1 -3
@@ -58,9 +58,7 @@
 #define ORC_TYPE_CALL			0
 #define ORC_TYPE_REGS			1
 #define ORC_TYPE_REGS_IRET		2
-#define UNWIND_HINT_TYPE_SAVE		3
-#define UNWIND_HINT_TYPE_RESTORE	4
-#define UNWIND_HINT_TYPE_RET_OFFSET	5
+#define UNWIND_HINT_TYPE_RET_OFFSET	3
 
 #ifndef __ASSEMBLY__
 /*
arch/x86/include/asm/unwind_hints.h +1 -12
@@ -86,15 +86,6 @@
 	UNWIND_HINT sp_offset=\sp_offset
 .endm
 
-.macro UNWIND_HINT_SAVE
-	UNWIND_HINT type=UNWIND_HINT_TYPE_SAVE
-.endm
-
-.macro UNWIND_HINT_RESTORE
-	UNWIND_HINT type=UNWIND_HINT_TYPE_RESTORE
-.endm
-
-
 /*
  * RET_OFFSET: Used on instructions that terminate a function; mostly RETURN
  * and sibling calls. On these, sp_offset denotes the expected offset from
@@ -118,9 +109,7 @@
 	".balign 4 \n\t"					\
 	".popsection\n\t"
 
-#define UNWIND_HINT_SAVE UNWIND_HINT(0, 0, UNWIND_HINT_TYPE_SAVE, 0)
-
-#define UNWIND_HINT_RESTORE UNWIND_HINT(0, 0, UNWIND_HINT_TYPE_RESTORE, 0)
+#define UNWIND_HINT_RET_OFFSET(offset) UNWIND_HINT(0, (offset), UNWIND_HINT_TYPE_RET_OFFSET, 0)
 
 #endif /* __ASSEMBLY__ */
 
tools/arch/x86/include/asm/orc_types.h +1 -3
@@ -58,9 +58,7 @@
 #define ORC_TYPE_CALL			0
 #define ORC_TYPE_REGS			1
 #define ORC_TYPE_REGS_IRET		2
-#define UNWIND_HINT_TYPE_SAVE		3
-#define UNWIND_HINT_TYPE_RESTORE	4
-#define UNWIND_HINT_TYPE_RET_OFFSET	5
+#define UNWIND_HINT_TYPE_RET_OFFSET	3
 
 #ifndef __ASSEMBLY__
 /*
tools/objtool/check.c +3 -39

55 lines not on this page · raw message

tools/objtool/check.h +1 -1
@@ -34,7 +34,7 @@ struct instruction {
 	enum insn_type type;
 	unsigned long immediate;
 	bool alt_group, dead_end, ignore, ignore_alts;
-	bool hint, save, restore;
+	bool hint;
 	bool retpoline_safe;
 	u8 visited;
 	u8 ret_offset;
merge bot Applied
#30 lore raw

[tip: objtool/core] objtool: Remove SAVE/RESTORE hints

The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     4dde773748d737bae48f21768fc511e7c1c02a97
Gitweb:        https://git.kernel.org/tip/4dde773748d737bae48f21768fc511e7c1c02a97
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Wed, 01 Apr 2020 16:54:26 +02:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Wed, 22 Apr 2020 23:10:06 +02:00

objtool: Remove SAVE/RESTORE hints

The SAVE/RESTORE hints are now unused; remove them.

---
 arch/x86/include/asm/orc_types.h       |  4 +--
 arch/x86/include/asm/unwind_hints.h    | 27 +----------------
 tools/arch/x86/include/asm/orc_types.h |  4 +--
 tools/objtool/check.c                  | 42 +------------------------
 tools/objtool/check.h                  |  2 +-
 5 files changed, 6 insertions(+), 73 deletions(-)
arch/x86/include/asm/orc_types.h +1 -3
index 5f18ca7..d255349 100644
@@ -58,9 +58,7 @@
 #define ORC_TYPE_CALL			0
 #define ORC_TYPE_REGS			1
 #define ORC_TYPE_REGS_IRET		2
-#define UNWIND_HINT_TYPE_SAVE		3
-#define UNWIND_HINT_TYPE_RESTORE	4
-#define UNWIND_HINT_TYPE_RET_OFFSET	5
+#define UNWIND_HINT_TYPE_RET_OFFSET	3
 
 #ifndef __ASSEMBLY__
 /*
arch/x86/include/asm/unwind_hints.h +0 -27
index aabf7ac..7d903fd 100644
@@ -86,15 +86,6 @@
 	UNWIND_HINT sp_offset=\sp_offset
 .endm
 
-.macro UNWIND_HINT_SAVE
-	UNWIND_HINT type=UNWIND_HINT_TYPE_SAVE
-.endm
-
-.macro UNWIND_HINT_RESTORE
-	UNWIND_HINT type=UNWIND_HINT_TYPE_RESTORE
-.endm
-
-
 /*
  * RET_OFFSET: Used on instructions that terminate a function; mostly RETURN
  * and sibling calls. On these, sp_offset denotes the expected offset from
@@ -104,24 +95,6 @@
 	UNWIND_HINT type=UNWIND_HINT_TYPE_RET_OFFSET sp_offset=\sp_offset
 .endm
 
-#else /* !__ASSEMBLY__ */
-
-#define UNWIND_HINT(sp_reg, sp_offset, type, end)		\
-	"987: \n\t"						\
-	".pushsection .discard.unwind_hints\n\t"		\
-	/* struct unwind_hint */				\
-	".long 987b - .\n\t"					\
-	".short " __stringify(sp_offset) "\n\t"			\
-	".byte " __stringify(sp_reg) "\n\t"			\
-	".byte " __stringify(type) "\n\t"			\
-	".byte " __stringify(end) "\n\t"			\
-	".balign 4 \n\t"					\
-	".popsection\n\t"
-
-#define UNWIND_HINT_SAVE UNWIND_HINT(0, 0, UNWIND_HINT_TYPE_SAVE, 0)
-
-#define UNWIND_HINT_RESTORE UNWIND_HINT(0, 0, UNWIND_HINT_TYPE_RESTORE, 0)
-
 #endif /* __ASSEMBLY__ */
 
 #endif /* _ASM_X86_UNWIND_HINTS_H */
tools/arch/x86/include/asm/orc_types.h +1 -3
index 5f18ca7..d255349 100644
@@ -58,9 +58,7 @@
 #define ORC_TYPE_CALL			0
 #define ORC_TYPE_REGS			1
 #define ORC_TYPE_REGS_IRET		2
-#define UNWIND_HINT_TYPE_SAVE		3
-#define UNWIND_HINT_TYPE_RESTORE	4
-#define UNWIND_HINT_TYPE_RET_OFFSET	5
+#define UNWIND_HINT_TYPE_RET_OFFSET	3
 
 #ifndef __ASSEMBLY__
 /*
tools/objtool/check.c +3 -39

55 lines not on this page · raw message

tools/objtool/check.h +1 -1
index 81ce27e..7c30760 100644
@@ -34,7 +34,7 @@ struct instruction {
 	enum insn_type type;
 	unsigned long immediate;
 	bool alt_group, dead_end, ignore, ignore_alts;
-	bool hint, save, restore;
+	bool hint;
 	bool retpoline_safe;
 	u8 visited;
 	u8 ret_offset;
merge bot Applied
#31 lore raw

[tip: objtool/core] objtool: Remove SAVE/RESTORE hints

In reply to Peter Zijlstra
The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     c536ed2fffd5dbf81fe2dede8ef294e0cbb08f72
Gitweb:        https://git.kernel.org/tip/c536ed2fffd5dbf81fe2dede8ef294e0cbb08f72
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Wed, 01 Apr 2020 16:54:26 +02:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Wed, 22 Apr 2020 10:53:50 +02:00

objtool: Remove SAVE/RESTORE hints

The SAVE/RESTORE hints are now unused; remove them.

---
 arch/x86/include/asm/orc_types.h       |  4 +--
 arch/x86/include/asm/unwind_hints.h    | 27 +----------------
 tools/arch/x86/include/asm/orc_types.h |  4 +--
 tools/objtool/check.c                  | 42 +------------------------
 tools/objtool/check.h                  |  2 +-
 5 files changed, 6 insertions(+), 73 deletions(-)
arch/x86/include/asm/orc_types.h +1 -3
index 5f18ca7..d255349 100644
@@ -58,9 +58,7 @@
 #define ORC_TYPE_CALL			0
 #define ORC_TYPE_REGS			1
 #define ORC_TYPE_REGS_IRET		2
-#define UNWIND_HINT_TYPE_SAVE		3
-#define UNWIND_HINT_TYPE_RESTORE	4
-#define UNWIND_HINT_TYPE_RET_OFFSET	5
+#define UNWIND_HINT_TYPE_RET_OFFSET	3
 
 #ifndef __ASSEMBLY__
 /*
arch/x86/include/asm/unwind_hints.h +0 -27
index aabf7ac..7d903fd 100644
@@ -86,15 +86,6 @@
 	UNWIND_HINT sp_offset=\sp_offset
 .endm
 
-.macro UNWIND_HINT_SAVE
-	UNWIND_HINT type=UNWIND_HINT_TYPE_SAVE
-.endm
-
-.macro UNWIND_HINT_RESTORE
-	UNWIND_HINT type=UNWIND_HINT_TYPE_RESTORE
-.endm
-
-
 /*
  * RET_OFFSET: Used on instructions that terminate a function; mostly RETURN
  * and sibling calls. On these, sp_offset denotes the expected offset from
@@ -104,24 +95,6 @@
 	UNWIND_HINT type=UNWIND_HINT_TYPE_RET_OFFSET sp_offset=\sp_offset
 .endm
 
-#else /* !__ASSEMBLY__ */
-
-#define UNWIND_HINT(sp_reg, sp_offset, type, end)		\
-	"987: \n\t"						\
-	".pushsection .discard.unwind_hints\n\t"		\
-	/* struct unwind_hint */				\
-	".long 987b - .\n\t"					\
-	".short " __stringify(sp_offset) "\n\t"			\
-	".byte " __stringify(sp_reg) "\n\t"			\
-	".byte " __stringify(type) "\n\t"			\
-	".byte " __stringify(end) "\n\t"			\
-	".balign 4 \n\t"					\
-	".popsection\n\t"
-
-#define UNWIND_HINT_SAVE UNWIND_HINT(0, 0, UNWIND_HINT_TYPE_SAVE, 0)
-
-#define UNWIND_HINT_RESTORE UNWIND_HINT(0, 0, UNWIND_HINT_TYPE_RESTORE, 0)
-
 #endif /* __ASSEMBLY__ */
 
 #endif /* _ASM_X86_UNWIND_HINTS_H */
tools/arch/x86/include/asm/orc_types.h +1 -3
index 5f18ca7..d255349 100644
@@ -58,9 +58,7 @@
 #define ORC_TYPE_CALL			0
 #define ORC_TYPE_REGS			1
 #define ORC_TYPE_REGS_IRET		2
-#define UNWIND_HINT_TYPE_SAVE		3
-#define UNWIND_HINT_TYPE_RESTORE	4
-#define UNWIND_HINT_TYPE_RET_OFFSET	5
+#define UNWIND_HINT_TYPE_RET_OFFSET	3
 
 #ifndef __ASSEMBLY__
 /*
tools/objtool/check.c +3 -39

55 lines not on this page · raw message

tools/objtool/check.h +1 -1
index 81ce27e..7c30760 100644
@@ -34,7 +34,7 @@ struct instruction {
 	enum insn_type type;
 	unsigned long immediate;
 	bool alt_group, dead_end, ignore, ignore_alts;
-	bool hint, save, restore;
+	bool hint;
 	bool retpoline_safe;
 	u8 visited;
 	u8 ret_offset;
author maintainer
#32 lore raw

PATCH v5 8/17 objtool: Rename struct cfi_state

In reply to Peter Zijlstra

There's going to be a new struct cfi_state, rename this one to make place.

---
 tools/objtool/arch.h            |    2 +-
 tools/objtool/arch/x86/decode.c |    2 +-
 tools/objtool/cfi.h             |    2 +-
 tools/objtool/check.c           |    2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)
tools/objtool/arch.h +1 -1
@@ -68,7 +68,7 @@ struct stack_op {
 	struct list_head list;
 };
 
-void arch_initial_func_cfi_state(struct cfi_state *state);
+void arch_initial_func_cfi_state(struct cfi_init_state *state);
 
 int arch_decode_instruction(struct elf *elf, struct section *sec,
 			    unsigned long offset, unsigned int maxlen,
tools/objtool/arch/x86/decode.c +1 -1
@@ -501,7 +501,7 @@ int arch_decode_instruction(struct elf *
 	return 0;
 }
 
-void arch_initial_func_cfi_state(struct cfi_state *state)
+void arch_initial_func_cfi_state(struct cfi_init_state *state)
 {
 	int i;
 
tools/objtool/cfi.h +1 -1
@@ -35,7 +35,7 @@ struct cfi_reg {
 	int offset;
 };
 
-struct cfi_state {
+struct cfi_init_state {
 	struct cfi_reg cfa;
 	struct cfi_reg regs[CFI_NUM_REGS];
 };
tools/objtool/check.c +1 -1
@@ -27,7 +27,7 @@ struct alternative {
 };
 
 const char *objname;
-struct cfi_state initial_func_cfi;
+struct cfi_init_state initial_func_cfi;
 
 struct instruction *find_insn(struct objtool_file *file,
 			      struct section *sec, unsigned long offset)
merge bot Applied
#33 lore raw

[tip: objtool/core] objtool: Rename struct cfi_state

The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     1079b70d332afdf5368804b0b95402822ba0a61d
Gitweb:        https://git.kernel.org/tip/1079b70d332afdf5368804b0b95402822ba0a61d
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Wed, 25 Mar 2020 15:34:50 +01:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Wed, 22 Apr 2020 23:10:06 +02:00

objtool: Rename struct cfi_state

There's going to be a new struct cfi_state, rename this one to make place.

---
 tools/objtool/arch.h            | 2 +-
 tools/objtool/arch/x86/decode.c | 2 +-
 tools/objtool/cfi.h             | 2 +-
 tools/objtool/check.c           | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)
tools/objtool/arch.h +1 -1
index 55396df..561c316 100644
@@ -70,7 +70,7 @@ struct stack_op {
 
 struct instruction;
 
-void arch_initial_func_cfi_state(struct cfi_state *state);
+void arch_initial_func_cfi_state(struct cfi_init_state *state);
 
 int arch_decode_instruction(struct elf *elf, struct section *sec,
 			    unsigned long offset, unsigned int maxlen,
tools/objtool/arch/x86/decode.c +1 -1
index 3273638..f0d42ad 100644
@@ -512,7 +512,7 @@ int arch_decode_instruction(struct elf *elf, struct section *sec,
 	return 0;
 }
 
-void arch_initial_func_cfi_state(struct cfi_state *state)
+void arch_initial_func_cfi_state(struct cfi_init_state *state)
 {
 	int i;
 
tools/objtool/cfi.h +1 -1
index 1a3e7b8..6faf976 100644
@@ -18,7 +18,7 @@ struct cfi_reg {
 	int offset;
 };
 
-struct cfi_state {
+struct cfi_init_state {
 	struct cfi_reg cfa;
 	struct cfi_reg regs[CFI_NUM_REGS];
 };
tools/objtool/check.c +1 -1
index 7e67bdb..8bc525f 100644
@@ -27,7 +27,7 @@ struct alternative {
 };
 
 const char *objname;
-struct cfi_state initial_func_cfi;
+struct cfi_init_state initial_func_cfi;
 
 struct instruction *find_insn(struct objtool_file *file,
 			      struct section *sec, unsigned long offset)
merge bot Applied
#34 lore raw

[tip: objtool/core] objtool: Rename struct cfi_state

In reply to Peter Zijlstra
The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     a3608f5954d07a40fb93764dc6d06195fa52eb14
Gitweb:        https://git.kernel.org/tip/a3608f5954d07a40fb93764dc6d06195fa52eb14
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Wed, 25 Mar 2020 15:34:50 +01:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Wed, 22 Apr 2020 10:53:50 +02:00

objtool: Rename struct cfi_state

There's going to be a new struct cfi_state, rename this one to make place.

---
 tools/objtool/arch.h            | 2 +-
 tools/objtool/arch/x86/decode.c | 2 +-
 tools/objtool/cfi.h             | 2 +-
 tools/objtool/check.c           | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)
tools/objtool/arch.h +1 -1
index 55396df..561c316 100644
@@ -70,7 +70,7 @@ struct stack_op {
 
 struct instruction;
 
-void arch_initial_func_cfi_state(struct cfi_state *state);
+void arch_initial_func_cfi_state(struct cfi_init_state *state);
 
 int arch_decode_instruction(struct elf *elf, struct section *sec,
 			    unsigned long offset, unsigned int maxlen,
tools/objtool/arch/x86/decode.c +1 -1
index 3273638..f0d42ad 100644
@@ -512,7 +512,7 @@ int arch_decode_instruction(struct elf *elf, struct section *sec,
 	return 0;
 }
 
-void arch_initial_func_cfi_state(struct cfi_state *state)
+void arch_initial_func_cfi_state(struct cfi_init_state *state)
 {
 	int i;
 
tools/objtool/cfi.h +1 -1
index 1a3e7b8..6faf976 100644
@@ -18,7 +18,7 @@ struct cfi_reg {
 	int offset;
 };
 
-struct cfi_state {
+struct cfi_init_state {
 	struct cfi_reg cfa;
 	struct cfi_reg regs[CFI_NUM_REGS];
 };
tools/objtool/check.c +1 -1
index 464f10c..538b462 100644
@@ -27,7 +27,7 @@ struct alternative {
 };
 
 const char *objname;
-struct cfi_state initial_func_cfi;
+struct cfi_init_state initial_func_cfi;
 
 struct instruction *find_insn(struct objtool_file *file,
 			      struct section *sec, unsigned long offset)
author maintainer
#35 lore raw

PATCH v5 9/17 objtool: Fix !CFI insn_state propagation

In reply to Peter Zijlstra

Objtool keeps per instruction CFI state in struct insn_state and will save/restore this where required. However, insn_state has grown some !CFI state, and this must not be saved/restored (that would loose/destroy state).

Fix this by moving the CFI specific parts of insn_state into struct cfi_state.

---
 tools/objtool/cfi.h     |   12 ++
 tools/objtool/check.c   |  264 ++++++++++++++++++++++++------------------------
 tools/objtool/check.h   |   13 --
 tools/objtool/orc_gen.c |    8 -
 4 files changed, 157 insertions(+), 140 deletions(-)
tools/objtool/cfi.h +12 -0
@@ -36,8 +36,20 @@ struct cfi_reg {
 };
 
 struct cfi_init_state {
+	struct cfi_reg regs[CFI_NUM_REGS];
 	struct cfi_reg cfa;
+};
+
+struct cfi_state {
 	struct cfi_reg regs[CFI_NUM_REGS];
+	struct cfi_reg vals[CFI_NUM_REGS];
+	struct cfi_reg cfa;
+	int stack_size;
+	int drap_reg, drap_offset;
+	unsigned char type;
+	bool bp_scratch;
+	bool drap;
+	bool end;
 };
 
 #endif /* _OBJTOOL_CFI_H */
tools/objtool/check.c +137 -127

614 lines not on this page · raw message

tools/objtool/check.h +4 -9
@@ -14,15 +14,10 @@
 #include <linux/hashtable.h>
 
 struct insn_state {
-	struct cfi_reg cfa;
-	struct cfi_reg regs[CFI_NUM_REGS];
-	int stack_size;
-	unsigned char type;
-	bool bp_scratch;
-	bool drap, end, uaccess, df;
+	struct cfi_state cfi;
 	unsigned int uaccess_stack;
-	int drap_reg, drap_offset;
-	struct cfi_reg vals[CFI_NUM_REGS];
+	bool uaccess;
+	bool df;
 };
 
 struct instruction {
@@ -45,7 +40,7 @@ struct instruction {
 	struct list_head alts;
 	struct symbol *func;
 	struct list_head stack_ops;
-	struct insn_state state;
+	struct cfi_state cfi;
 	struct orc_entry orc;
 };
 
tools/objtool/orc_gen.c +4 -4
@@ -16,10 +16,10 @@ int create_orc(struct objtool_file *file
 
 	for_each_insn(file, insn) {
 		struct orc_entry *orc = &insn->orc;
-		struct cfi_reg *cfa = &insn->state.cfa;
-		struct cfi_reg *bp = &insn->state.regs[CFI_BP];
+		struct cfi_reg *cfa = &insn->cfi.cfa;
+		struct cfi_reg *bp = &insn->cfi.regs[CFI_BP];
 
-		orc->end = insn->state.end;
+		orc->end = insn->cfi.end;
 
 		if (cfa->base == CFI_UNDEFINED) {
 			orc->sp_reg = ORC_REG_UNDEFINED;
@@ -75,7 +75,7 @@ int create_orc(struct objtool_file *file
 
 		orc->sp_offset = cfa->offset;
 		orc->bp_offset = bp->offset;
-		orc->type = insn->state.type;
+		orc->type = insn->cfi.type;
 	}
 
 	return 0;
merge bot Applied
#36 lore raw

[tip: objtool/core] objtool: Fix !CFI insn_state propagation

The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     4b6c0589e3938b6b09a16c29ab42d63d667f23b1
Gitweb:        https://git.kernel.org/tip/4b6c0589e3938b6b09a16c29ab42d63d667f23b1
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Wed, 25 Mar 2020 14:04:45 +01:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Wed, 22 Apr 2020 23:10:07 +02:00

objtool: Fix !CFI insn_state propagation

Objtool keeps per instruction CFI state in struct insn_state and will save/restore this where required. However, insn_state has grown some !CFI state, and this must not be saved/restored (that would loose/destroy state).

Fix this by moving the CFI specific parts of insn_state into struct cfi_state.

---
 tools/objtool/cfi.h     |  12 ++-
 tools/objtool/check.c   | 264 ++++++++++++++++++++-------------------
 tools/objtool/check.h   |  13 +--
 tools/objtool/orc_gen.c |   8 +-
 4 files changed, 157 insertions(+), 140 deletions(-)
tools/objtool/cfi.h +12 -0
index 6faf976..c7c59c6 100644
@@ -19,8 +19,20 @@ struct cfi_reg {
 };
 
 struct cfi_init_state {
+	struct cfi_reg regs[CFI_NUM_REGS];
 	struct cfi_reg cfa;
+};
+
+struct cfi_state {
 	struct cfi_reg regs[CFI_NUM_REGS];
+	struct cfi_reg vals[CFI_NUM_REGS];
+	struct cfi_reg cfa;
+	int stack_size;
+	int drap_reg, drap_offset;
+	unsigned char type;
+	bool bp_scratch;
+	bool drap;
+	bool end;
 };
 
 #endif /* _OBJTOOL_CFI_H */
tools/objtool/check.c +137 -127

612 lines not on this page · raw message

tools/objtool/check.h +4 -9
index 7c30760..99413d4 100644
@@ -14,15 +14,10 @@
 #include <linux/hashtable.h>
 
 struct insn_state {
-	struct cfi_reg cfa;
-	struct cfi_reg regs[CFI_NUM_REGS];
-	int stack_size;
-	unsigned char type;
-	bool bp_scratch;
-	bool drap, end, uaccess, df;
+	struct cfi_state cfi;
 	unsigned int uaccess_stack;
-	int drap_reg, drap_offset;
-	struct cfi_reg vals[CFI_NUM_REGS];
+	bool uaccess;
+	bool df;
 };
 
 struct instruction {
@@ -45,7 +40,7 @@ struct instruction {
 	struct list_head alts;
 	struct symbol *func;
 	struct list_head stack_ops;
-	struct insn_state state;
+	struct cfi_state cfi;
 	struct orc_entry orc;
 };
 
tools/objtool/orc_gen.c +4 -4
index 4c0dabd..2cf640f 100644
@@ -16,10 +16,10 @@ int create_orc(struct objtool_file *file)
 
 	for_each_insn(file, insn) {
 		struct orc_entry *orc = &insn->orc;
-		struct cfi_reg *cfa = &insn->state.cfa;
-		struct cfi_reg *bp = &insn->state.regs[CFI_BP];
+		struct cfi_reg *cfa = &insn->cfi.cfa;
+		struct cfi_reg *bp = &insn->cfi.regs[CFI_BP];
 
-		orc->end = insn->state.end;
+		orc->end = insn->cfi.end;
 
 		if (cfa->base == CFI_UNDEFINED) {
 			orc->sp_reg = ORC_REG_UNDEFINED;
@@ -75,7 +75,7 @@ int create_orc(struct objtool_file *file)
 
 		orc->sp_offset = cfa->offset;
 		orc->bp_offset = bp->offset;
-		orc->type = insn->state.type;
+		orc->type = insn->cfi.type;
 	}
 
 	return 0;
merge bot Applied
#37 lore raw

[tip: objtool/core] objtool: Fix !CFI insn_state propagation

In reply to Peter Zijlstra
The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     e7c0219b328c96746767f21b9532eed6a48f61c5
Gitweb:        https://git.kernel.org/tip/e7c0219b328c96746767f21b9532eed6a48f61c5
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Wed, 25 Mar 2020 14:04:45 +01:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Wed, 22 Apr 2020 10:53:50 +02:00

objtool: Fix !CFI insn_state propagation

Objtool keeps per instruction CFI state in struct insn_state and will save/restore this where required. However, insn_state has grown some !CFI state, and this must not be saved/restored (that would loose/destroy state).

Fix this by moving the CFI specific parts of insn_state into struct cfi_state.

---
 tools/objtool/cfi.h     |  12 ++-
 tools/objtool/check.c   | 264 ++++++++++++++++++++-------------------
 tools/objtool/check.h   |  13 +--
 tools/objtool/orc_gen.c |   8 +-
 4 files changed, 157 insertions(+), 140 deletions(-)
tools/objtool/cfi.h +12 -0
index 6faf976..c7c59c6 100644
@@ -19,8 +19,20 @@ struct cfi_reg {
 };
 
 struct cfi_init_state {
+	struct cfi_reg regs[CFI_NUM_REGS];
 	struct cfi_reg cfa;
+};
+
+struct cfi_state {
 	struct cfi_reg regs[CFI_NUM_REGS];
+	struct cfi_reg vals[CFI_NUM_REGS];
+	struct cfi_reg cfa;
+	int stack_size;
+	int drap_reg, drap_offset;
+	unsigned char type;
+	bool bp_scratch;
+	bool drap;
+	bool end;
 };
 
 #endif /* _OBJTOOL_CFI_H */
tools/objtool/check.c +137 -127

612 lines not on this page · raw message

tools/objtool/check.h +4 -9
index 7c30760..99413d4 100644
@@ -14,15 +14,10 @@
 #include <linux/hashtable.h>
 
 struct insn_state {
-	struct cfi_reg cfa;
-	struct cfi_reg regs[CFI_NUM_REGS];
-	int stack_size;
-	unsigned char type;
-	bool bp_scratch;
-	bool drap, end, uaccess, df;
+	struct cfi_state cfi;
 	unsigned int uaccess_stack;
-	int drap_reg, drap_offset;
-	struct cfi_reg vals[CFI_NUM_REGS];
+	bool uaccess;
+	bool df;
 };
 
 struct instruction {
@@ -45,7 +40,7 @@ struct instruction {
 	struct list_head alts;
 	struct symbol *func;
 	struct list_head stack_ops;
-	struct insn_state state;
+	struct cfi_state cfi;
 	struct orc_entry orc;
 };
 
tools/objtool/orc_gen.c +4 -4
index 4c0dabd..2cf640f 100644
@@ -16,10 +16,10 @@ int create_orc(struct objtool_file *file)
 
 	for_each_insn(file, insn) {
 		struct orc_entry *orc = &insn->orc;
-		struct cfi_reg *cfa = &insn->state.cfa;
-		struct cfi_reg *bp = &insn->state.regs[CFI_BP];
+		struct cfi_reg *cfa = &insn->cfi.cfa;
+		struct cfi_reg *bp = &insn->cfi.regs[CFI_BP];
 
-		orc->end = insn->state.end;
+		orc->end = insn->cfi.end;
 
 		if (cfa->base == CFI_UNDEFINED) {
 			orc->sp_reg = ORC_REG_UNDEFINED;
@@ -75,7 +75,7 @@ int create_orc(struct objtool_file *file)
 
 		orc->sp_offset = cfa->offset;
 		orc->bp_offset = bp->offset;
-		orc->type = insn->state.type;
+		orc->type = insn->cfi.type;
 	}
 
 	return 0;
author maintainer
#38 lore raw

PATCH v5 10/17 objtool: Implement noinstr validation

In reply to Peter Zijlstra

Validate that any call out of .noinstr.text is in between instr_begin() and instr_end() annotations.

This annotation is useful to ensure correct behaviour wrt tracing sensitive code like entry/exit and idle code. When we run code in a sensitive context we want a guarantee no unknown code is ran.

Since this validation relies on knowing the section of call destination symbols, we must run it on vmlinux.o instead of on individual object files.

Add two options:

-d/--duplicate "duplicate validation for vmlinux" -l/--vmlinux "vmlinux.o validation"

Where the latter auto-detects when objname ends with "vmlinux.o" and the former will force all validations, also those already done on !vmlinux object files.

---
 tools/objtool/builtin-check.c |   11 +++-
 tools/objtool/builtin.h       |    2 
 tools/objtool/check.c         |   98 ++++++++++++++++++++++++++++++++++++++++++
 tools/objtool/check.h         |    3 +
 tools/objtool/elf.h           |    2 
 5 files changed, 112 insertions(+), 4 deletions(-)
tools/objtool/builtin-check.c +9 -2
@@ -14,10 +14,11 @@
  */
 
 #include <subcmd/parse-options.h>
+#include <string.h>
 #include "builtin.h"
 #include "check.h"
 
-bool no_fp, no_unreachable, retpoline, module, backtrace, uaccess, stats;
+bool no_fp, no_unreachable, retpoline, module, backtrace, uaccess, stats, validate_dup, vmlinux;
 
 static const char * const check_usage[] = {
 	"objtool check [<options>] file.o",
@@ -32,12 +33,14 @@ const struct option check_options[] = {
 	OPT_BOOLEAN('b', "backtrace", &backtrace, "unwind on error"),
 	OPT_BOOLEAN('a', "uaccess", &uaccess, "enable uaccess checking"),
 	OPT_BOOLEAN('s', "stats", &stats, "print statistics"),
+	OPT_BOOLEAN('d', "duplicate", &validate_dup, "duplicate validation for vmlinux.o"),
+	OPT_BOOLEAN('l', "vmlinux", &vmlinux, "vmlinux.o validation"),
 	OPT_END(),
 };
 
 int cmd_check(int argc, const char **argv)
 {
-	const char *objname;
+	const char *objname, *s;
 
 	argc = parse_options(argc, argv, check_options, check_usage, 0);
 
@@ -46,5 +49,9 @@ int cmd_check(int argc, const char **arg
 
 	objname = argv[0];
 
+	s = strstr(objname, "vmlinux.o");
+	if (s && !s[9])
+		vmlinux = true;
+
 	return check(objname, false);
 }
tools/objtool/builtin.h +1 -1
@@ -8,7 +8,7 @@
 #include <subcmd/parse-options.h>
 
 extern const struct option check_options[];
-extern bool no_fp, no_unreachable, retpoline, module, backtrace, uaccess, stats;
+extern bool no_fp, no_unreachable, retpoline, module, backtrace, uaccess, stats, validate_dup, vmlinux;
 
 extern int cmd_check(int argc, const char **argv);
 extern int cmd_orc(int argc, const char **argv);
tools/objtool/check.c +98 -0

152 lines not on this page · raw message

tools/objtool/check.h +3 -0
@@ -18,6 +18,8 @@ struct insn_state {
 	unsigned int uaccess_stack;
 	bool uaccess;
 	bool df;
+	bool noinstr;
+	s8 instr;
 };
 
 struct instruction {
@@ -31,6 +33,7 @@ struct instruction {
 	bool alt_group, dead_end, ignore, ignore_alts;
 	bool hint;
 	bool retpoline_safe;
+	s8 instr;
 	u8 visited;
 	u8 ret_offset;
 	struct symbol *call_dest;
tools/objtool/elf.h +1 -1
@@ -39,7 +39,7 @@ struct section {
 	char *name;
 	int idx;
 	unsigned int len;
-	bool changed, text, rodata;
+	bool changed, text, rodata, noinstr;
 };
 
 struct symbol {
merge bot Applied
#39 lore raw

[tip: objtool/core] objtool: Implement noinstr validation

The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     1f6d80c69ef603a6b6b5b4eb8b0e77286b053702
Gitweb:        https://git.kernel.org/tip/1f6d80c69ef603a6b6b5b4eb8b0e77286b053702
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Tue, 10 Mar 2020 18:57:41 +01:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Wed, 22 Apr 2020 23:10:07 +02:00

objtool: Implement noinstr validation

Validate that any call out of .noinstr.text is in between instr_begin() and instr_end() annotations.

This annotation is useful to ensure correct behaviour wrt tracing sensitive code like entry/exit and idle code. When we run code in a sensitive context we want a guarantee no unknown code is ran.

Since this validation relies on knowing the section of call destination symbols, we must run it on vmlinux.o instead of on individual object files.

Add two options:

-d/--duplicate "duplicate validation for vmlinux" -l/--vmlinux "vmlinux.o validation"

Where the latter auto-detects when objname ends with "vmlinux.o" and the former will force all validations, also those already done on !vmlinux object files.

---
 tools/objtool/builtin-check.c | 11 +++-
 tools/objtool/builtin.h       |  2 +-
 tools/objtool/check.c         | 98 ++++++++++++++++++++++++++++++++++-
 tools/objtool/check.h         |  3 +-
 tools/objtool/elf.h           |  2 +-
 5 files changed, 112 insertions(+), 4 deletions(-)
tools/objtool/builtin-check.c +9 -2
index 10fbe75..be42b71 100644
@@ -14,10 +14,11 @@
  */
 
 #include <subcmd/parse-options.h>
+#include <string.h>
 #include "builtin.h"
 #include "check.h"
 
-bool no_fp, no_unreachable, retpoline, module, backtrace, uaccess, stats;
+bool no_fp, no_unreachable, retpoline, module, backtrace, uaccess, stats, validate_dup, vmlinux;
 
 static const char * const check_usage[] = {
 	"objtool check [<options>] file.o",
@@ -32,12 +33,14 @@ const struct option check_options[] = {
 	OPT_BOOLEAN('b', "backtrace", &backtrace, "unwind on error"),
 	OPT_BOOLEAN('a', "uaccess", &uaccess, "enable uaccess checking"),
 	OPT_BOOLEAN('s', "stats", &stats, "print statistics"),
+	OPT_BOOLEAN('d', "duplicate", &validate_dup, "duplicate validation for vmlinux.o"),
+	OPT_BOOLEAN('l', "vmlinux", &vmlinux, "vmlinux.o validation"),
 	OPT_END(),
 };
 
 int cmd_check(int argc, const char **argv)
 {
-	const char *objname;
+	const char *objname, *s;
 
 	argc = parse_options(argc, argv, check_options, check_usage, 0);
 
@@ -46,5 +49,9 @@ int cmd_check(int argc, const char **argv)
 
 	objname = argv[0];
 
+	s = strstr(objname, "vmlinux.o");
+	if (s && !s[9])
+		vmlinux = true;
+
 	return check(objname, false);
 }
tools/objtool/builtin.h +1 -1
index 0b90790..85c979c 100644
@@ -8,7 +8,7 @@
 #include <subcmd/parse-options.h>
 
 extern const struct option check_options[];
-extern bool no_fp, no_unreachable, retpoline, module, backtrace, uaccess, stats;
+extern bool no_fp, no_unreachable, retpoline, module, backtrace, uaccess, stats, validate_dup, vmlinux;
 
 extern int cmd_check(int argc, const char **argv);
 extern int cmd_orc(int argc, const char **argv);
tools/objtool/check.c +98 -0

152 lines not on this page · raw message

tools/objtool/check.h +3 -0
index 99413d4..12a9660 100644
@@ -18,6 +18,8 @@ struct insn_state {
 	unsigned int uaccess_stack;
 	bool uaccess;
 	bool df;
+	bool noinstr;
+	s8 instr;
 };
 
 struct instruction {
@@ -31,6 +33,7 @@ struct instruction {
 	bool alt_group, dead_end, ignore, ignore_alts;
 	bool hint;
 	bool retpoline_safe;
+	s8 instr;
 	u8 visited;
 	u8 ret_offset;
 	struct symbol *call_dest;
tools/objtool/elf.h +1 -1
index 0b79c23..eb79cb9 100644
@@ -39,7 +39,7 @@ struct section {
 	char *name;
 	int idx;
 	unsigned int len;
-	bool changed, text, rodata;
+	bool changed, text, rodata, noinstr;
 };
 
 struct symbol {
merge bot Applied
#40 lore raw

[tip: objtool/core] objtool: Implement noinstr validation

In reply to Peter Zijlstra
The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     c4a33939a7eb396acbb05569e57eebe4374cc57c
Gitweb:        https://git.kernel.org/tip/c4a33939a7eb396acbb05569e57eebe4374cc57c
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Tue, 10 Mar 2020 18:57:41 +01:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Wed, 22 Apr 2020 10:53:50 +02:00

objtool: Implement noinstr validation

Validate that any call out of .noinstr.text is in between instr_begin() and instr_end() annotations.

This annotation is useful to ensure correct behaviour wrt tracing sensitive code like entry/exit and idle code. When we run code in a sensitive context we want a guarantee no unknown code is ran.

Since this validation relies on knowing the section of call destination symbols, we must run it on vmlinux.o instead of on individual object files.

Add two options:

-d/--duplicate "duplicate validation for vmlinux" -l/--vmlinux "vmlinux.o validation"

Where the latter auto-detects when objname ends with "vmlinux.o" and the former will force all validations, also those already done on !vmlinux object files.

---
 tools/objtool/builtin-check.c | 11 +++-
 tools/objtool/builtin.h       |  2 +-
 tools/objtool/check.c         | 98 ++++++++++++++++++++++++++++++++++-
 tools/objtool/check.h         |  3 +-
 tools/objtool/elf.h           |  2 +-
 5 files changed, 112 insertions(+), 4 deletions(-)
tools/objtool/builtin-check.c +9 -2
index 10fbe75..be42b71 100644
@@ -14,10 +14,11 @@
  */
 
 #include <subcmd/parse-options.h>
+#include <string.h>
 #include "builtin.h"
 #include "check.h"
 
-bool no_fp, no_unreachable, retpoline, module, backtrace, uaccess, stats;
+bool no_fp, no_unreachable, retpoline, module, backtrace, uaccess, stats, validate_dup, vmlinux;
 
 static const char * const check_usage[] = {
 	"objtool check [<options>] file.o",
@@ -32,12 +33,14 @@ const struct option check_options[] = {
 	OPT_BOOLEAN('b', "backtrace", &backtrace, "unwind on error"),
 	OPT_BOOLEAN('a', "uaccess", &uaccess, "enable uaccess checking"),
 	OPT_BOOLEAN('s', "stats", &stats, "print statistics"),
+	OPT_BOOLEAN('d', "duplicate", &validate_dup, "duplicate validation for vmlinux.o"),
+	OPT_BOOLEAN('l', "vmlinux", &vmlinux, "vmlinux.o validation"),
 	OPT_END(),
 };
 
 int cmd_check(int argc, const char **argv)
 {
-	const char *objname;
+	const char *objname, *s;
 
 	argc = parse_options(argc, argv, check_options, check_usage, 0);
 
@@ -46,5 +49,9 @@ int cmd_check(int argc, const char **argv)
 
 	objname = argv[0];
 
+	s = strstr(objname, "vmlinux.o");
+	if (s && !s[9])
+		vmlinux = true;
+
 	return check(objname, false);
 }
tools/objtool/builtin.h +1 -1
index 0b90790..85c979c 100644
@@ -8,7 +8,7 @@
 #include <subcmd/parse-options.h>
 
 extern const struct option check_options[];
-extern bool no_fp, no_unreachable, retpoline, module, backtrace, uaccess, stats;
+extern bool no_fp, no_unreachable, retpoline, module, backtrace, uaccess, stats, validate_dup, vmlinux;
 
 extern int cmd_check(int argc, const char **argv);
 extern int cmd_orc(int argc, const char **argv);
tools/objtool/check.c +98 -0

152 lines not on this page · raw message

tools/objtool/check.h +3 -0
index 99413d4..12a9660 100644
@@ -18,6 +18,8 @@ struct insn_state {
 	unsigned int uaccess_stack;
 	bool uaccess;
 	bool df;
+	bool noinstr;
+	s8 instr;
 };
 
 struct instruction {
@@ -31,6 +33,7 @@ struct instruction {
 	bool alt_group, dead_end, ignore, ignore_alts;
 	bool hint;
 	bool retpoline_safe;
+	s8 instr;
 	u8 visited;
 	u8 ret_offset;
 	struct symbol *call_dest;
tools/objtool/elf.h +1 -1
index 0b79c23..eb79cb9 100644
@@ -39,7 +39,7 @@ struct section {
 	char *name;
 	int idx;
 	unsigned int len;
-	bool changed, text, rodata;
+	bool changed, text, rodata, noinstr;
 };
 
 struct symbol {
author maintainer
#41 lore raw

PATCH v5 11/17 objtool: Optimize !vmlinux.o again

In reply to Peter Zijlstra

When doing kbuild tests to see if the objtool changes affected those I found that there was a measurable regression:

pre post

  real    1m13.594        1m16.488s
  user    34m58.246s      35m23.947s
  sys     4m0.393s        4m27.312s

Perf showed that for small files the increased hash-table sizes were a measurable difference. Since we already have -l "vmlinux" to distinguish between the modes, make it also use a smaller portion of the hash-tables.

This flips it into a small win:

  real    1m14.143s
  user    34m49.292s
  sys     3m44.746s
---
 tools/objtool/elf.c     |   62 +++++++++++++++++++++++++++++++++---------------
 tools/objtool/elf.h     |   13 ++++++----
 tools/objtool/orc_gen.c |    3 --
 3 files changed, 52 insertions(+), 26 deletions(-)
tools/objtool/elf.c +43 -19
@@ -27,6 +27,22 @@ static inline u32 str_hash(const char *s
 	return jhash(str, strlen(str), 0);
 }
 
+static inline int elf_hash_bits(void)
+{
+	return vmlinux ? ELF_HASH_BITS : 16;
+}
+
+#define elf_hash_add(hashtable, node, key) \
+	hlist_add_head(node, &hashtable[hash_min(key, elf_hash_bits())])
+
+static void elf_hash_init(struct hlist_head *table)
+{
+	__hash_init(table, 1U << elf_hash_bits());
+}
+
+#define elf_hash_for_each_possible(name, obj, member, key)			\
+	hlist_for_each_entry(obj, &name[hash_min(key, elf_hash_bits())], member)
+
 static void rb_add(struct rb_root *tree, struct rb_node *node,
 		   int (*cmp)(struct rb_node *, const struct rb_node *))
 {
@@ -115,7 +131,7 @@ struct section *find_section_by_name(str
 {
 	struct section *sec;
 
-	hash_for_each_possible(elf->section_name_hash, sec, name_hash, str_hash(name))
+	elf_hash_for_each_possible(elf->section_name_hash, sec, name_hash, str_hash(name))
 		if (!strcmp(sec->name, name))
 			return sec;
 
@@ -127,7 +143,7 @@ static struct section *find_section_by_i
 {
 	struct section *sec;
 
-	hash_for_each_possible(elf->section_hash, sec, hash, idx)
+	elf_hash_for_each_possible(elf->section_hash, sec, hash, idx)
 		if (sec->idx == idx)
 			return sec;
 
@@ -138,7 +154,7 @@ static struct symbol *find_symbol_by_ind
 {
 	struct symbol *sym;
 
-	hash_for_each_possible(elf->symbol_hash, sym, hash, idx)
+	elf_hash_for_each_possible(elf->symbol_hash, sym, hash, idx)
 		if (sym->idx == idx)
 			return sym;
 
@@ -205,7 +221,7 @@ struct symbol *find_symbol_by_name(struc
 {
 	struct symbol *sym;
 
-	hash_for_each_possible(elf->symbol_name_hash, sym, name_hash, str_hash(name))
+	elf_hash_for_each_possible(elf->symbol_name_hash, sym, name_hash, str_hash(name))
 		if (!strcmp(sym->name, name))
 			return sym;
 
@@ -224,7 +240,7 @@ struct rela *find_rela_by_dest_range(str
 	sec = sec->rela;
 
 	for_offset_range(o, offset, offset + len) {
-		hash_for_each_possible(elf->rela_hash, rela, hash,
+		elf_hash_for_each_possible(elf->rela_hash, rela, hash,
 				       sec_offset_hash(sec, o)) {
 			if (rela->sec != sec)
 				continue;
@@ -309,8 +325,8 @@ static int read_sections(struct elf *elf
 		sec->len = sec->sh.sh_size;
 
 		list_add_tail(&sec->list, &elf->sections);
-		hash_add(elf->section_hash, &sec->hash, sec->idx);
-		hash_add(elf->section_name_hash, &sec->name_hash, str_hash(sec->name));
+		elf_hash_add(elf->section_hash, &sec->hash, sec->idx);
+		elf_hash_add(elf->section_name_hash, &sec->name_hash, str_hash(sec->name));
 	}
 
 	if (stats)
@@ -394,8 +410,8 @@ static int read_symbols(struct elf *elf)
 		else
 			entry = &sym->sec->symbol_list;
 		list_add(&sym->list, entry);
-		hash_add(elf->symbol_hash, &sym->hash, sym->idx);
-		hash_add(elf->symbol_name_hash, &sym->name_hash, str_hash(sym->name));
+		elf_hash_add(elf->symbol_hash, &sym->hash, sym->idx);
+		elf_hash_add(elf->symbol_name_hash, &sym->name_hash, str_hash(sym->name));
 	}
 
 	if (stats)
@@ -456,6 +472,14 @@ static int read_symbols(struct elf *elf)
 	return -1;
 }
 
+void elf_add_rela(struct elf *elf, struct rela *rela)
+{
+	struct section *sec = rela->sec;
+
+	list_add_tail(&rela->list, &sec->rela_list);
+	elf_hash_add(elf->rela_hash, &rela->hash, rela_hash(rela));
+}
+
 static int read_relas(struct elf *elf)
 {
 	struct section *sec;
@@ -503,8 +527,7 @@ static int read_relas(struct elf *elf)
 				return -1;
 			}
 
-			list_add_tail(&rela->list, &sec->rela_list);
-			hash_add(elf->rela_hash, &rela->hash, rela_hash(rela));
+			elf_add_rela(elf, rela);
 			nr_rela++;
 		}
 		max_rela = max(max_rela, nr_rela);
@@ -531,15 +554,16 @@ struct elf *elf_read(const char *name, i
 		perror("malloc");
 		return NULL;
 	}
-	memset(elf, 0, sizeof(*elf));
+	memset(elf, 0, offsetof(struct elf, sections));
 
-	hash_init(elf->symbol_hash);
-	hash_init(elf->symbol_name_hash);
-	hash_init(elf->section_hash);
-	hash_init(elf->section_name_hash);
-	hash_init(elf->rela_hash);
 	INIT_LIST_HEAD(&elf->sections);
 
+	elf_hash_init(elf->symbol_hash);
+	elf_hash_init(elf->symbol_name_hash);
+	elf_hash_init(elf->section_hash);
+	elf_hash_init(elf->section_name_hash);
+	elf_hash_init(elf->rela_hash);
+
 	elf->fd = open(name, flags);
 	if (elf->fd == -1) {
 		fprintf(stderr, "objtool: Can't open '%s': %s\n",
@@ -676,8 +700,8 @@ struct section *elf_create_section(struc
 	shstrtab->changed = true;
 
 	list_add_tail(&sec->list, &elf->sections);
-	hash_add(elf->section_hash, &sec->hash, sec->idx);
-	hash_add(elf->section_name_hash, &sec->name_hash, str_hash(sec->name));
+	elf_hash_add(elf->section_hash, &sec->hash, sec->idx);
+	elf_hash_add(elf->section_name_hash, &sec->name_hash, str_hash(sec->name));
 
 	return sec;
 }
tools/objtool/elf.h +8 -5
@@ -70,17 +70,19 @@ struct rela {
 	bool jump_table_start;
 };
 
+#define ELF_HASH_BITS	20
+
 struct elf {
 	Elf *elf;
 	GElf_Ehdr ehdr;
 	int fd;
 	char *name;
 	struct list_head sections;
-	DECLARE_HASHTABLE(symbol_hash, 20);
-	DECLARE_HASHTABLE(symbol_name_hash, 20);
-	DECLARE_HASHTABLE(section_hash, 16);
-	DECLARE_HASHTABLE(section_name_hash, 16);
-	DECLARE_HASHTABLE(rela_hash, 20);
+	DECLARE_HASHTABLE(symbol_hash, ELF_HASH_BITS);
+	DECLARE_HASHTABLE(symbol_name_hash, ELF_HASH_BITS);
+	DECLARE_HASHTABLE(section_hash, ELF_HASH_BITS);
+	DECLARE_HASHTABLE(section_name_hash, ELF_HASH_BITS);
+	DECLARE_HASHTABLE(rela_hash, ELF_HASH_BITS);
 };
 
 #define OFFSET_STRIDE_BITS	4
@@ -127,6 +129,7 @@ struct section *elf_create_rela_section(
 int elf_rebuild_rela_section(struct section *sec);
 int elf_write(struct elf *elf);
 void elf_close(struct elf *elf);
+void elf_add_rela(struct elf *elf, struct rela *rela);
 
 #define for_each_sec(file, sec)						\
 	list_for_each_entry(sec, &file->elf->sections, list)
tools/objtool/orc_gen.c +1 -2
@@ -111,8 +111,7 @@ static int create_orc_entry(struct elf *
 	rela->offset = idx * sizeof(int);
 	rela->sec = ip_relasec;
 
-	list_add_tail(&rela->list, &ip_relasec->rela_list);
-	hash_add(elf->rela_hash, &rela->hash, rela_hash(rela));
+	elf_add_rela(elf, rela);
 
 	return 0;
 }
merge bot Applied
#42 lore raw

[tip: objtool/core] objtool: Optimize !vmlinux.o again

The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     6700867cb19a52db07c4e9e86d51c6c8f01dcf8e
Gitweb:        https://git.kernel.org/tip/6700867cb19a52db07c4e9e86d51c6c8f01dcf8e
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Thu, 12 Mar 2020 14:29:38 +01:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Wed, 22 Apr 2020 23:10:07 +02:00

objtool: Optimize !vmlinux.o again

When doing kbuild tests to see if the objtool changes affected those I found that there was a measurable regression:

pre post

  real    1m13.594        1m16.488s
  user    34m58.246s      35m23.947s
  sys     4m0.393s        4m27.312s

Perf showed that for small files the increased hash-table sizes were a measurable difference. Since we already have -l "vmlinux" to distinguish between the modes, make it also use a smaller portion of the hash-tables.

This flips it into a small win:

  real    1m14.143s
  user    34m49.292s
  sys     3m44.746s
---
 tools/objtool/elf.c     | 62 +++++++++++++++++++++++++++-------------
 tools/objtool/elf.h     | 13 ++++----
 tools/objtool/orc_gen.c |  3 +--
 3 files changed, 52 insertions(+), 26 deletions(-)
tools/objtool/elf.c +43 -19
index 03e542d..d384b9e 100644
@@ -27,6 +27,22 @@ static inline u32 str_hash(const char *str)
 	return jhash(str, strlen(str), 0);
 }
 
+static inline int elf_hash_bits(void)
+{
+	return vmlinux ? ELF_HASH_BITS : 16;
+}
+
+#define elf_hash_add(hashtable, node, key) \
+	hlist_add_head(node, &hashtable[hash_min(key, elf_hash_bits())])
+
+static void elf_hash_init(struct hlist_head *table)
+{
+	__hash_init(table, 1U << elf_hash_bits());
+}
+
+#define elf_hash_for_each_possible(name, obj, member, key)			\
+	hlist_for_each_entry(obj, &name[hash_min(key, elf_hash_bits())], member)
+
 static void rb_add(struct rb_root *tree, struct rb_node *node,
 		   int (*cmp)(struct rb_node *, const struct rb_node *))
 {
@@ -115,7 +131,7 @@ struct section *find_section_by_name(struct elf *elf, const char *name)
 {
 	struct section *sec;
 
-	hash_for_each_possible(elf->section_name_hash, sec, name_hash, str_hash(name))
+	elf_hash_for_each_possible(elf->section_name_hash, sec, name_hash, str_hash(name))
 		if (!strcmp(sec->name, name))
 			return sec;
 
@@ -127,7 +143,7 @@ static struct section *find_section_by_index(struct elf *elf,
 {
 	struct section *sec;
 
-	hash_for_each_possible(elf->section_hash, sec, hash, idx)
+	elf_hash_for_each_possible(elf->section_hash, sec, hash, idx)
 		if (sec->idx == idx)
 			return sec;
 
@@ -138,7 +154,7 @@ static struct symbol *find_symbol_by_index(struct elf *elf, unsigned int idx)
 {
 	struct symbol *sym;
 
-	hash_for_each_possible(elf->symbol_hash, sym, hash, idx)
+	elf_hash_for_each_possible(elf->symbol_hash, sym, hash, idx)
 		if (sym->idx == idx)
 			return sym;
 
@@ -205,7 +221,7 @@ struct symbol *find_symbol_by_name(struct elf *elf, const char *name)
 {
 	struct symbol *sym;
 
-	hash_for_each_possible(elf->symbol_name_hash, sym, name_hash, str_hash(name))
+	elf_hash_for_each_possible(elf->symbol_name_hash, sym, name_hash, str_hash(name))
 		if (!strcmp(sym->name, name))
 			return sym;
 
@@ -224,7 +240,7 @@ struct rela *find_rela_by_dest_range(struct elf *elf, struct section *sec,
 	sec = sec->rela;
 
 	for_offset_range(o, offset, offset + len) {
-		hash_for_each_possible(elf->rela_hash, rela, hash,
+		elf_hash_for_each_possible(elf->rela_hash, rela, hash,
 				       sec_offset_hash(sec, o)) {
 			if (rela->sec != sec)
 				continue;
@@ -309,8 +325,8 @@ static int read_sections(struct elf *elf)
 		sec->len = sec->sh.sh_size;
 
 		list_add_tail(&sec->list, &elf->sections);
-		hash_add(elf->section_hash, &sec->hash, sec->idx);
-		hash_add(elf->section_name_hash, &sec->name_hash, str_hash(sec->name));
+		elf_hash_add(elf->section_hash, &sec->hash, sec->idx);
+		elf_hash_add(elf->section_name_hash, &sec->name_hash, str_hash(sec->name));
 	}
 
 	if (stats)
@@ -404,8 +420,8 @@ static int read_symbols(struct elf *elf)
 		else
 			entry = &sym->sec->symbol_list;
 		list_add(&sym->list, entry);
-		hash_add(elf->symbol_hash, &sym->hash, sym->idx);
-		hash_add(elf->symbol_name_hash, &sym->name_hash, str_hash(sym->name));
+		elf_hash_add(elf->symbol_hash, &sym->hash, sym->idx);
+		elf_hash_add(elf->symbol_name_hash, &sym->name_hash, str_hash(sym->name));
 	}
 
 	if (stats)
@@ -472,6 +488,14 @@ err:
 	return -1;
 }
 
+void elf_add_rela(struct elf *elf, struct rela *rela)
+{
+	struct section *sec = rela->sec;
+
+	list_add_tail(&rela->list, &sec->rela_list);
+	elf_hash_add(elf->rela_hash, &rela->hash, rela_hash(rela));
+}
+
 static int read_relas(struct elf *elf)
 {
 	struct section *sec;
@@ -519,8 +543,7 @@ static int read_relas(struct elf *elf)
 				return -1;
 			}
 
-			list_add_tail(&rela->list, &sec->rela_list);
-			hash_add(elf->rela_hash, &rela->hash, rela_hash(rela));
+			elf_add_rela(elf, rela);
 			nr_rela++;
 		}
 		max_rela = max(max_rela, nr_rela);
@@ -547,15 +570,16 @@ struct elf *elf_read(const char *name, int flags)
 		perror("malloc");
 		return NULL;
 	}
-	memset(elf, 0, sizeof(*elf));
+	memset(elf, 0, offsetof(struct elf, sections));
 
-	hash_init(elf->symbol_hash);
-	hash_init(elf->symbol_name_hash);
-	hash_init(elf->section_hash);
-	hash_init(elf->section_name_hash);
-	hash_init(elf->rela_hash);
 	INIT_LIST_HEAD(&elf->sections);
 
+	elf_hash_init(elf->symbol_hash);
+	elf_hash_init(elf->symbol_name_hash);
+	elf_hash_init(elf->section_hash);
+	elf_hash_init(elf->section_name_hash);
+	elf_hash_init(elf->rela_hash);
+
 	elf->fd = open(name, flags);
 	if (elf->fd == -1) {
 		fprintf(stderr, "objtool: Can't open '%s': %s\n",
@@ -692,8 +716,8 @@ struct section *elf_create_section(struct elf *elf, const char *name,
 	shstrtab->changed = true;
 
 	list_add_tail(&sec->list, &elf->sections);
-	hash_add(elf->section_hash, &sec->hash, sec->idx);
-	hash_add(elf->section_name_hash, &sec->name_hash, str_hash(sec->name));
+	elf_hash_add(elf->section_hash, &sec->hash, sec->idx);
+	elf_hash_add(elf->section_name_hash, &sec->name_hash, str_hash(sec->name));
 
 	return sec;
 }
tools/objtool/elf.h +8 -5
index eb79cb9..2811d04 100644
@@ -70,17 +70,19 @@ struct rela {
 	bool jump_table_start;
 };
 
+#define ELF_HASH_BITS	20
+
 struct elf {
 	Elf *elf;
 	GElf_Ehdr ehdr;
 	int fd;
 	char *name;
 	struct list_head sections;
-	DECLARE_HASHTABLE(symbol_hash, 20);
-	DECLARE_HASHTABLE(symbol_name_hash, 20);
-	DECLARE_HASHTABLE(section_hash, 16);
-	DECLARE_HASHTABLE(section_name_hash, 16);
-	DECLARE_HASHTABLE(rela_hash, 20);
+	DECLARE_HASHTABLE(symbol_hash, ELF_HASH_BITS);
+	DECLARE_HASHTABLE(symbol_name_hash, ELF_HASH_BITS);
+	DECLARE_HASHTABLE(section_hash, ELF_HASH_BITS);
+	DECLARE_HASHTABLE(section_name_hash, ELF_HASH_BITS);
+	DECLARE_HASHTABLE(rela_hash, ELF_HASH_BITS);
 };
 
 #define OFFSET_STRIDE_BITS	4
@@ -127,6 +129,7 @@ struct section *elf_create_rela_section(struct elf *elf, struct section *base);
 int elf_rebuild_rela_section(struct section *sec);
 int elf_write(struct elf *elf);
 void elf_close(struct elf *elf);
+void elf_add_rela(struct elf *elf, struct rela *rela);
 
 #define for_each_sec(file, sec)						\
 	list_for_each_entry(sec, &file->elf->sections, list)
tools/objtool/orc_gen.c +1 -2
index 2cf640f..9d2bf2d 100644
@@ -130,8 +130,7 @@ static int create_orc_entry(struct elf *elf, struct section *u_sec, struct secti
 	rela->offset = idx * sizeof(int);
 	rela->sec = ip_relasec;
 
-	list_add_tail(&rela->list, &ip_relasec->rela_list);
-	hash_add(elf->rela_hash, &rela->hash, rela_hash(rela));
+	elf_add_rela(elf, rela);
 
 	return 0;
 }
merge bot Applied
#43 lore raw

[tip: objtool/core] objtool: Optimize !vmlinux.o again

In reply to Peter Zijlstra
The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     34f7c96d96d5e11b03a612017fcc3a6e645bb481
Gitweb:        https://git.kernel.org/tip/34f7c96d96d5e11b03a612017fcc3a6e645bb481
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Thu, 12 Mar 2020 14:29:38 +01:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Wed, 22 Apr 2020 10:53:50 +02:00

objtool: Optimize !vmlinux.o again

When doing kbuild tests to see if the objtool changes affected those I found that there was a measurable regression:

pre post

  real    1m13.594        1m16.488s
  user    34m58.246s      35m23.947s
  sys     4m0.393s        4m27.312s

Perf showed that for small files the increased hash-table sizes were a measurable difference. Since we already have -l "vmlinux" to distinguish between the modes, make it also use a smaller portion of the hash-tables.

This flips it into a small win:

  real    1m14.143s
  user    34m49.292s
  sys     3m44.746s
---
 tools/objtool/elf.c     | 62 +++++++++++++++++++++++++++-------------
 tools/objtool/elf.h     | 13 ++++----
 tools/objtool/orc_gen.c |  3 +--
 3 files changed, 52 insertions(+), 26 deletions(-)
tools/objtool/elf.c +43 -19
index c4857fa..f26bb3e 100644
@@ -27,6 +27,22 @@ static inline u32 str_hash(const char *str)
 	return jhash(str, strlen(str), 0);
 }
 
+static inline int elf_hash_bits(void)
+{
+	return vmlinux ? ELF_HASH_BITS : 16;
+}
+
+#define elf_hash_add(hashtable, node, key) \
+	hlist_add_head(node, &hashtable[hash_min(key, elf_hash_bits())])
+
+static void elf_hash_init(struct hlist_head *table)
+{
+	__hash_init(table, 1U << elf_hash_bits());
+}
+
+#define elf_hash_for_each_possible(name, obj, member, key)			\
+	hlist_for_each_entry(obj, &name[hash_min(key, elf_hash_bits())], member)
+
 static void rb_add(struct rb_root *tree, struct rb_node *node,
 		   int (*cmp)(struct rb_node *, const struct rb_node *))
 {
@@ -115,7 +131,7 @@ struct section *find_section_by_name(struct elf *elf, const char *name)
 {
 	struct section *sec;
 
-	hash_for_each_possible(elf->section_name_hash, sec, name_hash, str_hash(name))
+	elf_hash_for_each_possible(elf->section_name_hash, sec, name_hash, str_hash(name))
 		if (!strcmp(sec->name, name))
 			return sec;
 
@@ -127,7 +143,7 @@ static struct section *find_section_by_index(struct elf *elf,
 {
 	struct section *sec;
 
-	hash_for_each_possible(elf->section_hash, sec, hash, idx)
+	elf_hash_for_each_possible(elf->section_hash, sec, hash, idx)
 		if (sec->idx == idx)
 			return sec;
 
@@ -138,7 +154,7 @@ static struct symbol *find_symbol_by_index(struct elf *elf, unsigned int idx)
 {
 	struct symbol *sym;
 
-	hash_for_each_possible(elf->symbol_hash, sym, hash, idx)
+	elf_hash_for_each_possible(elf->symbol_hash, sym, hash, idx)
 		if (sym->idx == idx)
 			return sym;
 
@@ -205,7 +221,7 @@ struct symbol *find_symbol_by_name(struct elf *elf, const char *name)
 {
 	struct symbol *sym;
 
-	hash_for_each_possible(elf->symbol_name_hash, sym, name_hash, str_hash(name))
+	elf_hash_for_each_possible(elf->symbol_name_hash, sym, name_hash, str_hash(name))
 		if (!strcmp(sym->name, name))
 			return sym;
 
@@ -224,7 +240,7 @@ struct rela *find_rela_by_dest_range(struct elf *elf, struct section *sec,
 	sec = sec->rela;
 
 	for_offset_range(o, offset, offset + len) {
-		hash_for_each_possible(elf->rela_hash, rela, hash,
+		elf_hash_for_each_possible(elf->rela_hash, rela, hash,
 				       sec_offset_hash(sec, o)) {
 			if (rela->sec != sec)
 				continue;
@@ -309,8 +325,8 @@ static int read_sections(struct elf *elf)
 		sec->len = sec->sh.sh_size;
 
 		list_add_tail(&sec->list, &elf->sections);
-		hash_add(elf->section_hash, &sec->hash, sec->idx);
-		hash_add(elf->section_name_hash, &sec->name_hash, str_hash(sec->name));
+		elf_hash_add(elf->section_hash, &sec->hash, sec->idx);
+		elf_hash_add(elf->section_name_hash, &sec->name_hash, str_hash(sec->name));
 	}
 
 	if (stats)
@@ -394,8 +410,8 @@ static int read_symbols(struct elf *elf)
 		else
 			entry = &sym->sec->symbol_list;
 		list_add(&sym->list, entry);
-		hash_add(elf->symbol_hash, &sym->hash, sym->idx);
-		hash_add(elf->symbol_name_hash, &sym->name_hash, str_hash(sym->name));
+		elf_hash_add(elf->symbol_hash, &sym->hash, sym->idx);
+		elf_hash_add(elf->symbol_name_hash, &sym->name_hash, str_hash(sym->name));
 	}
 
 	if (stats)
@@ -456,6 +472,14 @@ err:
 	return -1;
 }
 
+void elf_add_rela(struct elf *elf, struct rela *rela)
+{
+	struct section *sec = rela->sec;
+
+	list_add_tail(&rela->list, &sec->rela_list);
+	elf_hash_add(elf->rela_hash, &rela->hash, rela_hash(rela));
+}
+
 static int read_relas(struct elf *elf)
 {
 	struct section *sec;
@@ -503,8 +527,7 @@ static int read_relas(struct elf *elf)
 				return -1;
 			}
 
-			list_add_tail(&rela->list, &sec->rela_list);
-			hash_add(elf->rela_hash, &rela->hash, rela_hash(rela));
+			elf_add_rela(elf, rela);
 			nr_rela++;
 		}
 		max_rela = max(max_rela, nr_rela);
@@ -531,15 +554,16 @@ struct elf *elf_read(const char *name, int flags)
 		perror("malloc");
 		return NULL;
 	}
-	memset(elf, 0, sizeof(*elf));
+	memset(elf, 0, offsetof(struct elf, sections));
 
-	hash_init(elf->symbol_hash);
-	hash_init(elf->symbol_name_hash);
-	hash_init(elf->section_hash);
-	hash_init(elf->section_name_hash);
-	hash_init(elf->rela_hash);
 	INIT_LIST_HEAD(&elf->sections);
 
+	elf_hash_init(elf->symbol_hash);
+	elf_hash_init(elf->symbol_name_hash);
+	elf_hash_init(elf->section_hash);
+	elf_hash_init(elf->section_name_hash);
+	elf_hash_init(elf->rela_hash);
+
 	elf->fd = open(name, flags);
 	if (elf->fd == -1) {
 		fprintf(stderr, "objtool: Can't open '%s': %s\n",
@@ -676,8 +700,8 @@ struct section *elf_create_section(struct elf *elf, const char *name,
 	shstrtab->changed = true;
 
 	list_add_tail(&sec->list, &elf->sections);
-	hash_add(elf->section_hash, &sec->hash, sec->idx);
-	hash_add(elf->section_name_hash, &sec->name_hash, str_hash(sec->name));
+	elf_hash_add(elf->section_hash, &sec->hash, sec->idx);
+	elf_hash_add(elf->section_name_hash, &sec->name_hash, str_hash(sec->name));
 
 	return sec;
 }
tools/objtool/elf.h +8 -5
index eb79cb9..2811d04 100644
@@ -70,17 +70,19 @@ struct rela {
 	bool jump_table_start;
 };
 
+#define ELF_HASH_BITS	20
+
 struct elf {
 	Elf *elf;
 	GElf_Ehdr ehdr;
 	int fd;
 	char *name;
 	struct list_head sections;
-	DECLARE_HASHTABLE(symbol_hash, 20);
-	DECLARE_HASHTABLE(symbol_name_hash, 20);
-	DECLARE_HASHTABLE(section_hash, 16);
-	DECLARE_HASHTABLE(section_name_hash, 16);
-	DECLARE_HASHTABLE(rela_hash, 20);
+	DECLARE_HASHTABLE(symbol_hash, ELF_HASH_BITS);
+	DECLARE_HASHTABLE(symbol_name_hash, ELF_HASH_BITS);
+	DECLARE_HASHTABLE(section_hash, ELF_HASH_BITS);
+	DECLARE_HASHTABLE(section_name_hash, ELF_HASH_BITS);
+	DECLARE_HASHTABLE(rela_hash, ELF_HASH_BITS);
 };
 
 #define OFFSET_STRIDE_BITS	4
@@ -127,6 +129,7 @@ struct section *elf_create_rela_section(struct elf *elf, struct section *base);
 int elf_rebuild_rela_section(struct section *sec);
 int elf_write(struct elf *elf);
 void elf_close(struct elf *elf);
+void elf_add_rela(struct elf *elf, struct rela *rela);
 
 #define for_each_sec(file, sec)						\
 	list_for_each_entry(sec, &file->elf->sections, list)
tools/objtool/orc_gen.c +1 -2
index 2cf640f..9d2bf2d 100644
@@ -130,8 +130,7 @@ static int create_orc_entry(struct elf *elf, struct section *u_sec, struct secti
 	rela->offset = idx * sizeof(int);
 	rela->sec = ip_relasec;
 
-	list_add_tail(&rela->list, &ip_relasec->rela_list);
-	hash_add(elf->rela_hash, &rela->hash, rela_hash(rela));
+	elf_add_rela(elf, rela);
 
 	return 0;
 }
author maintainer
#44 lore raw

PATCH v5 12/17 objtool: Use sec_offset_hash() for insn_hash

In reply to Peter Zijlstra

In preparation for find_insn_containing(), change insn_hash to use sec_offset_hash().

This actually reduces runtime; probably because mixing in the section index reduces the collisions due to text sections all starting their instructions at offset 0.

Runtime on vmlinux.o from 3.1 to 2.5 seconds.

---
 tools/objtool/check.c |    5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
tools/objtool/check.c +3 -2
@@ -34,9 +34,10 @@ struct instruction *find_insn(struct obj
 {
 	struct instruction *insn;
 
-	hash_for_each_possible(file->insn_hash, insn, hash, offset)
+	hash_for_each_possible(file->insn_hash, insn, hash, sec_offset_hash(sec, offset)) {
 		if (insn->sec == sec && insn->offset == offset)
 			return insn;
+	}
 
 	return NULL;
 }
@@ -276,7 +277,7 @@ static int decode_instructions(struct ob
 			if (ret)
 				goto err;
 
-			hash_add(file->insn_hash, &insn->hash, insn->offset);
+			hash_add(file->insn_hash, &insn->hash, sec_offset_hash(sec, insn->offset));
 			list_add_tail(&insn->list, &file->insn_list);
 			nr_insns++;
 		}
merge bot Applied
#45 lore raw

[tip: objtool/core] objtool: Use sec_offset_hash() for insn_hash

The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     e499ccee47c429be75dee8a94d8f289aa133e3a0
Gitweb:        https://git.kernel.org/tip/e499ccee47c429be75dee8a94d8f289aa133e3a0
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Mon, 16 Mar 2020 15:47:27 +01:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Wed, 22 Apr 2020 23:10:07 +02:00

objtool: Use sec_offset_hash() for insn_hash

In preparation for find_insn_containing(), change insn_hash to use sec_offset_hash().

This actually reduces runtime; probably because mixing in the section index reduces the collisions due to text sections all starting their instructions at offset 0.

Runtime on vmlinux.o from 3.1 to 2.5 seconds.

---
 tools/objtool/check.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
tools/objtool/check.c +3 -2
index 39d1d4f..584f221 100644
@@ -34,9 +34,10 @@ struct instruction *find_insn(struct objtool_file *file,
 {
 	struct instruction *insn;
 
-	hash_for_each_possible(file->insn_hash, insn, hash, offset)
+	hash_for_each_possible(file->insn_hash, insn, hash, sec_offset_hash(sec, offset)) {
 		if (insn->sec == sec && insn->offset == offset)
 			return insn;
+	}
 
 	return NULL;
 }
@@ -282,7 +283,7 @@ static int decode_instructions(struct objtool_file *file)
 			if (ret)
 				goto err;
 
-			hash_add(file->insn_hash, &insn->hash, insn->offset);
+			hash_add(file->insn_hash, &insn->hash, sec_offset_hash(sec, insn->offset));
 			list_add_tail(&insn->list, &file->insn_list);
 			nr_insns++;
 		}
merge bot Applied
#46 lore raw

[tip: objtool/core] objtool: Use sec_offset_hash() for insn_hash

In reply to Peter Zijlstra
The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     87ecb582f0ac85886398dde8c3cdb2225cac7786
Gitweb:        https://git.kernel.org/tip/87ecb582f0ac85886398dde8c3cdb2225cac7786
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Mon, 16 Mar 2020 15:47:27 +01:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Wed, 22 Apr 2020 10:53:50 +02:00

objtool: Use sec_offset_hash() for insn_hash

In preparation for find_insn_containing(), change insn_hash to use sec_offset_hash().

This actually reduces runtime; probably because mixing in the section index reduces the collisions due to text sections all starting their instructions at offset 0.

Runtime on vmlinux.o from 3.1 to 2.5 seconds.

---
 tools/objtool/check.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
tools/objtool/check.c +3 -2
index 87e528c..923652b 100644
@@ -34,9 +34,10 @@ struct instruction *find_insn(struct objtool_file *file,
 {
 	struct instruction *insn;
 
-	hash_for_each_possible(file->insn_hash, insn, hash, offset)
+	hash_for_each_possible(file->insn_hash, insn, hash, sec_offset_hash(sec, offset)) {
 		if (insn->sec == sec && insn->offset == offset)
 			return insn;
+	}
 
 	return NULL;
 }
@@ -282,7 +283,7 @@ static int decode_instructions(struct objtool_file *file)
 			if (ret)
 				goto err;
 
-			hash_add(file->insn_hash, &insn->hash, insn->offset);
+			hash_add(file->insn_hash, &insn->hash, sec_offset_hash(sec, insn->offset));
 			list_add_tail(&insn->list, &file->insn_list);
 			nr_insns++;
 		}
author maintainer
#47 lore raw

PATCH v5 13/17 kbuild/objtool: Add objtool-vmlinux.o pass

In reply to Peter Zijlstra

Now that objtool is capable of processing vmlinux.o and actually has something useful to do there, (conditionally) add it to the final link pass.

This will increase build time by a few seconds.

---
 lib/Kconfig.debug       |    5 +++++
 scripts/link-vmlinux.sh |   24 ++++++++++++++++++++++++
 2 files changed, 29 insertions(+)
lib/Kconfig.debug +5 -0
@@ -379,6 +379,11 @@ config STACK_VALIDATION
 	  For more information, see
 	  tools/objtool/Documentation/stack-validation.txt.
 
+config VMLINUX_VALIDATION
+	bool
+	depends on STACK_VALIDATION && DEBUG_ENTRY && !PARAVIRT
+	default y
+
 config DEBUG_FORCE_WEAK_PER_CPU
 	bool "Force weak per-cpu definitions"
 	depends on DEBUG_KERNEL
scripts/link-vmlinux.sh +24 -0
@@ -55,6 +55,29 @@ modpost_link()
 	${LD} ${KBUILD_LDFLAGS} -r -o ${1} ${objects}
 }
 
+objtool_link()
+{
+	local objtoolopt;
+
+	if [ -n "${CONFIG_VMLINUX_VALIDATION}" ]; then
+		objtoolopt="check"
+		if [ -z "${CONFIG_FRAME_POINTER}" ]; then
+			objtoolopt="${objtoolopt} --no-fp"
+		fi
+		if [ -n "${CONFIG_GCOV_KERNEL}" ]; then
+			objtoolopt="${objtoolopt} --no-unreachable"
+		fi
+		if [ -n "${CONFIG_RETPOLINE}" ]; then
+			objtoolopt="${objtoolopt} --retpoline"
+		fi
+		if [ -n "${CONFIG_X86_SMAP}" ]; then
+			objtoolopt="${objtoolopt} --uaccess"
+		fi
+		info OBJTOOL ${1}
+		tools/objtool/objtool ${objtoolopt} ${1}
+	fi
+}
+
 # Link of vmlinux
 # ${1} - output file
 # ${2}, ${3}, ... - optional extra .o files
@@ -244,6 +267,7 @@ ${MAKE} -f "${srctree}/scripts/Makefile.
 #link vmlinux.o
 info LD vmlinux.o
 modpost_link vmlinux.o
+objtool_link vmlinux.o
 
 # modpost vmlinux.o to check for section mismatches
 ${MAKE} -f "${srctree}/scripts/Makefile.modpost" MODPOST_VMLINUX=1
merge bot Applied
#48 lore raw

[tip: objtool/core] kbuild/objtool: Add objtool-vmlinux.o pass

The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     1bd02c4f1a351989ee57a0735db1a0652238f991
Gitweb:        https://git.kernel.org/tip/1bd02c4f1a351989ee57a0735db1a0652238f991
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Wed, 18 Mar 2020 13:33:54 +01:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Wed, 22 Apr 2020 23:10:07 +02:00

kbuild/objtool: Add objtool-vmlinux.o pass

Now that objtool is capable of processing vmlinux.o and actually has something useful to do there, (conditionally) add it to the final link pass.

This will increase build time by a few seconds.

---
 lib/Kconfig.debug       |  5 +++++
 scripts/link-vmlinux.sh | 24 ++++++++++++++++++++++++
 2 files changed, 29 insertions(+)
lib/Kconfig.debug +5 -0
index 0b5cf05..3263b43 100644
@@ -369,6 +369,11 @@ config STACK_VALIDATION
 	  For more information, see
 	  tools/objtool/Documentation/stack-validation.txt.
 
+config VMLINUX_VALIDATION
+	bool
+	depends on STACK_VALIDATION && DEBUG_ENTRY && !PARAVIRT
+	default y
+
 config DEBUG_FORCE_WEAK_PER_CPU
 	bool "Force weak per-cpu definitions"
 	depends on DEBUG_KERNEL
scripts/link-vmlinux.sh +24 -0
index d09ab4a..3adef49 100755
@@ -55,6 +55,29 @@ modpost_link()
 	${LD} ${KBUILD_LDFLAGS} -r -o ${1} ${objects}
 }
 
+objtool_link()
+{
+	local objtoolopt;
+
+	if [ -n "${CONFIG_VMLINUX_VALIDATION}" ]; then
+		objtoolopt="check"
+		if [ -z "${CONFIG_FRAME_POINTER}" ]; then
+			objtoolopt="${objtoolopt} --no-fp"
+		fi
+		if [ -n "${CONFIG_GCOV_KERNEL}" ]; then
+			objtoolopt="${objtoolopt} --no-unreachable"
+		fi
+		if [ -n "${CONFIG_RETPOLINE}" ]; then
+			objtoolopt="${objtoolopt} --retpoline"
+		fi
+		if [ -n "${CONFIG_X86_SMAP}" ]; then
+			objtoolopt="${objtoolopt} --uaccess"
+		fi
+		info OBJTOOL ${1}
+		tools/objtool/objtool ${objtoolopt} ${1}
+	fi
+}
+
 # Link of vmlinux
 # ${1} - output file
 # ${2}, ${3}, ... - optional extra .o files
@@ -251,6 +274,7 @@ ${MAKE} -f "${srctree}/scripts/Makefile.build" obj=init need-builtin=1
 #link vmlinux.o
 info LD vmlinux.o
 modpost_link vmlinux.o
+objtool_link vmlinux.o
 
 # modpost vmlinux.o to check for section mismatches
 ${MAKE} -f "${srctree}/scripts/Makefile.modpost" MODPOST_VMLINUX=1
merge bot Applied
#49 lore raw

[tip: objtool/core] kbuild/objtool: Add objtool-vmlinux.o pass

In reply to Peter Zijlstra
The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     6804c1afd794c8135351faaa69e1503ee11393ac
Gitweb:        https://git.kernel.org/tip/6804c1afd794c8135351faaa69e1503ee11393ac
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Wed, 18 Mar 2020 13:33:54 +01:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Wed, 22 Apr 2020 10:53:51 +02:00

kbuild/objtool: Add objtool-vmlinux.o pass

Now that objtool is capable of processing vmlinux.o and actually has something useful to do there, (conditionally) add it to the final link pass.

This will increase build time by a few seconds.

---
 lib/Kconfig.debug       |  5 +++++
 scripts/link-vmlinux.sh | 24 ++++++++++++++++++++++++
 2 files changed, 29 insertions(+)
lib/Kconfig.debug +5 -0
index 21d9c5f..977b5a1 100644
@@ -369,6 +369,11 @@ config STACK_VALIDATION
 	  For more information, see
 	  tools/objtool/Documentation/stack-validation.txt.
 
+config VMLINUX_VALIDATION
+	bool
+	depends on STACK_VALIDATION && DEBUG_ENTRY && !PARAVIRT
+	default y
+
 config DEBUG_FORCE_WEAK_PER_CPU
 	bool "Force weak per-cpu definitions"
 	depends on DEBUG_KERNEL
scripts/link-vmlinux.sh +24 -0
index d09ab4a..3adef49 100755
@@ -55,6 +55,29 @@ modpost_link()
 	${LD} ${KBUILD_LDFLAGS} -r -o ${1} ${objects}
 }
 
+objtool_link()
+{
+	local objtoolopt;
+
+	if [ -n "${CONFIG_VMLINUX_VALIDATION}" ]; then
+		objtoolopt="check"
+		if [ -z "${CONFIG_FRAME_POINTER}" ]; then
+			objtoolopt="${objtoolopt} --no-fp"
+		fi
+		if [ -n "${CONFIG_GCOV_KERNEL}" ]; then
+			objtoolopt="${objtoolopt} --no-unreachable"
+		fi
+		if [ -n "${CONFIG_RETPOLINE}" ]; then
+			objtoolopt="${objtoolopt} --retpoline"
+		fi
+		if [ -n "${CONFIG_X86_SMAP}" ]; then
+			objtoolopt="${objtoolopt} --uaccess"
+		fi
+		info OBJTOOL ${1}
+		tools/objtool/objtool ${objtoolopt} ${1}
+	fi
+}
+
 # Link of vmlinux
 # ${1} - output file
 # ${2}, ${3}, ... - optional extra .o files
@@ -251,6 +274,7 @@ ${MAKE} -f "${srctree}/scripts/Makefile.build" obj=init need-builtin=1
 #link vmlinux.o
 info LD vmlinux.o
 modpost_link vmlinux.o
+objtool_link vmlinux.o
 
 # modpost vmlinux.o to check for section mismatches
 ${MAKE} -f "${srctree}/scripts/Makefile.modpost" MODPOST_VMLINUX=1
author maintainer
#50 lore raw

PATCH v5 14/17 objtool: Avoid iterating !text section symbols

In reply to Peter Zijlstra

validate_functions() iterates all sections their symbols; this is pointless to do for !text sections as they won't have instructions anyway.

---
 tools/objtool/check.c |    6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
tools/objtool/check.c +5 -1
@@ -2621,8 +2621,12 @@ static int validate_functions(struct obj
 	struct section *sec;
 	int warnings = 0;
 
-	for_each_sec(file, sec)
+	for_each_sec(file, sec) {
+		if (!(sec->sh.sh_flags & SHF_EXECINSTR))
+			continue;
+
 		warnings += validate_section(file, sec);
+	}
 
 	return warnings;
 }
merge bot Applied
#51 lore raw

[tip: objtool/core] objtool: Avoid iterating !text section symbols

The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     32cd779457c84c3d0ef34f9b45854f1c876395dc
Gitweb:        https://git.kernel.org/tip/32cd779457c84c3d0ef34f9b45854f1c876395dc
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Mon, 23 Mar 2020 21:11:14 +01:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Wed, 22 Apr 2020 23:10:08 +02:00

objtool: Avoid iterating !text section symbols

validate_functions() iterates all sections their symbols; this is pointless to do for !text sections as they won't have instructions anyway.

---
 tools/objtool/check.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
tools/objtool/check.c +5 -1
index 584f221..ef082a3 100644
@@ -2567,8 +2567,12 @@ static int validate_functions(struct objtool_file *file)
 	struct section *sec;
 	int warnings = 0;
 
-	for_each_sec(file, sec)
+	for_each_sec(file, sec) {
+		if (!(sec->sh.sh_flags & SHF_EXECINSTR))
+			continue;
+
 		warnings += validate_section(file, sec);
+	}
 
 	return warnings;
 }
merge bot Applied
#52 lore raw

[tip: objtool/core] objtool: Avoid iterating !text section symbols

In reply to Peter Zijlstra
The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     da837bd6f1994f780325649e8eee7d9b01c5ee4d
Gitweb:        https://git.kernel.org/tip/da837bd6f1994f780325649e8eee7d9b01c5ee4d
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Mon, 23 Mar 2020 21:11:14 +01:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Wed, 22 Apr 2020 10:53:51 +02:00

objtool: Avoid iterating !text section symbols

validate_functions() iterates all sections their symbols; this is pointless to do for !text sections as they won't have instructions anyway.

---
 tools/objtool/check.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
tools/objtool/check.c +5 -1
index 923652b..e201aa1 100644
@@ -2551,8 +2551,12 @@ static int validate_functions(struct objtool_file *file)
 	struct section *sec;
 	int warnings = 0;
 
-	for_each_sec(file, sec)
+	for_each_sec(file, sec) {
+		if (!(sec->sh.sh_flags & SHF_EXECINSTR))
+			continue;
+
 		warnings += validate_section(file, sec);
+	}
 
 	return warnings;
 }
author maintainer
#53 lore raw

PATCH v5 15/17 objtool: Rearrange validate_section()

In reply to Peter Zijlstra

In preparation of further changes, once again break out the loop body. No functional changes intended.

---
 tools/objtool/check.c |   51 ++++++++++++++++++++++++++++----------------------
 1 file changed, 29 insertions(+), 22 deletions(-)
tools/objtool/check.c +29 -22
@@ -2447,12 +2447,37 @@ static bool ignore_unreachable_insn(stru
 	return false;
 }
 
-static int validate_section(struct objtool_file *file, struct section *sec)
+static int validate_symbol(struct objtool_file *file, struct section *sec,
+			   struct symbol *sym, struct insn_state *state)
 {
-	struct symbol *func;
 	struct instruction *insn;
+	int ret;
+
+	if (!sym->len) {
+		WARN("%s() is missing an ELF size annotation", sym->name);
+		return 1;
+	}
+
+	if (sym->pfunc != sym || sym->alias != sym)
+		return 0;
+
+	insn = find_insn(file, sec, sym->offset);
+	if (!insn || insn->ignore || insn->visited)
+		return 0;
+
+	state->uaccess = sym->uaccess_safe;
+
+	ret = validate_branch(file, insn->func, insn, *state);
+	if (ret && backtrace)
+		BT_FUNC("<=== (sym)", insn);
+	return ret;
+}
+
+static int validate_section(struct objtool_file *file, struct section *sec)
+{
 	struct insn_state state;
-	int ret, warnings = 0;
+	struct symbol *func;
+	int warnings = 0;
 
 	clear_insn_state(&state);
 
@@ -2473,25 +2498,7 @@ static int validate_section(struct objto
 		if (func->type != STT_FUNC)
 			continue;
 
-		if (!func->len) {
-			WARN("%s() is missing an ELF size annotation",
-			     func->name);
-			warnings++;
-		}
-
-		if (func->pfunc != func || func->alias != func)
-			continue;
-
-		insn = find_insn(file, sec, func->offset);
-		if (!insn || insn->ignore || insn->visited)
-			continue;
-
-		state.uaccess = func->uaccess_safe;
-
-		ret = validate_branch(file, func, insn, state);
-		if (ret && backtrace)
-			BT_FUNC("<=== (func)", insn);
-		warnings += ret;
+		warnings += validate_symbol(file, sec, func, &state);
 	}
 
 	return warnings;
merge bot Applied
#54 lore raw

[tip: objtool/core] objtool: Rearrange validate_section()

The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     0e7f7f7c11588dd778f702da1405f0cc33cfea8c
Gitweb:        https://git.kernel.org/tip/0e7f7f7c11588dd778f702da1405f0cc33cfea8c
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Mon, 23 Mar 2020 21:17:50 +01:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Wed, 22 Apr 2020 23:10:08 +02:00

objtool: Rearrange validate_section()

In preparation of further changes, once again break out the loop body. No functional changes intended.

---
 tools/objtool/check.c | 51 +++++++++++++++++++++++-------------------
 1 file changed, 29 insertions(+), 22 deletions(-)
tools/objtool/check.c +29 -22
index ef082a3..1d455d6 100644
@@ -2502,12 +2502,37 @@ static bool ignore_unreachable_insn(struct instruction *insn)
 	return false;
 }
 
-static int validate_section(struct objtool_file *file, struct section *sec)
+static int validate_symbol(struct objtool_file *file, struct section *sec,
+			   struct symbol *sym, struct insn_state *state)
 {
-	struct symbol *func;
 	struct instruction *insn;
+	int ret;
+
+	if (!sym->len) {
+		WARN("%s() is missing an ELF size annotation", sym->name);
+		return 1;
+	}
+
+	if (sym->pfunc != sym || sym->alias != sym)
+		return 0;
+
+	insn = find_insn(file, sec, sym->offset);
+	if (!insn || insn->ignore || insn->visited)
+		return 0;
+
+	state->uaccess = sym->uaccess_safe;
+
+	ret = validate_branch(file, insn->func, insn, *state);
+	if (ret && backtrace)
+		BT_FUNC("<=== (sym)", insn);
+	return ret;
+}
+
+static int validate_section(struct objtool_file *file, struct section *sec)
+{
 	struct insn_state state;
-	int ret, warnings = 0;
+	struct symbol *func;
+	int warnings = 0;
 
 	/*
 	 * We need the full vmlinux for noinstr validation, otherwise we can
@@ -2521,31 +2546,13 @@ static int validate_section(struct objtool_file *file, struct section *sec)
 		if (func->type != STT_FUNC)
 			continue;
 
-		if (!func->len) {
-			WARN("%s() is missing an ELF size annotation",
-			     func->name);
-			warnings++;
-		}
-
-		if (func->pfunc != func || func->alias != func)
-			continue;
-
-		insn = find_insn(file, sec, func->offset);
-		if (!insn || insn->ignore || insn->visited)
-			continue;
-
 		clear_insn_state(&state);
 		state.cfi.cfa = initial_func_cfi.cfa;
 		memcpy(&state.cfi.regs, &initial_func_cfi.regs,
 		       CFI_NUM_REGS * sizeof(struct cfi_reg));
 		state.cfi.stack_size = initial_func_cfi.cfa.offset;
 
-		state.uaccess = func->uaccess_safe;
-
-		ret = validate_branch(file, func, insn, state);
-		if (ret && backtrace)
-			BT_FUNC("<=== (func)", insn);
-		warnings += ret;
+		warnings += validate_symbol(file, sec, func, &state);
 	}
 
 	return warnings;
merge bot Applied
#55 lore raw

[tip: objtool/core] objtool: Rearrange validate_section()

In reply to Peter Zijlstra
The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     4b5e2e7ffef87ae864f3f4546ee5753556e7550b
Gitweb:        https://git.kernel.org/tip/4b5e2e7ffef87ae864f3f4546ee5753556e7550b
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Mon, 23 Mar 2020 21:17:50 +01:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Wed, 22 Apr 2020 10:53:51 +02:00

objtool: Rearrange validate_section()

In preparation of further changes, once again break out the loop body. No functional changes intended.

---
 tools/objtool/check.c | 51 +++++++++++++++++++++++-------------------
 1 file changed, 29 insertions(+), 22 deletions(-)
tools/objtool/check.c +29 -22
index e201aa1..f49bf83 100644
@@ -2486,12 +2486,37 @@ static bool ignore_unreachable_insn(struct instruction *insn)
 	return false;
 }
 
-static int validate_section(struct objtool_file *file, struct section *sec)
+static int validate_symbol(struct objtool_file *file, struct section *sec,
+			   struct symbol *sym, struct insn_state *state)
 {
-	struct symbol *func;
 	struct instruction *insn;
+	int ret;
+
+	if (!sym->len) {
+		WARN("%s() is missing an ELF size annotation", sym->name);
+		return 1;
+	}
+
+	if (sym->pfunc != sym || sym->alias != sym)
+		return 0;
+
+	insn = find_insn(file, sec, sym->offset);
+	if (!insn || insn->ignore || insn->visited)
+		return 0;
+
+	state->uaccess = sym->uaccess_safe;
+
+	ret = validate_branch(file, insn->func, insn, *state);
+	if (ret && backtrace)
+		BT_FUNC("<=== (sym)", insn);
+	return ret;
+}
+
+static int validate_section(struct objtool_file *file, struct section *sec)
+{
 	struct insn_state state;
-	int ret, warnings = 0;
+	struct symbol *func;
+	int warnings = 0;
 
 	/*
 	 * We need the full vmlinux for noinstr validation, otherwise we can
@@ -2505,31 +2530,13 @@ static int validate_section(struct objtool_file *file, struct section *sec)
 		if (func->type != STT_FUNC)
 			continue;
 
-		if (!func->len) {
-			WARN("%s() is missing an ELF size annotation",
-			     func->name);
-			warnings++;
-		}
-
-		if (func->pfunc != func || func->alias != func)
-			continue;
-
-		insn = find_insn(file, sec, func->offset);
-		if (!insn || insn->ignore || insn->visited)
-			continue;
-
 		clear_insn_state(&state);
 		state.cfi.cfa = initial_func_cfi.cfa;
 		memcpy(&state.cfi.regs, &initial_func_cfi.regs,
 		       CFI_NUM_REGS * sizeof(struct cfi_reg));
 		state.cfi.stack_size = initial_func_cfi.cfa.offset;
 
-		state.uaccess = func->uaccess_safe;
-
-		ret = validate_branch(file, func, insn, state);
-		if (ret && backtrace)
-			BT_FUNC("<=== (func)", insn);
-		warnings += ret;
+		warnings += validate_symbol(file, sec, func, &state);
 	}
 
 	return warnings;
author maintainer
#56 lore raw

PATCH v5 16/17 objtool: Add STT_NOTYPE noinstr validation

In reply to Peter Zijlstra

Make sure to also check STT_NOTYPE symbols for noinstr violations.

---
 tools/objtool/check.c |   46 ++++++++++++++++++++++++++++++----------------
 1 file changed, 30 insertions(+), 16 deletions(-)
tools/objtool/check.c +30 -16
@@ -229,10 +229,18 @@ static void init_cfi_state(struct cfi_st
 	cfi->drap_offset = -1;
 }
 
-static void clear_insn_state(struct insn_state *state)
+static void init_insn_state(struct insn_state *state, struct section *sec)
 {
 	memset(state, 0, sizeof(*state));
 	init_cfi_state(&state->cfi);
+
+	/*
+	 * We need the full vmlinux for noinstr validation, otherwise we can
+	 * not correctly determine insn->call_dest->sec (external symbols do
+	 * not have a section).
+	 */
+	if (vmlinux && sec)
+		state->noinstr = sec->noinstr;
 }
 
 /*
@@ -2328,24 +2336,34 @@ static int validate_branch(struct objtoo
 	return 0;
 }
 
-static int validate_unwind_hints(struct objtool_file *file)
+static int validate_unwind_hints(struct objtool_file *file, struct section *sec)
 {
 	struct instruction *insn;
-	int ret, warnings = 0;
 	struct insn_state state;
+	int ret, warnings = 0;
 
 	if (!file->hints)
 		return 0;
 
-	clear_insn_state(&state);
+	init_insn_state(&state, sec);
 
-	for_each_insn(file, insn) {
+	if (sec) {
+		insn = find_insn(file, sec, 0);
+		if (!insn)
+			return 0;
+	} else {
+		insn = list_first_entry(&file->insn_list, typeof(*insn), list);
+	}
+
+	while (&insn->list != &file->insn_list && (!sec || insn->sec == sec)) {
 		if (insn->hint && !insn->visited) {
 			ret = validate_branch(file, insn->func, insn, state);
 			if (ret && backtrace)
 				BT_FUNC("<=== (hint)", insn);
 			warnings += ret;
 		}
+
+		insn = list_next_entry(insn, list);
 	}
 
 	return warnings;
@@ -2479,21 +2497,13 @@ static int validate_section(struct objto
 	struct symbol *func;
 	int warnings = 0;
 
-	clear_insn_state(&state);
+	init_insn_state(&state, sec);
 
 	state.cfi.cfa = initial_func_cfi.cfa;
 	memcpy(&state.cfi.regs, &initial_func_cfi.regs,
 	       CFI_NUM_REGS * sizeof(struct cfi_reg));
 	state.cfi.stack_size = initial_func_cfi.cfa.offset;
 
-	/*
-	 * We need the full vmlinux for noinstr validation, otherwise we can
-	 * not correctly determine insn->call_dest->sec (external symbols do
-	 * not have a section).
-	 */
-	if (vmlinux)
-		state.noinstr = sec->noinstr;
-
 	list_for_each_entry(func, &sec->symbol_list, list) {
 		if (func->type != STT_FUNC)
 			continue;
@@ -2507,12 +2517,16 @@ static int validate_section(struct objto
 static int validate_vmlinux_functions(struct objtool_file *file)
 {
 	struct section *sec;
+	int warnings = 0;
 
 	sec = find_section_by_name(file->elf, ".noinstr.text");
 	if (!sec)
 		return 0;
 
-	return validate_section(file, sec);
+	warnings += validate_section(file, sec);
+	warnings += validate_unwind_hints(file, sec);
+
+	return warnings;
 }
 
 static int validate_functions(struct objtool_file *file)
@@ -2597,7 +2611,7 @@ int check(const char *_objname, bool orc
 		goto out;
 	warnings += ret;
 
-	ret = validate_unwind_hints(&file);
+	ret = validate_unwind_hints(&file, NULL);
 	if (ret < 0)
 		goto out;
 	warnings += ret;
merge bot Applied
#57 lore raw

[tip: objtool/core] objtool: Add STT_NOTYPE noinstr validation

The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     d8ca70ffc3537f6fbc1c7e03ad80ff9ffe470076
Gitweb:        https://git.kernel.org/tip/d8ca70ffc3537f6fbc1c7e03ad80ff9ffe470076
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Mon, 23 Mar 2020 18:26:03 +01:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Wed, 22 Apr 2020 23:10:08 +02:00

objtool: Add STT_NOTYPE noinstr validation

Make sure to also check STT_NOTYPE symbols for noinstr violations.

---
 tools/objtool/check.c | 46 +++++++++++++++++++++++++++---------------
 1 file changed, 30 insertions(+), 16 deletions(-)
tools/objtool/check.c +30 -16
index 1d455d6..f51c325 100644
@@ -229,10 +229,18 @@ static void init_cfi_state(struct cfi_state *cfi)
 	cfi->drap_offset = -1;
 }
 
-static void clear_insn_state(struct insn_state *state)
+static void init_insn_state(struct insn_state *state, struct section *sec)
 {
 	memset(state, 0, sizeof(*state));
 	init_cfi_state(&state->cfi);
+
+	/*
+	 * We need the full vmlinux for noinstr validation, otherwise we can
+	 * not correctly determine insn->call_dest->sec (external symbols do
+	 * not have a section).
+	 */
+	if (vmlinux && sec)
+		state->noinstr = sec->noinstr;
 }
 
 /*
@@ -2370,24 +2378,34 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
 	return 0;
 }
 
-static int validate_unwind_hints(struct objtool_file *file)
+static int validate_unwind_hints(struct objtool_file *file, struct section *sec)
 {
 	struct instruction *insn;
-	int ret, warnings = 0;
 	struct insn_state state;
+	int ret, warnings = 0;
 
 	if (!file->hints)
 		return 0;
 
-	clear_insn_state(&state);
+	init_insn_state(&state, sec);
 
-	for_each_insn(file, insn) {
+	if (sec) {
+		insn = find_insn(file, sec, 0);
+		if (!insn)
+			return 0;
+	} else {
+		insn = list_first_entry(&file->insn_list, typeof(*insn), list);
+	}
+
+	while (&insn->list != &file->insn_list && (!sec || insn->sec == sec)) {
 		if (insn->hint && !insn->visited) {
 			ret = validate_branch(file, insn->func, insn, state);
 			if (ret && backtrace)
 				BT_FUNC("<=== (hint)", insn);
 			warnings += ret;
 		}
+
+		insn = list_next_entry(insn, list);
 	}
 
 	return warnings;
@@ -2534,19 +2552,11 @@ static int validate_section(struct objtool_file *file, struct section *sec)
 	struct symbol *func;
 	int warnings = 0;
 
-	/*
-	 * We need the full vmlinux for noinstr validation, otherwise we can
-	 * not correctly determine insn->call_dest->sec (external symbols do
-	 * not have a section).
-	 */
-	if (vmlinux)
-		state.noinstr = sec->noinstr;
-
 	list_for_each_entry(func, &sec->symbol_list, list) {
 		if (func->type != STT_FUNC)
 			continue;
 
-		clear_insn_state(&state);
+		init_insn_state(&state, sec);
 		state.cfi.cfa = initial_func_cfi.cfa;
 		memcpy(&state.cfi.regs, &initial_func_cfi.regs,
 		       CFI_NUM_REGS * sizeof(struct cfi_reg));
@@ -2561,12 +2571,16 @@ static int validate_section(struct objtool_file *file, struct section *sec)
 static int validate_vmlinux_functions(struct objtool_file *file)
 {
 	struct section *sec;
+	int warnings = 0;
 
 	sec = find_section_by_name(file->elf, ".noinstr.text");
 	if (!sec)
 		return 0;
 
-	return validate_section(file, sec);
+	warnings += validate_section(file, sec);
+	warnings += validate_unwind_hints(file, sec);
+
+	return warnings;
 }
 
 static int validate_functions(struct objtool_file *file)
@@ -2651,7 +2665,7 @@ int check(const char *_objname, bool orc)
 		goto out;
 	warnings += ret;
 
-	ret = validate_unwind_hints(&file);
+	ret = validate_unwind_hints(&file, NULL);
 	if (ret < 0)
 		goto out;
 	warnings += ret;
merge bot Applied
#58 lore raw

[tip: objtool/core] objtool: Add STT_NOTYPE noinstr validation

In reply to Peter Zijlstra
The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     932f8e987bfdcfc2365177978a30fdc0d6f6bd60
Gitweb:        https://git.kernel.org/tip/932f8e987bfdcfc2365177978a30fdc0d6f6bd60
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Mon, 23 Mar 2020 18:26:03 +01:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Wed, 22 Apr 2020 10:53:51 +02:00

objtool: Add STT_NOTYPE noinstr validation

Make sure to also check STT_NOTYPE symbols for noinstr violations.

---
 tools/objtool/check.c | 46 +++++++++++++++++++++++++++---------------
 1 file changed, 30 insertions(+), 16 deletions(-)
tools/objtool/check.c +30 -16
index f49bf83..0d9f9cf 100644
@@ -229,10 +229,18 @@ static void init_cfi_state(struct cfi_state *cfi)
 	cfi->drap_offset = -1;
 }
 
-static void clear_insn_state(struct insn_state *state)
+static void init_insn_state(struct insn_state *state, struct section *sec)
 {
 	memset(state, 0, sizeof(*state));
 	init_cfi_state(&state->cfi);
+
+	/*
+	 * We need the full vmlinux for noinstr validation, otherwise we can
+	 * not correctly determine insn->call_dest->sec (external symbols do
+	 * not have a section).
+	 */
+	if (vmlinux && sec)
+		state->noinstr = sec->noinstr;
 }
 
 /*
@@ -2354,24 +2362,34 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
 	return 0;
 }
 
-static int validate_unwind_hints(struct objtool_file *file)
+static int validate_unwind_hints(struct objtool_file *file, struct section *sec)
 {
 	struct instruction *insn;
-	int ret, warnings = 0;
 	struct insn_state state;
+	int ret, warnings = 0;
 
 	if (!file->hints)
 		return 0;
 
-	clear_insn_state(&state);
+	init_insn_state(&state, sec);
 
-	for_each_insn(file, insn) {
+	if (sec) {
+		insn = find_insn(file, sec, 0);
+		if (!insn)
+			return 0;
+	} else {
+		insn = list_first_entry(&file->insn_list, typeof(*insn), list);
+	}
+
+	while (&insn->list != &file->insn_list && (!sec || insn->sec == sec)) {
 		if (insn->hint && !insn->visited) {
 			ret = validate_branch(file, insn->func, insn, state);
 			if (ret && backtrace)
 				BT_FUNC("<=== (hint)", insn);
 			warnings += ret;
 		}
+
+		insn = list_next_entry(insn, list);
 	}
 
 	return warnings;
@@ -2518,19 +2536,11 @@ static int validate_section(struct objtool_file *file, struct section *sec)
 	struct symbol *func;
 	int warnings = 0;
 
-	/*
-	 * We need the full vmlinux for noinstr validation, otherwise we can
-	 * not correctly determine insn->call_dest->sec (external symbols do
-	 * not have a section).
-	 */
-	if (vmlinux)
-		state.noinstr = sec->noinstr;
-
 	list_for_each_entry(func, &sec->symbol_list, list) {
 		if (func->type != STT_FUNC)
 			continue;
 
-		clear_insn_state(&state);
+		init_insn_state(&state, sec);
 		state.cfi.cfa = initial_func_cfi.cfa;
 		memcpy(&state.cfi.regs, &initial_func_cfi.regs,
 		       CFI_NUM_REGS * sizeof(struct cfi_reg));
@@ -2545,12 +2555,16 @@ static int validate_section(struct objtool_file *file, struct section *sec)
 static int validate_vmlinux_functions(struct objtool_file *file)
 {
 	struct section *sec;
+	int warnings = 0;
 
 	sec = find_section_by_name(file->elf, ".noinstr.text");
 	if (!sec)
 		return 0;
 
-	return validate_section(file, sec);
+	warnings += validate_section(file, sec);
+	warnings += validate_unwind_hints(file, sec);
+
+	return warnings;
 }
 
 static int validate_functions(struct objtool_file *file)
@@ -2635,7 +2649,7 @@ int check(const char *_objname, bool orc)
 		goto out;
 	warnings += ret;
 
-	ret = validate_unwind_hints(&file);
+	ret = validate_unwind_hints(&file, NULL);
 	if (ret < 0)
 		goto out;
 	warnings += ret;
author maintainer
#59 lore raw

PATCH v5 17/17 objtool: Also consider .entry.text as noinstr

In reply to Peter Zijlstra

From: Thomas Gleixner <[email protected]>

Consider all of .entry.text as noinstr. This gets us coverage across the PTI boundary. While we could add everything .noinstr.text into .entry.text that would bloat the amount of code in the user mapping.

---
 tools/objtool/check.c |   18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)
tools/objtool/check.c +12 -6
@@ -285,7 +285,8 @@ static int decode_instructions(struct ob
 		    strncmp(sec->name, ".discard.", 9))
 			sec->text = true;
 
-		if (!strcmp(sec->name, ".noinstr.text"))
+		if (!strcmp(sec->name, ".noinstr.text") ||
+		    !strcmp(sec->name, ".entry.text"))
 			sec->noinstr = true;
 
 		for (offset = 0; offset < sec->len; offset += insn->len) {
@@ -2065,7 +2066,7 @@ static inline const char *call_dest_name
 static int validate_call(struct instruction *insn, struct insn_state *state)
 {
 	if (state->noinstr && state->instr <= 0 &&
-	    (!insn->call_dest || insn->call_dest->sec != insn->sec)) {
+	    (!insn->call_dest || !insn->call_dest->sec->noinstr)) {
 		WARN_FUNC("call to %s() leaves .noinstr.text section",
 				insn->sec, insn->offset, call_dest_name(insn));
 		return 1;
@@ -2636,11 +2637,16 @@ static int validate_vmlinux_functions(st
 	int warnings = 0;
 
 	sec = find_section_by_name(file->elf, ".noinstr.text");
-	if (!sec)
-		return 0;
+	if (sec) {
+		warnings += validate_section(file, sec);
+		warnings += validate_unwind_hints(file, sec);
+	}
 
-	warnings += validate_section(file, sec);
-	warnings += validate_unwind_hints(file, sec);
+	sec = find_section_by_name(file->elf, ".entry.text");
+	if (sec) {
+		warnings += validate_section(file, sec);
+		warnings += validate_unwind_hints(file, sec);
+	}
 
 	return warnings;
 }
#60 lore raw

[tip: objtool/core] objtool: Also consider .entry.text as noinstr

The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     b164b813e866b98387815eec40c6653a2edbc38a
Gitweb:        https://git.kernel.org/tip/b164b813e866b98387815eec40c6653a2edbc38a
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Wed, 25 Mar 2020 17:18:17 +01:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Wed, 22 Apr 2020 23:10:08 +02:00

objtool: Also consider .entry.text as noinstr

Consider all of .entry.text as noinstr. This gets us coverage across the PTI boundary. While we could add everything .noinstr.text into .entry.text that would bloat the amount of code in the user mapping.

---
 tools/objtool/check.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)
tools/objtool/check.c +12 -6
index f51c325..89b208c 100644
@@ -266,7 +266,8 @@ static int decode_instructions(struct objtool_file *file)
 		    strncmp(sec->name, ".discard.", 9))
 			sec->text = true;
 
-		if (!strcmp(sec->name, ".noinstr.text"))
+		if (!strcmp(sec->name, ".noinstr.text") ||
+		    !strcmp(sec->name, ".entry.text"))
 			sec->noinstr = true;
 
 		for (offset = 0; offset < sec->len; offset += insn->len) {
@@ -2087,7 +2088,7 @@ static inline const char *call_dest_name(struct instruction *insn)
 static int validate_call(struct instruction *insn, struct insn_state *state)
 {
 	if (state->noinstr && state->instr <= 0 &&
-	    (!insn->call_dest || insn->call_dest->sec != insn->sec)) {
+	    (!insn->call_dest || !insn->call_dest->sec->noinstr)) {
 		WARN_FUNC("call to %s() leaves .noinstr.text section",
 				insn->sec, insn->offset, call_dest_name(insn));
 		return 1;
@@ -2574,11 +2575,16 @@ static int validate_vmlinux_functions(struct objtool_file *file)
 	int warnings = 0;
 
 	sec = find_section_by_name(file->elf, ".noinstr.text");
-	if (!sec)
-		return 0;
+	if (sec) {
+		warnings += validate_section(file, sec);
+		warnings += validate_unwind_hints(file, sec);
+	}
 
-	warnings += validate_section(file, sec);
-	warnings += validate_unwind_hints(file, sec);
+	sec = find_section_by_name(file->elf, ".entry.text");
+	if (sec) {
+		warnings += validate_section(file, sec);
+		warnings += validate_unwind_hints(file, sec);
+	}
 
 	return warnings;
 }
#61 lore raw

[tip: objtool/core] objtool: Also consider .entry.text as noinstr

In reply to Peter Zijlstra
The following commit has been merged into the objtool/core branch of tip:

The following commit has been merged into the objtool/core branch of tip:

Commit-ID:     0cc9ac8db0b447922d9af77916cd7941fc784b64
Gitweb:        https://git.kernel.org/tip/0cc9ac8db0b447922d9af77916cd7941fc784b64
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Wed, 25 Mar 2020 17:18:17 +01:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Wed, 22 Apr 2020 10:53:51 +02:00

objtool: Also consider .entry.text as noinstr

Consider all of .entry.text as noinstr. This gets us coverage across the PTI boundary. While we could add everything .noinstr.text into .entry.text that would bloat the amount of code in the user mapping.

---
 tools/objtool/check.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)
tools/objtool/check.c +12 -6
index 0d9f9cf..f2a8427 100644
@@ -266,7 +266,8 @@ static int decode_instructions(struct objtool_file *file)
 		    strncmp(sec->name, ".discard.", 9))
 			sec->text = true;
 
-		if (!strcmp(sec->name, ".noinstr.text"))
+		if (!strcmp(sec->name, ".noinstr.text") ||
+		    !strcmp(sec->name, ".entry.text"))
 			sec->noinstr = true;
 
 		for (offset = 0; offset < sec->len; offset += insn->len) {
@@ -2071,7 +2072,7 @@ static inline const char *call_dest_name(struct instruction *insn)
 static int validate_call(struct instruction *insn, struct insn_state *state)
 {
 	if (state->noinstr && state->instr <= 0 &&
-	    (!insn->call_dest || insn->call_dest->sec != insn->sec)) {
+	    (!insn->call_dest || !insn->call_dest->sec->noinstr)) {
 		WARN_FUNC("call to %s() leaves .noinstr.text section",
 				insn->sec, insn->offset, call_dest_name(insn));
 		return 1;
@@ -2558,11 +2559,16 @@ static int validate_vmlinux_functions(struct objtool_file *file)
 	int warnings = 0;
 
 	sec = find_section_by_name(file->elf, ".noinstr.text");
-	if (!sec)
-		return 0;
+	if (sec) {
+		warnings += validate_section(file, sec);
+		warnings += validate_unwind_hints(file, sec);
+	}
 
-	warnings += validate_section(file, sec);
-	warnings += validate_unwind_hints(file, sec);
+	sec = find_section_by_name(file->elf, ".entry.text");
+	if (sec) {
+		warnings += validate_section(file, sec);
+		warnings += validate_unwind_hints(file, sec);
+	}
 
 	return warnings;
 }
Reviewed-by on the v5 series
#62 lore raw
In reply to Peter Zijlstra
Quoted 11 lines · Peter Zijlstra

As should be familiar by now; these patches implement the noinstr (no-instrument) validation in objtool as requested by Thomas, to ensure critical code (entry for now, idle later) run no unexpected code.

Functions are marked with: noinstr, which implies notrace, noinline and sticks things in the .noinstr.text section. Such functions can then use instr_begin() and instr_end() to allow calls to code outside of this section in sanctioned areas.

On the way of getting there, it also fixes the x86/ftrace trampiline ORC unwind.

I think I've already done that, but there are some changes, so

M

Reviewed-by on the v5 series
#63 lore raw
In reply to Peter Zijlstra
Quoted 4 lines · Peter Zijlstra

As should be familiar by now; these patches implement the noinstr (no-instrument) validation in objtool as requested by Thomas, to ensure critical code (entry for now, idle later) run no unexpected code.

Quoted 8 lines

Functions are marked with: noinstr, which implies notrace, noinline and sticks things in the .noinstr.text section. Such functions can then use instr_begin() and instr_end() to allow calls to code outside of this section in sanctioned areas.

On the way of getting there, it also fixes the x86/ftrace trampiline ORC unwind.

I have sent a few very minor comments; you can add my review for all patches:

alex.