Skip to contentSkip to conversation

perf/x86: Add perf text poke event

PATCH Superseded Adrian Hunter 2 participants 6 messages Dec 18, 2019 – Dec 19, 2019

Subsystems (2)

v1 of 7 Superseded Latest is v7 →

lore
Share & tools
lore

Adrian Hunter

author reviewer
#0 lore raw

cover perf/x86: Add perf text poke event

Hi

Here are patches to add a text poke event to record changes to kernel text (i.e. self-modifying code) in order to support tracers like Intel PT decoding through jump labels.

The first patch makes the kernel change and the subsequent patches are tools changes.

The second patch adds support for updating perf tools' data cache with the changed bytes.

The last patch is an Intel PT specific tools change.

The patches are based on linux-next.

Changes since RFC:

Dropped 'flags' from the new event. The consensus seemed to be that text pokes should emply a scheme similar to x86's INT3 method instead.

dropped tools patches already applied.

Example:

For jump labels, the kernel needs
CONFIG_JUMP_LABEL=y
and also an easy to flip jump label is in sysctl_schedstats() which needs
CONFIG_SCHEDSTATS=y
CONFIG_PROC_SYSCTL=y

Also note the 'sudo perf record' is put into the background which, as written, needs sudo credential caching (otherwise the background task will stop awaiting the sudo password), hence the 'sudo echo' to start.

Before:

  $ sudo echo
  $ sudo perf record -o perf.data.before --kcore -a -e intel_pt//k -m,64M &
  [1] 1640
  $ cat /proc/sys/kernel/sched_schedstats
  0
  $ sudo bash -c 'echo 1 > /proc/sys/kernel/sched_schedstats'
  $ cat /proc/sys/kernel/sched_schedstats
  1
  $ sudo bash -c 'echo 0 > /proc/sys/kernel/sched_schedstats'
  $ cat /proc/sys/kernel/sched_schedstats
  0
  $ sudo kill 1640
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 16.635 MB perf.data.before ]
  $ perf script -i perf.data.before --itrace=e >/dev/null
  Warning:
  1946 instruction trace errors

After:

  $ sudo echo
  $ sudo perf record -o perf.data.after --kcore -a -e intel_pt//k -m,64M &
  [1] 1882
  $ cat /proc/sys/kernel/sched_schedstats
  0
  $ sudo bash -c 'echo 1 > /proc/sys/kernel/sched_schedstats'
  $ cat /proc/sys/kernel/sched_schedstats
  1
  $ sudo bash -c 'echo 0 > /proc/sys/kernel/sched_schedstats'
  $ cat /proc/sys/kernel/sched_schedstats
  0
  $ sudo kill 1882
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 10.893 MB perf.data.after ]
  $ perf script -i perf.data.after --itrace=e
  $

Adrian Hunter (3):
perf/x86: Add perf text poke event
perf tools: Add support for PERF_RECORD_TEXT_POKE perf intel-pt: Add support for text poke events

 arch/x86/kernel/alternative.c             | 37 ++++++++++++-
 include/linux/perf_event.h                |  6 +++
 include/uapi/linux/perf_event.h           | 19 ++++++-
 kernel/events/core.c                      | 87 ++++++++++++++++++++++++++++++-
 tools/include/uapi/linux/perf_event.h     | 19 ++++++-
 tools/perf/arch/x86/util/intel-pt.c       |  4 ++
 tools/perf/builtin-record.c               | 45 ++++++++++++++++
 tools/perf/lib/include/perf/event.h       |  8 +++
 tools/perf/util/event.c                   | 39 ++++++++++++++
 tools/perf/util/event.h                   |  5 ++
 tools/perf/util/evlist.h                  |  1 +
 tools/perf/util/intel-pt.c                | 71 +++++++++++++++++++++++++
 tools/perf/util/machine.c                 | 34 ++++++++++++
 tools/perf/util/machine.h                 |  3 ++
 tools/perf/util/perf_event_attr_fprintf.c |  1 +
 tools/perf/util/record.c                  | 10 ++++
 tools/perf/util/record.h                  |  1 +
 tools/perf/util/session.c                 | 20 +++++++
 tools/perf/util/tool.h                    |  3 +-
 19 files changed, 408 insertions(+), 5 deletions(-)

Regards
Adrian

author reviewer
#1 lore raw

PATCH 1/3 perf/x86: Add perf text poke event

Record changes to kernel text (i.e. self-modifying code) in order to support tracers like Intel PT decoding through jump labels.

A copy of the running kernel code is needed as a reference point (e.g. from /proc/kcore). The text poke event records the old bytes and the new bytes so that the event can be processed forwards or backwards.

In the case of Intel PT tracing, the executable code must be walked to reconstruct the control flow. For x86 a jump label text poke begins:
- write INT3 byte
- IPI-SYNC
- write instruction tail
At this point the actual control flow will be through the INT3 and handler and not hit the old or new instruction. Intel PT outputs FUP/TIP packets for the INT3, so the flow can still be decoded. Subsequently:
- emit RECORD_TEXT_POKE with the new instruction
- IPI-SYNC
- write first byte
- IPI-SYNC
So before the text poke event timestamp, the decoder will see either the old instruction flow or FUP/TIP of INT3. After the text poke event timestamp, the decoder will see either the new instruction flow or FUP/TIP of INT3. Thus decoders can use the timestamp as the point at which to modify the executable code.

---
 arch/x86/kernel/alternative.c   | 37 +++++++++++++-
 include/linux/perf_event.h      |  6 +++
 include/uapi/linux/perf_event.h | 19 ++++++-
 kernel/events/core.c            | 87 ++++++++++++++++++++++++++++++++-
 4 files changed, 146 insertions(+), 3 deletions(-)
arch/x86/kernel/alternative.c +36 -1
index 30e86730655c..f932f8f4caf4 100644
@@ -3,6 +3,7 @@
 
 #include <linux/module.h>
 #include <linux/sched.h>
+#include <linux/perf_event.h>
 #include <linux/mutex.h>
 #include <linux/list.h>
 #include <linux/stringify.h>
@@ -946,6 +947,7 @@ struct text_poke_loc {
 	s32 rel32;
 	u8 opcode;
 	const u8 text[POKE_MAX_OPCODE_SIZE];
+	u8 old;
 };
 
 static struct bp_patching_desc {
@@ -1087,8 +1089,10 @@ static void text_poke_bp_batch(struct text_poke_loc *tp, unsigned int nr_entries
 	/*
 	 * First step: add a int3 trap to the address that will be patched.
 	 */
-	for (i = 0; i < nr_entries; i++)
+	for (i = 0; i < nr_entries; i++) {
+		tp[i].old = *(u8 *)text_poke_addr(&tp[i]);
 		text_poke(text_poke_addr(&tp[i]), &int3, INT3_INSN_SIZE);
+	}
 
 	text_poke_sync();
 
@@ -1096,14 +1100,45 @@ static void text_poke_bp_batch(struct text_poke_loc *tp, unsigned int nr_entries
 	 * Second step: update all but the first byte of the patched range.
 	 */
 	for (do_sync = 0, i = 0; i < nr_entries; i++) {
+		u8 old[POKE_MAX_OPCODE_SIZE] = { tp[i].old, };
 		int len = text_opcode_size(tp[i].opcode);
 
 		if (len - INT3_INSN_SIZE > 0) {
+			memcpy(old + INT3_INSN_SIZE,
+			       text_poke_addr(&tp[i]) + INT3_INSN_SIZE,
+			       len - INT3_INSN_SIZE);
 			text_poke(text_poke_addr(&tp[i]) + INT3_INSN_SIZE,
 				  (const char *)tp[i].text + INT3_INSN_SIZE,
 				  len - INT3_INSN_SIZE);
 			do_sync++;
 		}
+
+		/*
+		 * Emit a perf event to record the text poke, primarily to
+		 * support Intel PT decoding which must walk the executable code
+		 * to reconstruct the trace. The flow up to here is:
+		 *   - write INT3 byte
+		 *   - IPI-SYNC
+		 *   - write instruction tail
+		 * At this point the actual control flow will be through the
+		 * INT3 and handler and not hit the old or new instruction.
+		 * Intel PT outputs FUP/TIP packets for the INT3, so the flow
+		 * can still be decoded. Subsequently:
+		 *   - emit RECORD_TEXT_POKE with the new instruction
+		 *   - IPI-SYNC
+		 *   - write first byte
+		 *   - IPI-SYNC
+		 * So before the text poke event timestamp, the decoder will see
+		 * either the old instruction flow or FUP/TIP of INT3. After the
+		 * text poke event timestamp, the decoder will see either the
+		 * new instruction flow or FUP/TIP of INT3. Thus decoders can
+		 * use the timestamp as the point at which to modify the
+		 * executable code.
+		 * The old instruction is recorded so that the event can be
+		 * processed forwards or backwards.
+		 */
+		perf_event_text_poke((unsigned long)text_poke_addr(&tp[i]), old,
+				     tp[i].text, len);
 	}
 
 	if (do_sync) {
include/linux/perf_event.h +6 -0
index 6d4c22aee384..2cfc127e3534 100644
@@ -1212,6 +1212,8 @@ extern void perf_event_exec(void);
 extern void perf_event_comm(struct task_struct *tsk, bool exec);
 extern void perf_event_namespaces(struct task_struct *tsk);
 extern void perf_event_fork(struct task_struct *tsk);
+extern void perf_event_text_poke(unsigned long addr, const void *old_bytes,
+				 const void *new_bytes, size_t len);
 
 /* Callchains */
 DECLARE_PER_CPU(struct perf_callchain_entry, perf_callchain_entry);
@@ -1462,6 +1464,10 @@ static inline void perf_event_exec(void)				{ }
 static inline void perf_event_comm(struct task_struct *tsk, bool exec)	{ }
 static inline void perf_event_namespaces(struct task_struct *tsk)	{ }
 static inline void perf_event_fork(struct task_struct *tsk)		{ }
+static inline void perf_event_text_poke(unsigned long addr,
+					const void *old_bytes,
+					const void *new_bytes,
+					size_t len)			{ }
 static inline void perf_event_init(void)				{ }
 static inline int  perf_swevent_get_recursion_context(void)		{ return -1; }
 static inline void perf_swevent_put_recursion_context(int rctx)		{ }
include/uapi/linux/perf_event.h +18 -1
index 377d794d3105..d8422ebf128c 100644
@@ -377,7 +377,8 @@ struct perf_event_attr {
 				ksymbol        :  1, /* include ksymbol events */
 				bpf_event      :  1, /* include bpf events */
 				aux_output     :  1, /* generate AUX records instead of events */
-				__reserved_1   : 32;
+				text_poke      :  1, /* include text poke events */
+				__reserved_1   : 31;
 
 	union {
 		__u32		wakeup_events;	  /* wakeup every n events */
@@ -1006,6 +1007,22 @@ enum perf_event_type {
 	 */
 	PERF_RECORD_BPF_EVENT			= 18,
 
+	/*
+	 * Records changes to kernel text i.e. self-modified code.
+	 * 'len' is the number of old bytes, which is the same as the number
+	 * of new bytes. 'bytes' contains the old bytes followed immediately
+	 * by the new bytes.
+	 *
+	 * struct {
+	 *	struct perf_event_header	header;
+	 *	u64				addr;
+	 *	u16				len;
+	 *	u8				bytes[];
+	 *	struct sample_id		sample_id;
+	 * };
+	 */
+	PERF_RECORD_TEXT_POKE			= 19,
+
 	PERF_RECORD_MAX,			/* non-ABI */
 };
 
kernel/events/core.c +86 -1

117 lines not on this page · raw message

Signature · 1 line
2.17.1
maintainer
#2 lore raw
Quoted 2 lines · Adrian Hunter

Record changes to kernel text (i.e. self-modifying code) in order to support tracers like Intel PT decoding through jump labels.

I don't get the obsession with just jump-labels, we need a solution all modifying code. The fentry site usage is increasing, and optprobes are also fairly popular with a bunch of people.

Quoted 13 lines

A copy of the running kernel code is needed as a reference point (e.g. from /proc/kcore). The text poke event records the old bytes and the new bytes so that the event can be processed forwards or backwards.

In the case of Intel PT tracing, the executable code must be walked to reconstruct the control flow. For x86 a jump label text poke begins:
- write INT3 byte
- IPI-SYNC
- write instruction tail
At this point the actual control flow will be through the INT3 and handler and not hit the old or new instruction. Intel PT outputs FUP/TIP packets for the INT3, so the flow can still be decoded. Subsequently:
- emit RECORD_TEXT_POKE with the new instruction

Quoted 8 lines

- IPI-SYNC
- write first byte
- IPI-SYNC
So before the text poke event timestamp, the decoder will see either the old instruction flow or FUP/TIP of INT3. After the text poke event timestamp, the decoder will see either the new instruction flow or FUP/TIP of INT3. Thus decoders can use the timestamp as the point at which to modify the executable code.

I feel a much better justification for the design can be found in the discussion we've had around ARM-CS.

Basically SMP instruction coherency mandates something like this, it is just a happy accident x86 already had all the bits in place.

How is something like:

"Record (single instruction) changes to the kernel text (i.e. self-modifying code) in order to support tracers like Intel PT and ARM CoreSight.

A copy of the running kernel code is needed as a reference point (e.g. from /proc/kcore). The text poke event records the old bytes and the new bytes so that the event can be processed forwards or backwards.

The basic problem is recording the modified instruction in an unambiguous manner given SMP instruction cache (in)coherence. That is, when modifying an instruction concurrently any solution with one or multiple timestamps is not sufficient:

	CPU0				CPU1

0
1 write insn A
2 execute insn A
3 sync-I$
4

Due to I$, CPU1 might execute either the old or new A. No matter where we record tracepoints on CPU0, one simply cannot tell what CPU1 will have observed, except that at 0 it must be the old one and at 4 it must be the new one.

To solve this, take inspiration from x86 text poking, which has to solve this exact problem due to variable length instruction encoding and I-fetch windows.

1) overwrite the instruction with a breakpoint and sync I$

This guarantees that that code flow will never hit the target instruction anymore, on any CPU (or rather, it will cause an exception).

2) issue the TEXT_POKE event

3) overwrite the breakpoint with the new instruction and sync I$

Now we know that any execution after the TEXT_POKE event will either observe the breakpoint (and hit the exception) or the new instruction.

So by guarding the TEXT_POKE event with an exception on either side; we can now tell, without doubt, which instruction another CPU will have observed."

?

Quoted 3 lines
---
 arch/x86/kernel/alternative.c   | 37 +++++++++++++-

I'm thinking it might make sense to do this x86 part in a separate patch, and just present the generic thing first:

Quoted 11 lines
 include/linux/perf_event.h      |  6 +++
 include/uapi/linux/perf_event.h | 19 ++++++-
 kernel/events/core.c            | 87 ++++++++++++++++++++++++++++++++-
 4 files changed, 146 insertions(+), 3 deletions(-)
include/uapi/linux/perf_event.h +1 -0
@@ -1006,6 +1007,22 @@ enum perf_event_type {
 	 */
 	PERF_RECORD_BPF_EVENT			= 18,
 
+	/*

include/uapi/linux/perf_event.h enum perf_event_type { · line 1020

Quoted 10 lines
include/uapi/linux/perf_event.h +10 -0
+	 * Records changes to kernel text i.e. self-modified code.
+	 * 'len' is the number of old bytes, which is the same as the number
+	 * of new bytes. 'bytes' contains the old bytes followed immediately
+	 * by the new bytes.
+	 *
+	 * struct {
+	 *	struct perf_event_header	header;
+	 *	u64				addr;
+	 *	u16				len;
+	 *	u8				bytes[];

Would it make sense to have something like:

	 *	u16				old_len;
	 *	u16				new_len;
	 *	u8				old_bytes[old_len];
	 *	u8				new_bytes[new_len];

That would allow using this for (short) trampolines (ftrace, optprobes etc..). {old_len=0, new_len>0} would indicate a new trampoline, while {old_len>0, new_len=0} would indicate the dissapearance of a trampoline.

include/uapi/linux/perf_event.h enum perf_event_type { · line 1027

Quoted 7 lines
include/uapi/linux/perf_event.h +5 -0
+	 *	struct sample_id		sample_id;
+	 * };
+	 */
+	PERF_RECORD_TEXT_POKE			= 19,
+
 	PERF_RECORD_MAX,			/* non-ABI */
 };

Then the x86 patch, hooking up the event, can also cover kprobes and stuff.

author reviewer
#3 lore raw
Quoted 13 lines

-----Original Message-----
From: Peter Zijlstra <[email protected]>
Sent: Thursday, December 19, 2019 3:09 PM
To: Hunter, Adrian <[email protected]>
Cc: Ingo Molnar <[email protected]>; Borislav Petkov <[email protected]>; H . Peter Anvin <[email protected]>; [email protected]; Mark Rutland <[email protected]>; Alexander Shishkin <[email protected]>; Mathieu Poirier <[email protected]>; Leo Yan <[email protected]>; Arnaldo Carvalho de Melo <[email protected]>; Jiri Olsa <[email protected]>; linux- [email protected]
Subject: Re: [PATCH 1/3] perf/x86: Add perf text poke event

Quoted 7 lines
Quoted 2 lines · Adrian Hunter

Record changes to kernel text (i.e. self-modifying code) in order to support tracers like Intel PT decoding through jump labels.

I don't get the obsession with just jump-labels, we need a solution all modifying code. The fentry site usage is increasing, and optprobes are also fairly popular with a bunch of people.

Yes we need a solution for all modifying code. Jump labels are just a good place to start because they are the biggest problem by far.

Quoted 70 lines
Quoted 5 lines

A copy of the running kernel code is needed as a reference point (e.g. from /proc/kcore). The text poke event records the old bytes and the new bytes so that the event can be processed forwards or backwards.

Quoted 8 lines

In the case of Intel PT tracing, the executable code must be walked to reconstruct the control flow. For x86 a jump label text poke begins:
- write INT3 byte
- IPI-SYNC
- write instruction tail
At this point the actual control flow will be through the INT3 and handler and not hit the old or new instruction. Intel PT outputs FUP/TIP packets for the INT3, so the flow can still be decoded.

Subsequently:

Quoted 9 lines

- emit RECORD_TEXT_POKE with the new instruction
- IPI-SYNC
- write first byte
- IPI-SYNC
So before the text poke event timestamp, the decoder will see either the old instruction flow or FUP/TIP of INT3. After the text poke event timestamp, the decoder will see either the new instruction flow or FUP/TIP of INT3. Thus decoders can use the timestamp as the point at which to modify the executable code.

I feel a much better justification for the design can be found in the discussion we've had around ARM-CS.

Basically SMP instruction coherency mandates something like this, it is just a happy accident x86 already had all the bits in place.

How is something like:

"Record (single instruction) changes to the kernel text (i.e. self-modifying code) in order to support tracers like Intel PT and ARM CoreSight.

A copy of the running kernel code is needed as a reference point (e.g. from /proc/kcore). The text poke event records the old bytes and the new bytes so that the event can be processed forwards or backwards.

The basic problem is recording the modified instruction in an unambiguous manner given SMP instruction cache (in)coherence. That is, when modifying an instruction concurrently any solution with one or multiple timestamps is not sufficient:

	CPU0				CPU1

0
1 write insn A
2 execute insn A
3 sync-I$
4

Due to I$, CPU1 might execute either the old or new A. No matter where we record tracepoints on CPU0, one simply cannot tell what CPU1 will have observed, except that at 0 it must be the old one and at 4 it must be the new one.

To solve this, take inspiration from x86 text poking, which has to solve this exact problem due to variable length instruction encoding and I-fetch windows.

1) overwrite the instruction with a breakpoint and sync I$

This guarantees that that code flow will never hit the target instruction anymore, on any CPU (or rather, it will cause an exception).

2) issue the TEXT_POKE event

3) overwrite the breakpoint with the new instruction and sync I$

Quoted 8 lines

Now we know that any execution after the TEXT_POKE event will either observe the breakpoint (and hit the exception) or the new instruction.

So by guarding the TEXT_POKE event with an exception on either side; we can now tell, without doubt, which instruction another CPU will have observed."

?

Ok

Quoted 32 lines
Quoted 4 lines
---
 arch/x86/kernel/alternative.c   | 37 +++++++++++++-

I'm thinking it might make sense to do this x86 part in a separate patch, and just present the generic thing first:

Quoted 11 lines
 include/linux/perf_event.h      |  6 +++
 include/uapi/linux/perf_event.h | 19 ++++++-
 kernel/events/core.c            | 87 ++++++++++++++++++++++++++++++++-
 4 files changed, 146 insertions(+), 3 deletions(-)
include/uapi/linux/perf_event.h +1 -0
@@ -1006,6 +1007,22 @@ enum perf_event_type {
 	 */
 	PERF_RECORD_BPF_EVENT			= 18,
 
+	/*

include/uapi/linux/perf_event.h enum perf_event_type { · line 1020

Quoted 10 lines
include/uapi/linux/perf_event.h +10 -0
+	 * Records changes to kernel text i.e. self-modified code.
+	 * 'len' is the number of old bytes, which is the same as the number
+	 * of new bytes. 'bytes' contains the old bytes followed immediately
+	 * by the new bytes.
+	 *
+	 * struct {
+	 *	struct perf_event_header	header;
+	 *	u64				addr;
+	 *	u16				len;
+	 *	u8				bytes[];

Would it make sense to have something like:

Quoted 8 lines
	 *	u16				old_len;
	 *	u16				new_len;
	 *	u8				old_bytes[old_len];
	 *	u8				new_bytes[new_len];

That would allow using this for (short) trampolines (ftrace, optprobes etc..). {old_len=0, new_len>0} would indicate a new trampoline, while {old_len>0, new_len=0} would indicate the dissapearance of a trampoline.

Yes that makes sense.

Quoted 10 lines

include/uapi/linux/perf_event.h enum perf_event_type { · line 1027

Quoted 8 lines
include/uapi/linux/perf_event.h +5 -0
+	 *	struct sample_id		sample_id;
+	 * };
+	 */
+	PERF_RECORD_TEXT_POKE			= 19,
+
 	PERF_RECORD_MAX,			/* non-ABI */
 };

Then the x86 patch, hooking up the event, can also cover kprobes and stuff.

Ok

author reviewer
#4 lore raw

PATCH 2/3 perf tools: Add support for PERF_RECORD_TEXT_POKE

In reply to Adrian Hunter

Add processing for PERF_RECORD_TEXT_POKE events. When a text poke event is processed, then the kernel dso data cache is updated with the poked bytes.

---
 tools/include/uapi/linux/perf_event.h     | 19 +++++++++-
 tools/perf/builtin-record.c               | 45 +++++++++++++++++++++++
 tools/perf/lib/include/perf/event.h       |  8 ++++
 tools/perf/util/event.c                   | 39 ++++++++++++++++++++
 tools/perf/util/event.h                   |  5 +++
 tools/perf/util/evlist.h                  |  1 +
 tools/perf/util/machine.c                 | 34 +++++++++++++++++
 tools/perf/util/machine.h                 |  3 ++
 tools/perf/util/perf_event_attr_fprintf.c |  1 +
 tools/perf/util/record.c                  | 10 +++++
 tools/perf/util/record.h                  |  1 +
 tools/perf/util/session.c                 | 20 ++++++++++
 tools/perf/util/tool.h                    |  3 +-
 13 files changed, 187 insertions(+), 2 deletions(-)
tools/include/uapi/linux/perf_event.h +18 -1
index 377d794d3105..d8422ebf128c 100644
@@ -377,7 +377,8 @@ struct perf_event_attr {
 				ksymbol        :  1, /* include ksymbol events */
 				bpf_event      :  1, /* include bpf events */
 				aux_output     :  1, /* generate AUX records instead of events */
-				__reserved_1   : 32;
+				text_poke      :  1, /* include text poke events */
+				__reserved_1   : 31;
 
 	union {
 		__u32		wakeup_events;	  /* wakeup every n events */
@@ -1006,6 +1007,22 @@ enum perf_event_type {
 	 */
 	PERF_RECORD_BPF_EVENT			= 18,
 
+	/*
+	 * Records changes to kernel text i.e. self-modified code.
+	 * 'len' is the number of old bytes, which is the same as the number
+	 * of new bytes. 'bytes' contains the old bytes followed immediately
+	 * by the new bytes.
+	 *
+	 * struct {
+	 *	struct perf_event_header	header;
+	 *	u64				addr;
+	 *	u16				len;
+	 *	u8				bytes[];
+	 *	struct sample_id		sample_id;
+	 * };
+	 */
+	PERF_RECORD_TEXT_POKE			= 19,
+
 	PERF_RECORD_MAX,			/* non-ABI */
 };
 
tools/perf/builtin-record.c +45 -0

57 lines not on this page · raw message

tools/perf/lib/include/perf/event.h +8 -0
index 18106899cb4e..ecd2087e122e 100644
@@ -105,6 +105,13 @@ struct perf_record_bpf_event {
 	__u8			 tag[BPF_TAG_SIZE];  // prog tag
 };
 
+struct perf_record_text_poke_event {
+	struct perf_event_header header;
+	__u64			addr;
+	__u16			len;
+	__u8			bytes[];
+};
+
 struct perf_record_sample {
 	struct perf_event_header header;
 	__u64			 array[];
@@ -360,6 +367,7 @@ union perf_event {
 	struct perf_record_sample		sample;
 	struct perf_record_bpf_event		bpf;
 	struct perf_record_ksymbol		ksymbol;
+	struct perf_record_text_poke_event	text_poke;
 	struct perf_record_header_attr		attr;
 	struct perf_record_event_update		event_update;
 	struct perf_record_header_event_type	event_type;
tools/perf/util/event.c +39 -0
index c5447ff516a2..b0d934cdda31 100644
@@ -31,6 +31,7 @@
 #include "stat.h"
 #include "session.h"
 #include "bpf-event.h"
+#include "print_binary.h"
 #include "tool.h"
 #include "../perf.h"
 
@@ -54,6 +55,7 @@ static const char *perf_event__names[] = {
 	[PERF_RECORD_NAMESPACES]		= "NAMESPACES",
 	[PERF_RECORD_KSYMBOL]			= "KSYMBOL",
 	[PERF_RECORD_BPF_EVENT]			= "BPF_EVENT",
+	[PERF_RECORD_TEXT_POKE]			= "TEXT_POKE",
 	[PERF_RECORD_HEADER_ATTR]		= "ATTR",
 	[PERF_RECORD_HEADER_EVENT_TYPE]		= "EVENT_TYPE",
 	[PERF_RECORD_HEADER_TRACING_DATA]	= "TRACING_DATA",
@@ -252,6 +254,14 @@ int perf_event__process_bpf(struct perf_tool *tool __maybe_unused,
 	return machine__process_bpf(machine, event, sample);
 }
 
+int perf_event__process_text_poke(struct perf_tool *tool __maybe_unused,
+				  union perf_event *event,
+				  struct perf_sample *sample,
+				  struct machine *machine)
+{
+	return machine__process_text_poke(machine, event, sample);
+}
+
 size_t perf_event__fprintf_mmap(union perf_event *event, FILE *fp)
 {
 	return fprintf(fp, " %d/%d: [%#" PRI_lx64 "(%#" PRI_lx64 ") @ %#" PRI_lx64 "]: %c %s\n",
@@ -398,6 +408,32 @@ size_t perf_event__fprintf_bpf(union perf_event *event, FILE *fp)
 		       event->bpf.type, event->bpf.flags, event->bpf.id);
 }
 
+static int text_poke_printer(enum binary_printer_ops op, unsigned int val,
+			     void *extra __maybe_unused, FILE *fp)
+{
+	if (op == BINARY_PRINT_NUM_DATA)
+		return fprintf(fp, " %02x", val);
+	if (op == BINARY_PRINT_LINE_END)
+		return fprintf(fp, "\n");
+	return 0;
+}
+
+size_t perf_event__fprintf_text_poke(union perf_event *event, FILE *fp)
+{
+	size_t ret = fprintf(fp, " addr %#" PRI_lx64 " len %u\n",
+			     event->text_poke.addr,
+			     event->text_poke.len);
+
+	ret += fprintf(fp, "     old bytes:");
+	ret += binary__fprintf(event->text_poke.bytes, event->text_poke.len, 16,
+			       text_poke_printer, NULL, fp);
+	ret += fprintf(fp, "     new bytes:");
+	ret += binary__fprintf(event->text_poke.bytes + event->text_poke.len,
+			       event->text_poke.len, 16, text_poke_printer,
+			       NULL, fp);
+	return ret;
+}
+
 size_t perf_event__fprintf(union perf_event *event, FILE *fp)
 {
 	size_t ret = fprintf(fp, "PERF_RECORD_%s",
@@ -439,6 +475,9 @@ size_t perf_event__fprintf(union perf_event *event, FILE *fp)
 	case PERF_RECORD_BPF_EVENT:
 		ret += perf_event__fprintf_bpf(event, fp);
 		break;
+	case PERF_RECORD_TEXT_POKE:
+		ret += perf_event__fprintf_text_poke(event, fp);
+		break;
 	default:
 		ret += fprintf(fp, "\n");
 	}
tools/perf/util/event.h +5 -0
index 85223159737c..cbe98e22527f 100644
@@ -345,6 +345,10 @@ int perf_event__process_bpf(struct perf_tool *tool,
 			    union perf_event *event,
 			    struct perf_sample *sample,
 			    struct machine *machine);
+int perf_event__process_text_poke(struct perf_tool *tool,
+				  union perf_event *event,
+				  struct perf_sample *sample,
+				  struct machine *machine);
 int perf_event__process(struct perf_tool *tool,
 			union perf_event *event,
 			struct perf_sample *sample,
@@ -378,6 +382,7 @@ size_t perf_event__fprintf_cpu_map(union perf_event *event, FILE *fp);
 size_t perf_event__fprintf_namespaces(union perf_event *event, FILE *fp);
 size_t perf_event__fprintf_ksymbol(union perf_event *event, FILE *fp);
 size_t perf_event__fprintf_bpf(union perf_event *event, FILE *fp);
+size_t perf_event__fprintf_text_poke(union perf_event *event, FILE *fp);
 size_t perf_event__fprintf(union perf_event *event, FILE *fp);
 
 int kallsyms__get_function_start(const char *kallsyms_filename,
tools/perf/util/evlist.h +1 -0
index f5bd5c386df1..93e9b35e53b0 100644
@@ -175,6 +175,7 @@ struct callchain_param;
 void perf_evlist__set_id_pos(struct evlist *evlist);
 bool perf_can_sample_identifier(void);
 bool perf_can_record_switch_events(void);
+bool perf_can_record_text_poke_events(void);
 bool perf_can_record_cpu_wide(void);
 bool perf_can_aux_sample(void);
 void perf_evlist__config(struct evlist *evlist, struct record_opts *opts,
tools/perf/util/machine.c +34 -0
index c8c5410315e8..3449af7f6b81 100644
@@ -770,6 +770,38 @@ int machine__process_ksymbol(struct machine *machine __maybe_unused,
 	return machine__process_ksymbol_register(machine, event, sample);
 }
 
+int machine__process_text_poke(struct machine *machine, union perf_event *event,
+			       struct perf_sample *sample __maybe_unused)
+{
+	struct map *map = maps__find(&machine->kmaps, event->text_poke.addr);
+
+	if (dump_trace)
+		perf_event__fprintf_text_poke(event, stdout);
+
+	if (map) {
+		u8 *new_bytes = event->text_poke.bytes + event->text_poke.len;
+		int ret;
+
+		/*
+		 * Kernel maps might be changed when loading symbols so loading
+		 * must be done prior to using kernel maps.
+		 */
+		map__load(map);
+		ret = dso__data_write_cache_addr(map->dso, map, machine,
+						 event->text_poke.addr,
+						 new_bytes,
+						 event->text_poke.len);
+		if (ret != event->text_poke.len)
+			pr_err("Failed to write kernel text poke at %#" PRI_lx64 "\n",
+			       event->text_poke.addr);
+	} else {
+		pr_err("Failed to find kernel text poke address map for %#" PRI_lx64 "\n",
+		       event->text_poke.addr);
+	}
+
+	return 0;
+}
+
 static struct map *machine__addnew_module_map(struct machine *machine, u64 start,
 					      const char *filename)
 {
@@ -1901,6 +1933,8 @@ int machine__process_event(struct machine *machine, union perf_event *event,
 		ret = machine__process_ksymbol(machine, event, sample); break;
 	case PERF_RECORD_BPF_EVENT:
 		ret = machine__process_bpf(machine, event, sample); break;
+	case PERF_RECORD_TEXT_POKE:
+		ret = machine__process_text_poke(machine, event, sample); break;
 	default:
 		ret = -1;
 		break;
tools/perf/util/machine.h +3 -0
index be0a930eca89..0bb086acb7ed 100644
@@ -135,6 +135,9 @@ int machine__process_mmap2_event(struct machine *machine, union perf_event *even
 int machine__process_ksymbol(struct machine *machine,
 			     union perf_event *event,
 			     struct perf_sample *sample);
+int machine__process_text_poke(struct machine *machine,
+			       union perf_event *event,
+			       struct perf_sample *sample);
 int machine__process_event(struct machine *machine, union perf_event *event,
 				struct perf_sample *sample);
 
tools/perf/util/perf_event_attr_fprintf.c +1 -0
index 651203126c71..4c6c700335a6 100644
@@ -144,6 +144,7 @@ int perf_event_attr__fprintf(FILE *fp, struct perf_event_attr *attr,
 	PRINT_ATTRf(aux_watermark, p_unsigned);
 	PRINT_ATTRf(sample_max_stack, p_unsigned);
 	PRINT_ATTRf(aux_sample_size, p_unsigned);
+	PRINT_ATTRf(text_poke, p_unsigned);
 
 	return ret;
 }
tools/perf/util/record.c +10 -0
index 7def66168503..207ba2a65008 100644
@@ -97,6 +97,11 @@ static void perf_probe_context_switch(struct evsel *evsel)
 	evsel->core.attr.context_switch = 1;
 }
 
+static void perf_probe_text_poke(struct evsel *evsel)
+{
+	evsel->core.attr.text_poke = 1;
+}
+
 bool perf_can_sample_identifier(void)
 {
 	return perf_probe_api(perf_probe_sample_identifier);
@@ -112,6 +117,11 @@ bool perf_can_record_switch_events(void)
 	return perf_probe_api(perf_probe_context_switch);
 }
 
+bool perf_can_record_text_poke_events(void)
+{
+	return perf_probe_api(perf_probe_text_poke);
+}
+
 bool perf_can_record_cpu_wide(void)
 {
 	struct perf_event_attr attr = {
tools/perf/util/record.h +1 -0
index 5421fd2ad383..127b4ca11fd3 100644
@@ -46,6 +46,7 @@ struct record_opts {
 	bool	      sample_id;
 	bool	      no_bpf_event;
 	bool	      kcore;
+	bool	      text_poke;
 	unsigned int  freq;
 	unsigned int  mmap_pages;
 	unsigned int  auxtrace_mmap_pages;
tools/perf/util/session.c +20 -0
index d0d7d25b23e3..c0cd316d8e28 100644
@@ -489,6 +489,8 @@ void perf_tool__fill_defaults(struct perf_tool *tool)
 		tool->ksymbol = perf_event__process_ksymbol;
 	if (tool->bpf == NULL)
 		tool->bpf = perf_event__process_bpf;
+	if (tool->text_poke == NULL)
+		tool->text_poke = perf_event__process_text_poke;
 	if (tool->read == NULL)
 		tool->read = process_event_sample_stub;
 	if (tool->throttle == NULL)
@@ -658,6 +660,21 @@ static void perf_event__switch_swap(union perf_event *event, bool sample_id_all)
 		swap_sample_id_all(event, &event->context_switch + 1);
 }
 
+static void perf_event__text_poke_swap(union perf_event *event, bool sample_id_all)
+{
+	event->text_poke.addr  = bswap_64(event->text_poke.addr);
+	event->text_poke.len   = bswap_16(event->text_poke.len);
+
+	if (sample_id_all) {
+		size_t len = sizeof(event->text_poke.len) +
+			     event->text_poke.len * 2;
+		void *data = &event->text_poke.len;
+
+		data += PERF_ALIGN(len, sizeof(u64));
+		swap_sample_id_all(event, data);
+	}
+}
+
 static void perf_event__throttle_swap(union perf_event *event,
 				      bool sample_id_all)
 {
@@ -931,6 +948,7 @@ static perf_event__swap_op perf_event__swap_ops[] = {
 	[PERF_RECORD_SWITCH]		  = perf_event__switch_swap,
 	[PERF_RECORD_SWITCH_CPU_WIDE]	  = perf_event__switch_swap,
 	[PERF_RECORD_NAMESPACES]	  = perf_event__namespaces_swap,
+	[PERF_RECORD_TEXT_POKE]		  = perf_event__text_poke_swap,
 	[PERF_RECORD_HEADER_ATTR]	  = perf_event__hdr_attr_swap,
 	[PERF_RECORD_HEADER_EVENT_TYPE]	  = perf_event__event_type_swap,
 	[PERF_RECORD_HEADER_TRACING_DATA] = perf_event__tracing_data_swap,
@@ -1470,6 +1488,8 @@ static int machines__deliver_event(struct machines *machines,
 		return tool->ksymbol(tool, event, sample, machine);
 	case PERF_RECORD_BPF_EVENT:
 		return tool->bpf(tool, event, sample, machine);
+	case PERF_RECORD_TEXT_POKE:
+		return tool->text_poke(tool, event, sample, machine);
 	default:
 		++evlist->stats.nr_unknown_events;
 		return -1;
tools/perf/util/tool.h +2 -1
index 2abbf668b8de..006182923bbe 100644
@@ -56,7 +56,8 @@ struct perf_tool {
 			throttle,
 			unthrottle,
 			ksymbol,
-			bpf;
+			bpf,
+			text_poke;
 
 	event_attr_op	attr;
 	event_attr_op	event_update;
Signature · 1 line
2.17.1
author reviewer
#5 lore raw

PATCH 3/3 perf intel-pt: Add support for text poke events

In reply to Adrian Hunter

Select text poke events when available and the kernel is being traced. Process text poke events to invalidate entries in Intel PT's instruction cache.

---
 tools/perf/arch/x86/util/intel-pt.c |  4 ++
 tools/perf/util/intel-pt.c          | 71 +++++++++++++++++++++++++++++
 2 files changed, 75 insertions(+)
tools/perf/arch/x86/util/intel-pt.c +4 -0
index 20df442fdf36..ec75445ef7cf 100644
@@ -827,6 +827,10 @@ static int intel_pt_recording_options(struct auxtrace_record *itr,
 		}
 	}
 
+	if (have_timing_info && !intel_pt_evsel->core.attr.exclude_kernel &&
+	    perf_can_record_text_poke_events() && perf_can_record_cpu_wide())
+		opts->text_poke = true;
+
 	if (intel_pt_evsel) {
 		/*
 		 * To obtain the auxtrace buffer file descriptor, the auxtrace
tools/perf/util/intel-pt.c +71 -0

89 lines not on this page · raw message

Signature · 1 line
2.17.1