Skip to contentSkip to conversation

tools/lib/traceevent, tools/perf: Move struct tep_handler definition in a local header file

PATCH Applied Steven Rostedt 5 participants 12 messages Oct 5, 2018 – Aug 1, 2019 perf events

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 tools/lib/traceevent, tools/perf: Move struct tep_handler definition in a local header file

From: Tzvetomir Stoyanov <[email protected]>

As traceevent is going to be transferred into a proper library, its local data should be protected from the library users. This patch encapsulates struct tep_handler into a local header, not visible outside of the library. It implements also a bunch of new APIs, which library users can use to access tep_handler members.

---
 tools/lib/traceevent/Build               |   1 +
 tools/lib/traceevent/event-parse-api.c   | 275 +++++++++++++++++++++++++++++++
 tools/lib/traceevent/event-parse-local.h |  92 +++++++++++
 tools/lib/traceevent/event-parse.c       |   2 +
 tools/lib/traceevent/event-parse.h       | 228 ++++---------------------
 tools/lib/traceevent/event-plugin.c      |   1 +
 tools/lib/traceevent/parse-filter.c      |   1 +
 tools/perf/util/trace-event-parse.c      |  25 +--
 tools/perf/util/trace-event-read.c       |   2 +-
 9 files changed, 416 insertions(+), 211 deletions(-)
 create mode 100644 tools/lib/traceevent/event-parse-api.c
 create mode 100644 tools/lib/traceevent/event-parse-local.h
tools/lib/traceevent/Build +1 -0
index 0050c145d806..ba54bfce0b0b 100644
@@ -5,6 +5,7 @@ libtraceevent-y += parse-filter.o
 libtraceevent-y += parse-utils.o
 libtraceevent-y += kbuffer-parse.o
 libtraceevent-y += tep_strerror.o
+libtraceevent-y += event-parse-api.o
 
 plugin_jbd2-y         += plugin_jbd2.o
 plugin_hrtimer-y      += plugin_hrtimer.o
tools/lib/traceevent/event-parse-api.c new file +275 -0

275 lines not on this page · raw message

tools/lib/traceevent/event-parse-local.h new file +92 -0

92 lines not on this page · raw message

tools/lib/traceevent/event-parse.c +2 -0
index 233179a712d6..3692f29fee46 100644
@@ -22,6 +22,8 @@
 
 #include <netinet/in.h>
 #include "event-parse.h"
+
+#include "event-parse-local.h"
 #include "event-utils.h"
 #include "trace-seq.h"
 
tools/lib/traceevent/event-parse.h +28 -200

253 lines not on this page · raw message

tools/lib/traceevent/event-plugin.c +1 -0
index 46eb64eb0c2e..e74f16c88398 100644
@@ -14,6 +14,7 @@
 #include <unistd.h>
 #include <dirent.h>
 #include "event-parse.h"
+#include "event-parse-local.h"
 #include "event-utils.h"
 #include "trace-seq.h"
 
tools/lib/traceevent/parse-filter.c +1 -0
index d64b6128fa7d..ed87cb56713d 100644
@@ -11,6 +11,7 @@
 #include <sys/types.h>
 
 #include "event-parse.h"
+#include "event-parse-local.h"
 #include "event-utils.h"
 
 #define COMM "COMM"
tools/perf/util/trace-event-parse.c +15 -10
index a4d7de1c96d1..9b78160e23a2 100644
@@ -37,10 +37,11 @@ static int get_common_field(struct scripting_context *context,
 	struct tep_format_field *field;
 
 	if (!*size) {
-		if (!pevent->events)
+
+		event = tep_get_first_event(pevent);
+		if (!event)
 			return 0;
 
-		event = pevent->events[0];
 		field = tep_find_common_field(event, type);
 		if (!field)
 			return 0;
@@ -192,25 +193,29 @@ struct tep_event_format *trace_find_next_event(struct tep_handle *pevent,
 					       struct tep_event_format *event)
 {
 	static int idx;
+	int events_count;
+	struct tep_event_format *all_events;
 
-	if (!pevent || !pevent->events)
+	all_events = tep_get_first_event(pevent);
+	events_count = tep_get_events_count(pevent);
+	if (!pevent || !all_events || events_count < 1)
 		return NULL;
 
 	if (!event) {
 		idx = 0;
-		return pevent->events[0];
+		return all_events;
 	}
 
-	if (idx < pevent->nr_events && event == pevent->events[idx]) {
+	if (idx < events_count && event == (all_events + idx)) {
 		idx++;
-		if (idx == pevent->nr_events)
+		if (idx == events_count)
 			return NULL;
-		return pevent->events[idx];
+		return (all_events + idx);
 	}
 
-	for (idx = 1; idx < pevent->nr_events; idx++) {
-		if (event == pevent->events[idx - 1])
-			return pevent->events[idx];
+	for (idx = 1; idx < events_count; idx++) {
+		if (event == (all_events + (idx - 1)))
+			return (all_events + idx);
 	}
 	return NULL;
 }
tools/perf/util/trace-event-read.c +1 -1
index b98ee2a2eb44..cdf6de82a507 100644
@@ -241,7 +241,7 @@ static int read_header_files(struct tep_handle *pevent)
 		 * The commit field in the page is of type long,
 		 * use that instead, since it represents the kernel.
 		 */
-		tep_set_long_size(pevent, pevent->header_page_size_size);
+		tep_set_long_size(pevent, tep_get_header_page_size(pevent));
 	}
 	free(header_page);
 
Signature · 1 line
2.13.6
#1 lore raw
Quoted 5 lines · On Fri, 5 Oct 2018 12:22:25 -0400 Steven Rostedt <[email protected]> wrote:
tools/lib/traceevent/event-parse-api.c new file +5 -0
+/**
+ * tep_get_first_event - returns the first event in the events array
+ * @tep: a handle to the tep_handle
+ *
+ * This returns pointer to the first element of the events array

tools/lib/traceevent/event-parse-api.c line 25

Quoted 10 lines
tools/lib/traceevent/event-parse-api.c new file +10 -0
+ * If @tep is NULL, NULL is returned.
+ */
+struct tep_event_format *tep_get_first_event(struct tep_handle *tep)
+{
+	if(tep && tep->events)
+		return tep->events[0];
+
+	return NULL;
+}
+

Hold off on pulling this in. This patch has some whitespace issues that need to be cleaned up.

Thanks!

-- Steve

#2 lore raw

Em Fri, Oct 05, 2018 at 12:28:46PM -0400, Steven Rostedt escreveu:

Quoted 13 lines
Quoted 10 lines · On Fri, 5 Oct 2018 12:22:25 -0400 Steven Rostedt <[email protected]> wrote:
tools/lib/traceevent/event-parse-api.c new file +10 -0
+/**
+ * tep_get_first_event - returns the first event in the events array
+ * @tep: a handle to the tep_handle
+ *
+ * This returns pointer to the first element of the events array
+ * If @tep is NULL, NULL is returned.
+ */
+struct tep_event_format *tep_get_first_event(struct tep_handle *tep)
+{
+	if(tep && tep->events)
Quoted 8 lines

tools/lib/traceevent/event-parse-api.c line 25

Quoted 5 lines
tools/lib/traceevent/event-parse-api.c new file +5 -0
+		return tep->events[0];
+
+	return NULL;
+}
+

Hold off on pulling this in. This patch has some whitespace issues that need to be cleaned up.

Ah, that if( thing? ?I can fix it here, holler if there is more than that.

#3 lore raw
Quoted 23 lines · On Mon, 8 Oct 2018 14:32:24 -0300 Arnaldo Carvalho de Melo <[email protected]>…

Em Fri, Oct 05, 2018 at 12:28:46PM -0400, Steven Rostedt escreveu:

Quoted 13 lines
Quoted 10 lines · On Fri, 5 Oct 2018 12:22:25 -0400 Steven Rostedt <[email protected]> wrote:
(quoted diff) +10 -0
+/**
+ * tep_get_first_event - returns the first event in the events array
+ * @tep: a handle to the tep_handle
+ *
+ * This returns pointer to the first element of the events array
+ * If @tep is NULL, NULL is returned.
+ */
+struct tep_event_format *tep_get_first_event(struct tep_handle *tep)
+{
+	if(tep && tep->events)
Quoted 8 lines

Review

Quoted 5 lines
(quoted diff) +5 -0
+		return tep->events[0];
+
+	return NULL;
+}
+  

Hold off on pulling this in. This patch has some whitespace issues that need to be cleaned up.

Quoted 2 lines

Ah, that if( thing? ?I can fix it here, holler if there is more than that.

Well, here's a fixed up version anyway...

-- Steve

From 96dd7937a31d3a715c450c9a67f62a0022cd6a7c Mon Sep 17 00:00:00 2001
From: Tzvetomir Stoyanov <[email protected]>
Date: Mon, 8 Oct 2018 15:55:00 +0300
Subject: [PATCH] tools/lib/traceevent, tools/perf: Move struct tep_handler
definition in a local header file

As traceevent is going to be transferred into a proper library, its local data should be protected from the library users. This patch capsulates struct tep_handler into a local header, not visible outside of the library. It implements also a bunch of new APIs, which library users can use to access tep_handler members. Version 3 of the patch - fixed whitespace issues.

---
 tools/lib/traceevent/Build               |   1 +
 tools/lib/traceevent/event-parse-api.c   | 275 +++++++++++++++++++++++++++++++
 tools/lib/traceevent/event-parse-local.h |  92 +++++++++++
 tools/lib/traceevent/event-parse.c       |   2 +
 tools/lib/traceevent/event-parse.h       | 230 ++++----------------------
 tools/lib/traceevent/event-plugin.c      |   1 +
 tools/lib/traceevent/parse-filter.c      |   1 +
 tools/perf/util/trace-event-parse.c      |  25 +--
 tools/perf/util/trace-event-read.c       |   2 +-
 9 files changed, 417 insertions(+), 212 deletions(-)
 create mode 100644 tools/lib/traceevent/event-parse-api.c
 create mode 100644 tools/lib/traceevent/event-parse-local.h
tools/lib/traceevent/Build +1 -0
index 0050c145d806..ba54bfce0b0b 100644
@@ -5,6 +5,7 @@ libtraceevent-y += parse-filter.o
 libtraceevent-y += parse-utils.o
 libtraceevent-y += kbuffer-parse.o
 libtraceevent-y += tep_strerror.o
+libtraceevent-y += event-parse-api.o
 
 plugin_jbd2-y         += plugin_jbd2.o
 plugin_hrtimer-y      += plugin_hrtimer.o
tools/lib/traceevent/event-parse-api.c new file +275 -0

275 lines not on this page · raw message

tools/lib/traceevent/event-parse-local.h new file +92 -0

92 lines not on this page · raw message

tools/lib/traceevent/event-parse.c +2 -0
index 233179a712d6..3692f29fee46 100644
@@ -22,6 +22,8 @@
 
 #include <netinet/in.h>
 #include "event-parse.h"
+
+#include "event-parse-local.h"
 #include "event-utils.h"
 #include "trace-seq.h"
 
tools/lib/traceevent/event-parse.h +29 -201

259 lines not on this page · raw message

tools/lib/traceevent/event-plugin.c +1 -0
index 46eb64eb0c2e..e74f16c88398 100644
@@ -14,6 +14,7 @@
 #include <unistd.h>
 #include <dirent.h>
 #include "event-parse.h"
+#include "event-parse-local.h"
 #include "event-utils.h"
 #include "trace-seq.h"
 
tools/lib/traceevent/parse-filter.c +1 -0
index d64b6128fa7d..ed87cb56713d 100644
@@ -11,6 +11,7 @@
 #include <sys/types.h>
 
 #include "event-parse.h"
+#include "event-parse-local.h"
 #include "event-utils.h"
 
 #define COMM "COMM"
tools/perf/util/trace-event-parse.c +15 -10
index a4d7de1c96d1..9b78160e23a2 100644
@@ -37,10 +37,11 @@ static int get_common_field(struct scripting_context *context,
 	struct tep_format_field *field;
 
 	if (!*size) {
-		if (!pevent->events)
+
+		event = tep_get_first_event(pevent);
+		if (!event)
 			return 0;
 
-		event = pevent->events[0];
 		field = tep_find_common_field(event, type);
 		if (!field)
 			return 0;
@@ -192,25 +193,29 @@ struct tep_event_format *trace_find_next_event(struct tep_handle *pevent,
 					       struct tep_event_format *event)
 {
 	static int idx;
+	int events_count;
+	struct tep_event_format *all_events;
 
-	if (!pevent || !pevent->events)
+	all_events = tep_get_first_event(pevent);
+	events_count = tep_get_events_count(pevent);
+	if (!pevent || !all_events || events_count < 1)
 		return NULL;
 
 	if (!event) {
 		idx = 0;
-		return pevent->events[0];
+		return all_events;
 	}
 
-	if (idx < pevent->nr_events && event == pevent->events[idx]) {
+	if (idx < events_count && event == (all_events + idx)) {
 		idx++;
-		if (idx == pevent->nr_events)
+		if (idx == events_count)
 			return NULL;
-		return pevent->events[idx];
+		return (all_events + idx);
 	}
 
-	for (idx = 1; idx < pevent->nr_events; idx++) {
-		if (event == pevent->events[idx - 1])
-			return pevent->events[idx];
+	for (idx = 1; idx < events_count; idx++) {
+		if (event == (all_events + (idx - 1)))
+			return (all_events + idx);
 	}
 	return NULL;
 }
tools/perf/util/trace-event-read.c +1 -1
index b98ee2a2eb44..cdf6de82a507 100644
@@ -241,7 +241,7 @@ static int read_header_files(struct tep_handle *pevent)
 		 * The commit field in the page is of type long,
 		 * use that instead, since it represents the kernel.
 		 */
-		tep_set_long_size(pevent, pevent->header_page_size_size);
+		tep_set_long_size(pevent, tep_get_header_page_size(pevent));
 	}
 	free(header_page);
 
Signature · 1 line
2.13.6
maintainer Applied
#4 lore raw
In reply to Steven Rostedt

Em Fri, Oct 05, 2018 at 12:22:25PM -0400, Steven Rostedt escreveu:

Quoted 11 lines

From: Tzvetomir Stoyanov <[email protected]>

As traceevent is going to be transferred into a proper library, its local data should be protected from the library users. This patch encapsulates struct tep_handler into a local header, not visible outside of the library. It implements also a bunch of new APIs, which library users can use to access tep_handler members.

Thanks, applied to perf/core.

- Arnaldo

#5 lore raw

[tip:perf/core] tools lib traceevent, perf tools: Move struct tep_handler definition in a local header file

In reply to Steven Rostedt
tools lib traceevent, perf tools: Move struct tep_handler definition in a local header file
Commit-ID:  bb3dd7e7c4d5e024d607c0ec06c2a2fb9408cc99
Gitweb:     https://git.kernel.org/tip/bb3dd7e7c4d5e024d607c0ec06c2a2fb9408cc99
Author:     Tzvetomir Stoyanov <[email protected]>
AuthorDate: Fri, 5 Oct 2018 12:22:25 -0400
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Mon, 8 Oct 2018 15:05:37 -0300

tools lib traceevent, perf tools: Move struct tep_handler definition in a local header file

As traceevent is going to be transferred into a proper library, its local data should be protected from the library users. This patch encapsulates struct tep_handler into a local header, not visible outside of the library. It implements also a bunch of new APIs, which library users can use to access tep_handler members.

---
 tools/lib/traceevent/Build               |   1 +
 tools/lib/traceevent/event-parse-api.c   | 275 +++++++++++++++++++++++++++++++
 tools/lib/traceevent/event-parse-local.h |  92 +++++++++++
 tools/lib/traceevent/event-parse.c       |   2 +
 tools/lib/traceevent/event-parse.h       | 228 ++++---------------------
 tools/lib/traceevent/event-plugin.c      |   1 +
 tools/lib/traceevent/parse-filter.c      |   1 +
 tools/perf/util/trace-event-parse.c      |  25 +--
 tools/perf/util/trace-event-read.c       |   2 +-
 9 files changed, 416 insertions(+), 211 deletions(-)
tools/lib/traceevent/Build +1 -0
index 0050c145d806..ba54bfce0b0b 100644
@@ -5,6 +5,7 @@ libtraceevent-y += parse-filter.o
 libtraceevent-y += parse-utils.o
 libtraceevent-y += kbuffer-parse.o
 libtraceevent-y += tep_strerror.o
+libtraceevent-y += event-parse-api.o
 
 plugin_jbd2-y         += plugin_jbd2.o
 plugin_hrtimer-y      += plugin_hrtimer.o
tools/lib/traceevent/event-parse-api.c new file +275 -0

275 lines not on this page · raw message

tools/lib/traceevent/event-parse-local.h new file +92 -0

92 lines not on this page · raw message

tools/lib/traceevent/event-parse.c +2 -0
index 233179a712d6..3692f29fee46 100644
@@ -22,6 +22,8 @@
 
 #include <netinet/in.h>
 #include "event-parse.h"
+
+#include "event-parse-local.h"
 #include "event-utils.h"
 #include "trace-seq.h"
 
tools/lib/traceevent/event-parse.h +28 -200

253 lines not on this page · raw message

tools/lib/traceevent/event-plugin.c +1 -0
index 46eb64eb0c2e..e74f16c88398 100644
@@ -14,6 +14,7 @@
 #include <unistd.h>
 #include <dirent.h>
 #include "event-parse.h"
+#include "event-parse-local.h"
 #include "event-utils.h"
 #include "trace-seq.h"
 
tools/lib/traceevent/parse-filter.c +1 -0
index d64b6128fa7d..ed87cb56713d 100644
@@ -11,6 +11,7 @@
 #include <sys/types.h>
 
 #include "event-parse.h"
+#include "event-parse-local.h"
 #include "event-utils.h"
 
 #define COMM "COMM"
tools/perf/util/trace-event-parse.c +15 -10
index 02f97f5dd588..32e558a65af3 100644
@@ -37,10 +37,11 @@ static int get_common_field(struct scripting_context *context,
 	struct tep_format_field *field;
 
 	if (!*size) {
-		if (!pevent->events)
+
+		event = tep_get_first_event(pevent);
+		if (!event)
 			return 0;
 
-		event = pevent->events[0];
 		field = tep_find_common_field(event, type);
 		if (!field)
 			return 0;
@@ -193,25 +194,29 @@ struct tep_event_format *trace_find_next_event(struct tep_handle *pevent,
 					       struct tep_event_format *event)
 {
 	static int idx;
+	int events_count;
+	struct tep_event_format *all_events;
 
-	if (!pevent || !pevent->events)
+	all_events = tep_get_first_event(pevent);
+	events_count = tep_get_events_count(pevent);
+	if (!pevent || !all_events || events_count < 1)
 		return NULL;
 
 	if (!event) {
 		idx = 0;
-		return pevent->events[0];
+		return all_events;
 	}
 
-	if (idx < pevent->nr_events && event == pevent->events[idx]) {
+	if (idx < events_count && event == (all_events + idx)) {
 		idx++;
-		if (idx == pevent->nr_events)
+		if (idx == events_count)
 			return NULL;
-		return pevent->events[idx];
+		return (all_events + idx);
 	}
 
-	for (idx = 1; idx < pevent->nr_events; idx++) {
-		if (event == pevent->events[idx - 1])
-			return pevent->events[idx];
+	for (idx = 1; idx < events_count; idx++) {
+		if (event == (all_events + (idx - 1)))
+			return (all_events + idx);
 	}
 	return NULL;
 }
tools/perf/util/trace-event-read.c +1 -1
index add8441de579..76f12c705ef9 100644
@@ -241,7 +241,7 @@ static int read_header_files(struct tep_handle *pevent)
 		 * The commit field in the page is of type long,
 		 * use that instead, since it represents the kernel.
 		 */
-		tep_set_long_size(pevent, pevent->header_page_size_size);
+		tep_set_long_size(pevent, tep_get_header_page_size(pevent));
 	}
 	free(header_page);
 
#6 lore raw
In reply to Steven Rostedt

Hi,

Quoted 7 lines · Steven Rostedt

From: Tzvetomir Stoyanov <[email protected]>

As traceevent is going to be transferred into a proper library, its local data should be protected from the library users. This patch encapsulates struct tep_handler into a local header, not visible outside of the library. It implements also a bunch of new APIs, which library users can use to access tep_handler members.

This commit appears to have broken perf script --gen-script / trace_find_next_event(). As far as I can tell:

@ -192,25 +193,29 @@ struct tep_event_format *trace_find_next_event(struct tep_handle *pevent,

Quoted 27 lines
(quoted diff) +9 -5
 					       struct tep_event_format *event)
 {
 	static int idx;
+	int events_count;
+	struct tep_event_format *all_events;
 
-	if (!pevent || !pevent->events)
+	all_events = tep_get_first_event(pevent);
+	events_count = tep_get_events_count(pevent);
+	if (!pevent || !all_events || events_count < 1)
 		return NULL;
 
 	if (!event) {
 		idx = 0;
-		return pevent->events[0];
+		return all_events;
 	}
 
-	if (idx < pevent->nr_events && event == pevent->events[idx]) {
+	if (idx < events_count && event == (all_events + idx)) {
 		idx++;
-		if (idx == pevent->nr_events)
+		if (idx == events_count)
 			return NULL;
-		return pevent->events[idx];
+		return (all_events + idx);
 	}

Review

Quoted 10 lines
(quoted diff) +3 -3
 
-	for (idx = 1; idx < pevent->nr_events; idx++) {
-		if (event == pevent->events[idx - 1])
-			return pevent->events[idx];
+	for (idx = 1; idx < events_count; idx++) {
+		if (event == (all_events + (idx - 1)))
+			return (all_events + idx);
 	}
 	return NULL;
 }

Is just plain wrong, as:

Review

Quoted 2 lines
(quoted diff) +1 -1
-		return pevent->events[idx];
+		return (all_events + idx);

that's not a valid conversion. ->events isn't an array of tep_handle, it's an array of tep_handle* (and even if it were, the previous notation would have needed to dereference the value to make it comparable). What this does is look idx behind the individually allocated event. Which is incorrect.

To reproduce the crash, just generating a trace file with at least two events suffices:

perf record -e raw_syscalls:sys_enter,raw_syscalls:sys_exit sleep 1

gdb --args perf script -g py

Program received signal SIGSEGV, Segmentation fault.
__strlen_avx2 () at ../sysdeps/x86_64/multiarch/strlen-avx2.S:96
96 ../sysdeps/x86_64/multiarch/strlen-avx2.S: No such file or directory.
(gdb) bt
#0 __strlen_avx2 () at ../sysdeps/x86_64/multiarch/strlen-avx2.S:96
#1 0x00007ffff726f9ef in _IO_vfprintf_internal (s=s@entry=0x555555c23970, format=format@entry=0x5555558fed76 "def %s__%s(",

    ap=ap@entry=0x7fffffffb900) at vfprintf.c:1638
#2  0x00007ffff7326536 in ___fprintf_chk (fp=fp@entry=0x555555c23970, flag=flag@entry=1,
    format=format@entry=0x5555558fed76 "def %s__%s(") at fprintf_chk.c:35
#3  0x000055555587065c in fprintf (__fmt=0x5555558fed76 "def %s__%s(", __stream=0x555555c23970)
    at /usr/include/x86_64-linux-gnu/bits/stdio2.h:100
#4  python_generate_script (pevent=0x555555c2e620, outfile=<optimized out>)
    at util/scripting-engines/trace-event-python.c:1651
#5  0x0000555555745762 in cmd_script (argc=<optimized out>, argv=0x7fffffffe4d0) at builtin-script.c:3743
#6  0x000055555579821d in run_builtin (p=0x555555abeb38 <commands+408>, argc=3, argv=0x7fffffffe4d0) at perf.c:303
#7  0x0000555555712b7a in handle_internal_command (argv=0x7fffffffe4d0, argc=3) at perf.c:355
#8  run_argv (argcp=<synthetic pointer>, argv=<synthetic pointer>) at perf.c:399
#9  main (argc=3, argv=0x7fffffffe4d0) at perf.c:521

The fix (in recent kernel versions) appears to just bee to use tep_get_event(), instead of the old pevent->events[...]. But it appears to me that
commit 80c5526c8544ae76cba31fb9702ab8accac1f0f3

Author: Tzvetomir Stoyanov <[email protected]>
Date:   2019-04-01 12:43:12 -0400

tools lib traceevent: Implement new traceevent APIs for accessing struct tep_handler fields

ommitted adding it to event-parse.h. It appears to be intended as public API however, given that it got documented in

commit 747e942c3925bb85e2865371664499a98fca83b0

Author: Tzvetomir Stoyanov <[email protected]>
Date:   2019-05-10 15:56:23 -0400

tools lib traceevent: Man pages for libtraceevent event get APIs

The following patch fixes this for me. I can polish it up, but I'm wondering if I'm missing something here?

tools/lib/traceevent/event-parse.h +1 -0
index 642f68ab5fb2..7ebc9b5308d4 100644
@@ -517,6 +517,7 @@ int tep_read_number_field(struct tep_format_field *field, const void *data,
 			  unsigned long long *value);
 
 struct tep_event *tep_get_first_event(struct tep_handle *tep);
+struct tep_event *tep_get_event(struct tep_handle *tep, int index);
 int tep_get_events_count(struct tep_handle *tep);
 struct tep_event *tep_find_event(struct tep_handle *tep, int id);
 
tools/perf/util/trace-event-parse.c +6 -8
index 62bc61155dd1..6a035ffd58ac 100644
@@ -179,28 +179,26 @@ struct tep_event *trace_find_next_event(struct tep_handle *pevent,
 {
 	static int idx;
 	int events_count;
-	struct tep_event *all_events;
 
-	all_events = tep_get_first_event(pevent);
 	events_count = tep_get_events_count(pevent);
-	if (!pevent || !all_events || events_count < 1)
+	if (!pevent || events_count < 1)
 		return NULL;
 
 	if (!event) {
 		idx = 0;
-		return all_events;
+		return tep_get_event(pevent, 0);
 	}
 
-	if (idx < events_count && event == (all_events + idx)) {
+	if (idx < events_count && event == tep_get_event(pevent, idx)) {
 		idx++;
 		if (idx == events_count)
 			return NULL;
-		return (all_events + idx);
+		return tep_get_event(pevent, idx);
 	}
 
 	for (idx = 1; idx < events_count; idx++) {
-		if (event == (all_events + (idx - 1)))
-			return (all_events + idx);
+		if (event == tep_get_event(pevent, idx - 1))
+			return tep_get_event(pevent, idx);
 	}
 	return NULL;
 }

Not related to this crash, but it also seems that the whole trace_find_next_event() API ought to be removed. Back when it was

-struct event *trace_find_next_event(struct event *event) -{
- if (!event)
- return event_list;
-
- return event->next;
-}

it made some sense, but the changes in

commit aaf045f72335653b24784d6042be8e4aee114403

Author: Steven Rostedt <[email protected]>
Date:   2012-04-06 00:47:56 +0200

perf: Have perf use the new libtraceevent.a library

seem to make the current API somewhat absurd, as evidenced by the complexity in trace_find_next_event(). I mean even just removing that function and changing the two callsites to simple for loops with tep_get_events_count() & tep_get_event() ought to be a lot better.

Greetings,

Andres Freund

#7 lore raw

Hi Andres,

Quoted 3 lines · Andres Freund

Hi,

...

Quoted 12 lines

Is just plain wrong, as:

Review

Quoted 2 lines
(quoted diff) +1 -1
-             return pevent->events[idx];
+             return (all_events + idx);

that's not a valid conversion. ->events isn't an array of tep_handle, it's an array of tep_handle* (and even if it were, the previous notation would have needed to dereference the value to make it comparable). What this does is look idx behind the individually allocated event. Which is incorrect.

You are right, it is a bug.

Quoted 1 line

...

Quoted 47 lines

The following patch fixes this for me. I can polish it up, but I'm wondering if I'm missing something here?

tools/lib/traceevent/event-parse.h +1 -0
index 642f68ab5fb2..7ebc9b5308d4 100644
@@ -517,6 +517,7 @@ int tep_read_number_field(struct tep_format_field *field, const void *data,
                          unsigned long long *value);
 
 struct tep_event *tep_get_first_event(struct tep_handle *tep);
+struct tep_event *tep_get_event(struct tep_handle *tep, int index);
 int tep_get_events_count(struct tep_handle *tep);
 struct tep_event *tep_find_event(struct tep_handle *tep, int id);
 
tools/perf/util/trace-event-parse.c +4 -6
index 62bc61155dd1..6a035ffd58ac 100644
@@ -179,28 +179,26 @@ struct tep_event *trace_find_next_event(struct tep_handle *pevent,
 {
        static int idx;
        int events_count;
-       struct tep_event *all_events;
 
-       all_events = tep_get_first_event(pevent);
        events_count = tep_get_events_count(pevent);
-       if (!pevent || !all_events || events_count < 1)
+       if (!pevent || events_count < 1)
                return NULL;
 
        if (!event) {
                idx = 0;
-               return all_events;
+               return tep_get_event(pevent, 0);
        }
 
-       if (idx < events_count && event == (all_events + idx)) {
+       if (idx < events_count && event == tep_get_event(pevent, idx)) {
                idx++;
                if (idx == events_count)
                        return NULL;
-               return (all_events + idx);
+               return tep_get_event(pevent, idx);

tools/perf/util/trace-event-parse.c struct tep_event *trace_find_next_event(struct tep_handle *pevent, · line 204

Quoted 10 lines
tools/perf/util/trace-event-parse.c +2 -2
        }
 
        for (idx = 1; idx < events_count; idx++) {
-               if (event == (all_events + (idx - 1)))
-                       return (all_events + idx);
+               if (event == tep_get_event(pevent, idx - 1))
+                       return tep_get_event(pevent, idx);
        }
        return NULL;
 }

I'm OK with the first part of the patch, the changes in tools/lib/traceevent/event-parse.h.
tep_get_event() is an API and is omitted in the header by mistake, it should be there.

As for the fix for the crash itself, I think it would be better to implement your suggestion -
removing trace_find_next_event() at all and replacing it with tep_get_events_count() and
tep_get_event() loop.

Quoted 21 lines

Not related to this crash, but it also seems that the whole trace_find_next_event() API ought to be removed. Back when it was

tools/perf/util/trace-event-parse.c +0 -7
-struct event *trace_find_next_event(struct event *event)
-{
-       if (!event)
-               return event_list;
-
-       return event->next;
-}

it made some sense, but the changes in

commit aaf045f72335653b24784d6042be8e4aee114403

Author: Steven Rostedt <[email protected]>
Date:   2012-04-06 00:47:56 +0200

perf: Have perf use the new libtraceevent.a library

Quoted 8 lines

seem to make the current API somewhat absurd, as evidenced by the complexity in trace_find_next_event(). I mean even just removing that function and changing the two callsites to simple for loops with tep_get_events_count() & tep_get_event() ought to be a lot better.

Greetings,

Andres Freund

Thanks!

Signature · 2 lines
Tzvetomir (Ceco) Stoyanov
VMware Open Source Technology Center
#8 lore raw
In reply to Andres Freund
Quoted 2 lines · On Thu, 25 Jul 2019 20:58:29 -0700 Andres Freund <[email protected]> wrote:

Hi,

Hi Andres,

Quoted 26 lines · Steven Rostedt
Quoted 7 lines

From: Tzvetomir Stoyanov <[email protected]>

As traceevent is going to be transferred into a proper library, its local data should be protected from the library users. This patch encapsulates struct tep_handler into a local header, not visible outside of the library. It implements also a bunch of new APIs, which library users can use to access tep_handler members.

This commit appears to have broken perf script --gen-script / trace_find_next_event(). As far as I can tell:

@ -192,25 +193,29 @@ struct tep_event_format *trace_find_next_event(struct tep_handle *pevent,

Review

Quoted 9 lines
(quoted diff) +4 -1
 					       struct tep_event_format *event)
 {
 	static int idx;
+	int events_count;
+	struct tep_event_format *all_events;
 
-	if (!pevent || !pevent->events)
+	all_events = tep_get_first_event(pevent);
+	events_count = tep_get_events_count(pevent);

Is just plain wrong, as:

Quoted 5 lines

Review

Quoted 2 lines
(quoted diff) +1 -1
-		return pevent->events[idx];
+		return (all_events + idx);  

that's not a valid conversion. ->events isn't an array of tep_handle, it's an array of tep_handle* (and even if it were, the previous notation

You're right, it is wrong, but it's not tep_handle* but tep_event_format*.

The tep_get_first_event() returns a pointer to the first event, but that's not an array. We can't use that to index to other events.

Quoted 13 lines

would have needed to dereference the value to make it comparable). What this does is look idx behind the individually allocated event. Which is incorrect.

The fix (in recent kernel versions) appears to just bee to use tep_get_event(), instead of the old pevent->events[...]. But it appears to me that
commit 80c5526c8544ae76cba31fb9702ab8accac1f0f3

Quoted 7 lines
Author: Tzvetomir Stoyanov <[email protected]>
Date:   2019-04-01 12:43:12 -0400

tools lib traceevent: Implement new traceevent APIs for accessing struct tep_handler fields

ommitted adding it to event-parse.h. It appears to be intended as public API however, given that it got documented in

And it appears that we missed to add it ;-)

Quoted 16 lines

commit 747e942c3925bb85e2865371664499a98fca83b0

Author: Tzvetomir Stoyanov <[email protected]>
Date:   2019-05-10 15:56:23 -0400

tools lib traceevent: Man pages for libtraceevent event get APIs

The following patch fixes this for me. I can polish it up, but I'm wondering if I'm missing something here?

tools/lib/traceevent/event-parse.h +0 -0
index 642f68ab5fb2..7ebc9b5308d4 100644

tools/lib/traceevent/event-parse.h int tep_read_number_field(struct tep_format_field *field, const void *data, · line 520

Quoted 4 lines
tools/lib/traceevent/event-parse.h +1 -0
@@ -517,6 +517,7 @@ int tep_read_number_field(struct tep_format_field *field, const void *data,
 			  unsigned long long *value);
 
 struct tep_event *tep_get_first_event(struct tep_handle *tep);
+struct tep_event *tep_get_event(struct tep_handle *tep, int index);

I was looking at the tep_get_event() code, and we should switch that to "unsigned int index" otherwise passing in a negative number will return an address outside the array.

Quoted 8 lines
tools/lib/traceevent/event-parse.h +0 -0
 int tep_get_events_count(struct tep_handle *tep);
 struct tep_event *tep_find_event(struct tep_handle *tep, int id);
 
tools/perf/util/trace-event-parse.c +0 -0
index 62bc61155dd1..6a035ffd58ac 100644

tools/perf/util/trace-event-parse.c struct tep_event *trace_find_next_event(struct tep_handle *pevent, · line 183

Quoted 7 lines
tools/perf/util/trace-event-parse.c +0 -2
@@ -179,28 +179,26 @@ struct tep_event *trace_find_next_event(struct tep_handle *pevent,
 {
 	static int idx;
 	int events_count;
-	struct tep_event *all_events;
 
-	all_events = tep_get_first_event(pevent);
 	events_count = tep_get_events_count(pevent);

I think we can get rid of the events_count and all its checks, as the same check is done within tep_get_event().

tools/perf/util/trace-event-parse.c struct tep_event *trace_find_next_event(struct tep_handle *pevent, · line 184

Quoted 2 lines
tools/perf/util/trace-event-parse.c +1 -1
-	if (!pevent || !all_events || events_count < 1)
+	if (!pevent || events_count < 1)

if (!pevent)

tools/perf/util/trace-event-parse.c struct tep_event *trace_find_next_event(struct tep_handle *pevent, · line 192

Quoted 10 lines
tools/perf/util/trace-event-parse.c +2 -2
 		return NULL;
 
 	if (!event) {
 		idx = 0;
-		return all_events;
+		return tep_get_event(pevent, 0);
 	}
 
-	if (idx < events_count && event == (all_events + idx)) {
+	if (idx < events_count && event == tep_get_event(pevent, idx)) {
	if (event == tep_get_event(pevent, idx))
		return tep_get_event(pevent, ++idx);

tools/perf/util/trace-event-parse.c struct tep_event *trace_find_next_event(struct tep_handle *pevent, · line 198

Quoted 7 lines
tools/perf/util/trace-event-parse.c +1 -1
 		idx++;
 		if (idx == events_count)
 			return NULL;
-		return (all_events + idx);
+		return tep_get_event(pevent, idx);
 	}
 
	struct tep_event_format *next_event;

	for (idx = 0; next_event = tep_get_event(pevent, idx); idx++)
		if (event == next_event)
			return tep_get_event(pevent, ++idx);

Also, I think setting the idx to 1 in the loop is wrong. Why? think of this:

	first_event = trace_find_next_event(pevent, NULL);

	next_event = trace_find_next_event(pevent, first_event);
	for (i = 0; i < 5; i++)
		next_event = trace_find_next_event(pevent, next_event);

	second_event = trace_find_next_event(pevent, first_event);

second_event would become NULL.

Care to send a formal patch?

Thanks!

-- Steve

Quoted 40 lines
tools/perf/util/trace-event-parse.c +2 -2
 	for (idx = 1; idx < events_count; idx++) {
-		if (event == (all_events + (idx - 1)))
-			return (all_events + idx);
+		if (event == tep_get_event(pevent, idx - 1))
+			return tep_get_event(pevent, idx);
 	}
 
 	return NULL;

}

Not related to this crash, but it also seems that the whole trace_find_next_event() API ought to be removed. Back when it was

tools/perf/util/trace-event-parse.c +0 -7
-struct event *trace_find_next_event(struct event *event)
-{
-       if (!event)
-               return event_list;
-
-       return event->next;
-}

it made some sense, but the changes in

commit aaf045f72335653b24784d6042be8e4aee114403

Author: Steven Rostedt <[email protected]>
Date:   2012-04-06 00:47:56 +0200

perf: Have perf use the new libtraceevent.a library

seem to make the current API somewhat absurd, as evidenced by the complexity in trace_find_next_event(). I mean even just removing that function and changing the two callsites to simple for loops with tep_get_events_count() & tep_get_event() ought to be a lot better.

Greetings,

Andres Freund

#9 lore raw

Hi,

Quoted 5 lines · Steven Rostedt
Quoted 3 lines · On Thu, 25 Jul 2019 20:58:29 -0700 Andres Freund <[email protected]> wrote:

Is just plain wrong, as:

Quoted 8 lines
Quoted 5 lines

Review

Quoted 2 lines
(quoted diff) +1 -1
-		return pevent->events[idx];
+		return (all_events + idx);  

that's not a valid conversion. ->events isn't an array of tep_handle, it's an array of tep_handle* (and even if it were, the previous notation

You're right, it is wrong, but it's not tep_handle* but tep_event_format*.

Err, yea. Typo.

Quoted 7 lines
Quoted 7 lines
tools/lib/traceevent/event-parse.h +0 -0
index 642f68ab5fb2..7ebc9b5308d4 100644
@@ -517,6 +517,7 @@ int tep_read_number_field(struct tep_format_field *field, const void *data,
 			  unsigned long long *value);
 
Quoted 6 lines

tools/lib/traceevent/event-parse.h int tep_read_number_field(struct tep_format_field *field, const void *data, · line 520

Quoted 2 lines
tools/lib/traceevent/event-parse.h +1 -0
 struct tep_event *tep_get_first_event(struct tep_handle *tep);
+struct tep_event *tep_get_event(struct tep_handle *tep, int index);

I was looking at the tep_get_event() code, and we should switch that to "unsigned int index" otherwise passing in a negative number will return an address outside the array.

Makes sense.

Quoted 63 lines
Quoted 8 lines
tools/lib/traceevent/event-parse.h +0 -0
 int tep_get_events_count(struct tep_handle *tep);
 struct tep_event *tep_find_event(struct tep_handle *tep, int id);
 
tools/perf/util/trace-event-parse.c +0 -0
index 62bc61155dd1..6a035ffd58ac 100644

tools/perf/util/trace-event-parse.c struct tep_event *trace_find_next_event(struct tep_handle *pevent, · line 183

Quoted 7 lines
tools/perf/util/trace-event-parse.c +0 -2
@@ -179,28 +179,26 @@ struct tep_event *trace_find_next_event(struct tep_handle *pevent,
 {
 	static int idx;
 	int events_count;
-	struct tep_event *all_events;
 
-	all_events = tep_get_first_event(pevent);
 	events_count = tep_get_events_count(pevent);

I think we can get rid of the events_count and all its checks, as the same check is done within tep_get_event().

tools/perf/util/trace-event-parse.c struct tep_event *trace_find_next_event(struct tep_handle *pevent, · line 184

Quoted 2 lines
tools/perf/util/trace-event-parse.c +1 -1
-	if (!pevent || !all_events || events_count < 1)
+	if (!pevent || events_count < 1)

if (!pevent)

tools/perf/util/trace-event-parse.c struct tep_event *trace_find_next_event(struct tep_handle *pevent, · line 192

Quoted 10 lines
tools/perf/util/trace-event-parse.c +2 -2
 		return NULL;
 
 	if (!event) {
 		idx = 0;
-		return all_events;
+		return tep_get_event(pevent, 0);
 	}
 
-	if (idx < events_count && event == (all_events + idx)) {
+	if (idx < events_count && event == tep_get_event(pevent, idx)) {
	if (event == tep_get_event(pevent, idx))
		return tep_get_event(pevent, ++idx);

tools/perf/util/trace-event-parse.c struct tep_event *trace_find_next_event(struct tep_handle *pevent, · line 198

Quoted 7 lines
tools/perf/util/trace-event-parse.c +1 -1
 		idx++;
 		if (idx == events_count)
 			return NULL;
-		return (all_events + idx);
+		return tep_get_event(pevent, idx);
 	}
 
	struct tep_event_format *next_event;

	for (idx = 0; next_event = tep_get_event(pevent, idx); idx++)
		if (event == next_event)
			return tep_get_event(pevent, ++idx);

Also, I think setting the idx to 1 in the loop is wrong. Why? think of this:

	first_event = trace_find_next_event(pevent, NULL);

	next_event = trace_find_next_event(pevent, first_event);
	for (i = 0; i < 5; i++)
		next_event = trace_find_next_event(pevent, next_event);

	second_event = trace_find_next_event(pevent, first_event);
Quoted 1 line

second_event would become NULL.

How about my proposal to instead change the loops in trace-event-{python,perl}.c, the only callers of trace_find_next_event, to be something akin to

[idx_type_for_tep_get_event] event_count = tep_get_events_count(pevent); for ([idx_type_for_tep_get_event] idx = 0; idx < event_count; idx++)

{
	struct tep_event *event = tep_get_events(...);

}

and just removing trace_find_next_event()? It's not a nice API imo, and seems unnecessary given that the events aren't a linked list anymore.

Quoted 1 line

Care to send a formal patch?

Will do.

Greetings,

Andres Freund

#10 lore raw
Quoted 6 lines · On Fri, 26 Jul 2019 13:55:44 -0700 Andres Freund <[email protected]> wrote:

How about my proposal to instead change the loops in trace-event-{python,perl}.c, the only callers of trace_find_next_event, to be something akin to

[idx_type_for_tep_get_event] event_count = tep_get_events_count(pevent); for ([idx_type_for_tep_get_event] idx = 0; idx < event_count; idx++)

Quoted 7 lines
{
	struct tep_event *event = tep_get_events(...);

}

and just removing trace_find_next_event()? It's not a nice API imo, and seems unnecessary given that the events aren't a linked list anymore.

Yep, go for it :-)

Quoted 5 lines
Quoted 3 lines

Care to send a formal patch?

Will do.

Thanks!

-- Steve