Skip to contentSkip to conversation

perf: convert to ctf: support converting syscalls:* tracepoints.

PATCH In review Wang Nan 3 participants 11 messages Jan 26, 2015 – Jan 30, 2015 perf events

lore
Share & tools
lore

Wang Nan

author
#0 lore raw

v2 cover perf: convert to ctf: support converting syscalls:* tracepoints.

Babeltrace now support bt_ctf_validate_identifier() to allow us detect keyword confliction before real add it. I update patch 2/3 to use that new interface. Also, I group all 3 patches I made during converting syscalls:* tracepoints samples together for review.

Wang Nan (3):
tools lib traceevent: add priv field to truct format_field.
perf: convert: fix duplicate field names and avoid reserved keywords.
perf: convert: fix signess of value.

 tools/lib/traceevent/event-parse.c |   2 +
 tools/lib/traceevent/event-parse.h |   2 +
 tools/perf/util/data-convert-bt.c  | 161 +++++++++++++++++++++++++++++++++----
 3 files changed, 149 insertions(+), 16 deletions(-)
Signature · 1 line
1.8.4
author
#1 lore raw

PATCH v2 1/3 tools lib traceevent: add priv field to struct format_field.

Introduce a priv field to 'struct format_field' for futher expansion.

(In https://lkml.org/lkml/2015/1/21/383 , Jiri Olsa gives a suggestion
about changing lib traceevent to solve a bug of perf-convert-to-ctf, which is related to duplicated field names. I think his suggestion should be something like this patch. )

---
 tools/lib/traceevent/event-parse.c | 2 ++
 tools/lib/traceevent/event-parse.h | 2 ++
 2 files changed, 4 insertions(+)
tools/lib/traceevent/event-parse.c +2 -0
index cf3a44b..5f76003 100644
@@ -5909,6 +5909,8 @@ static void free_format_fields(struct format_field *field)
 		free(field->type);
 		free(field->name);
 		free(field);
+		if (field->destroy_priv)
+			field->destroy_priv(field);
 		field = next;
 	}
 }
tools/lib/traceevent/event-parse.h +2 -0
index 7a3873f..928d801 100644
@@ -190,6 +190,8 @@ struct format_field {
 	unsigned int		arraylen;
 	unsigned int		elementsize;
 	unsigned long		flags;
+	void			*priv;
+	void 			(*destroy_priv)(struct format_field *);
 };
 
 struct format {
Signature · 1 line
1.8.4
#2 lore raw

Re: [PATCH v2 1/3] tools lib traceevent: add priv field to struct format_field.

Quoted 6 lines · Wang Nan

Introduce a priv field to 'struct format_field' for futher expansion.

(In https://lkml.org/lkml/2015/1/21/383 , Jiri Olsa gives a suggestion
about changing lib traceevent to solve a bug of perf-convert-to-ctf, which is related to duplicated field names. I think his suggestion should be something like this patch. )

Hi Steven,
any feedback on this one?

thanks,
jirka

Quoted 36 lines
---
 tools/lib/traceevent/event-parse.c | 2 ++
 tools/lib/traceevent/event-parse.h | 2 ++
 2 files changed, 4 insertions(+)
tools/lib/traceevent/event-parse.c +2 -0
index cf3a44b..5f76003 100644
@@ -5909,6 +5909,8 @@ static void free_format_fields(struct format_field *field)
 		free(field->type);
 		free(field->name);
 		free(field);
+		if (field->destroy_priv)
+			field->destroy_priv(field);
 		field = next;
 	}
 }
tools/lib/traceevent/event-parse.h +2 -0
index 7a3873f..928d801 100644
@@ -190,6 +190,8 @@ struct format_field {
 	unsigned int		arraylen;
 	unsigned int		elementsize;
 	unsigned long		flags;
+	void			*priv;
+	void 			(*destroy_priv)(struct format_field *);
 };
 
 struct format {
Signature · 1 line
1.8.4
#3 lore raw

Re: [PATCH v2 1/3] tools lib traceevent: add priv field to struct format_field.

In reply to Wang Nan

tools/lib/traceevent/event-parse.c static void free_format_fields(struct format_field *field) · line 5913

Quoted 10 lines · On Mon, 26 Jan 2015 18:38:23 +0800 Wang Nan <[email protected]> wrote:
tools/lib/traceevent/event-parse.c +2 -0
index cf3a44b..5f76003 100644
@@ -5909,6 +5909,8 @@ static void free_format_fields(struct format_field *field)
 		free(field->type);
 		free(field->name);
 		free(field);
+		if (field->destroy_priv)
+			field->destroy_priv(field);

I think you want to call field->destroy_priv() *before* you free field.

-- Steve

Quoted 3 lines
tools/lib/traceevent/event-parse.c +0 -0
 		field = next;
 	}
 }
#4 lore raw

Re: [PATCH v2 1/3] tools lib traceevent: add priv field to struct format_field.

Quoted 4 lines · Steven Rostedt

On Mon, 26 Jan 2015 18:38:23 +0800
Wang Nan <[email protected]> wrote:

Quoted 12 lines

tools/lib/traceevent/event-parse.c static void free_format_fields(struct format_field *field) · line 5913

Quoted 10 lines
tools/lib/traceevent/event-parse.c +2 -0
index cf3a44b..5f76003 100644
@@ -5909,6 +5909,8 @@ static void free_format_fields(struct format_field *field)
 		free(field->type);
 		free(field->name);
 		free(field);
+		if (field->destroy_priv)
+			field->destroy_priv(field);

I think you want to call field->destroy_priv() *before* you free field.

argh.. missed that :-\ will fix

thanks,
jirka

Quoted 6 lines

-- Steve

Quoted 3 lines
tools/lib/traceevent/event-parse.c +0 -0
 		field = next;
 	}
 }
#5 lore raw

PATCH v3 1/1 tools lib traceevent: Add priv field to struct format_field

Quoted 18 lines · Jiri Olsa
Quoted 4 lines · Steven Rostedt

On Mon, 26 Jan 2015 18:38:23 +0800
Wang Nan <[email protected]> wrote:

Quoted 12 lines

tools/lib/traceevent/event-parse.c static void free_format_fields(struct format_field *field) · line 5913

Quoted 10 lines
tools/lib/traceevent/event-parse.c +2 -0
index cf3a44b..5f76003 100644
@@ -5909,6 +5909,8 @@ static void free_format_fields(struct format_field *field)
 		free(field->type);
 		free(field->name);
 		free(field);
+		if (field->destroy_priv)
+			field->destroy_priv(field);

I think you want to call field->destroy_priv() *before* you free field.

Quoted 2 lines

argh.. missed that :-\ will fix

fixed version

jirka

---
From: Wang Nan <[email protected]>

Introduce a priv field to 'struct format_field' for futher expansion.

(In https://lkml.org/lkml/2015/1/21/383 , Jiri Olsa gives a suggestion
about changing lib traceevent to solve a bug of perf-convert-to-ctf, which is related to duplicated field names. I think his suggestion should be something like this patch. )

Cc Arnaldo Carvalho de Melo <[email protected]>
Cc David Ahern <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Sebastian Andrzej Siewior <[email protected]>
Cc Tom Zanussi <[email protected]>
[ moved field release after destroy callback call ]
---
 tools/lib/traceevent/event-parse.c | 2 ++
 tools/lib/traceevent/event-parse.h | 2 ++
 2 files changed, 4 insertions(+)
tools/lib/traceevent/event-parse.c +2 -0
index afe20ed9fac8..64d40b7e0582 100644
@@ -6236,6 +6236,8 @@ static void free_format_fields(struct format_field *field)
 		next = field->next;
 		free(field->type);
 		free(field->name);
+		if (field->destroy_priv)
+			field->destroy_priv(field);
 		free(field);
 		field = next;
 	}
tools/lib/traceevent/event-parse.h +2 -0
index 7a3873ff9a4f..928d801444ab 100644
@@ -190,6 +190,8 @@ struct format_field {
 	unsigned int		arraylen;
 	unsigned int		elementsize;
 	unsigned long		flags;
+	void			*priv;
+	void 			(*destroy_priv)(struct format_field *);
 };
 
 struct format {
Signature · 1 line
1.9.3
author
#6 lore raw

PATCH v2 2/3 perf: convert: fix duplicate field names and avoid reserved keywords.

In reply to Wang Nan

(If Steven Rostedt accept the previous patch which introduce a priv
field to 'struct format_field', we can use a relative simple method for name conversion. If not , perf must track name conversion by itself.)

Some parameters of syscall tracepoints named as 'nr', 'event', etc. When dealing with them, perf convert to ctf meets some problem:

1. If a parameter with name 'nr', it will duplicate syscall's common field 'nr'. One such syscall is io_submit().

2. If a parameter with name 'event', it is denied to be inserted because 'event' is a babeltrace keywork. One such syscall is epoll_ctl.

This patch appends '_dupl_X' suffix to avoid problem 1, prepend a '_' prefix to avoid problem 2.

---
 tools/perf/util/data-convert-bt.c | 98 +++++++++++++++++++++++++++++++++++++--
 1 file changed, 94 insertions(+), 4 deletions(-)
tools/perf/util/data-convert-bt.c +94 -4
index ddecce8..934bd9b 100644
@@ -14,6 +14,7 @@
 #include <babeltrace/ctf-writer/event.h>
 #include <babeltrace/ctf-writer/event-types.h>
 #include <babeltrace/ctf-writer/event-fields.h>
+#include <babeltrace/ctf-ir/utils.h>
 #include <babeltrace/ctf/events.h>
 #include <traceevent/event-parse.h>
 #include "asm/bug.h"
@@ -184,6 +185,7 @@ static int add_tracepoint_field_value(struct ctf_writer *cw,
 	unsigned int len;
 	int ret;
 
+	name = fmtf->priv ? (const char *)fmtf->priv : fmtf->name;
 	offset = fmtf->offset;
 	len = fmtf->size;
 	if (flags & FIELD_IS_STRING)
@@ -567,6 +569,94 @@ static int process_sample_event(struct perf_tool *tool,
 	return cs ? 0 : -1;
 }
 
+/* If dup < 0, add a prefix. Else, add _dupl_X suffix. */
+static char *change_name(char *name, char *orig_name, int dup)
+{
+	char *new_name = NULL;
+	size_t len;
+
+	if (!name)
+		name = orig_name;
+
+	if (dup >= 10)
+		goto out;
+	/*
+	 * Add '_' prefix to potential keywork.  According to
+	 * Mathieu Desnoyers (https://lkml.org/lkml/2015/1/23/652),
+	 * futher CTF spec updating may require us to use '$'.
+	 */
+	if (dup < 0)
+		len = strlen(name) + sizeof("_");
+	else
+		len = strlen(orig_name) + sizeof("_dupl_X");
+
+	new_name = malloc(len);
+	if (!new_name)
+		goto out;
+
+	if (dup < 0)
+		snprintf(new_name, len, "_%s", name);
+	else
+		snprintf(new_name, len, "%s_dupl_%d", orig_name, dup);
+
+out:
+	if (name != orig_name)
+		free(name);
+	return new_name;
+}
+
+static void destroy_field_priv(struct format_field *field)
+{
+	if (!field->priv)
+		return;
+
+	if (field->priv != field->name)
+		free(field->priv);
+
+	field->priv = NULL;
+	field->destroy_priv = NULL;
+}
+
+static int event_class_add_field(struct bt_ctf_event_class *event_class,
+		struct bt_ctf_field_type *type,
+		struct format_field *field)
+{
+	struct bt_ctf_field_type *t = NULL;
+	char *name;
+	int dup = 1;
+	int ret;
+
+	if (field->priv)
+		return bt_ctf_event_class_add_field(event_class, type,
+				(char *)field->priv);
+
+	name = field->name;
+
+	/* If 'name' is a keywork, add prefix. */
+	if (bt_ctf_validate_identifier(name))
+		name = change_name(name, field->name, -1);
+
+	if (!name) {
+		pr_err("Failed to fix invalid identifier.");
+		return -1;
+	}
+	while ((t = bt_ctf_event_class_get_field_by_name(event_class, name))) {
+		bt_ctf_field_type_put(t);
+		name = change_name(name, field->name, dup++);
+		if (!name) {
+			pr_err("Failed to create dup name for '%s'\n", field->name);
+			return -1;
+		}
+	}
+
+	ret = bt_ctf_event_class_add_field(event_class, type, name);
+
+	field->priv = name;
+	field->destroy_priv = destroy_field_priv;
+
+	return ret;
+}
+
 static int add_tracepoint_fields_types(struct ctf_writer *cw,
 				       struct format_field *fields,
 				       struct bt_ctf_event_class *event_class)
@@ -595,14 +685,14 @@ static int add_tracepoint_fields_types(struct ctf_writer *cw,
 		if (flags & FIELD_IS_ARRAY)
 			type = bt_ctf_field_type_array_create(type, field->arraylen);
 
-		ret = bt_ctf_event_class_add_field(event_class, type,
-				field->name);
+		ret = event_class_add_field(event_class, type, field);
 
 		if (flags & FIELD_IS_ARRAY)
 			bt_ctf_field_type_put(type);
 
 		if (ret) {
-			pr_err("Failed to add field '%s\n", field->name);
+			pr_err("Failed to add field '%s': %d\n",
+					field->name, ret);
 			return -1;
 		}
 	}
@@ -646,7 +736,7 @@ static int add_generic_types(struct ctf_writer *cw, struct perf_evsel *evsel,
 	do {								\
 		pr2("  field '%s'\n", n);				\
 		if (bt_ctf_event_class_add_field(cl, t, n)) {		\
-			pr_err("Failed to add field '%s;\n", n);	\
+			pr_err("Failed to add field '%s';\n", n);	\
 			return -1;					\
 		}							\
 	} while (0)
Signature · 1 line
1.8.4
#7 lore raw

Re: [PATCH v2 2/3] perf: convert: fix duplicate field names and avoid reserved keywords.

Quoted 4 lines · Wang Nan

(If Steven Rostedt accept the previous patch which introduce a priv
field to 'struct format_field', we can use a relative simple method for name conversion. If not , perf must track name conversion by itself.)

Sorry for coming in so late here.

Quoted 7 lines

Some parameters of syscall tracepoints named as 'nr', 'event', etc. When dealing with them, perf convert to ctf meets some problem:

1. If a parameter with name 'nr', it will duplicate syscall's common field 'nr'. One such syscall is io_submit().

Quoted 6 lines

2. If a parameter with name 'event', it is denied to be inserted because 'event' is a babeltrace keywork. One such syscall is epoll_ctl.

This patch appends '_dupl_X' suffix to avoid problem 1, prepend a '_' prefix to avoid problem 2.

Actually, I don't like this approach. That is, to have this private data structure. Why not just add an "alias" to format_field. In other words, instead of hiding this interaction behind a void pointer and needing to create a function pointer to free it, just add another field to format field and be done with it. I think it would make the code a hell of a lot simpler and easier to understand.

struct format_field {

 	struct format_field	*next;
	struct event_format	*event;
	char			*type;
	char			*name;
+	char			*alias;
	int			offset;
	int			size;
	unsigned int		arraylen;
	unsigned int		elementsize;
	unsigned long		flags;
};

And put the logic in event parse to free alias if need be. Heck, you could even add a pevent_*() func to assign the alias using strdup, or what not.

-- Steve

#8 lore raw

Re: [PATCH v2 2/3] perf: convert: fix duplicate field names and avoid reserved keywords.

Quoted 42 lines · Steven Rostedt
Quoted 4 lines · Wang Nan

(If Steven Rostedt accept the previous patch which introduce a priv
field to 'struct format_field', we can use a relative simple method for name conversion. If not , perf must track name conversion by itself.)

Sorry for coming in so late here.

Quoted 7 lines

Some parameters of syscall tracepoints named as 'nr', 'event', etc. When dealing with them, perf convert to ctf meets some problem:

1. If a parameter with name 'nr', it will duplicate syscall's common field 'nr'. One such syscall is io_submit().

Quoted 6 lines

2. If a parameter with name 'event', it is denied to be inserted because 'event' is a babeltrace keywork. One such syscall is epoll_ctl.

This patch appends '_dupl_X' suffix to avoid problem 1, prepend a '_' prefix to avoid problem 2.

Actually, I don't like this approach. That is, to have this private data structure. Why not just add an "alias" to format_field. In other words, instead of hiding this interaction behind a void pointer and needing to create a function pointer to free it, just add another field to format field and be done with it. I think it would make the code a hell of a lot simpler and easier to understand.

struct format_field {

 	struct format_field	*next;
	struct event_format	*event;
	char			*type;
	char			*name;
(quoted diff) +1 -0
+	char			*alias;
	int			offset;
	int			size;
	unsigned int		arraylen;
	unsigned int		elementsize;
	unsigned long		flags;
};
Quoted 3 lines

And put the logic in event parse to free alias if need be. Heck, you could even add a pevent_*() func to assign the alias using strdup, or what not.

ok, sounds good

thanks,
jirka

author
#9 lore raw

PATCH v2 3/3 perf: convert: fix signess of value.

In reply to Wang Nan

When converting int values, perf first extractes it to a ulonglong, then feeds it to babeltrace as a signed value. For negative 32 bit values (for example, return values of failed syscalls), the extracted data should be something like 0xfffffffe (-2). It becomes a large int64 value. Babeltrace denies to insert it with bt_ctf_field_signed_integer_set_value() because it is larger than 0x7fffffff, the largest positive value a signed 32 bit int can be.

This patch introduces adjust_signess(), which fills high bits of ulonglong with 1 if the value is negative.

---
 tools/perf/util/data-convert-bt.c | 63 +++++++++++++++++++++++++++++++--------
 1 file changed, 51 insertions(+), 12 deletions(-)
tools/perf/util/data-convert-bt.c +51 -12
index 934bd9b..a89f879 100644
@@ -166,6 +166,43 @@ get_tracepoint_field_type(struct ctf_writer *cw, struct format_field *field)
 		return cw->data.u32;
 }
 
+static unsigned long long adjust_signess(unsigned long long value_int, int size)
+{
+	unsigned long long value_mask;
+
+	/*
+	 * value_mask = (1 << (size * 8 - 1)) - 1.
+	 * Directly set value_mask for code readers.
+	 */
+	switch (size) {
+	case 1:
+		value_mask = 0x7fULL;
+		break;
+	case 2:
+		value_mask = 0x7fffULL;
+		break;
+	case 4:
+		value_mask = 0x7fffffffULL;
+		break;
+	case 8:
+		/*
+		 * For 64 bit value, return it self. There is no need
+		 * to fill high bit.
+		 */
+		/* Fall through */
+	default:
+		/* BUG! */
+		return value_int;
+	}
+
+	/* If it is a positive value, don't adjust. */
+	if ((value_int & (~0ULL - value_mask)) == 0)
+		return value_int;
+
+	/* Fill upper part of value_int with 1 to make it a negative long long. */
+	return (value_int & value_mask) | ~value_mask;
+}
+
 static int add_tracepoint_field_value(struct ctf_writer *cw,
 				      struct bt_ctf_event_class *event_class,
 				      struct bt_ctf_event *event,
@@ -177,7 +214,6 @@ static int add_tracepoint_field_value(struct ctf_writer *cw,
 	struct bt_ctf_field *field;
 	const char *name = fmtf->name;
 	void *data = sample->raw_data;
-	unsigned long long value_int;
 	unsigned long flags = fmtf->flags;
 	unsigned int n_items;
 	unsigned int i;
@@ -222,11 +258,6 @@ static int add_tracepoint_field_value(struct ctf_writer *cw,
 	type = get_tracepoint_field_type(cw, fmtf);
 
 	for (i = 0; i < n_items; i++) {
-		if (!(flags & FIELD_IS_STRING))
-			value_int = pevent_read_number(
-					fmtf->event->pevent,
-					data + offset + i * len, len);
-
 		if (flags & FIELD_IS_ARRAY)
 			field = bt_ctf_field_array_get_field(array_field, i);
 		else
@@ -240,12 +271,20 @@ static int add_tracepoint_field_value(struct ctf_writer *cw,
 		if (flags & FIELD_IS_STRING)
 			ret = bt_ctf_field_string_set_value(field,
 					data + offset + i * len);
-		else if (!(flags & FIELD_IS_SIGNED))
-			ret = bt_ctf_field_unsigned_integer_set_value(
-					field, value_int);
-		else
-			ret = bt_ctf_field_signed_integer_set_value(
-					field, value_int);
+		else {
+			unsigned long long value_int;
+			value_int = pevent_read_number(
+					fmtf->event->pevent,
+					data + offset + i * len, len);
+
+			if (!(flags & FIELD_IS_SIGNED))
+				ret = bt_ctf_field_unsigned_integer_set_value(
+						field, value_int);
+			else
+				ret = bt_ctf_field_signed_integer_set_value(
+						field, adjust_signess(value_int, len));
+		}
+
 		if (ret) {
 			pr_err("failed to set file value %s\n", name);
 			goto err_put_field;
Signature · 1 line
1.8.4
#10 lore raw
In reply to Wang Nan
Quoted 9 lines · Wang Nan

Babeltrace now support bt_ctf_validate_identifier() to allow us detect keyword confliction before real add it. I update patch 2/3 to use that new interface. Also, I group all 3 patches I made during converting syscalls:* tracepoints samples together for review.

Wang Nan (3):
tools lib traceevent: add priv field to truct format_field.
perf: convert: fix duplicate field names and avoid reserved keywords.
perf: convert: fix signess of value.

heya,
looks ok to me.. I applied/pushed this on the top fo my 'perf/core_ctf_convert' branch and will repost together with CTF v4 post, once we figure out the:
http://marc.info/?l=linux-kernel&m=142219345232713&w=2

and get feedback from Steven about the libtrace change ;-)

thanks,
jirka