perf tools: Add perf data CTF conversion

hi,
this is follow up on original RFC patchset:
http://marc.info/?t=140732735600004&r=1&w=2

Basically we are adding 'perf data convert' command to allow conversion of perf data file into CTF [1] data.

v6 changes:
- first bunch of patches got accepted, adding the
rest plus additional fixes
- rebased to latest acme's perf/core
- removed acked/reviewed tags from patch 2, due to small change - changed ctf_writer__flush_streams to return error value
- it all works with recent tracecompass and libbabeltrace git tree
- after recent changes in build system, it needs to be compiled with:

      $ make LIBBABELTRACE=1 LIBBABELTRACE_DIR=/opt/libbabeltrace/
    this will change once new version libabeltrace is released
    and adopted by common distros

v5 changes:
- added struct field_format::alias instead of priv void*
- rebased to latest acme's perf/core

v4 changes:
- sorted out babeltrace new code regressions
- rebased to latest acme's perf/core
- includes fixes from Wang Nan, patches 9/10/11
- patch 1/11 changelog updated with babeltrace
compilation instructions [Arnaldo]

v3 changes:
- rebased to latest acme's perf/core

v2 changes:
- addressed comments from Namhyung
- rebased to latest acme's perf/core

Changes from RFC:
- able to generate CTF data, that are possible to be displayed under tracecompas GUI [3], please check several screenshots in here [4]
- storing CTF data streams per cpu
- several cleanups

Examples:
- Catch default perf data (cycles event):

  $ perf record ls
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.012 MB perf.data (~546 samples) ]

- To display converted CTF data run [2]:

  $ babeltrace ./ctf-data/
  [03:19:13.962125533] (+?.?????????) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 }
  [03:19:13.962130001] (+0.000004468) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 }
  [03:19:13.962131936] (+0.000001935) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 8 }
  [03:19:13.962133732] (+0.000001796) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 114 }
  [03:19:13.962135557] (+0.000001825) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 2087 }
  [03:19:13.962137627] (+0.000002070) cycles: { }, { ip = 0xFFFFFFFF81361938, tid = 20714, pid = 20714, period = 37582 }
  [03:19:13.962161091] (+0.000023464) cycles: { }, { ip = 0xFFFFFFFF8124218F, tid = 20714, pid = 20714, period = 600246 }
  [03:19:13.962517569] (+0.000356478) cycles: { }, { ip = 0xFFFFFFFF811A75DB, tid = 20714, pid = 20714, period = 1325731 }
  [03:19:13.969518008] (+0.007000439) cycles: { }, { ip = 0x34080917B2, tid = 20714, pid = 20714, period = 1144298 }

- To get some nice output in tracecompas GUI [3], please capture sched:* and syscall tracepoints like:

  # perf record -e 'sched:*,raw_syscalls:*' -a
  ^C[ perf record: Woken up 0 times to write data ]
  [ perf record: Captured and wrote 412.347 MB perf.data (~18015721 samples) ]

- To convert perf data file run:

  # perf data convert --to-ctf=./ctf 
  [ perf data convert: Converted 'perf.data' into CTF data './ctf' ]
  [ perf data convert: Converted and wrote 408.421 MB (3964792 samples) ]

- To display converted CTF data run [2]:

  # babeltrace ./ctf-data/
  [23:32:20.165354855] (+0.000000507) sched:sched_wakeup: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF810BCA72, perf_tid = 0, perf_pid = 0, perf_id = 462554, perf_period = 1, common_type = 265, ...
  [23:32:20.165359078] (+0.000001181) sched:sched_switch: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8172A110, perf_tid = 0, perf_pid = 0, perf_id = 462562, perf_period = 1, common_type = 263, ...
  [23:32:20.165364686] (+0.000000328) sched:sched_stat_runtime: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF810C8AE5, perf_tid = 5326, perf_pid = 5326, perf_id = 462610, perf_period = 11380, ...
  [23:32:20.165366067] (+0.000001205) sched:sched_switch: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8172A110, perf_tid = 5326, perf_pid = 5326, perf_id = 462562, perf_period = 1, common_type  ...
  [23:32:20.165723312] (+0.000001479) sched:sched_stat_runtime: { cpu_id = 2 }, { perf_ip = 0xFFFFFFFF810C8AE5, perf_tid = 11821, perf_pid = 11821, perf_id = 462612, perf_period = 1000265, ...
  [23:32:20.065282391] (+?.?????????) raw_syscalls:sys_enter: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF810230AF, perf_tid = 26155, perf_pid = 26155, perf_id = 462635, perf_period = 1, ...
  [23:32:20.065286422] (+0.000004031) raw_syscalls:sys_exit: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF810231D8, perf_tid = 26155, perf_pid = 26155, perf_id = 462639, perf_period = 1, ...

- Or run tracecompass and open the CTF data ;-)

Changes are also reachable in here:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/ctf

thanks,
jirka

[1] Common Trace Format - http://www.efficios.com/ctf
[2] babeltrace - http://www.efficios.com/babeltrace
[3] Trace compass - http://projects.eclipse.org/projects/tools.tracecompass
[4] screenshots - http://people.redhat.com/~jolsa/tracecompass-perf/

---
He Kuang (1):
perf data: Show error message when conversion failed

Jiri Olsa (3):
perf data: Enable stream flush within processing perf data: Add support for setting ordered_events queue size tools lib traceevent: Add alias field to struct format_field

Sebastian Andrzej Siewior (1):
perf data: Switch to multiple cpu stream files

Wang Nan (2):
perf data: Fix duplicate field names and avoid reserved keywords perf data: Fix signess of value

 tools/lib/traceevent/event-parse.c |   6 +-
 tools/lib/traceevent/event-parse.h |   1 +
 tools/perf/util/data-convert-bt.c  | 410 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----------
 3 files changed, 373 insertions(+), 44 deletions(-)

From: He Kuang <[email protected]>

Show message when errors occurred during conversion setup and conversion process.

Before this patch:

  $ ./perf data convert --to-ctf=ctf
  $ echo $?
  255

After this patch:

  $ ./perf data convert --to-ctf=ctf
  Error during conversion setup.

---
 tools/perf/util/data-convert-bt.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

index dd17c9a32fbc..db4f709af18d 100644
@@ -837,6 +837,8 @@ int bt_convert__perf2ctf(const char *input, const char *path, bool force)
 	err = perf_session__process_events(session);
 	if (!err)
 		err = bt_ctf_stream_flush(cw->stream);
+	else
+		pr_err("Error during conversion.\n");
 
 	fprintf(stderr,
 		"[ perf data convert: Converted '%s' into CTF data '%s' ]\n",

@@ -847,11 +849,15 @@ int bt_convert__perf2ctf(const char *input, const char *path, bool force)
 		(double) c.events_size / 1024.0 / 1024.0,
 		c.events_count);
 
-	/* its all good */
-free_session:
 	perf_session__delete(session);
+	ctf_writer__cleanup(cw);
 
+	return err;
+
+free_session:
+	perf_session__delete(session);
 free_writer:
 	ctf_writer__cleanup(cw);
+	pr_err("Error during conversion setup.\n");
 	return err;
 }

1.9.3

From: Sebastian Andrzej Siewior <[email protected]>

Currently we store the data into single data strea/file. The cpu if data is stored within the event sample. The lttng puts the CPU number that belongs to the event into the packet context instead into the event.

This patch makes sure that the trace produce by perf does look the same way. We now use one stream per-CPU. Having it all in one stream increased the total size of the resulting file. The test went from 416KiB (with perf_cpu event member) to 24MiB due to the required (and pointless) flush. With the per-cpu streams the total size went up to 588KiB.

---
 tools/perf/util/data-convert-bt.c | 206 +++++++++++++++++++++++++++++++++-----
 1 file changed, 182 insertions(+), 24 deletions(-)

index db4f709af18d..0a2d7a5b70f9 100644
@@ -38,12 +38,20 @@ struct evsel_priv {
 	struct bt_ctf_event_class *event_class;
 };
 
+#define MAX_CPUS	4096
+
+struct ctf_stream {
+	struct bt_ctf_stream *stream;
+	int cpu;
+};
+
 struct ctf_writer {
 	/* writer primitives */
-	struct bt_ctf_writer		*writer;
-	struct bt_ctf_stream		*stream;
-	struct bt_ctf_stream_class	*stream_class;
-	struct bt_ctf_clock		*clock;
+	struct bt_ctf_writer		 *writer;
+	struct ctf_stream		**stream;
+	int				  stream_cnt;
+	struct bt_ctf_stream_class	 *stream_class;
+	struct bt_ctf_clock		 *clock;
 
 	/* data types */
 	union {

@@ -346,12 +354,6 @@ static int add_generic_values(struct ctf_writer *cw,
 			return -1;
 	}
 
-	if (type & PERF_SAMPLE_CPU) {
-		ret = value_set_u32(cw, event, "perf_cpu", sample->cpu);
-		if (ret)
-			return -1;
-	}
-
 	if (type & PERF_SAMPLE_PERIOD) {
 		ret = value_set_u64(cw, event, "perf_period", sample->period);
 		if (ret)

@@ -381,6 +383,113 @@ static int add_generic_values(struct ctf_writer *cw,
 	return 0;
 }
 
+static int ctf_stream__flush(struct ctf_stream *cs)
+{
+	int err = 0;
+
+	if (cs) {
+		err = bt_ctf_stream_flush(cs->stream);
+		if (err)
+			pr_err("CTF stream %d flush failed\n", cs->cpu);
+
+		pr("Flush stream for cpu %d\n", cs->cpu);
+	}
+
+	return err;
+}
+
+static struct ctf_stream *ctf_stream__create(struct ctf_writer *cw, int cpu)
+{
+	struct ctf_stream *cs;
+	struct bt_ctf_field *pkt_ctx   = NULL;
+	struct bt_ctf_field *cpu_field = NULL;
+	struct bt_ctf_stream *stream   = NULL;
+	int ret;
+
+	cs = zalloc(sizeof(*cs));
+	if (!cs) {
+		pr_err("Failed to allocate ctf stream\n");
+		return NULL;
+	}
+
+	stream = bt_ctf_writer_create_stream(cw->writer, cw->stream_class);
+	if (!stream) {
+		pr_err("Failed to create CTF stream\n");
+		goto out;
+	}
+
+	pkt_ctx = bt_ctf_stream_get_packet_context(stream);
+	if (!pkt_ctx) {
+		pr_err("Failed to obtain packet context\n");
+		goto out;
+	}
+
+	cpu_field = bt_ctf_field_structure_get_field(pkt_ctx, "cpu_id");
+	bt_ctf_field_put(pkt_ctx);
+	if (!cpu_field) {
+		pr_err("Failed to obtain cpu field\n");
+		goto out;
+	}
+
+	ret = bt_ctf_field_unsigned_integer_set_value(cpu_field, (u32) cpu);
+	if (ret) {
+		pr_err("Failed to update CPU number\n");
+		goto out;
+	}
+
+	bt_ctf_field_put(cpu_field);
+
+	cs->cpu    = cpu;
+	cs->stream = stream;
+	return cs;
+
+out:
+	if (cpu_field)
+		bt_ctf_field_put(cpu_field);
+	if (stream)
+		bt_ctf_stream_put(stream);
+
+	free(cs);
+	return NULL;
+}
+
+static void ctf_stream__delete(struct ctf_stream *cs)
+{
+	if (cs) {
+		bt_ctf_stream_put(cs->stream);
+		free(cs);
+	}
+}
+
+static struct ctf_stream *ctf_stream(struct ctf_writer *cw, int cpu)
+{
+	struct ctf_stream *cs = cw->stream[cpu];
+
+	if (!cs) {
+		cs = ctf_stream__create(cw, cpu);
+		cw->stream[cpu] = cs;
+	}
+
+	return cs;
+}
+
+static int get_sample_cpu(struct ctf_writer *cw, struct perf_sample *sample,
+			  struct perf_evsel *evsel)
+{
+	int cpu = 0;
+
+	if (evsel->attr.sample_type & PERF_SAMPLE_CPU)
+		cpu = sample->cpu;
+
+	if (cpu > cw->stream_cnt) {
+		pr_err("Event was recorded for CPU %d, limit is at %d.\n",
+			cpu, cw->stream_cnt);
+		cpu = 0;
+	}
+
+	return cpu;
+}
+
 static int process_sample_event(struct perf_tool *tool,
 				union perf_event *_event __maybe_unused,
 				struct perf_sample *sample,

@@ -390,6 +499,7 @@ static int process_sample_event(struct perf_tool *tool,
 	struct convert *c = container_of(tool, struct convert, tool);
 	struct evsel_priv *priv = evsel->priv;
 	struct ctf_writer *cw = &c->writer;
+	struct ctf_stream *cs;
 	struct bt_ctf_event_class *event_class;
 	struct bt_ctf_event *event;
 	int ret;

@@ -424,9 +534,12 @@ static int process_sample_event(struct perf_tool *tool,
 			return -1;
 	}
 
-	bt_ctf_stream_append_event(cw->stream, event);
+	cs = ctf_stream(cw, get_sample_cpu(cw, sample, evsel));
+	if (cs)
+		bt_ctf_stream_append_event(cs->stream, event);
+
 	bt_ctf_event_put(event);
-	return 0;
+	return cs ? 0 : -1;
 }
 
 static int add_tracepoint_fields_types(struct ctf_writer *cw,

@@ -528,9 +641,6 @@ static int add_generic_types(struct ctf_writer *cw, struct perf_evsel *evsel,
 	if (type & PERF_SAMPLE_STREAM_ID)
 		ADD_FIELD(event_class, cw->data.u64, "perf_stream_id");
 
-	if (type & PERF_SAMPLE_CPU)
-		ADD_FIELD(event_class, cw->data.u32, "perf_cpu");
-
 	if (type & PERF_SAMPLE_PERIOD)
 		ADD_FIELD(event_class, cw->data.u64, "perf_period");
 

@@ -604,6 +714,39 @@ static int setup_events(struct ctf_writer *cw, struct perf_session *session)
 	return 0;
 }
 
+static int setup_streams(struct ctf_writer *cw, struct perf_session *session)
+{
+	struct ctf_stream **stream;
+	struct perf_header *ph = &session->header;
+	int ncpus;
+
+	/*
+	 * Try to get the number of cpus used in the data file,
+	 * if not present fallback to the MAX_CPUS.
+	 */
+	ncpus = ph->env.nr_cpus_avail ?: MAX_CPUS;
+
+	stream = zalloc(sizeof(*stream) * ncpus);
+	if (!stream) {
+		pr_err("Failed to allocate streams.\n");
+		return -ENOMEM;
+	}
+
+	cw->stream     = stream;
+	cw->stream_cnt = ncpus;
+	return 0;
+}
+
+static void free_streams(struct ctf_writer *cw)
+{
+	int cpu;
+
+	for (cpu = 0; cpu < cw->stream_cnt; cpu++)
+		ctf_stream__delete(cw->stream[cpu]);
+
+	free(cw->stream);
+}
+
 static int ctf_writer__setup_env(struct ctf_writer *cw,
 				 struct perf_session *session)
 {

@@ -713,7 +856,7 @@ static void ctf_writer__cleanup(struct ctf_writer *cw)
 	ctf_writer__cleanup_data(cw);
 
 	bt_ctf_clock_put(cw->clock);
-	bt_ctf_stream_put(cw->stream);
+	free_streams(cw);
 	bt_ctf_stream_class_put(cw->stream_class);
 	bt_ctf_writer_put(cw->writer);
 

@@ -725,8 +868,9 @@ static int ctf_writer__init(struct ctf_writer *cw, const char *path)
 {
 	struct bt_ctf_writer		*writer;
 	struct bt_ctf_stream_class	*stream_class;
-	struct bt_ctf_stream		*stream;
 	struct bt_ctf_clock		*clock;
+	struct bt_ctf_field_type	*pkt_ctx_type;
+	int				ret;
 
 	/* CTF writer */
 	writer = bt_ctf_writer_create(path);

@@ -767,14 +911,15 @@ static int ctf_writer__init(struct ctf_writer *cw, const char *path)
 	if (ctf_writer__init_data(cw))
 		goto err_cleanup;
 
-	/* CTF stream instance */
-	stream = bt_ctf_writer_create_stream(writer, stream_class);
-	if (!stream) {
-		pr("Failed to create CTF stream.\n");
+	/* Add cpu_id for packet context */
+	pkt_ctx_type = bt_ctf_stream_class_get_packet_context_type(stream_class);
+	if (!pkt_ctx_type)
 		goto err_cleanup;
-	}
 
-	cw->stream = stream;
+	ret = bt_ctf_field_type_structure_add_field(pkt_ctx_type, cw->data.u32, "cpu_id");
+	bt_ctf_field_type_put(pkt_ctx_type);
+	if (ret)
+		goto err_cleanup;
 
 	/* CTF clock writer setup */
 	if (bt_ctf_writer_add_clock(writer, clock)) {

@@ -791,6 +936,16 @@ err:
 	return -1;
 }
 
+static int ctf_writer__flush_streams(struct ctf_writer *cw)
+{
+	int cpu, ret = 0;
+
+	for (cpu = 0; cpu < cw->stream_cnt && !ret; cpu++)
+		ret = ctf_stream__flush(cw->stream[cpu]);
+
+	return ret;
+}
+
 int bt_convert__perf2ctf(const char *input, const char *path, bool force)
 {
 	struct perf_session *session;

@@ -834,9 +989,12 @@ int bt_convert__perf2ctf(const char *input, const char *path, bool force)
 	if (setup_events(cw, session))
 		goto free_session;
 
+	if (setup_streams(cw, session))
+		goto free_session;
+
 	err = perf_session__process_events(session);
 	if (!err)
-		err = bt_ctf_stream_flush(cw->stream);
+		err = ctf_writer__flush_streams(cw);
 	else
 		pr_err("Error during conversion.\n");
 

1.9.3

Em Sat, Apr 18, 2015 at 05:50:15PM +0200, Jiri Olsa escreveu:

With latest babeltrace.git:

[acme@zoo babeltrace]$ git log --oneline | head -1 48d711a204f6 Fix: Missing stdint.h in stream-class.h

But I noticed I have this installed, as well:

[acme@zoo babeltrace]$ rpm -qa | grep babeltr
libbabeltrace-1.2.1-1.fc20.x86_64
[acme@zoo babeltrace]$

But:

[acme@zoo babeltrace]$ ldd ~/bin/perf | grep babeltrace

	libbabeltrace-ctf.so.1 => /opt/libbabeltrace/lib/libbabeltrace-ctf.so.1 (0x00007fb52713b000)
	libbabeltrace.so.1 => /opt/libbabeltrace/lib/libbabeltrace.so.1 (0x00007fb526889000)

[acme@zoo babeltrace]$

Will try removing the rpm as well...

[acme@zoo linux]$ perf data convert --to-ctf=ctf
perf: event-types.c:1855: bt_ctf_field_type_set_native_byte_order: Assertion `byte_order == 1234 || byte_order == 4321' failed. Aborted (core dumped)
[acme@zoo linux]$

Core was generated by `perf data convert --to-ctf=ctf'.
Program terminated with signal SIGABRT, Aborted.
#0 0x00007f843378a877 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56 return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
Missing separate debuginfos, use: debuginfo-install bzip2-libs-1.0.6-9.fc20.x86_64 libunwind-1.1-3.fc20.x86_64 libuuid-2.24.2-2.fc20.x86_64 nss-softokn-freebl-3.18.0-1.fc20.x86_64 numactl-libs-2.0.9-2.fc20.x86_64 popt-1.16-2.fc20.x86_64 python-libs-2.7.5-15.fc20.x86_64 slang-2.2.4-11.fc20.x86_64 xz-libs-5.1.2-12alpha.fc20.x86_64 (gdb) bt

#0  0x00007f843378a877 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x00007f843378bf68 in __GI_abort () at abort.c:89
#2  0x00007f84337837d6 in __assert_fail_base (fmt=0x7f84338d0ce8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", 
    assertion=assertion@entry=0x7f8432b3f510 "byte_order == 1234 || byte_order == 4321", file=file@entry=0x7f8432b3f3ba "event-types.c", 
    line=line@entry=1855, function=function@entry=0x7f8432b3f540 <__PRETTY_FUNCTION__.12860> "bt_ctf_field_type_set_native_byte_order")
    at assert.c:92
#3  0x00007f8433783882 in __GI___assert_fail (assertion=assertion@entry=0x7f8432b3f510 "byte_order == 1234 || byte_order == 4321", 
    file=file@entry=0x7f8432b3f3ba "event-types.c", line=line@entry=1855, 
    function=function@entry=0x7f8432b3f540 <__PRETTY_FUNCTION__.12860> "bt_ctf_field_type_set_native_byte_order") at assert.c:101
#4  0x00007f8432b313fa in bt_ctf_field_type_set_native_byte_order (type=<optimized out>, byte_order=<optimized out>) at event-types.c:1855
#5  0x00007f8432b32c09 in bt_ctf_stream_class_freeze (stream_class=0x23a5b50) at stream-class.c:478
#6  0x00007f8432b2ce75 in bt_ctf_event_create (event_class=event_class@entry=0x23aabc0) at event.c:324
#7  0x00000000004e71bd in process_sample_event (tool=0x7ffe2413eaa0, _event=<optimized out>, sample=0x7ffe2413e650, evsel=0x23a9eb0, 
    machine=<optimized out>) at util/data-convert-bt.c:518
#8  0x00000000004b0b35 in perf_evlist__deliver_sample (machine=0x23a92e0, evsel=0x23a9eb0, sample=0x7ffe2413e650, event=0x7f84360962b0, 
    tool=0x7ffe2413eaa0, evlist=0x23a94d0) at util/session.c:883

#9 machines__deliver_event (file_offset=<optimized out>, tool=<optimized out>, sample=0x7ffe2413e650, event=<optimized out>,
evlist=<optimized out>, machines=<optimized out>) at util/session.c:920
#10 ordered_events__deliver_event (oe=<optimized out>, event=<optimized out>) at util/session.c:108
#11 0x00000000004b4e54 in __ordered_events__flush (oe=0x23a9410) at util/ordered-events.c:207
#12 ordered_events__flush (oe=oe@entry=0x23a9410, how=how@entry=OE_FLUSH__FINAL) at util/ordered-events.c:271
#13 0x00000000004b3ad8 in __perf_session__process_events (file_size=18224, data_size=<optimized out>, data_offset=<optimized out>, session=0x23a9220)

    at util/session.c:1404
#14 perf_session__process_events (session=session@entry=0x23a9220) at util/session.c:1422
#15 0x00000000004e81db in bt_convert__perf2ctf (input=<optimized out>, path=0x7ffe241402ff "ctf", force=<optimized out>)
    at util/data-convert-bt.c:995

#16 0x00000000004497c5 in cmd_data_convert (argc=<optimized out>, argv=<optimized out>, prefix=<optimized out>) at builtin-data.c:81 #17 0x0000000000472165 in run_builtin (p=p@entry=0x838640 <commands+576>, argc=argc@entry=3, argv=argv@entry=0x7ffe2413f190) at perf.c:370 #18 0x000000000041e660 in handle_internal_command (argv=0x7ffe2413f190, argc=3) at perf.c:429
#19 run_argv (argv=0x7ffe2413ef10, argcp=0x7ffe2413ef1c) at perf.c:473
#20 main (argc=3, argv=0x7ffe2413f190) at perf.c:588
(gdb)

[acme@zoo linux]$ ls -la ctf
total 60
drwxrwx---. 2 acme acme 4096 Abr 20 11:57 .
drwxrwxr-x. 30 acme acme 20480 Abr 20 16:54 ..
-rw-rw----. 1 acme acme 0 Abr 20 16:54 metadata
-rw-rw----. 1 acme acme 32768 Abr 20 16:49 perf_stream_0 [acme@zoo linux]$ babeltrace ctf
[warning] Ill-shapen or missing "/* CTF x.y" header for text-only metadata. [warning] Unsupported CTF specification version 32547.6313276. Trying anyway. [error] at line 1: token "": syntax error, unexpected $end

[error] Error creating AST
[warning] Unable to open trace metadata for path "ctf". [warning] [Context] Cannot open_trace of format ctf at path ctf. [warning] [Context] cannot open trace "ctf" from ctf for reading. [error] Cannot open any trace for reading.

[error] opening trace "ctf" for reading.

[error] none of the specified trace paths could be opened.

[acme@zoo linux]$

Em Mon, Apr 20, 2015 at 04:58:54PM -0300, Arnaldo Carvalho de Melo escreveu:

Didn't help, removing this patch makes it work again:

[acme@zoo linux]$ perf data convert --to-ctf=ctf
[ perf data convert: Converted 'perf.data' into CTF data 'ctf' ]
[ perf data convert: Converted and wrote 0.000 MB (8 samples) ]
[acme@zoo linux]$ babeltrace ctf
[20:29:41.303589157] (+?.?????????) cycles: { }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 23321, perf_pid = 23321, perf_period = 1 } [20:29:41.303591929] (+0.000002772) cycles: { }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 23321, perf_pid = 23321, perf_period = 1 } [20:29:41.303592989] (+0.000001060) cycles: { }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 23321, perf_pid = 23321, perf_period = 11 } [20:29:41.303594530] (+0.000001541) cycles: { }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 23321, perf_pid = 23321, perf_period = 275 } [20:29:41.303873972] (+0.000279442) cycles: { }, { perf_ip = 0xFFFFFFFF810D659E, perf_tid = 23321, perf_pid = 23321, perf_period = 5458 } [20:29:41.303876335] (+0.000002363) cycles: { }, { perf_ip = 0xFFFFFFFF8109F508, perf_tid = 23321, perf_pid = 23321, perf_period = 5388 } [20:29:41.303878723] (+0.000002388) cycles: { }, { perf_ip = 0x7F19FA50063F, perf_tid = 23321, perf_pid = 23321, perf_period = 73411 } [20:29:41.303902598] (+0.000023875) cycles: { }, { perf_ip = 0xFFFFFFFF810DB934, perf_tid = 23321, perf_pid = 23321, perf_period = 1026523 } [acme@zoo linux]$

- Arnaldo

hum, this is rather old one (Mar 10th), current is:

[jolsa@krava babeltrace]$ git show --oneline | head -1 d8f190b24d52 Add tests for bt_ctf_field_string_append_len()

jirka

Em Mon, Apr 20, 2015 at 10:19:44PM +0200, Jiri Olsa escreveu:

I did a git remove update...

Lemme double check:

[acme@zoo babeltrace]$ git remote update Fetching origin
[acme@zoo babeltrace]$ git remote -v
origin git://git.efficios.com/babeltrace.git (fetch)
origin git://git.efficios.com/babeltrace.git (push)
[acme@zoo babeltrace]$ git log --oneline | head -1 48d711a204f6 Fix: Missing stdint.h in stream-class.h [acme@zoo babeltrace]$

What is wrong here? Is it in a different branch?

/me checks again that... Yeah, I probably created my local branch from some other branch, now that I did:

[acme@zoo babeltrace]$ git checkout -b master origin/master Branch master set up to track remote branch master from origin. Switched to a new branch 'master'
[acme@zoo babeltrace]$ git log --oneline | head -1 d8f190b24d52 Add tests for bt_ctf_field_string_append_len() [acme@zoo babeltrace]$

will try again those patches...

- ARnaldo

Em Mon, Apr 20, 2015 at 05:41:29PM -0300, Arnaldo Carvalho de Melo escreveu:

<SNIP>

Works now, continuing to process the others.o

- Arnaldo

[root@zoo ~]# perf record -a usleep 1
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 1.343 MB perf.data (24 samples) ] [root@zoo ~]#
[root@zoo ~]# perf data convert --to-ctf=ctf
[ perf data convert: Converted 'perf.data' into CTF data 'ctf' ] [ perf data convert: Converted and wrote 0.001 MB (24 samples) ] [root@zoo ~]# ls -la ctf
total 176
drwxr-x---. 2 root root 4096 Apr 20 17:48 . dr-xr-x---. 21 root root 36864 Apr 20 17:48 ..

-rw-r-----.  1 root root  2468 Apr 20 17:48 metadata
-rw-r-----.  1 root root 32768 Apr 20 17:48 perf_stream_0
-rw-r-----.  1 root root 32768 Apr 20 17:48 perf_stream_1
-rw-r-----.  1 root root 32768 Apr 20 17:48 perf_stream_2
-rw-r-----.  1 root root 32768 Apr 20 17:48 perf_stream_3

[root@zoo ~]# babeltrace ctf
bash: babeltrace: command not found...
[root@zoo ~]# /opt/libbabeltrace/bin/babeltrace ctf
[23:15:23.559883150] (+?.?????????) cycles: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 2344, perf_pid = 2344, perf_period = 1 } [23:15:23.559885913] (+0.000002763) cycles: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 2344, perf_pid = 2344, perf_period = 1 } [23:15:23.559886994] (+0.000001081) cycles: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 2344, perf_pid = 2344, perf_period = 11 } [23:15:23.559889237] (+0.000002243) cycles: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 2344, perf_pid = 2344, perf_period = 281 } [23:15:23.559928349] (+0.000039112) cycles: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 0, perf_pid = 0, perf_period = 1 } [23:15:23.559930427] (+0.000002078) cycles: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 0, perf_pid = 0, perf_period = 1 } [23:15:23.559931532] (+0.000001105) cycles: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 0, perf_pid = 0, perf_period = 14 } [23:15:23.559932945] (+0.000001413) cycles: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 0, perf_pid = 0, perf_period = 382 } [23:15:23.559940818] (+0.000007873) cycles: { cpu_id = 2 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 0, perf_pid = 0, perf_period = 1 } [23:15:23.559942796] (+0.000001978) cycles: { cpu_id = 2 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 0, perf_pid = 0, perf_period = 1 } [23:15:23.559944288] (+0.000001492) cycles: { cpu_id = 2 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 0, perf_pid = 0, perf_period = 16 } [23:15:23.559946113] (+0.000001825) cycles: { cpu_id = 2 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 0, perf_pid = 0, perf_period = 344 } [23:15:23.559964274] (+0.000018161) cycles: { cpu_id = 3 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 0, perf_pid = 0, perf_period = 1 } [23:15:23.559966375] (+0.000002101) cycles: { cpu_id = 3 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 0, perf_pid = 0, perf_period = 1 } [23:15:23.559967491] (+0.000001116) cycles: { cpu_id = 3 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 0, perf_pid = 0, perf_period = 15 } [23:15:23.559968894] (+0.000001403) cycles: { cpu_id = 3 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 0, perf_pid = 0, perf_period = 404 } [23:15:23.560102516] (+0.000133622) cycles: { cpu_id = 3 }, { perf_ip = 0xFFFFFFFF810D659E, perf_tid = 2345, perf_pid = 2345, perf_period = 8401 } [23:15:23.560105701] (+0.000003185) cycles: { cpu_id = 3 }, { perf_ip = 0xFFFFFFFF810CDDC0, perf_tid = 2345, perf_pid = 2345, perf_period = 9317 } [23:15:23.560109298] (+0.000003597) cycles: { cpu_id = 3 }, { perf_ip = 0xFFFFFFFF8131793E, perf_tid = 2345, perf_pid = 2345, perf_period = 97191 } [23:15:23.560140949] (+0.000031651) cycles: { cpu_id = 3 }, { perf_ip = 0xFFFFFFFF81100D70, perf_tid = 0, perf_pid = 0, perf_period = 929183 } [23:15:23.562123834] (+0.001982885) cycles: { cpu_id = 2 }, { perf_ip = 0xFFFFFFFF8109F5F5, perf_tid = 2345, perf_pid = 2345, perf_period = 5662 } [23:15:23.562133948] (+0.000010114) cycles: { cpu_id = 2 }, { perf_ip = 0xFFFFFFFF8104C859, perf_tid = 2345, perf_pid = 2345, perf_period = 5662 } [23:15:23.562143711] (+0.000009763) cycles: { cpu_id = 2 }, { perf_ip = 0xFFFFFFFF810CDFA5, perf_tid = 2345, perf_pid = 2345, perf_period = 21836 } [23:15:23.562174958] (+0.000031247) cycles: { cpu_id = 2 }, { perf_ip = 0xFFFFFFFF813A2B50, perf_tid = 2345, perf_pid = 2345, perf_period = 88041 } [root@zoo ~]#

For big data files the size of data allocated for stream instance could get really high. It's needed to flush the data out of the stream once in a while.

Unfortunately there's no size indication in the stream object, so we govern the flush based on the number of stored events. Current flush limit is set ot 100000 events.

---
 tools/perf/util/data-convert-bt.c | 26 ++++++++++++++++++++++++--
 1 file changed, 24 insertions(+), 2 deletions(-)

index 0a2d7a5b70f9..4f942b37067b 100644
@@ -43,6 +43,7 @@ struct evsel_priv {
 struct ctf_stream {
 	struct bt_ctf_stream *stream;
 	int cpu;
+	u32 count;
 };
 
 struct ctf_writer {

@@ -392,7 +393,10 @@ static int ctf_stream__flush(struct ctf_stream *cs)
 		if (err)
 			pr_err("CTF stream %d flush failed\n", cs->cpu);
 
-		pr("Flush stream for cpu %d\n", cs->cpu);
+		pr("Flush stream for cpu %d (%u samples)\n",
+		   cs->cpu, cs->count);
+
+		cs->count = 0;
 	}
 
 	return err;

@@ -490,6 +494,19 @@ static int get_sample_cpu(struct ctf_writer *cw, struct perf_sample *sample,
 	return cpu;
 }
 
+#define STREAM_FLUSH_COUNT 100000
+
+/*
+ * Currently we have no other way to determine the
+ * time for the stream flush other than keep track
+ * of the number of events and check it against
+ * threshold.
+ */
+static bool is_flush_needed(struct ctf_stream *cs)
+{
+	return cs->count >= STREAM_FLUSH_COUNT;
+}
+
 static int process_sample_event(struct perf_tool *tool,
 				union perf_event *_event __maybe_unused,
 				struct perf_sample *sample,

@@ -535,8 +552,13 @@ static int process_sample_event(struct perf_tool *tool,
 	}
 
 	cs = ctf_stream(cw, get_sample_cpu(cw, sample, evsel));
-	if (cs)
+	if (cs) {
+		if (is_flush_needed(cs))
+			ctf_stream__flush(cs);
+
+		cs->count++;
 		bt_ctf_stream_append_event(cs->stream, event);
+	}
 
 	bt_ctf_event_put(event);
 	return cs ? 0 : -1;

1.9.3

Adding support to limit the size of ordered_events queue, so we could control allocation size of perf data files without proper finished round events.

---
 tools/perf/util/data-convert-bt.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

index 4f942b37067b..b35c8d6c291d 100644
@@ -74,6 +74,9 @@ struct convert {
 
 	u64			events_size;
 	u64			events_count;
+
+	/* Ordered events configured queue size. */
+	u64			queue_size;
 };
 
 static int value_set(struct bt_ctf_field_type *type,

@@ -968,6 +971,18 @@ static int ctf_writer__flush_streams(struct ctf_writer *cw)
 	return ret;
 }
 
+static int convert__config(const char *var, const char *value, void *cb)
+{
+	struct convert *c = cb;
+
+	if (!strcmp(var, "convert.queue-size")) {
+		c->queue_size = perf_config_u64(var, value);
+		return 0;
+	}
+
+	return perf_default_config(var, value, cb);
+}
+
 int bt_convert__perf2ctf(const char *input, const char *path, bool force)
 {
 	struct perf_session *session;

@@ -994,6 +1009,8 @@ int bt_convert__perf2ctf(const char *input, const char *path, bool force)
 	struct ctf_writer *cw = &c.writer;
 	int err = -1;
 
+	perf_config(convert__config, &c);
+
 	/* CTF writer */
 	if (ctf_writer__init(cw, path))
 		return -1;

@@ -1003,6 +1020,11 @@ int bt_convert__perf2ctf(const char *input, const char *path, bool force)
 	if (!session)
 		goto free_writer;
 
+	if (c.queue_size) {
+		ordered_events__set_alloc_size(&session->ordered_events,
+					       c.queue_size);
+	}
+
 	/* CTF writer env/clock setup  */
 	if (ctf_writer__setup_env(cw, session))
 		goto free_session;

1.9.3

Introduce an 'alias' field to 'struct format_field' to be able to use alternative name for the field.

It is initialized with same string pointer as 'name' field. The free logic checks the 'alias' pointer being reset by user and frees it.

This will be handy when converting data into CTF, where each field within event needs to have a unique name (while this is not required for tracepoint). Converter can easily assign unique name into the format_field struct.

---
 tools/lib/traceevent/event-parse.c | 6 ++++--
 tools/lib/traceevent/event-parse.h | 1 +
 2 files changed, 5 insertions(+), 2 deletions(-)

index 12a7e2a40c89..1ec56b459bc0 100644
@@ -1387,7 +1387,7 @@ static int event_read_fields(struct event_format *event, struct format_field **f
 			do_warning_event(event, "%s: no type found", __func__);
 			goto fail;
 		}
-		field->name = last_token;
+		field->name = field->alias = last_token;
 
 		if (test_type(type, EVENT_OP))
 			goto fail;

@@ -1469,7 +1469,7 @@ static int event_read_fields(struct event_format *event, struct format_field **f
 				size_dynamic = type_size(field->name);
 				free_token(field->name);
 				strcat(field->type, brackets);
-				field->name = token;
+				field->name = field->alias = token;
 				type = read_token(&token);
 			} else {
 				char *new_type;

@@ -6444,6 +6444,8 @@ void pevent_ref(struct pevent *pevent)
 void pevent_free_format_field(struct format_field *field)
 {
 	free(field->type);
+	if (field->alias != field->name)
+		free(field->alias);
 	free(field->name);
 	free(field);
 }

index 86a5839fb048..063b1971eb35 100644
@@ -191,6 +191,7 @@ struct format_field {
 	struct event_format	*event;
 	char			*type;
 	char			*name;
+	char			*alias;
 	int			offset;
 	int			size;
 	unsigned int		arraylen;

1.9.3

-- Steve

From: Wang Nan <[email protected]>

Some parameters of syscall tracepoints named as 'nr', 'event', etc. When dealing with them, perf convert to ctf meets some problem:

1. If a parameter with name 'nr', it will duplicate syscall's common field 'nr'. One such syscall is io_submit().

2. If a parameter with name 'event', it is denied to be inserted because 'event' is a babeltrace keywork. One such syscall is epoll_ctl.

This patch appends '_dupl_X' suffix to avoid problem 1, prepend a '_' prefix to avoid problem 2.

---
 tools/perf/util/data-convert-bt.c | 86 +++++++++++++++++++++++++++++++++++++--
 1 file changed, 82 insertions(+), 4 deletions(-)

index b35c8d6c291d..8eda4ed628e7 100644
@@ -14,6 +14,7 @@
 #include <babeltrace/ctf-writer/event.h>
 #include <babeltrace/ctf-writer/event-types.h>
 #include <babeltrace/ctf-writer/event-fields.h>
+#include <babeltrace/ctf-ir/utils.h>
 #include <babeltrace/ctf/events.h>
 #include <traceevent/event-parse.h>
 #include "asm/bug.h"

@@ -184,6 +185,7 @@ static int add_tracepoint_field_value(struct ctf_writer *cw,
 	unsigned int len;
 	int ret;
 
+	name = fmtf->alias;
 	offset = fmtf->offset;
 	len = fmtf->size;
 	if (flags & FIELD_IS_STRING)

@@ -567,6 +569,82 @@ static int process_sample_event(struct perf_tool *tool,
 	return cs ? 0 : -1;
 }
 
+/* If dup < 0, add a prefix. Else, add _dupl_X suffix. */
+static char *change_name(char *name, char *orig_name, int dup)
+{
+	char *new_name = NULL;
+	size_t len;
+
+	if (!name)
+		name = orig_name;
+
+	if (dup >= 10)
+		goto out;
+	/*
+	 * Add '_' prefix to potential keywork.  According to
+	 * Mathieu Desnoyers (https://lkml.org/lkml/2015/1/23/652),
+	 * futher CTF spec updating may require us to use '$'.
+	 */
+	if (dup < 0)
+		len = strlen(name) + sizeof("_");
+	else
+		len = strlen(orig_name) + sizeof("_dupl_X");
+
+	new_name = malloc(len);
+	if (!new_name)
+		goto out;
+
+	if (dup < 0)
+		snprintf(new_name, len, "_%s", name);
+	else
+		snprintf(new_name, len, "%s_dupl_%d", orig_name, dup);
+
+out:
+	if (name != orig_name)
+		free(name);
+	return new_name;
+}
+
+static int event_class_add_field(struct bt_ctf_event_class *event_class,
+		struct bt_ctf_field_type *type,
+		struct format_field *field)
+{
+	struct bt_ctf_field_type *t = NULL;
+	char *name;
+	int dup = 1;
+	int ret;
+
+	/* alias was already assigned */
+	if (field->alias != field->name)
+		return bt_ctf_event_class_add_field(event_class, type,
+				(char *)field->alias);
+
+	name = field->name;
+
+	/* If 'name' is a keywork, add prefix. */
+	if (bt_ctf_validate_identifier(name))
+		name = change_name(name, field->name, -1);
+
+	if (!name) {
+		pr_err("Failed to fix invalid identifier.");
+		return -1;
+	}
+	while ((t = bt_ctf_event_class_get_field_by_name(event_class, name))) {
+		bt_ctf_field_type_put(t);
+		name = change_name(name, field->name, dup++);
+		if (!name) {
+			pr_err("Failed to create dup name for '%s'\n", field->name);
+			return -1;
+		}
+	}
+
+	ret = bt_ctf_event_class_add_field(event_class, type, name);
+	if (!ret)
+		field->alias = name;
+
+	return ret;
+}
+
 static int add_tracepoint_fields_types(struct ctf_writer *cw,
 				       struct format_field *fields,
 				       struct bt_ctf_event_class *event_class)

@@ -595,14 +673,14 @@ static int add_tracepoint_fields_types(struct ctf_writer *cw,
 		if (flags & FIELD_IS_ARRAY)
 			type = bt_ctf_field_type_array_create(type, field->arraylen);
 
-		ret = bt_ctf_event_class_add_field(event_class, type,
-				field->name);
+		ret = event_class_add_field(event_class, type, field);
 
 		if (flags & FIELD_IS_ARRAY)
 			bt_ctf_field_type_put(type);
 
 		if (ret) {
-			pr_err("Failed to add field '%s\n", field->name);
+			pr_err("Failed to add field '%s': %d\n",
+					field->name, ret);
 			return -1;
 		}
 	}

@@ -646,7 +724,7 @@ static int add_generic_types(struct ctf_writer *cw, struct perf_evsel *evsel,
 	do {								\
 		pr2("  field '%s'\n", n);				\
 		if (bt_ctf_event_class_add_field(cl, t, n)) {		\
-			pr_err("Failed to add field '%s;\n", n);	\
+			pr_err("Failed to add field '%s';\n", n);	\
 			return -1;					\
 		}							\
 	} while (0)

1.9.3

Minor nitpick: "event" is a reserved keyword in the CTF specification [1], the rule is only enforced by Babeltrace. Also, keywork -> keyword.

Jérémie

[1] http://diamon.org/docs/ctf/v1.8.2/#specC.1.2

Jérémie Galarneau
EfficiOS Inc.
http://www.efficios.com

thanks,

Arnaldo, any chance you could change it? (with ack from He Kuang of course), or do you need a respin..

jirka

Em Sat, Apr 18, 2015 at 10:45:36PM +0200, Jiri Olsa escreveu:

I'll do it

From: Wang Nan <[email protected]>

When converting int values, perf first extractes it to a ulonglong, then feeds it to babeltrace as a signed value. For negative 32 bit values (for example, return values of failed syscalls), the extracted data should be something like 0xfffffffe (-2). It becomes a large int64 value. Babeltrace denies to insert it with bt_ctf_field_signed_integer_set_value() because it is larger than 0x7fffffff, the largest positive value a signed 32 bit int can be.

This patch introduces adjust_signess(), which fills high bits of ulonglong with 1 if the value is negative.

---
 tools/perf/util/data-convert-bt.c | 64 +++++++++++++++++++++++++++++++--------
 1 file changed, 52 insertions(+), 12 deletions(-)

index 8eda4ed628e7..977cc3f98d8f 100644
@@ -166,6 +166,43 @@ get_tracepoint_field_type(struct ctf_writer *cw, struct format_field *field)
 		return cw->data.u32;
 }
 
+static unsigned long long adjust_signess(unsigned long long value_int, int size)
+{
+	unsigned long long value_mask;
+
+	/*
+	 * value_mask = (1 << (size * 8 - 1)) - 1.
+	 * Directly set value_mask for code readers.
+	 */
+	switch (size) {
+	case 1:
+		value_mask = 0x7fULL;
+		break;
+	case 2:
+		value_mask = 0x7fffULL;
+		break;
+	case 4:
+		value_mask = 0x7fffffffULL;
+		break;
+	case 8:
+		/*
+		 * For 64 bit value, return it self. There is no need
+		 * to fill high bit.
+		 */
+		/* Fall through */
+	default:
+		/* BUG! */
+		return value_int;
+	}
+
+	/* If it is a positive value, don't adjust. */
+	if ((value_int & (~0ULL - value_mask)) == 0)
+		return value_int;
+
+	/* Fill upper part of value_int with 1 to make it a negative long long. */
+	return (value_int & value_mask) | ~value_mask;
+}
+
 static int add_tracepoint_field_value(struct ctf_writer *cw,
 				      struct bt_ctf_event_class *event_class,
 				      struct bt_ctf_event *event,

@@ -177,7 +214,6 @@ static int add_tracepoint_field_value(struct ctf_writer *cw,
 	struct bt_ctf_field *field;
 	const char *name = fmtf->name;
 	void *data = sample->raw_data;
-	unsigned long long value_int;
 	unsigned long flags = fmtf->flags;
 	unsigned int n_items;
 	unsigned int i;

@@ -222,11 +258,6 @@ static int add_tracepoint_field_value(struct ctf_writer *cw,
 	type = get_tracepoint_field_type(cw, fmtf);
 
 	for (i = 0; i < n_items; i++) {
-		if (!(flags & FIELD_IS_STRING))
-			value_int = pevent_read_number(
-					fmtf->event->pevent,
-					data + offset + i * len, len);
-
 		if (flags & FIELD_IS_ARRAY)
 			field = bt_ctf_field_array_get_field(array_field, i);
 		else

@@ -240,12 +271,21 @@ static int add_tracepoint_field_value(struct ctf_writer *cw,
 		if (flags & FIELD_IS_STRING)
 			ret = bt_ctf_field_string_set_value(field,
 					data + offset + i * len);
-		else if (!(flags & FIELD_IS_SIGNED))
-			ret = bt_ctf_field_unsigned_integer_set_value(
-					field, value_int);
-		else
-			ret = bt_ctf_field_signed_integer_set_value(
-					field, value_int);
+		else {
+			unsigned long long value_int;
+
+			value_int = pevent_read_number(
+					fmtf->event->pevent,
+					data + offset + i * len, len);
+
+			if (!(flags & FIELD_IS_SIGNED))
+				ret = bt_ctf_field_unsigned_integer_set_value(
+						field, value_int);
+			else
+				ret = bt_ctf_field_signed_integer_set_value(
+						field, adjust_signess(value_int, len));
+		}
+
 		if (ret) {
 			pr_err("failed to set file value %s\n", name);
 			goto err_put_field;

1.9.3

Em Sat, Apr 18, 2015 at 05:50:20PM +0200, Jiri Olsa escreveu:

There is no such word "signess", it is "signedness", fixing this up. Humm, it seems there is such a word indeed:

http://www.urbandictionary.com/define.php?term=Signess

But I bet this is the one we want:

http://en.wikipedia.org/wiki/Signedness

Right? :-)

- Arnaldo

right ;-) thanks

jirka

Sorry for the bad English. Please help me to fix it. Thank you.

Em Tue, Apr 21, 2015 at 10:41:55AM +0800, Wang Nan escreveu:

It is not a problem, thanks for your contributions, it all works that way, we find mistakes, we fix mistakes, then we make some more...

I only mentioned that urban dictionary link because I am not a native speaker as well, we speak portuguese down here, so I went to check if my feeling was right and found it, felt it was funny and unexpected, mentioned here, no offense intended :-)

- Arnaldo

Em Sat, Apr 18, 2015 at 05:50:13PM +0200, Jiri Olsa escreveu:

Patches 1 and 5 applied, the others don't apply because patch 2 failed my testing, reported.

- Arnaldo