Skip to contentSkip to conversation

perf tool: add meta-data header support for pipe-mode

PATCH Superseded David Carrillo-Cisneros 3 participants 43 messages May 23, 2017 – Jun 6, 2017 1 message not in the archive perf events

v2 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

v2 cover perf tool: add meta-data header support for pipe-mode

v2: - Finer patch splitting.
- Add only one record type with a feature id instead of one record per new feature (as suggested by Jiri).
- Add perf.data documentation.

(This is a rebased and updated version of Stephane Eranian's version
in https://patchwork.kernel.org/patch/1499081/)

Up until now, meta-data was only available when perf record was used in "regular" mode, i.e., generating a perf.data file. For users depending on pipe mode, neither host or event header information were gathered. This patch addresses this limitation.

The difficulty in pipe mode is that information needs to be written sequentially to the pipe. Meta data headers are usually generated (and also expected) at the beginning of the file (or piped output). To solve this problem, we introduce new synthetic record types, one for each meta-data type. The approach is similar to what is *ALREADY* used for BUILD_ID and TRACING_DATA.

We have modified util/header.c such that the same routines are used to generate and read the meta-data information regardless of pipe-mode vs. regular mode. To make this work, we added a new struct called feat_fd which encapsulates all the information necessary to read or write meta-data information to a file/pipe or from a file/pipe.

With this series, it is possible to get:

  $ perf record -o - -e cycles sleep 1 | perf report --stdio --header
  # ========
  # captured on: Mon May 22 16:33:43 2017
  # ========
  #
  # hostname : my_hostname
  # os release : 4.11.0-dbx-up_perf
  # perf version : 4.11.rc6.g6277c80
  # arch : x86_64
  # nrcpus online : 72
  # nrcpus avail : 72
  # cpudesc : Intel(R) Xeon(R) CPU E5-2696 v3 @ 2.30GHz
  # cpuid : GenuineIntel,6,63,2
  # total memory : 263457192 kB
  # cmdline : /root/perf record -o - -e cycles -c 100000 sleep 1
  # HEADER_CPU_TOPOLOGY info available, use -I to display
  # HEADER_NUMA_TOPOLOGY info available, use -I to display
  # pmu mappings: intel_bts = 6, uncore_imc_4 = 22, uncore_sbox_1 = 47, uncore_cbox_5 = 33, uncore_ha_0 = 16, uncore_cbox
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.000 MB - ]
  ...

Only patch 13/13 has a significant effect in perf's behavior. All other are transparent preparatory changes or bug fixes.

David Carrillo-Cisneros (13):
perf header: encapsulate read and swap
perf header: add PROCESS_STR_FUN macro
perf header: fail on write_padded error
perf util: add const modifier to buf in "writen" function perf header: revamp do_write
perf header: add struct feat_fd for write
perf header: use struct feat_fd for print
perf header: use struct feat_fd to process header records perf header: use struct feat_fd in read header records perf header: add a buffer to struct feat_fd
perf header: change FEAT_OP* macros
perf tool: add show_feature_header to perf_tool perf tools: add feature header record to pipe-mode

 tools/perf/Documentation/perf.data-file-format.txt |  10 +-
 tools/perf/builtin-annotate.c                      |   1 +
 tools/perf/builtin-inject.c                        |   1 +
 tools/perf/builtin-record.c                        |   7 +
 tools/perf/builtin-report.c                        |  12 +-
 tools/perf/builtin-script.c                        |   4 +
 tools/perf/util/build-id.c                         |  10 +-
 tools/perf/util/build-id.h                         |   4 +-
 tools/perf/util/event.c                            |   1 +
 tools/perf/util/event.h                            |   8 +
 tools/perf/util/header.c                           | 877 +++++++++++----------
 tools/perf/util/header.h                           |  16 +-
 tools/perf/util/session.c                          |  12 +
 tools/perf/util/tool.h                             |  10 +-
 tools/perf/util/util.c                             |   6 +-
 tools/perf/util/util.h                             |   2 +-
 16 files changed, 564 insertions(+), 417 deletions(-)
Signature · 1 line
2.13.0.219.gdb65acc882-goog
#1 lore raw

PATCH v2 1/13 perf header: encapsulate read and swap

Most callers of readn in perf header read either a 32 or a 64 bits number, error check it and swap it, if necessary.

Create do_read_u32 and do_read_u64 to simplify this usage.

---
 tools/perf/util/header.c | 212 +++++++++++++++++------------------------------
 1 file changed, 76 insertions(+), 136 deletions(-)
tools/perf/util/header.c +76 -136
index 948b2c5efb65..1dd4dbe13f88 100644
@@ -117,27 +117,56 @@ static int do_write_string(int fd, const char *str)
 	return write_padded(fd, str, olen, len);
 }
 
+static int __do_read(int fd, void *addr, ssize_t size)
+{
+	ssize_t ret = readn(fd, addr, size);
+
+	if (ret != (ssize_t)size)
+		return ret < 0 ? (int)ret : -1;
+	return 0;
+}
+
+static int do_read_u32(int fd, struct perf_header *ph, u32 *addr)
+{
+	int ret;
+
+	ret = __do_read(fd, addr, sizeof(*addr));
+	if (ret)
+		return ret;
+
+	if (ph->needs_swap)
+		*addr = bswap_32(*addr);
+	return 0;
+}
+
+static int do_read_u64(int fd, struct perf_header *ph, u64 *addr)
+{
+	int ret;
+
+	ret = __do_read(fd, addr, sizeof(*addr));
+	if (ret)
+		return ret;
+
+	if (ph->needs_swap)
+		*addr = bswap_64(*addr);
+	return 0;
+}
+
 static char *do_read_string(int fd, struct perf_header *ph)
 {
-	ssize_t sz, ret;
 	u32 len;
 	char *buf;
 
-	sz = readn(fd, &len, sizeof(len));
-	if (sz < (ssize_t)sizeof(len))
+	if (do_read_u32(fd, ph, &len))
 		return NULL;
 
-	if (ph->needs_swap)
-		len = bswap_32(len);
-
 	buf = malloc(len);
 	if (!buf)
 		return NULL;
 
-	ret = readn(fd, buf, len);
-	if (ret == (ssize_t)len) {
+	if (!__do_read(fd, buf, len)) {
 		/*
-		 * strings are padded by zeroes
+		 * note that strings are padded by zeroes
 		 * thus the actual strlen of buf
 		 * may be less than len
 		 */
@@ -1190,20 +1219,12 @@ read_event_desc(struct perf_header *ph, int fd)
 	size_t msz;
 
 	/* number of events */
-	ret = readn(fd, &nre, sizeof(nre));
-	if (ret != (ssize_t)sizeof(nre))
+	if (do_read_u32(fd, ph, &nre))
 		goto error;
 
-	if (ph->needs_swap)
-		nre = bswap_32(nre);
-
-	ret = readn(fd, &sz, sizeof(sz));
-	if (ret != (ssize_t)sizeof(sz))
+	if (do_read_u32(fd, ph, &sz))
 		goto error;
 
-	if (ph->needs_swap)
-		sz = bswap_32(sz);
-
 	/* buffer to hold on file attr struct */
 	buf = malloc(sz);
 	if (!buf)
@@ -1225,7 +1246,7 @@ read_event_desc(struct perf_header *ph, int fd)
 		 * must read entire on-file attr struct to
 		 * sync up with layout.
 		 */
-		ret = readn(fd, buf, sz);
+		ret = __do_read(fd, buf, sz);
 		if (ret != (ssize_t)sz)
 			goto error;
 
@@ -1234,14 +1255,11 @@ read_event_desc(struct perf_header *ph, int fd)
 
 		memcpy(&evsel->attr, buf, msz);
 
-		ret = readn(fd, &nr, sizeof(nr));
-		if (ret != (ssize_t)sizeof(nr))
+		if (do_read_u32(fd, ph, &nr))
 			goto error;
 
-		if (ph->needs_swap) {
-			nr = bswap_32(nr);
+		if (ph->needs_swap)
 			evsel->needs_swap = true;
-		}
 
 		evsel->name = do_read_string(fd, ph);
 
@@ -1255,11 +1273,8 @@ read_event_desc(struct perf_header *ph, int fd)
 		evsel->id = id;
 
 		for (j = 0 ; j < nr; j++) {
-			ret = readn(fd, id, sizeof(*id));
-			if (ret != (ssize_t)sizeof(*id))
+			if (do_read_u64(fd, ph, id))
 				goto error;
-			if (ph->needs_swap)
-				*id = bswap_64(*id);
 			id++;
 		}
 	}
@@ -1631,26 +1646,18 @@ static int process_nrcpus(struct perf_file_section *section __maybe_unused,
 			  struct perf_header *ph, int fd,
 			  void *data __maybe_unused)
 {
-	ssize_t ret;
-	u32 nr;
-
-	ret = readn(fd, &nr, sizeof(nr));
-	if (ret != sizeof(nr))
-		return -1;
-
-	if (ph->needs_swap)
-		nr = bswap_32(nr);
-
-	ph->env.nr_cpus_avail = nr;
-
-	ret = readn(fd, &nr, sizeof(nr));
-	if (ret != sizeof(nr))
-		return -1;
+	int ret;
+	u32 nr_cpus_avail, nr_cpus_online;
 
-	if (ph->needs_swap)
-		nr = bswap_32(nr);
+	ret = do_read_u32(fd, ph, &nr_cpus_avail);
+	if (ret)
+		return ret;
 
-	ph->env.nr_cpus_online = nr;
+	ret = do_read_u32(fd, ph, &nr_cpus_online);
+	if (ret)
+		return ret;
+	ph->env.nr_cpus_avail = (int)nr_cpus_avail;
+	ph->env.nr_cpus_online = (int)nr_cpus_online;
 	return 0;
 }
 
@@ -1674,17 +1681,13 @@ static int process_total_mem(struct perf_file_section *section __maybe_unused,
 			     struct perf_header *ph, int fd,
 			     void *data __maybe_unused)
 {
-	uint64_t mem;
-	ssize_t ret;
+	u64 total_mem;
+	int ret;
 
-	ret = readn(fd, &mem, sizeof(mem));
-	if (ret != sizeof(mem))
+	ret = do_read_u64(fd, ph, &total_mem);
+	if (ret)
 		return -1;
-
-	if (ph->needs_swap)
-		mem = bswap_64(mem);
-
-	ph->env.total_mem = mem;
+	ph->env.total_mem = (unsigned long long)total_mem;
 	return 0;
 }
 
@@ -1744,17 +1747,12 @@ static int process_cmdline(struct perf_file_section *section,
 			   struct perf_header *ph, int fd,
 			   void *data __maybe_unused)
 {
-	ssize_t ret;
 	char *str, *cmdline = NULL, **argv = NULL;
 	u32 nr, i, len = 0;
 
-	ret = readn(fd, &nr, sizeof(nr));
-	if (ret != sizeof(nr))
+	if (do_read_u32(fd, ph, &nr))
 		return -1;
 
-	if (ph->needs_swap)
-		nr = bswap_32(nr);
-
 	ph->env.nr_cmdline = nr;
 
 	cmdline = zalloc(section->size + nr + 1);
@@ -1789,7 +1787,6 @@ static int process_cpu_topology(struct perf_file_section *section,
 				struct perf_header *ph, int fd,
 				void *data __maybe_unused)
 {
-	ssize_t ret;
 	u32 nr, i;
 	char *str;
 	struct strbuf sb;
@@ -1800,15 +1797,10 @@ static int process_cpu_topology(struct perf_file_section *section,
 	if (!ph->env.cpu)
 		return -1;
 
-	ret = readn(fd, &nr, sizeof(nr));
-	if (ret != sizeof(nr))
+	if (do_read_u32(fd, ph, &nr))
 		goto free_cpu;
 
-	if (ph->needs_swap)
-		nr = bswap_32(nr);
-
 	ph->env.nr_sibling_cores = nr;
-	size += sizeof(u32);
 	if (strbuf_init(&sb, 128) < 0)
 		goto free_cpu;
 
@@ -1820,20 +1812,14 @@ static int process_cpu_topology(struct perf_file_section *section,
 		/* include a NULL character at the end */
 		if (strbuf_add(&sb, str, strlen(str) + 1) < 0)
 			goto error;
-		size += string_size(str);
 		free(str);
 	}
 	ph->env.sibling_cores = strbuf_detach(&sb, NULL);
 
-	ret = readn(fd, &nr, sizeof(nr));
-	if (ret != sizeof(nr))
+	if (do_read_u32(fd, ph, &nr))
 		return -1;
 
-	if (ph->needs_swap)
-		nr = bswap_32(nr);
-
 	ph->env.nr_sibling_threads = nr;
-	size += sizeof(u32);
 
 	for (i = 0; i < nr; i++) {
 		str = do_read_string(fd, ph);
@@ -1843,7 +1829,6 @@ static int process_cpu_topology(struct perf_file_section *section,
 		/* include a NULL character at the end */
 		if (strbuf_add(&sb, str, strlen(str) + 1) < 0)
 			goto error;
-		size += string_size(str);
 		free(str);
 	}
 	ph->env.sibling_threads = strbuf_detach(&sb, NULL);
@@ -1858,22 +1843,14 @@ static int process_cpu_topology(struct perf_file_section *section,
 	}
 
 	for (i = 0; i < (u32)cpu_nr; i++) {
-		ret = readn(fd, &nr, sizeof(nr));
-		if (ret != sizeof(nr))
+		if (do_read_u32(fd, ph, &nr))
 			goto free_cpu;
 
-		if (ph->needs_swap)
-			nr = bswap_32(nr);
-
 		ph->env.cpu[i].core_id = nr;
 
-		ret = readn(fd, &nr, sizeof(nr));
-		if (ret != sizeof(nr))
+		if (do_read_u32(fd, ph, &nr))
 			goto free_cpu;
 
-		if (ph->needs_swap)
-			nr = bswap_32(nr);
-
 		if (nr != (u32)-1 && nr > (u32)cpu_nr) {
 			pr_debug("socket_id number is too big."
 				 "You may need to upgrade the perf tool.\n");
@@ -1897,18 +1874,13 @@ static int process_numa_topology(struct perf_file_section *section __maybe_unuse
 				 void *data __maybe_unused)
 {
 	struct numa_node *nodes, *n;
-	ssize_t ret;
 	u32 nr, i;
 	char *str;
 
 	/* nr nodes */
-	ret = readn(fd, &nr, sizeof(nr));
-	if (ret != sizeof(nr))
+	if (do_read_u32(fd, ph, &nr))
 		return -1;
 
-	if (ph->needs_swap)
-		nr = bswap_32(nr);
-
 	nodes = zalloc(sizeof(*nodes) * nr);
 	if (!nodes)
 		return -ENOMEM;
@@ -1917,24 +1889,15 @@ static int process_numa_topology(struct perf_file_section *section __maybe_unuse
 		n = &nodes[i];
 
 		/* node number */
-		ret = readn(fd, &n->node, sizeof(u32));
-		if (ret != sizeof(n->node))
+		if (do_read_u32(fd, ph, &n->node))
 			goto error;
 
-		ret = readn(fd, &n->mem_total, sizeof(u64));
-		if (ret != sizeof(u64))
+		if (do_read_u64(fd, ph, &n->mem_total))
 			goto error;
 
-		ret = readn(fd, &n->mem_free, sizeof(u64));
-		if (ret != sizeof(u64))
+		if (do_read_u64(fd, ph, &n->mem_free))
 			goto error;
 
-		if (ph->needs_swap) {
-			n->node      = bswap_32(n->node);
-			n->mem_total = bswap_64(n->mem_total);
-			n->mem_free  = bswap_64(n->mem_free);
-		}
-
 		str = do_read_string(fd, ph);
 		if (!str)
 			goto error;
@@ -1958,19 +1921,14 @@ static int process_pmu_mappings(struct perf_file_section *section __maybe_unused
 				struct perf_header *ph, int fd,
 				void *data __maybe_unused)
 {
-	ssize_t ret;
 	char *name;
 	u32 pmu_num;
 	u32 type;
 	struct strbuf sb;
 
-	ret = readn(fd, &pmu_num, sizeof(pmu_num));
-	if (ret != sizeof(pmu_num))
+	if (do_read_u32(fd, ph, &pmu_num))
 		return -1;
 
-	if (ph->needs_swap)
-		pmu_num = bswap_32(pmu_num);
-
 	if (!pmu_num) {
 		pr_debug("pmu mappings not available\n");
 		return 0;
@@ -1981,10 +1939,8 @@ static int process_pmu_mappings(struct perf_file_section *section __maybe_unused
 		return -1;
 
 	while (pmu_num) {
-		if (readn(fd, &type, sizeof(type)) != sizeof(type))
+		if (do_read_u32(fd, ph, &type))
 			goto error;
-		if (ph->needs_swap)
-			type = bswap_32(type);
 
 		name = do_read_string(fd, ph);
 		if (!name)
@@ -2024,12 +1980,9 @@ static int process_group_desc(struct perf_file_section *section __maybe_unused,
 		u32 nr_members;
 	} *desc;
 
-	if (readn(fd, &nr_groups, sizeof(nr_groups)) != sizeof(nr_groups))
+	if (do_read_u32(fd, ph, &nr_groups))
 		return -1;
 
-	if (ph->needs_swap)
-		nr_groups = bswap_32(nr_groups);
-
 	ph->env.nr_groups = nr_groups;
 	if (!nr_groups) {
 		pr_debug("group desc not available\n");
@@ -2045,16 +1998,11 @@ static int process_group_desc(struct perf_file_section *section __maybe_unused,
 		if (!desc[i].name)
 			goto out_free;
 
-		if (readn(fd, &desc[i].leader_idx, sizeof(u32)) != sizeof(u32))
+		if (do_read_u32(fd, ph, &desc[i].leader_idx))
 			goto out_free;
 
-		if (readn(fd, &desc[i].nr_members, sizeof(u32)) != sizeof(u32))
+		if (do_read_u32(fd, ph, &desc[i].nr_members))
 			goto out_free;
-
-		if (ph->needs_swap) {
-			desc[i].leader_idx = bswap_32(desc[i].leader_idx);
-			desc[i].nr_members = bswap_32(desc[i].nr_members);
-		}
 	}
 
 	/*
@@ -2127,21 +2075,15 @@ static int process_cache(struct perf_file_section *section __maybe_unused,
 	struct cpu_cache_level *caches;
 	u32 cnt, i, version;
 
-	if (readn(fd, &version, sizeof(version)) != sizeof(version))
+	if (do_read_u32(fd, ph, &version))
 		return -1;
 
-	if (ph->needs_swap)
-		version = bswap_32(version);
-
 	if (version != 1)
 		return -1;
 
-	if (readn(fd, &cnt, sizeof(cnt)) != sizeof(cnt))
+	if (do_read_u32(fd, ph, &cnt))
 		return -1;
 
-	if (ph->needs_swap)
-		cnt = bswap_32(cnt);
-
 	caches = zalloc(sizeof(*caches) * cnt);
 	if (!caches)
 		return -1;
@@ -2150,10 +2092,8 @@ static int process_cache(struct perf_file_section *section __maybe_unused,
 		struct cpu_cache_level c;
 
 		#define _R(v)						\
-			if (readn(fd, &c.v, sizeof(u32)) != sizeof(u32))\
+			if (do_read_u32(fd, ph, &c.v))\
 				goto out_free_caches;			\
-			if (ph->needs_swap)				\
-				c.v = bswap_32(c.v);			\
 
 		_R(level)
 		_R(line_size)
Signature · 1 line
2.13.0.219.gdb65acc882-goog
maintainer
#2 lore raw

Re: [PATCH v2 01/13] perf header: encapsulate read and swap

Quoted 15 lines · David Carrillo-Cisneros

Most callers of readn in perf header read either a 32 or a 64 bits number, error check it and swap it, if necessary.

Create do_read_u32 and do_read_u64 to simplify this usage.

---
 tools/perf/util/header.c | 212 +++++++++++++++++------------------------------
 1 file changed, 76 insertions(+), 136 deletions(-)
tools/perf/util/header.c +0 -0
index 948b2c5efb65..1dd4dbe13f88 100644

tools/perf/util/header.c static int do_write_string(int fd, const char *str) · line 124

Quoted 8 lines
tools/perf/util/header.c +5 -0
@@ -117,27 +117,56 @@ static int do_write_string(int fd, const char *str)
 	return write_padded(fd, str, olen, len);
 }
 
+static int __do_read(int fd, void *addr, ssize_t size)
+{
+	ssize_t ret = readn(fd, addr, size);
+
+	if (ret != (ssize_t)size)

The 'size' is already ssize_t.

Thanks,
Namhyung

Quoted 3 lines
tools/perf/util/header.c +3 -0
+		return ret < 0 ? (int)ret : -1;
+	return 0;
+}
#3 lore raw

Re: [PATCH v2 01/13] perf header: encapsulate read and swap

Quoted 19 lines · Namhyung Kim
Quoted 18 lines · David Carrillo-Cisneros

Most callers of readn in perf header read either a 32 or a 64 bits number, error check it and swap it, if necessary.

Create do_read_u32 and do_read_u64 to simplify this usage.

---
 tools/perf/util/header.c | 212 +++++++++++++++++------------------------------
 1 file changed, 76 insertions(+), 136 deletions(-)
tools/perf/util/header.c +0 -0
index 948b2c5efb65..1dd4dbe13f88 100644
@@ -117,27 +117,56 @@ static int do_write_string(int fd, const char *str)
      return write_padded(fd, str, olen, len);
 }
 
Quoted 7 lines

tools/perf/util/header.c static int do_write_string(int fd, const char *str) · line 124

Quoted 5 lines
tools/perf/util/header.c +5 -0
+static int __do_read(int fd, void *addr, ssize_t size)
+{
+     ssize_t ret = readn(fd, addr, size);
+
+     if (ret != (ssize_t)size)

The 'size' is already ssize_t.

Will fix.

#4 lore raw

Re: [PATCH v2 01/13] perf header: encapsulate read and swap

In reply to David Carrillo-Cisneros

On Tue, May 23, 2017 at 12:48:41AM -0700, David Carrillo-Cisneros wrote:

SNIP

Quoted 9 lines
tools/perf/util/header.c +0 -4
-	if (ph->needs_swap)
-		nr = bswap_32(nr);
-
 	ph->env.nr_sibling_cores = nr;
-	size += sizeof(u32);
 	if (strbuf_init(&sb, 128) < 0)
 		goto free_cpu;

tools/perf/util/header.c static int process_cpu_topology(struct perf_file_section *section, · line 1816

Quoted 6 lines
tools/perf/util/header.c +0 -1
@@ -1820,20 +1812,14 @@ static int process_cpu_topology(struct perf_file_section *section,
 		/* include a NULL character at the end */
 		if (strbuf_add(&sb, str, strlen(str) + 1) < 0)
 			goto error;
-		size += string_size(str);
 		free(str);
 	}

hum, you disabled the size check logic by removing those size calculations, there's following code:

        /*
         * The header may be from old perf,
         * which doesn't include core id and socket id information.
         */
        if (section->size <= size) {
                zfree(&ph->env.cpu);
                return 0;
        }

that recognize earlier version of the header

jirka

#5 lore raw

Re: [PATCH v2 01/13] perf header: encapsulate read and swap

Quoted 33 lines · Jiri Olsa

On Tue, May 23, 2017 at 12:48:41AM -0700, David Carrillo-Cisneros wrote:

SNIP

Quoted 9 lines
tools/perf/util/header.c +0 -4
-     if (ph->needs_swap)
-             nr = bswap_32(nr);
-
      ph->env.nr_sibling_cores = nr;
-     size += sizeof(u32);
      if (strbuf_init(&sb, 128) < 0)
              goto free_cpu;

tools/perf/util/header.c static int process_cpu_topology(struct perf_file_section *section, · line 1816

Quoted 6 lines
tools/perf/util/header.c +0 -1
@@ -1820,20 +1812,14 @@ static int process_cpu_topology(struct perf_file_section *section,
              /* include a NULL character at the end */
              if (strbuf_add(&sb, str, strlen(str) + 1) < 0)
                      goto error;
-             size += string_size(str);
              free(str);
      }

hum, you disabled the size check logic by removing those size calculations, there's following code:

        /*
         * The header may be from old perf,
         * which doesn't include core id and socket id information.
         */
        if (section->size <= size) {
                zfree(&ph->env.cpu);
                return 0;
        }
Quoted 1 line

that recognize earlier version of the header

True. My bad. Will fix.

Quoted 2 lines

jirka

#6 lore raw

PATCH v2 2/13 perf header: add PROCESS_STR_FUN macro

In reply to David Carrillo-Cisneros

Simplify code by adding a macro to handle the common case of processing header features that a simple string.

---
 tools/perf/util/header.c | 65 +++++++++++++-----------------------------------
 1 file changed, 17 insertions(+), 48 deletions(-)
tools/perf/util/header.c +17 -48
index 1dd4dbe13f88..8f9200724979 100644
@@ -1593,6 +1593,23 @@ static int perf_header__read_build_ids(struct perf_header *header,
 	return err;
 }
 
+/* Macro for features that simply need to read and store a string. */
+#define FEAT_PROCESS_STR_FUN(__feat, __feat_env) \
+static int process_##__feat(struct perf_file_section *section __maybe_unused, \
+			    struct perf_header *ph, int fd,		      \
+			    void *data __maybe_unused) \
+{\
+	ph->env.__feat_env = do_read_string(fd, ph); \
+	return ph->env.__feat_env ? 0 : -ENOMEM; \
+}
+
+FEAT_PROCESS_STR_FUN(hostname, hostname);
+FEAT_PROCESS_STR_FUN(osrelease, os_release);
+FEAT_PROCESS_STR_FUN(version, version);
+FEAT_PROCESS_STR_FUN(arch, arch);
+FEAT_PROCESS_STR_FUN(cpudesc, cpu_desc);
+FEAT_PROCESS_STR_FUN(cpuid, cpuid);
+
 static int process_tracing_data(struct perf_file_section *section __maybe_unused,
 				struct perf_header *ph __maybe_unused,
 				int fd, void *data)
@@ -1610,38 +1627,6 @@ static int process_build_id(struct perf_file_section *section,
 	return 0;
 }
 
-static int process_hostname(struct perf_file_section *section __maybe_unused,
-			    struct perf_header *ph, int fd,
-			    void *data __maybe_unused)
-{
-	ph->env.hostname = do_read_string(fd, ph);
-	return ph->env.hostname ? 0 : -ENOMEM;
-}
-
-static int process_osrelease(struct perf_file_section *section __maybe_unused,
-			     struct perf_header *ph, int fd,
-			     void *data __maybe_unused)
-{
-	ph->env.os_release = do_read_string(fd, ph);
-	return ph->env.os_release ? 0 : -ENOMEM;
-}
-
-static int process_version(struct perf_file_section *section __maybe_unused,
-			   struct perf_header *ph, int fd,
-			   void *data __maybe_unused)
-{
-	ph->env.version = do_read_string(fd, ph);
-	return ph->env.version ? 0 : -ENOMEM;
-}
-
-static int process_arch(struct perf_file_section *section __maybe_unused,
-			struct perf_header *ph,	int fd,
-			void *data __maybe_unused)
-{
-	ph->env.arch = do_read_string(fd, ph);
-	return ph->env.arch ? 0 : -ENOMEM;
-}
-
 static int process_nrcpus(struct perf_file_section *section __maybe_unused,
 			  struct perf_header *ph, int fd,
 			  void *data __maybe_unused)
@@ -1661,22 +1646,6 @@ static int process_nrcpus(struct perf_file_section *section __maybe_unused,
 	return 0;
 }
 
-static int process_cpudesc(struct perf_file_section *section __maybe_unused,
-			   struct perf_header *ph, int fd,
-			   void *data __maybe_unused)
-{
-	ph->env.cpu_desc = do_read_string(fd, ph);
-	return ph->env.cpu_desc ? 0 : -ENOMEM;
-}
-
-static int process_cpuid(struct perf_file_section *section __maybe_unused,
-			 struct perf_header *ph,  int fd,
-			 void *data __maybe_unused)
-{
-	ph->env.cpuid = do_read_string(fd, ph);
-	return ph->env.cpuid ? 0 : -ENOMEM;
-}
-
 static int process_total_mem(struct perf_file_section *section __maybe_unused,
 			     struct perf_header *ph, int fd,
 			     void *data __maybe_unused)
Signature · 1 line
2.13.0.219.gdb65acc882-goog
#7 lore raw

PATCH v2 3/13 perf header: fail on write_padded error

In reply to David Carrillo-Cisneros

Do not proceed if write_padded error failed.

Also, add comments to remind that return value of write_* functions in util/header.c do not return number of bytes written.

---
 tools/perf/util/header.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
tools/perf/util/header.c +5 -1
index 8f9200724979..6890f7c51814 100644
@@ -72,6 +72,7 @@ bool perf_header__has_feat(const struct perf_header *header, int feat)
 	return test_bit(feat, header->adds_features);
 }
 
+/* Return: 0 if succeded, -ERR if failed. */
 static int do_write(int fd, const void *buf, size_t size)
 {
 	while (size) {
@@ -87,6 +88,7 @@ static int do_write(int fd, const void *buf, size_t size)
 	return 0;
 }
 
+/* Return: 0 if succeded, -ERR if failed. */
 int write_padded(int fd, const void *bf, size_t count, size_t count_aligned)
 {
 	static const char zero_buf[NAME_ALIGN];
@@ -101,6 +103,7 @@ int write_padded(int fd, const void *bf, size_t count, size_t count_aligned)
 #define string_size(str)						\
 	(PERF_ALIGN((strlen(str) + 1), NAME_ALIGN) + sizeof(u32))
 
+/* Return: 0 if succeded, -ERR if failed. */
 static int do_write_string(int fd, const char *str)
 {
 	u32 len, olen;
@@ -3186,7 +3189,8 @@ int perf_event__synthesize_tracing_data(struct perf_tool *tool, int fd,
 	 */
 	tracing_data_put(tdata);
 
-	write_padded(fd, NULL, 0, padding);
+	if (write_padded(fd, NULL, 0, padding))
+		return -1;
 
 	return aligned_size;
 }
Signature · 1 line
2.13.0.219.gdb65acc882-goog
#8 lore raw

PATCH v2 4/13 perf util: add const modifier to buf in "writen" function

In reply to David Carrillo-Cisneros

Make buf in helper function "writen" constant to simplify the life of its callers.

This requires to hack a cast of buf prior to passing it to "ion" which is simpler than the alternative of reworking the "ion" function to provide a read and a write paths, the latter with constant buf argument.

---
 tools/perf/util/util.c | 6 ++++--
 tools/perf/util/util.h | 2 +-
 2 files changed, 5 insertions(+), 3 deletions(-)
tools/perf/util/util.c +4 -2
index 6450c75a6f5b..31ca7661e5fb 100644
@@ -260,6 +260,7 @@ static ssize_t ion(bool is_read, int fd, void *buf, size_t n)
 	size_t left = n;
 
 	while (left) {
+		/* buf must be treated as const if !is_read. */
 		ssize_t ret = is_read ? read(fd, buf, left) :
 					write(fd, buf, left);
 
@@ -287,9 +288,10 @@ ssize_t readn(int fd, void *buf, size_t n)
 /*
  * Write exactly 'n' bytes or return an error.
  */
-ssize_t writen(int fd, void *buf, size_t n)
+ssize_t writen(int fd, const void *buf, size_t n)
 {
-	return ion(false, fd, buf, n);
+	/* ion does not modify buf. */
+	return ion(false, fd, (void *)buf, n);
 }
 
 size_t hex_width(u64 v)
tools/perf/util/util.h +1 -1
index 3852b6d3270a..e0881bf67fc9 100644
@@ -57,7 +57,7 @@ int copyfile_mode(const char *from, const char *to, mode_t mode);
 int copyfile_offset(int fromfd, loff_t from_ofs, int tofd, loff_t to_ofs, u64 size);
 
 ssize_t readn(int fd, void *buf, size_t n);
-ssize_t writen(int fd, void *buf, size_t n);
+ssize_t writen(int fd, const void *buf, size_t n);
 
 struct perf_event_attr;
 
Signature · 1 line
2.13.0.219.gdb65acc882-goog
#9 lore raw

PATCH v2 5/13 perf header: revamp do_write

In reply to David Carrillo-Cisneros

Now that writen takes a const buffer, use it in do_write instead of duplicating readn functionality..

Export do_write to use it consistently in header.c and build_id.c .

---
 tools/perf/util/build-id.c |  2 +-
 tools/perf/util/header.c   | 14 +++++---------
 tools/perf/util/header.h   |  2 ++
 3 files changed, 8 insertions(+), 10 deletions(-)
tools/perf/util/build-id.c +1 -1
index 168cc49654e7..84d2ea51e557 100644
@@ -345,7 +345,7 @@ static int write_buildid(const char *name, size_t name_len, u8 *build_id,
 	b.header.misc = misc;
 	b.header.size = sizeof(b) + len;
 
-	err = writen(fd, &b, sizeof(b));
+	err = do_write(fd, &b, sizeof(b));
 	if (err < 0)
 		return err;
 
tools/perf/util/header.c +5 -9
index 6890f7c51814..292fb2156a59 100644
@@ -73,17 +73,13 @@ bool perf_header__has_feat(const struct perf_header *header, int feat)
 }
 
 /* Return: 0 if succeded, -ERR if failed. */
-static int do_write(int fd, const void *buf, size_t size)
+int do_write(int fd, const void *buf, size_t size)
 {
-	while (size) {
-		int ret = write(fd, buf, size);
-
-		if (ret < 0)
-			return -errno;
+	ssize_t ret;
 
-		size -= ret;
-		buf += ret;
-	}
+	ret  = writen(fd, buf, size);
+	if (ret != (ssize_t)size)
+		return ret < 0 ? (int)ret: -1;
 
 	return 0;
 }
tools/perf/util/header.h +2 -0
index d30109b421ee..e98489c8bba7 100644
@@ -144,6 +144,8 @@ bool is_perf_magic(u64 magic);
 
 #define NAME_ALIGN 64
 
+int do_write(int fd, const void *buf, size_t size);
+
 int write_padded(int fd, const void *bf, size_t count, size_t count_aligned);
 
 /*
Signature · 1 line
2.13.0.219.gdb65acc882-goog
#10 lore raw

Re: [PATCH v2 05/13] perf header: revamp do_write

Quoted 2 lines · David Carrillo-Cisneros

Now that writen takes a const buffer, use it in do_write instead of duplicating readn functionality..

readn?

jirka

#11 lore raw

Re: [PATCH v2 05/13] perf header: revamp do_write

Quoted 5 lines · Jiri Olsa
Quoted 2 lines · David Carrillo-Cisneros

Now that writen takes a const buffer, use it in do_write instead of duplicating readn functionality..

readn?

Will fix.

Quoted 2 lines

jirka

#12 lore raw

PATCH v2 6/13 perf header: add struct feat_fd for write

In reply to David Carrillo-Cisneros

Introduce struct feat_fd. This patch uses it as a wrapper around fd in write_* functions for feature headers. Next patches will extend its functionality to other feature header functions.

This patch does not change behavior.

---
 tools/perf/util/build-id.c |   8 ++-
 tools/perf/util/build-id.h |   4 +-
 tools/perf/util/header.c   | 134 ++++++++++++++++++++++++++-------------------
 tools/perf/util/header.h   |   7 ++-
 4 files changed, 90 insertions(+), 63 deletions(-)
tools/perf/util/build-id.c +5 -3
index 84d2ea51e557..292e90db3924 100644
@@ -330,7 +330,7 @@ bool dso__build_id_is_kmod(const struct dso *dso, char *bf, size_t size)
 		else
 
 static int write_buildid(const char *name, size_t name_len, u8 *build_id,
-			 pid_t pid, u16 misc, int fd)
+			 pid_t pid, u16 misc, struct feat_fd *fd)
 {
 	int err;
 	struct build_id_event b;
@@ -352,7 +352,8 @@ static int write_buildid(const char *name, size_t name_len, u8 *build_id,
 	return write_padded(fd, name, name_len + 1, len);
 }
 
-static int machine__write_buildid_table(struct machine *machine, int fd)
+static int machine__write_buildid_table(struct machine *machine,
+					struct feat_fd *fd)
 {
 	int err = 0;
 	char nm[PATH_MAX];
@@ -397,7 +398,8 @@ static int machine__write_buildid_table(struct machine *machine, int fd)
 	return err;
 }
 
-int perf_session__write_buildid_table(struct perf_session *session, int fd)
+int perf_session__write_buildid_table(struct perf_session *session,
+				      struct feat_fd *fd)
 {
 	struct rb_node *nd;
 	int err = machine__write_buildid_table(&session->machines.host, fd);
tools/perf/util/build-id.h +3 -1
index a96081121179..84e5e8a52970 100644
@@ -9,6 +9,7 @@
 
 extern struct perf_tool build_id__mark_dso_hit_ops;
 struct dso;
+struct feat_fd;
 
 int build_id__sprintf(const u8 *build_id, int len, char *bf);
 int sysfs__sprintf_build_id(const char *root_dir, char *sbuild_id);
@@ -26,7 +27,8 @@ int build_id__mark_dso_hit(struct perf_tool *tool, union perf_event *event,
 int dsos__hit_all(struct perf_session *session);
 
 bool perf_session__read_build_ids(struct perf_session *session, bool with_hits);
-int perf_session__write_buildid_table(struct perf_session *session, int fd);
+int perf_session__write_buildid_table(struct perf_session *session,
+				      struct feat_fd *fd);
 int perf_session__cache_build_ids(struct perf_session *session);
 
 char *build_id_cache__origname(const char *sbuild_id);
tools/perf/util/header.c +77 -57

426 lines not on this page · raw message

tools/perf/util/header.h +5 -2
index e98489c8bba7..9d8dcd5eb727 100644
@@ -144,9 +144,12 @@ bool is_perf_magic(u64 magic);
 
 #define NAME_ALIGN 64
 
-int do_write(int fd, const void *buf, size_t size);
+struct feat_fd;
 
-int write_padded(int fd, const void *bf, size_t count, size_t count_aligned);
+int do_write(struct feat_fd *fd, const void *buf, size_t size);
+
+int write_padded(struct feat_fd *fd, const void *bf,
+		 size_t count, size_t count_aligned);
 
 /*
  * arch specific callback
Signature · 1 line
2.13.0.219.gdb65acc882-goog
#13 lore raw

Re: [PATCH v2 06/13] perf header: add struct feat_fd for write

On Tue, May 23, 2017 at 12:48:46AM -0700, David Carrillo-Cisneros wrote:

SNIP

Quoted 28 lines
tools/perf/util/header.c +5 -4
@@ -2211,18 +2214,18 @@ static int do_write_feat(int fd, struct perf_header *h, int type,
 		if (!feat_ops[type].write)
 			return -1;
 
-		(*p)->offset = lseek(fd, 0, SEEK_CUR);
+		(*p)->offset = lseek(fd->fd, 0, SEEK_CUR);
 
-		err = feat_ops[type].write(fd, h, evlist);
+		err = feat_ops[type].write(fd, evlist);
 		if (err < 0) {
 			pr_debug("failed to write feature %s\n", feat_ops[type].name);
 
 			/* undo anything written */
-			lseek(fd, (*p)->offset, SEEK_SET);
+			lseek(fd->fd, (*p)->offset, SEEK_SET);
 
 			return -1;
 		}
-		(*p)->size = lseek(fd, 0, SEEK_CUR) - (*p)->offset;
+		(*p)->size = lseek(fd->fd, 0, SEEK_CUR) - (*p)->offset;
 		(*p)++;
 	}
 	return ret;
@@ -2232,12 +2235,18 @@ static int perf_header__adds_write(struct perf_header *header,
 				   struct perf_evlist *evlist, int fd)
 {
 	int nr_sections;
+	struct feat_fd fdd;

tools/perf/util/header.c static int perf_header__adds_write(struct perf_header *header, · line 2248

Quoted 10 lines
tools/perf/util/header.c +4 -0
 	struct perf_file_section *feat_sec, *p;
 	int sec_size;
 	u64 sec_start;
 	int feat;
 	int err;
 
+	fdd = (struct feat_fd){
+		.fd  = fd,
+		.ph = header,
+	};

could you unite the naming for struct feat_fd variables?

it's slightly confusing, especialy when you keep the 'fd' name in some cases.. ff? ;-)

jirka

#14 lore raw

Re: [PATCH v2 06/13] perf header: add struct feat_fd for write

Quoted 45 lines · Jiri Olsa

On Tue, May 23, 2017 at 12:48:46AM -0700, David Carrillo-Cisneros wrote:

SNIP

Quoted 28 lines
tools/perf/util/header.c +5 -4
@@ -2211,18 +2214,18 @@ static int do_write_feat(int fd, struct perf_header *h, int type,
              if (!feat_ops[type].write)
                      return -1;
 
-             (*p)->offset = lseek(fd, 0, SEEK_CUR);
+             (*p)->offset = lseek(fd->fd, 0, SEEK_CUR);
 
-             err = feat_ops[type].write(fd, h, evlist);
+             err = feat_ops[type].write(fd, evlist);
              if (err < 0) {
                      pr_debug("failed to write feature %s\n", feat_ops[type].name);
 
                      /* undo anything written */
-                     lseek(fd, (*p)->offset, SEEK_SET);
+                     lseek(fd->fd, (*p)->offset, SEEK_SET);
 
                      return -1;
              }
-             (*p)->size = lseek(fd, 0, SEEK_CUR) - (*p)->offset;
+             (*p)->size = lseek(fd->fd, 0, SEEK_CUR) - (*p)->offset;
              (*p)++;
      }
      return ret;
@@ -2232,12 +2235,18 @@ static int perf_header__adds_write(struct perf_header *header,
                                 struct perf_evlist *evlist, int fd)
 {
      int nr_sections;
+     struct feat_fd fdd;

tools/perf/util/header.c static int perf_header__adds_write(struct perf_header *header, · line 2248

Quoted 10 lines
tools/perf/util/header.c +4 -0
      struct perf_file_section *feat_sec, *p;
      int sec_size;
      u64 sec_start;
      int feat;
      int err;
 
+     fdd = (struct feat_fd){
+             .fd  = fd,
+             .ph = header,
+     };

could you unite the naming for struct feat_fd variables?

Quoted 3 lines

it's slightly confusing, especialy when you keep the 'fd' name in some cases.. ff? ;-)

Thank you for the review :) . I'll do the rename.

Quoted 1 line

jirka

#15 lore raw

PATCH v2 7/13 perf header: use struct feat_fd for print

In reply to David Carrillo-Cisneros

As preparation for using header records in pipe mode, replace int fd with struct feat_fd fd in print functions for all header record types.

Add offset and size to struct feat_fd to provide pipe-mode with variables for offset and size that in file-mode are stored in struct perf_file_section.

---
 tools/perf/util/header.c | 106 +++++++++++++++++++++++------------------------
 1 file changed, 51 insertions(+), 55 deletions(-)
tools/perf/util/header.c +51 -55
index 4934ef543aef..3d0c61027170 100644
@@ -60,6 +60,8 @@ struct perf_file_attr {
 struct feat_fd {
 	struct perf_header *ph;
 	int fd;
+	ssize_t offset;
+	size_t size;
 };
 
 void perf_header__set_feat(struct perf_header *header, int feat)
@@ -1114,62 +1116,56 @@ static int write_stat(struct feat_fd *fd __maybe_unused,
 	return 0;
 }
 
-static void print_hostname(struct perf_header *ph, int fd __maybe_unused,
-			   FILE *fp)
+static void print_hostname(struct feat_fd *fd, FILE *fp)
 {
-	fprintf(fp, "# hostname : %s\n", ph->env.hostname);
+	fprintf(fp, "# hostname : %s\n", fd->ph->env.hostname);
 }
 
-static void print_osrelease(struct perf_header *ph, int fd __maybe_unused,
-			    FILE *fp)
+static void print_osrelease(struct feat_fd *fd, FILE *fp)
 {
-	fprintf(fp, "# os release : %s\n", ph->env.os_release);
+	fprintf(fp, "# os release : %s\n", fd->ph->env.os_release);
 }
 
-static void print_arch(struct perf_header *ph, int fd __maybe_unused, FILE *fp)
+static void print_arch(struct feat_fd *fd, FILE *fp)
 {
-	fprintf(fp, "# arch : %s\n", ph->env.arch);
+	fprintf(fp, "# arch : %s\n", fd->ph->env.arch);
 }
 
-static void print_cpudesc(struct perf_header *ph, int fd __maybe_unused,
-			  FILE *fp)
+static void print_cpudesc(struct feat_fd *fd, FILE *fp)
 {
-	fprintf(fp, "# cpudesc : %s\n", ph->env.cpu_desc);
+	fprintf(fp, "# cpudesc : %s\n", fd->ph->env.cpu_desc);
 }
 
-static void print_nrcpus(struct perf_header *ph, int fd __maybe_unused,
-			 FILE *fp)
+static void print_nrcpus(struct feat_fd *fd, FILE *fp)
 {
-	fprintf(fp, "# nrcpus online : %u\n", ph->env.nr_cpus_online);
-	fprintf(fp, "# nrcpus avail : %u\n", ph->env.nr_cpus_avail);
+	fprintf(fp, "# nrcpus online : %u\n", fd->ph->env.nr_cpus_online);
+	fprintf(fp, "# nrcpus avail : %u\n", fd->ph->env.nr_cpus_avail);
 }
 
-static void print_version(struct perf_header *ph, int fd __maybe_unused,
-			  FILE *fp)
+static void print_version(struct feat_fd *fd, FILE *fp)
 {
-	fprintf(fp, "# perf version : %s\n", ph->env.version);
+	fprintf(fp, "# perf version : %s\n", fd->ph->env.version);
 }
 
-static void print_cmdline(struct perf_header *ph, int fd __maybe_unused,
-			  FILE *fp)
+static void print_cmdline(struct feat_fd *fd, FILE *fp)
 {
 	int nr, i;
 
-	nr = ph->env.nr_cmdline;
+	nr = fd->ph->env.nr_cmdline;
 
 	fprintf(fp, "# cmdline : ");
 
 	for (i = 0; i < nr; i++)
-		fprintf(fp, "%s ", ph->env.cmdline_argv[i]);
+		fprintf(fp, "%s ", fd->ph->env.cmdline_argv[i]);
 	fputc('\n', fp);
 }
 
-static void print_cpu_topology(struct perf_header *ph, int fd __maybe_unused,
-			       FILE *fp)
+static void print_cpu_topology(struct feat_fd *fd, FILE *fp)
 {
+	struct perf_header *ph = fd->ph;
+	int cpu_nr = ph->env.nr_cpus_avail;
 	int nr, i;
 	char *str;
-	int cpu_nr = ph->env.nr_cpus_avail;
 
 	nr = ph->env.nr_sibling_cores;
 	str = ph->env.sibling_cores;
@@ -1295,9 +1291,9 @@ static int __desc_attr__fprintf(FILE *fp, const char *name, const char *val,
 	return fprintf(fp, ", %s = %s", name, val);
 }
 
-static void print_event_desc(struct perf_header *ph, int fd, FILE *fp)
+static void print_event_desc(struct feat_fd *fd, FILE *fp)
 {
-	struct perf_evsel *evsel, *events = read_event_desc(ph, fd);
+	struct perf_evsel *evsel, *events = read_event_desc(fd->ph, fd->fd);
 	u32 j;
 	u64 *id;
 
@@ -1327,20 +1323,18 @@ static void print_event_desc(struct perf_header *ph, int fd, FILE *fp)
 	free_event_desc(events);
 }
 
-static void print_total_mem(struct perf_header *ph, int fd __maybe_unused,
-			    FILE *fp)
+static void print_total_mem(struct feat_fd *fd, FILE *fp)
 {
-	fprintf(fp, "# total memory : %Lu kB\n", ph->env.total_mem);
+	fprintf(fp, "# total memory : %llu kB\n", fd->ph->env.total_mem);
 }
 
-static void print_numa_topology(struct perf_header *ph, int fd __maybe_unused,
-				FILE *fp)
+static void print_numa_topology(struct feat_fd *fd, FILE *fp)
 {
 	int i;
 	struct numa_node *n;
 
-	for (i = 0; i < ph->env.nr_numa_nodes; i++) {
-		n = &ph->env.numa_nodes[i];
+	for (i = 0; i < fd->ph->env.nr_numa_nodes; i++) {
+		n = &fd->ph->env.numa_nodes[i];
 
 		fprintf(fp, "# node%u meminfo  : total = %"PRIu64" kB,"
 			    " free = %"PRIu64" kB\n",
@@ -1351,56 +1345,51 @@ static void print_numa_topology(struct perf_header *ph, int fd __maybe_unused,
 	}
 }
 
-static void print_cpuid(struct perf_header *ph, int fd __maybe_unused, FILE *fp)
+static void print_cpuid(struct feat_fd *fd, FILE *fp)
 {
-	fprintf(fp, "# cpuid : %s\n", ph->env.cpuid);
+	fprintf(fp, "# cpuid : %s\n", fd->ph->env.cpuid);
 }
 
-static void print_branch_stack(struct perf_header *ph __maybe_unused,
-			       int fd __maybe_unused, FILE *fp)
+static void print_branch_stack(struct feat_fd *fd __maybe_unused, FILE *fp)
 {
 	fprintf(fp, "# contains samples with branch stack\n");
 }
 
-static void print_auxtrace(struct perf_header *ph __maybe_unused,
-			   int fd __maybe_unused, FILE *fp)
+static void print_auxtrace(struct feat_fd *fd __maybe_unused, FILE *fp)
 {
 	fprintf(fp, "# contains AUX area data (e.g. instruction trace)\n");
 }
 
-static void print_stat(struct perf_header *ph __maybe_unused,
-		       int fd __maybe_unused, FILE *fp)
+static void print_stat(struct feat_fd *fd __maybe_unused, FILE *fp)
 {
 	fprintf(fp, "# contains stat data\n");
 }
 
-static void print_cache(struct perf_header *ph __maybe_unused,
-			int fd __maybe_unused, FILE *fp __maybe_unused)
+static void print_cache(struct feat_fd *fd, FILE *fp __maybe_unused)
 {
 	int i;
 
 	fprintf(fp, "# CPU cache info:\n");
-	for (i = 0; i < ph->env.caches_cnt; i++) {
+	for (i = 0; i < fd->ph->env.caches_cnt; i++) {
 		fprintf(fp, "#  ");
-		cpu_cache_level__fprintf(fp, &ph->env.caches[i]);
+		cpu_cache_level__fprintf(fp, &fd->ph->env.caches[i]);
 	}
 }
 
-static void print_pmu_mappings(struct perf_header *ph, int fd __maybe_unused,
-			       FILE *fp)
+static void print_pmu_mappings(struct feat_fd *fd, FILE *fp)
 {
 	const char *delimiter = "# pmu mappings: ";
 	char *str, *tmp;
 	u32 pmu_num;
 	u32 type;
 
-	pmu_num = ph->env.nr_pmu_mappings;
+	pmu_num = fd->ph->env.nr_pmu_mappings;
 	if (!pmu_num) {
 		fprintf(fp, "# pmu mappings: not available\n");
 		return;
 	}
 
-	str = ph->env.pmu_mappings;
+	str = fd->ph->env.pmu_mappings;
 
 	while (pmu_num) {
 		type = strtoul(str, &tmp, 0);
@@ -1423,14 +1412,13 @@ static void print_pmu_mappings(struct perf_header *ph, int fd __maybe_unused,
 	fprintf(fp, "# pmu mappings: unable to read\n");
 }
 
-static void print_group_desc(struct perf_header *ph, int fd __maybe_unused,
-			     FILE *fp)
+static void print_group_desc(struct feat_fd *fd, FILE *fp)
 {
 	struct perf_session *session;
 	struct perf_evsel *evsel;
 	u32 nr = 0;
 
-	session = container_of(ph, struct perf_session, header);
+	session = container_of(fd->ph, struct perf_session, header);
 
 	evlist__for_each_entry(session->evlist, evsel) {
 		if (perf_evsel__is_group_leader(evsel) &&
@@ -2095,7 +2083,7 @@ static int process_cache(struct perf_file_section *section __maybe_unused,
 
 struct feature_ops {
 	int (*write)(struct feat_fd *fd, struct perf_evlist *evlist);
-	void (*print)(struct perf_header *h, int fd, FILE *fp);
+	void (*print)(struct feat_fd *fd, FILE *fp);
 	int (*process)(struct perf_file_section *section,
 		       struct perf_header *h, int fd, void *data);
 	const char *name;
@@ -2148,6 +2136,7 @@ static int perf_file_section__fprintf_info(struct perf_file_section *section,
 					   int feat, int fd, void *data)
 {
 	struct header_print_data *hd = data;
+	struct feat_fd fdd;
 
 	if (lseek(fd, section->offset, SEEK_SET) == (off_t)-1) {
 		pr_debug("Failed to lseek to %" PRIu64 " offset for feature "
@@ -2161,8 +2150,15 @@ static int perf_file_section__fprintf_info(struct perf_file_section *section,
 	if (!feat_ops[feat].print)
 		return 0;
 
+	fdd = (struct  feat_fd) {
+		.fd = fd,
+		.ph = ph,
+		.size = section->size,
+		.offset = section->offset,
+	};
+
 	if (!feat_ops[feat].full_only || hd->full)
-		feat_ops[feat].print(ph, fd, hd->fp);
+		feat_ops[feat].print(&fdd, hd->fp);
 	else
 		fprintf(hd->fp, "# %s info available, use -I to display\n",
 			feat_ops[feat].name);
Signature · 1 line
2.13.0.219.gdb65acc882-goog
#16 lore raw

Re: [PATCH v2 07/13] perf header: use struct feat_fd for print

Quoted 7 lines · David Carrillo-Cisneros

As preparation for using header records in pipe mode, replace int fd with struct feat_fd fd in print functions for all header record types.

Add offset and size to struct feat_fd to provide pipe-mode with variables for offset and size that in file-mode are stored in struct perf_file_section.

please put that change in the pipe-mode change related patch
perf header: add a buffer to struct feat_fd

thanks,
jirka

#17 lore raw

Re: [PATCH v2 07/13] perf header: use struct feat_fd for print

Quoted 5 lines · Jiri Olsa
Quoted 4 lines · David Carrillo-Cisneros

As preparation for using header records in pipe mode, replace int fd with struct feat_fd fd in print functions for all header record types.

Quoted 7 lines
Quoted 3 lines

Add offset and size to struct feat_fd to provide pipe-mode with variables for offset and size that in file-mode are stored in struct perf_file_section.

please put that change in the pipe-mode change related patch
perf header: add a buffer to struct feat_fd

This is needed here to convert perf_file_section__fprintf_info . i will explain that in the log.

#18 lore raw

Re: [PATCH v2 07/13] perf header: use struct feat_fd for print

Quoted 10 lines · David Carrillo-Cisneros
Quoted 9 lines · Jiri Olsa
Quoted 7 lines · David Carrillo-Cisneros

As preparation for using header records in pipe mode, replace int fd with struct feat_fd fd in print functions for all header record types.

Add offset and size to struct feat_fd to provide pipe-mode with variables for offset and size that in file-mode are stored in struct perf_file_section.

Quoted 6 lines
Quoted 3 lines

please put that change in the pipe-mode change related patch
perf header: add a buffer to struct feat_fd

This is needed here to convert perf_file_section__fprintf_info . i will explain that in the log.

Never mind. Will move.

#19 lore raw

PATCH v2 8/13 perf header: use struct feat_fd to process header records

In reply to David Carrillo-Cisneros

As preparation for using header records in pipe-mode, replace int fd with struct feat_fd in process functions for all header record types.

This patch does not change behavior.

---
 tools/perf/util/header.c | 171 +++++++++++++++++++++--------------------------
 1 file changed, 78 insertions(+), 93 deletions(-)
tools/perf/util/header.c +78 -93
index 3d0c61027170..dc923f5639f8 100644
@@ -1585,12 +1585,10 @@ static int perf_header__read_build_ids(struct perf_header *header,
 
 /* Macro for features that simply need to read and store a string. */
 #define FEAT_PROCESS_STR_FUN(__feat, __feat_env) \
-static int process_##__feat(struct perf_file_section *section __maybe_unused, \
-			    struct perf_header *ph, int fd,		      \
-			    void *data __maybe_unused) \
+static int process_##__feat(struct feat_fd *fd, void *data __maybe_unused) \
 {\
-	ph->env.__feat_env = do_read_string(fd, ph); \
-	return ph->env.__feat_env ? 0 : -ENOMEM; \
+	fd->ph->env.__feat_env = do_read_string(fd->fd, fd->ph); \
+	return fd->ph->env.__feat_env ? 0 : -ENOMEM; \
 }
 
 FEAT_PROCESS_STR_FUN(hostname, hostname);
@@ -1600,53 +1598,46 @@ FEAT_PROCESS_STR_FUN(arch, arch);
 FEAT_PROCESS_STR_FUN(cpudesc, cpu_desc);
 FEAT_PROCESS_STR_FUN(cpuid, cpuid);
 
-static int process_tracing_data(struct perf_file_section *section __maybe_unused,
-				struct perf_header *ph __maybe_unused,
-				int fd, void *data)
+static int process_build_id(struct feat_fd *fd, void *data __maybe_unused)
 {
-	ssize_t ret = trace_report(fd, data, false);
-	return ret < 0 ? -1 : 0;
+	if (perf_header__read_build_ids(fd->ph, fd->fd, fd->offset, fd->size))
+		pr_debug("Failed to read buildids, continuing...\n");
+	return 0;
 }
 
-static int process_build_id(struct perf_file_section *section,
-			    struct perf_header *ph, int fd,
-			    void *data __maybe_unused)
+static int process_tracing_data(struct feat_fd *fd, void *data)
 {
-	if (perf_header__read_build_ids(ph, fd, section->offset, section->size))
-		pr_debug("Failed to read buildids, continuing...\n");
-	return 0;
+	ssize_t ret = trace_report(fd->fd, data, false);
+
+	return ret < 0 ? -1 : 0;
 }
 
-static int process_nrcpus(struct perf_file_section *section __maybe_unused,
-			  struct perf_header *ph, int fd,
-			  void *data __maybe_unused)
+static int process_nrcpus(struct feat_fd *fd, void *data __maybe_unused)
 {
 	int ret;
 	u32 nr_cpus_avail, nr_cpus_online;
 
-	ret = do_read_u32(fd, ph, &nr_cpus_avail);
+	ret = do_read_u32(fd->fd, fd->ph, &nr_cpus_avail);
 	if (ret)
 		return ret;
 
-	ret = do_read_u32(fd, ph, &nr_cpus_online);
+	ret = do_read_u32(fd->fd, fd->ph, &nr_cpus_online);
 	if (ret)
 		return ret;
-	ph->env.nr_cpus_avail = (int)nr_cpus_avail;
-	ph->env.nr_cpus_online = (int)nr_cpus_online;
+	fd->ph->env.nr_cpus_avail = (int)nr_cpus_avail;
+	fd->ph->env.nr_cpus_online = (int)nr_cpus_online;
 	return 0;
 }
 
-static int process_total_mem(struct perf_file_section *section __maybe_unused,
-			     struct perf_header *ph, int fd,
-			     void *data __maybe_unused)
+static int process_total_mem(struct feat_fd *fd, void *data __maybe_unused)
 {
 	u64 total_mem;
 	int ret;
 
-	ret = do_read_u64(fd, ph, &total_mem);
+	ret = do_read_u64(fd->fd, fd->ph, &total_mem);
 	if (ret)
 		return -1;
-	ph->env.total_mem = (unsigned long long)total_mem;
+	fd->ph->env.total_mem = (unsigned long long)total_mem;
 	return 0;
 }
 
@@ -1683,17 +1674,15 @@ perf_evlist__set_event_name(struct perf_evlist *evlist,
 }
 
 static int
-process_event_desc(struct perf_file_section *section __maybe_unused,
-		   struct perf_header *header, int fd,
-		   void *data __maybe_unused)
+process_event_desc(struct feat_fd *fd, void *data __maybe_unused)
 {
 	struct perf_session *session;
-	struct perf_evsel *evsel, *events = read_event_desc(header, fd);
+	struct perf_evsel *evsel, *events = read_event_desc(fd->ph, fd->fd);
 
 	if (!events)
 		return 0;
 
-	session = container_of(header, struct perf_session, header);
+	session = container_of(fd->ph, struct perf_session, header);
 	for (evsel = events; evsel->attr.size; evsel++)
 		perf_evlist__set_event_name(session->evlist, evsel);
 
@@ -1702,19 +1691,17 @@ process_event_desc(struct perf_file_section *section __maybe_unused,
 	return 0;
 }
 
-static int process_cmdline(struct perf_file_section *section,
-			   struct perf_header *ph, int fd,
-			   void *data __maybe_unused)
+static int process_cmdline(struct feat_fd *fd, void *data __maybe_unused)
 {
 	char *str, *cmdline = NULL, **argv = NULL;
 	u32 nr, i, len = 0;
 
-	if (do_read_u32(fd, ph, &nr))
+	if (do_read_u32(fd->fd, fd->ph, &nr))
 		return -1;
 
-	ph->env.nr_cmdline = nr;
+	fd->ph->env.nr_cmdline = nr;
 
-	cmdline = zalloc(section->size + nr + 1);
+	cmdline = zalloc(fd->size + nr + 1);
 	if (!cmdline)
 		return -1;
 
@@ -1723,7 +1710,7 @@ static int process_cmdline(struct perf_file_section *section,
 		goto error;
 
 	for (i = 0; i < nr; i++) {
-		str = do_read_string(fd, ph);
+		str = do_read_string(fd->fd, fd->ph);
 		if (!str)
 			goto error;
 
@@ -1732,8 +1719,8 @@ static int process_cmdline(struct perf_file_section *section,
 		len += strlen(str) + 1;
 		free(str);
 	}
-	ph->env.cmdline = cmdline;
-	ph->env.cmdline_argv = (const char **) argv;
+	fd->ph->env.cmdline = cmdline;
+	fd->ph->env.cmdline_argv = (const char **) argv;
 	return 0;
 
 error:
@@ -1742,21 +1729,20 @@ static int process_cmdline(struct perf_file_section *section,
 	return -1;
 }
 
-static int process_cpu_topology(struct perf_file_section *section,
-				struct perf_header *ph, int fd,
-				void *data __maybe_unused)
+static int process_cpu_topology(struct feat_fd *fd, void *data __maybe_unused)
 {
 	u32 nr, i;
 	char *str;
 	struct strbuf sb;
-	int cpu_nr = ph->env.nr_cpus_avail;
-	u64 size = 0;
+	int cpu_nr = fd->ph->env.nr_cpus_avail;
+	struct perf_header *ph = fd->ph;
+	u64 start_offset = fd->offset;
 
 	ph->env.cpu = calloc(cpu_nr, sizeof(*ph->env.cpu));
 	if (!ph->env.cpu)
 		return -1;
 
-	if (do_read_u32(fd, ph, &nr))
+	if (do_read_u32(fd->fd, fd->ph, &nr))
 		goto free_cpu;
 
 	ph->env.nr_sibling_cores = nr;
@@ -1764,7 +1750,7 @@ static int process_cpu_topology(struct perf_file_section *section,
 		goto free_cpu;
 
 	for (i = 0; i < nr; i++) {
-		str = do_read_string(fd, ph);
+		str = do_read_string(fd->fd, fd->ph);
 		if (!str)
 			goto error;
 
@@ -1775,13 +1761,13 @@ static int process_cpu_topology(struct perf_file_section *section,
 	}
 	ph->env.sibling_cores = strbuf_detach(&sb, NULL);
 
-	if (do_read_u32(fd, ph, &nr))
+	if (do_read_u32(fd->fd, fd->ph, &nr))
 		return -1;
 
 	ph->env.nr_sibling_threads = nr;
 
 	for (i = 0; i < nr; i++) {
-		str = do_read_string(fd, ph);
+		str = do_read_string(fd->fd, fd->ph);
 		if (!str)
 			goto error;
 
@@ -1796,18 +1782,18 @@ static int process_cpu_topology(struct perf_file_section *section,
 	 * The header may be from old perf,
 	 * which doesn't include core id and socket id information.
 	 */
-	if (section->size <= size) {
+	if (fd->size <= fd->offset - start_offset) {
 		zfree(&ph->env.cpu);
 		return 0;
 	}
 
 	for (i = 0; i < (u32)cpu_nr; i++) {
-		if (do_read_u32(fd, ph, &nr))
+		if (do_read_u32(fd->fd, fd->ph, &nr))
 			goto free_cpu;
 
 		ph->env.cpu[i].core_id = nr;
 
-		if (do_read_u32(fd, ph, &nr))
+		if (do_read_u32(fd->fd, fd->ph, &nr))
 			goto free_cpu;
 
 		if (nr != (u32)-1 && nr > (u32)cpu_nr) {
@@ -1828,16 +1814,15 @@ static int process_cpu_topology(struct perf_file_section *section,
 	return -1;
 }
 
-static int process_numa_topology(struct perf_file_section *section __maybe_unused,
-				 struct perf_header *ph, int fd,
-				 void *data __maybe_unused)
+static int process_numa_topology(struct feat_fd *fd, void *data __maybe_unused)
 {
+	struct perf_header *ph = fd->ph;
 	struct numa_node *nodes, *n;
 	u32 nr, i;
 	char *str;
 
 	/* nr nodes */
-	if (do_read_u32(fd, ph, &nr))
+	if (do_read_u32(fd->fd, fd->ph, &nr))
 		return -1;
 
 	nodes = zalloc(sizeof(*nodes) * nr);
@@ -1848,16 +1833,16 @@ static int process_numa_topology(struct perf_file_section *section __maybe_unuse
 		n = &nodes[i];
 
 		/* node number */
-		if (do_read_u32(fd, ph, &n->node))
+		if (do_read_u32(fd->fd, fd->ph, &n->node))
 			goto error;
 
-		if (do_read_u64(fd, ph, &n->mem_total))
+		if (do_read_u64(fd->fd, fd->ph, &n->mem_total))
 			goto error;
 
-		if (do_read_u64(fd, ph, &n->mem_free))
+		if (do_read_u64(fd->fd, fd->ph, &n->mem_free))
 			goto error;
 
-		str = do_read_string(fd, ph);
+		str = do_read_string(fd->fd, fd->ph);
 		if (!str)
 			goto error;
 
@@ -1876,16 +1861,15 @@ static int process_numa_topology(struct perf_file_section *section __maybe_unuse
 	return -1;
 }
 
-static int process_pmu_mappings(struct perf_file_section *section __maybe_unused,
-				struct perf_header *ph, int fd,
-				void *data __maybe_unused)
+static int process_pmu_mappings(struct feat_fd *fd, void *data __maybe_unused)
 {
+	struct perf_header *ph = fd->ph;
 	char *name;
 	u32 pmu_num;
 	u32 type;
 	struct strbuf sb;
 
-	if (do_read_u32(fd, ph, &pmu_num))
+	if (do_read_u32(fd->fd, fd->ph, &pmu_num))
 		return -1;
 
 	if (!pmu_num) {
@@ -1898,10 +1882,10 @@ static int process_pmu_mappings(struct perf_file_section *section __maybe_unused
 		return -1;
 
 	while (pmu_num) {
-		if (do_read_u32(fd, ph, &type))
+		if (do_read_u32(fd->fd, fd->ph, &type))
 			goto error;
 
-		name = do_read_string(fd, ph);
+		name = do_read_string(fd->fd, fd->ph);
 		if (!name)
 			goto error;
 
@@ -1925,10 +1909,9 @@ static int process_pmu_mappings(struct perf_file_section *section __maybe_unused
 	return -1;
 }
 
-static int process_group_desc(struct perf_file_section *section __maybe_unused,
-			      struct perf_header *ph, int fd,
-			      void *data __maybe_unused)
+static int process_group_desc(struct feat_fd *fd, void *data __maybe_unused)
 {
+	struct perf_header *ph = fd->ph;
 	size_t ret = -1;
 	u32 i, nr, nr_groups;
 	struct perf_session *session;
@@ -1939,7 +1922,7 @@ static int process_group_desc(struct perf_file_section *section __maybe_unused,
 		u32 nr_members;
 	} *desc;
 
-	if (do_read_u32(fd, ph, &nr_groups))
+	if (do_read_u32(fd->fd, fd->ph, &nr_groups))
 		return -1;
 
 	ph->env.nr_groups = nr_groups;
@@ -1953,14 +1936,14 @@ static int process_group_desc(struct perf_file_section *section __maybe_unused,
 		return -1;
 
 	for (i = 0; i < nr_groups; i++) {
-		desc[i].name = do_read_string(fd, ph);
+		desc[i].name = do_read_string(fd->fd, fd->ph);
 		if (!desc[i].name)
 			goto out_free;
 
-		if (do_read_u32(fd, ph, &desc[i].leader_idx))
+		if (do_read_u32(fd->fd, fd->ph, &desc[i].leader_idx))
 			goto out_free;
 
-		if (do_read_u32(fd, ph, &desc[i].nr_members))
+		if (do_read_u32(fd->fd, fd->ph, &desc[i].nr_members))
 			goto out_free;
 	}
 
@@ -2011,36 +1994,32 @@ static int process_group_desc(struct perf_file_section *section __maybe_unused,
 	return ret;
 }
 
-static int process_auxtrace(struct perf_file_section *section,
-			    struct perf_header *ph, int fd,
-			    void *data __maybe_unused)
+static int process_auxtrace(struct feat_fd *fd, void *data __maybe_unused)
 {
 	struct perf_session *session;
 	int err;
 
-	session = container_of(ph, struct perf_session, header);
+	session = container_of(fd->ph, struct perf_session, header);
 
-	err = auxtrace_index__process(fd, section->size, session,
-				      ph->needs_swap);
+	err = auxtrace_index__process(fd->fd, fd->size, session,
+				      fd->ph->needs_swap);
 	if (err < 0)
 		pr_err("Failed to process auxtrace index\n");
 	return err;
 }
 
-static int process_cache(struct perf_file_section *section __maybe_unused,
-			 struct perf_header *ph __maybe_unused, int fd __maybe_unused,
-			 void *data __maybe_unused)
+static int process_cache(struct feat_fd *fd, void *data __maybe_unused)
 {
 	struct cpu_cache_level *caches;
 	u32 cnt, i, version;
 
-	if (do_read_u32(fd, ph, &version))
+	if (do_read_u32(fd->fd, fd->ph, &version))
 		return -1;
 
 	if (version != 1)
 		return -1;
 
-	if (do_read_u32(fd, ph, &cnt))
+	if (do_read_u32(fd->fd, fd->ph, &cnt))
 		return -1;
 
 	caches = zalloc(sizeof(*caches) * cnt);
@@ -2051,7 +2030,7 @@ static int process_cache(struct perf_file_section *section __maybe_unused,
 		struct cpu_cache_level c;
 
 		#define _R(v)						\
-			if (do_read_u32(fd, ph, &c.v))\
+			if (do_read_u32(fd->fd, fd->ph, &c.v))\
 				goto out_free_caches;			\
 
 		_R(level)
@@ -2061,7 +2040,7 @@ static int process_cache(struct perf_file_section *section __maybe_unused,
 		#undef _R
 
 		#define _R(v)				\
-			c.v = do_read_string(fd, ph);	\
+			c.v = do_read_string(fd->fd, fd->ph);	\
 			if (!c.v)			\
 				goto out_free_caches;
 
@@ -2073,8 +2052,8 @@ static int process_cache(struct perf_file_section *section __maybe_unused,
 		caches[i] = c;
 	}
 
-	ph->env.caches = caches;
-	ph->env.caches_cnt = cnt;
+	fd->ph->env.caches = caches;
+	fd->ph->env.caches_cnt = cnt;
 	return 0;
 out_free_caches:
 	free(caches);
@@ -2084,8 +2063,7 @@ static int process_cache(struct perf_file_section *section __maybe_unused,
 struct feature_ops {
 	int (*write)(struct feat_fd *fd, struct perf_evlist *evlist);
 	void (*print)(struct feat_fd *fd, FILE *fp);
-	int (*process)(struct perf_file_section *section,
-		       struct perf_header *h, int fd, void *data);
+	int (*process)(struct feat_fd *fd, void *data);
 	const char *name;
 	bool full_only;
 };
@@ -2618,6 +2596,13 @@ static int perf_file_section__process(struct perf_file_section *section,
 				      struct perf_header *ph,
 				      int feat, int fd, void *data)
 {
+	struct feat_fd fdd = {
+		.fd = fd,
+		.ph = ph,
+		.size = section->size,
+		.offset = section->offset,
+	};
+
 	if (lseek(fd, section->offset, SEEK_SET) == (off_t)-1) {
 		pr_debug("Failed to lseek to %" PRIu64 " offset for feature "
 			  "%d, continuing...\n", section->offset, feat);
@@ -2632,7 +2617,7 @@ static int perf_file_section__process(struct perf_file_section *section,
 	if (!feat_ops[feat].process)
 		return 0;
 
-	return feat_ops[feat].process(section, ph, fd, data);
+	return feat_ops[feat].process(&fdd, data);
 }
 
 static int perf_file_header__read_pipe(struct perf_pipe_file_header *header,
Signature · 1 line
2.13.0.219.gdb65acc882-goog
#20 lore raw

PATCH v2 9/13 perf header: use struct feat_fd in read header records

In reply to David Carrillo-Cisneros

As preparation for using header records in-pipe mode, replace int fd with struct feat_fd in read functions for all header record types.

This patch does not change behavior.

---
 tools/perf/util/header.c | 93 ++++++++++++++++++++++++------------------------
 1 file changed, 46 insertions(+), 47 deletions(-)
tools/perf/util/header.c +46 -47
index dc923f5639f8..65cd2d1f1721 100644
@@ -124,16 +124,16 @@ static int do_write_string(struct feat_fd *fd, const char *str)
 	return write_padded(fd, str, olen, len);
 }
 
-static int __do_read(int fd, void *addr, ssize_t size)
+static int __do_read(struct feat_fd *fd, void *addr, ssize_t size)
 {
-	ssize_t ret = readn(fd, addr, size);
+	ssize_t ret = readn(fd->fd, addr, size);
 
 	if (ret != (ssize_t)size)
 		return ret < 0 ? (int)ret : -1;
 	return 0;
 }
 
-static int do_read_u32(int fd, struct perf_header *ph, u32 *addr)
+static int do_read_u32(struct feat_fd *fd, u32 *addr)
 {
 	int ret;
 
@@ -141,12 +141,12 @@ static int do_read_u32(int fd, struct perf_header *ph, u32 *addr)
 	if (ret)
 		return ret;
 
-	if (ph->needs_swap)
+	if (fd->ph->needs_swap)
 		*addr = bswap_32(*addr);
 	return 0;
 }
 
-static int do_read_u64(int fd, struct perf_header *ph, u64 *addr)
+static int do_read_u64(struct feat_fd *fd, u64 *addr)
 {
 	int ret;
 
@@ -154,17 +154,17 @@ static int do_read_u64(int fd, struct perf_header *ph, u64 *addr)
 	if (ret)
 		return ret;
 
-	if (ph->needs_swap)
+	if (fd->ph->needs_swap)
 		*addr = bswap_64(*addr);
 	return 0;
 }
 
-static char *do_read_string(int fd, struct perf_header *ph)
+static char *do_read_string(struct feat_fd *fd)
 {
 	u32 len;
 	char *buf;
 
-	if (do_read_u32(fd, ph, &len))
+	if (do_read_u32(fd, &len))
 		return NULL;
 
 	buf = malloc(len);
@@ -1206,8 +1206,7 @@ static void free_event_desc(struct perf_evsel *events)
 	free(events);
 }
 
-static struct perf_evsel *
-read_event_desc(struct perf_header *ph, int fd)
+static struct perf_evsel *read_event_desc(struct feat_fd *fd)
 {
 	struct perf_evsel *evsel, *events = NULL;
 	u64 *id;
@@ -1217,10 +1216,10 @@ read_event_desc(struct perf_header *ph, int fd)
 	size_t msz;
 
 	/* number of events */
-	if (do_read_u32(fd, ph, &nre))
+	if (do_read_u32(fd, &nre))
 		goto error;
 
-	if (do_read_u32(fd, ph, &sz))
+	if (do_read_u32(fd, &sz))
 		goto error;
 
 	/* buffer to hold on file attr struct */
@@ -1248,18 +1247,18 @@ read_event_desc(struct perf_header *ph, int fd)
 		if (ret != (ssize_t)sz)
 			goto error;
 
-		if (ph->needs_swap)
+		if (fd->ph->needs_swap)
 			perf_event__attr_swap(buf);
 
 		memcpy(&evsel->attr, buf, msz);
 
-		if (do_read_u32(fd, ph, &nr))
+		if (do_read_u32(fd, &nr))
 			goto error;
 
-		if (ph->needs_swap)
+		if (fd->ph->needs_swap)
 			evsel->needs_swap = true;
 
-		evsel->name = do_read_string(fd, ph);
+		evsel->name = do_read_string(fd);
 
 		if (!nr)
 			continue;
@@ -1271,7 +1270,7 @@ read_event_desc(struct perf_header *ph, int fd)
 		evsel->id = id;
 
 		for (j = 0 ; j < nr; j++) {
-			if (do_read_u64(fd, ph, id))
+			if (do_read_u64(fd, id))
 				goto error;
 			id++;
 		}
@@ -1293,7 +1292,7 @@ static int __desc_attr__fprintf(FILE *fp, const char *name, const char *val,
 
 static void print_event_desc(struct feat_fd *fd, FILE *fp)
 {
-	struct perf_evsel *evsel, *events = read_event_desc(fd->ph, fd->fd);
+	struct perf_evsel *evsel, *events = read_event_desc(fd);
 	u32 j;
 	u64 *id;
 
@@ -1587,7 +1586,7 @@ static int perf_header__read_build_ids(struct perf_header *header,
 #define FEAT_PROCESS_STR_FUN(__feat, __feat_env) \
 static int process_##__feat(struct feat_fd *fd, void *data __maybe_unused) \
 {\
-	fd->ph->env.__feat_env = do_read_string(fd->fd, fd->ph); \
+	fd->ph->env.__feat_env = do_read_string(fd); \
 	return fd->ph->env.__feat_env ? 0 : -ENOMEM; \
 }
 
@@ -1617,11 +1616,11 @@ static int process_nrcpus(struct feat_fd *fd, void *data __maybe_unused)
 	int ret;
 	u32 nr_cpus_avail, nr_cpus_online;
 
-	ret = do_read_u32(fd->fd, fd->ph, &nr_cpus_avail);
+	ret = do_read_u32(fd, &nr_cpus_avail);
 	if (ret)
 		return ret;
 
-	ret = do_read_u32(fd->fd, fd->ph, &nr_cpus_online);
+	ret = do_read_u32(fd, &nr_cpus_online);
 	if (ret)
 		return ret;
 	fd->ph->env.nr_cpus_avail = (int)nr_cpus_avail;
@@ -1634,7 +1633,7 @@ static int process_total_mem(struct feat_fd *fd, void *data __maybe_unused)
 	u64 total_mem;
 	int ret;
 
-	ret = do_read_u64(fd->fd, fd->ph, &total_mem);
+	ret = do_read_u64(fd, &total_mem);
 	if (ret)
 		return -1;
 	fd->ph->env.total_mem = (unsigned long long)total_mem;
@@ -1677,7 +1676,7 @@ static int
 process_event_desc(struct feat_fd *fd, void *data __maybe_unused)
 {
 	struct perf_session *session;
-	struct perf_evsel *evsel, *events = read_event_desc(fd->ph, fd->fd);
+	struct perf_evsel *evsel, *events = read_event_desc(fd);
 
 	if (!events)
 		return 0;
@@ -1696,7 +1695,7 @@ static int process_cmdline(struct feat_fd *fd, void *data __maybe_unused)
 	char *str, *cmdline = NULL, **argv = NULL;
 	u32 nr, i, len = 0;
 
-	if (do_read_u32(fd->fd, fd->ph, &nr))
+	if (do_read_u32(fd, &nr))
 		return -1;
 
 	fd->ph->env.nr_cmdline = nr;
@@ -1710,7 +1709,7 @@ static int process_cmdline(struct feat_fd *fd, void *data __maybe_unused)
 		goto error;
 
 	for (i = 0; i < nr; i++) {
-		str = do_read_string(fd->fd, fd->ph);
+		str = do_read_string(fd);
 		if (!str)
 			goto error;
 
@@ -1742,7 +1741,7 @@ static int process_cpu_topology(struct feat_fd *fd, void *data __maybe_unused)
 	if (!ph->env.cpu)
 		return -1;
 
-	if (do_read_u32(fd->fd, fd->ph, &nr))
+	if (do_read_u32(fd, &nr))
 		goto free_cpu;
 
 	ph->env.nr_sibling_cores = nr;
@@ -1750,7 +1749,7 @@ static int process_cpu_topology(struct feat_fd *fd, void *data __maybe_unused)
 		goto free_cpu;
 
 	for (i = 0; i < nr; i++) {
-		str = do_read_string(fd->fd, fd->ph);
+		str = do_read_string(fd);
 		if (!str)
 			goto error;
 
@@ -1761,13 +1760,13 @@ static int process_cpu_topology(struct feat_fd *fd, void *data __maybe_unused)
 	}
 	ph->env.sibling_cores = strbuf_detach(&sb, NULL);
 
-	if (do_read_u32(fd->fd, fd->ph, &nr))
+	if (do_read_u32(fd, &nr))
 		return -1;
 
 	ph->env.nr_sibling_threads = nr;
 
 	for (i = 0; i < nr; i++) {
-		str = do_read_string(fd->fd, fd->ph);
+		str = do_read_string(fd);
 		if (!str)
 			goto error;
 
@@ -1788,12 +1787,12 @@ static int process_cpu_topology(struct feat_fd *fd, void *data __maybe_unused)
 	}
 
 	for (i = 0; i < (u32)cpu_nr; i++) {
-		if (do_read_u32(fd->fd, fd->ph, &nr))
+		if (do_read_u32(fd, &nr))
 			goto free_cpu;
 
 		ph->env.cpu[i].core_id = nr;
 
-		if (do_read_u32(fd->fd, fd->ph, &nr))
+		if (do_read_u32(fd, &nr))
 			goto free_cpu;
 
 		if (nr != (u32)-1 && nr > (u32)cpu_nr) {
@@ -1822,7 +1821,7 @@ static int process_numa_topology(struct feat_fd *fd, void *data __maybe_unused)
 	char *str;
 
 	/* nr nodes */
-	if (do_read_u32(fd->fd, fd->ph, &nr))
+	if (do_read_u32(fd, &nr))
 		return -1;
 
 	nodes = zalloc(sizeof(*nodes) * nr);
@@ -1833,16 +1832,16 @@ static int process_numa_topology(struct feat_fd *fd, void *data __maybe_unused)
 		n = &nodes[i];
 
 		/* node number */
-		if (do_read_u32(fd->fd, fd->ph, &n->node))
+		if (do_read_u32(fd, &n->node))
 			goto error;
 
-		if (do_read_u64(fd->fd, fd->ph, &n->mem_total))
+		if (do_read_u64(fd, &n->mem_total))
 			goto error;
 
-		if (do_read_u64(fd->fd, fd->ph, &n->mem_free))
+		if (do_read_u64(fd, &n->mem_free))
 			goto error;
 
-		str = do_read_string(fd->fd, fd->ph);
+		str = do_read_string(fd);
 		if (!str)
 			goto error;
 
@@ -1869,7 +1868,7 @@ static int process_pmu_mappings(struct feat_fd *fd, void *data __maybe_unused)
 	u32 type;
 	struct strbuf sb;
 
-	if (do_read_u32(fd->fd, fd->ph, &pmu_num))
+	if (do_read_u32(fd, &pmu_num))
 		return -1;
 
 	if (!pmu_num) {
@@ -1882,10 +1881,10 @@ static int process_pmu_mappings(struct feat_fd *fd, void *data __maybe_unused)
 		return -1;
 
 	while (pmu_num) {
-		if (do_read_u32(fd->fd, fd->ph, &type))
+		if (do_read_u32(fd, &type))
 			goto error;
 
-		name = do_read_string(fd->fd, fd->ph);
+		name = do_read_string(fd);
 		if (!name)
 			goto error;
 
@@ -1922,7 +1921,7 @@ static int process_group_desc(struct feat_fd *fd, void *data __maybe_unused)
 		u32 nr_members;
 	} *desc;
 
-	if (do_read_u32(fd->fd, fd->ph, &nr_groups))
+	if (do_read_u32(fd, &nr_groups))
 		return -1;
 
 	ph->env.nr_groups = nr_groups;
@@ -1936,14 +1935,14 @@ static int process_group_desc(struct feat_fd *fd, void *data __maybe_unused)
 		return -1;
 
 	for (i = 0; i < nr_groups; i++) {
-		desc[i].name = do_read_string(fd->fd, fd->ph);
+		desc[i].name = do_read_string(fd);
 		if (!desc[i].name)
 			goto out_free;
 
-		if (do_read_u32(fd->fd, fd->ph, &desc[i].leader_idx))
+		if (do_read_u32(fd, &desc[i].leader_idx))
 			goto out_free;
 
-		if (do_read_u32(fd->fd, fd->ph, &desc[i].nr_members))
+		if (do_read_u32(fd, &desc[i].nr_members))
 			goto out_free;
 	}
 
@@ -2013,13 +2012,13 @@ static int process_cache(struct feat_fd *fd, void *data __maybe_unused)
 	struct cpu_cache_level *caches;
 	u32 cnt, i, version;
 
-	if (do_read_u32(fd->fd, fd->ph, &version))
+	if (do_read_u32(fd, &version))
 		return -1;
 
 	if (version != 1)
 		return -1;
 
-	if (do_read_u32(fd->fd, fd->ph, &cnt))
+	if (do_read_u32(fd, &cnt))
 		return -1;
 
 	caches = zalloc(sizeof(*caches) * cnt);
@@ -2030,7 +2029,7 @@ static int process_cache(struct feat_fd *fd, void *data __maybe_unused)
 		struct cpu_cache_level c;
 
 		#define _R(v)						\
-			if (do_read_u32(fd->fd, fd->ph, &c.v))\
+			if (do_read_u32(fd, &c.v))\
 				goto out_free_caches;			\
 
 		_R(level)
@@ -2040,7 +2039,7 @@ static int process_cache(struct feat_fd *fd, void *data __maybe_unused)
 		#undef _R
 
 		#define _R(v)				\
-			c.v = do_read_string(fd->fd, fd->ph);	\
+			c.v = do_read_string(fd);	\
 			if (!c.v)			\
 				goto out_free_caches;
 
Signature · 1 line
2.13.0.219.gdb65acc882-goog
#21 lore raw

PATCH v2 10/13 perf header: add a buffer to struct feat_fd

In reply to David Carrillo-Cisneros

Extend struct feat_fd to use a temporal buffer in pipe-mode, rather than a perf.data file.

Revamp write_pmu_mappings to avoid seeking so that is compatible with pipe-mode.

Print an error when trying to use buf in features that do not support pipe-mode.

---
 tools/perf/util/header.c | 79 +++++++++++++++++++++++++++++++++++++-----------
 1 file changed, 62 insertions(+), 17 deletions(-)
tools/perf/util/header.c +62 -17
index 65cd2d1f1721..b7704b30ed52 100644
@@ -60,6 +60,7 @@ struct perf_file_attr {
 struct feat_fd {
 	struct perf_header *ph;
 	int fd;
+	void *buf;	/* Either buf != NULL or fd >= 0 */
 	ssize_t offset;
 	size_t size;
 };
@@ -82,11 +83,27 @@ bool perf_header__has_feat(const struct perf_header *header, int feat)
 /* Return: 0 if succeded, -ERR if failed. */
 int do_write(struct feat_fd *fd, const void *buf, size_t size)
 {
-	ssize_t ret;
+	void *addr;
 
-	ret  = writen(fd->fd, buf, size);
-	if (ret != (ssize_t)size)
-		return ret < 0 ? (int)ret: -1;
+	if (!fd->buf) {
+		ssize_t ret = writen(fd->fd, buf, size);
+
+		if (ret != (ssize_t)size)
+			return ret < 0 ? (int)ret : -1;
+		return 0;
+	}
+retry:
+	if (size > (fd->size - fd->offset)) {
+		addr = realloc(fd->buf, fd->size << 1);
+		if (!addr)
+			return -ENOSPC;
+		fd->buf = addr;
+		fd->size <<= 1;
+		goto retry;
+	}
+
+	memcpy(fd->buf + fd->offset, buf, size);
+	fd->offset += size;
 
 	return 0;
 }
@@ -126,10 +143,21 @@ static int do_write_string(struct feat_fd *fd, const char *str)
 
 static int __do_read(struct feat_fd *fd, void *addr, ssize_t size)
 {
-	ssize_t ret = readn(fd->fd, addr, size);
+	if (!fd->buf) {
+		ssize_t ret = readn(fd->fd, addr, size);
+
+		if (ret != (ssize_t)size)
+			return ret < 0 ? (int)ret : -1;
+		return 0;
+	}
+
+	assert((ssize_t)fd->size > fd->offset);
+	if (size > (ssize_t)fd->size - fd->offset)
+		return -1;
+
+	memcpy(addr, fd->buf + fd->offset, size);
+	fd->offset += size;
 
-	if (ret != (ssize_t)size)
-		return ret < 0 ? (int)ret : -1;
 	return 0;
 }
 
@@ -187,6 +215,10 @@ static char *do_read_string(struct feat_fd *fd)
 static int write_tracing_data(struct feat_fd *fd,
 			      struct perf_evlist *evlist)
 {
+	if (fd->buf) {
+		pr_err("Unsupported write_tracing_data to memory buffer.\n");
+		return -1;
+	}
 	return read_tracing_data(fd->fd, &evlist->entries);
 }
 
@@ -201,6 +233,10 @@ static int write_build_id(struct feat_fd *fd,
 	if (!perf_session__read_build_ids(session, true))
 		return -1;
 
+	if (fd->buf) {
+		pr_err("Unsupported write_build_id to memory buffer.\n");
+		return -1;
+	}
 	err = perf_session__write_buildid_table(session, fd);
 	if (err < 0) {
 		pr_debug("failed to write buildid table\n");
@@ -795,11 +831,19 @@ static int write_pmu_mappings(struct feat_fd *fd,
 			      struct perf_evlist *evlist __maybe_unused)
 {
 	struct perf_pmu *pmu = NULL;
-	off_t offset = lseek(fd->fd, 0, SEEK_CUR);
-	__u32 pmu_num = 0;
+	u32 pmu_num = 0;
 	int ret;
 
-	/* write real pmu_num later */
+	/*
+	 * Do a first pass to count number of pmu to avoid lseek so this
+	 * works in pipe mode as well.
+	 */
+	while ((pmu = perf_pmu__scan(pmu))) {
+		if (!pmu->name)
+			continue;
+		pmu_num++;
+	}
+
 	ret = do_write(fd, &pmu_num, sizeof(pmu_num));
 	if (ret < 0)
 		return ret;
@@ -807,7 +851,6 @@ static int write_pmu_mappings(struct feat_fd *fd,
 	while ((pmu = perf_pmu__scan(pmu))) {
 		if (!pmu->name)
 			continue;
-		pmu_num++;
 
 		ret = do_write(fd, &pmu->type, sizeof(pmu->type));
 		if (ret < 0)
@@ -818,12 +861,6 @@ static int write_pmu_mappings(struct feat_fd *fd,
 			return ret;
 	}
 
-	if (pwrite(fd->fd, &pmu_num, sizeof(pmu_num), offset) != sizeof(pmu_num)) {
-		/* discard all */
-		lseek(fd->fd, offset, SEEK_SET);
-		return -1;
-	}
-
 	return 0;
 }
 
@@ -909,6 +946,10 @@ static int write_auxtrace(struct feat_fd *fd,
 	struct perf_session *session;
 	int err;
 
+	if (fd->buf) {
+		pr_err("Unsupported write_auxtrace to memory buffer.\n");
+		return -1;
+	}
 	session = container_of(fd->ph, struct perf_session, header);
 
 	err = auxtrace_index__write(fd->fd, &session->auxtrace_index);
@@ -2187,6 +2228,10 @@ static int do_write_feat(struct feat_fd *fd, struct perf_header *h, int type,
 		if (!feat_ops[type].write)
 			return -1;
 
+		if (fd->buf) {
+			pr_err("Called %s for memory buffer.\n", __func__);
+			return -1;
+		}
 		(*p)->offset = lseek(fd->fd, 0, SEEK_CUR);
 
 		err = feat_ops[type].write(fd, evlist);
Signature · 1 line
2.13.0.219.gdb65acc882-goog
#22 lore raw

Re: [PATCH v2 10/13] perf header: add a buffer to struct feat_fd

On Tue, May 23, 2017 at 12:48:50AM -0700, David Carrillo-Cisneros wrote:

SNIP

Quoted 10 lines
tools/perf/util/header.c +3 -2
+
+	memcpy(addr, fd->buf + fd->offset, size);
+	fd->offset += size;
 
-	if (ret != (ssize_t)size)
-		return ret < 0 ? (int)ret : -1;
 	return 0;
 }

tools/perf/util/header.c static char *do_read_string(struct feat_fd *fd) · line 221

Quoted 7 lines
tools/perf/util/header.c +4 -0
@@ -187,6 +215,10 @@ static char *do_read_string(struct feat_fd *fd)
 static int write_tracing_data(struct feat_fd *fd,
 			      struct perf_evlist *evlist)
 {
+	if (fd->buf) {
+		pr_err("Unsupported write_tracing_data to memory buffer.\n");
+		return -1;
+	}

could those messsages mention the pipe mode, this one does not give clue it's pipe mode related

also together with your following patches, this condition should never hit right? more like the assert stuff.. WARN_ON/WARN_ON_ONCE maybe

thanks,
jirka

#23 lore raw

Re: [PATCH v2 10/13] perf header: add a buffer to struct feat_fd

In reply to David Carrillo-Cisneros

On Tue, May 23, 2017 at 12:48:50AM -0700, David Carrillo-Cisneros wrote:

SNIP

Quoted 21 lines
tools/perf/util/header.c +12 -4
@@ -82,11 +83,27 @@ bool perf_header__has_feat(const struct perf_header *header, int feat)
 /* Return: 0 if succeded, -ERR if failed. */
 int do_write(struct feat_fd *fd, const void *buf, size_t size)
 {
-	ssize_t ret;
+	void *addr;
 
-	ret  = writen(fd->fd, buf, size);
-	if (ret != (ssize_t)size)
-		return ret < 0 ? (int)ret: -1;
+	if (!fd->buf) {
+		ssize_t ret = writen(fd->fd, buf, size);
+
+		if (ret != (ssize_t)size)
+			return ret < 0 ? (int)ret : -1;
+		return 0;
+	}
+retry:
+	if (size > (fd->size - fd->offset)) {
+		addr = realloc(fd->buf, fd->size << 1);
+		if (!addr)

tools/perf/util/header.c bool perf_header__has_feat(const struct perf_header *header, int feat) · line 108

Quoted 10 lines
tools/perf/util/header.c +8 -0
+			return -ENOSPC;
+		fd->buf = addr;
+		fd->size <<= 1;
+		goto retry;
+	}
+
+	memcpy(fd->buf + fd->offset, buf, size);
+	fd->offset += size;
 
 	return 0;

please put those 2 cases in separate functions

Quoted 15 lines
tools/perf/util/header.c +9 -1
 }
@@ -126,10 +143,21 @@ static int do_write_string(struct feat_fd *fd, const char *str)
 
 static int __do_read(struct feat_fd *fd, void *addr, ssize_t size)
 {
-	ssize_t ret = readn(fd->fd, addr, size);
+	if (!fd->buf) {
+		ssize_t ret = readn(fd->fd, addr, size);
+
+		if (ret != (ssize_t)size)
+			return ret < 0 ? (int)ret : -1;
+		return 0;
+	}
+
+	assert((ssize_t)fd->size > fd->offset);

tools/perf/util/header.c static int do_write_string(struct feat_fd *fd, const char *str) · line 162

Quoted 10 lines
tools/perf/util/header.c +5 -2
+	if (size > (ssize_t)fd->size - fd->offset)
+		return -1;
+
+	memcpy(addr, fd->buf + fd->offset, size);
+	fd->offset += size;
 
-	if (ret != (ssize_t)size)
-		return ret < 0 ? (int)ret : -1;
 	return 0;
 }

same for the read

thanks,
jirka

#24 lore raw

Re: [PATCH v2 10/13] perf header: add a buffer to struct feat_fd

In reply to David Carrillo-Cisneros
Quoted 8 lines · David Carrillo-Cisneros

Extend struct feat_fd to use a temporal buffer in pipe-mode, rather than a perf.data file.

Revamp write_pmu_mappings to avoid seeking so that is compatible with pipe-mode.

Print an error when trying to use buf in features that do not support pipe-mode.

could you please just list those features in changelog plus the reason they don't support this

thanks,
jirka

#25 lore raw

Re: [PATCH v2 10/13] perf header: add a buffer to struct feat_fd

In reply to David Carrillo-Cisneros

On Tue, May 23, 2017 at 12:48:50AM -0700, David Carrillo-Cisneros wrote:

SNIP

Quoted 9 lines
tools/perf/util/header.c +1 -2
@@ -795,11 +831,19 @@ static int write_pmu_mappings(struct feat_fd *fd,
 			      struct perf_evlist *evlist __maybe_unused)
 {
 	struct perf_pmu *pmu = NULL;
-	off_t offset = lseek(fd->fd, 0, SEEK_CUR);
-	__u32 pmu_num = 0;
+	u32 pmu_num = 0;
 	int ret;
 

tools/perf/util/header.c static int write_pmu_mappings(struct feat_fd *fd, · line 845

Quoted 10 lines
tools/perf/util/header.c +9 -1
-	/* write real pmu_num later */
+	/*
+	 * Do a first pass to count number of pmu to avoid lseek so this
+	 * works in pipe mode as well.
+	 */
+	while ((pmu = perf_pmu__scan(pmu))) {
+		if (!pmu->name)
+			continue;
+		pmu_num++;
+	}

please put this functionality into preceeding separate patch

thanks,
jirka

#26 lore raw

PATCH v2 11/13 perf header: change FEAT_OP* macros

In reply to David Carrillo-Cisneros

There are three FEAT_OP* macros:
- FEAT_OPA: for features without process record.
- FEAT_OPP: for features with process record.
- FEAT_OPF: like FEAT_OPP but to show only if show_full_info flags is set.

To add pipe-mode headers we need to specify whether a feature generates a corresponding sample record. This is an extra variation to the existing macros. To simplify, redefine macros instead so that:
- show_full_info is specified as an argument (to remove the FEAT_OPF variation) and,
- it always sets "process" handler (to remove the FEAT_OPA variation). Individual process handlers can be NULLed individually.

This allows to define two variations only:
- FEAT_OPR: has sample record.
- FEAT_OPN: doesn't has sample record.

---
 tools/perf/util/header.c | 72 +++++++++++++++++++++++++++++-------------------
 1 file changed, 44 insertions(+), 28 deletions(-)
tools/perf/util/header.c +44 -28
index b7704b30ed52..7aa84a02b9bd 100644
@@ -11,6 +11,7 @@
 #include <linux/list.h>
 #include <linux/kernel.h>
 #include <linux/bitops.h>
+#include <linux/stringify.h>
 #include <sys/stat.h>
 #include <sys/types.h>
 #include <sys/utsname.h>
@@ -2106,42 +2107,57 @@ struct feature_ops {
 	int (*process)(struct feat_fd *fd, void *data);
 	const char *name;
 	bool full_only;
+	bool has_record;
 };
 
-#define FEAT_OPA(n, func) \
-	[n] = { .name = #n, .write = write_##func, .print = print_##func }
-#define FEAT_OPP(n, func) \
-	[n] = { .name = #n, .write = write_##func, .print = print_##func, \
-		.process = process_##func }
-#define FEAT_OPF(n, func) \
-	[n] = { .name = #n, .write = write_##func, .print = print_##func, \
-		.process = process_##func, .full_only = true }
+#define FEAT_OPR(n, func, __full_only) \
+	[HEADER_##n] = {					\
+		.name = __stringify(HEADER_##n),		\
+		.write = write_##func,				\
+		.print = print_##func,				\
+		.full_only = __full_only,			\
+		.process = process_##func,			\
+		.has_record = true				\
+	}
+
+#define FEAT_OPN(n, func, __full_only) \
+	[HEADER_##n] = {					\
+		.name = __stringify(HEADER_##n),		\
+		.write = write_##func,				\
+		.print = print_##func,				\
+		.full_only = __full_only,			\
+		.process = process_##func			\
+	}
 
 /* feature_ops not implemented: */
 #define print_tracing_data	NULL
 #define print_build_id		NULL
 
+#define process_branch_stack	NULL
+#define process_stat		NULL
+
+
 static const struct feature_ops feat_ops[HEADER_LAST_FEATURE] = {
-	FEAT_OPP(HEADER_TRACING_DATA,	tracing_data),
-	FEAT_OPP(HEADER_BUILD_ID,	build_id),
-	FEAT_OPP(HEADER_HOSTNAME,	hostname),
-	FEAT_OPP(HEADER_OSRELEASE,	osrelease),
-	FEAT_OPP(HEADER_VERSION,	version),
-	FEAT_OPP(HEADER_ARCH,		arch),
-	FEAT_OPP(HEADER_NRCPUS,		nrcpus),
-	FEAT_OPP(HEADER_CPUDESC,	cpudesc),
-	FEAT_OPP(HEADER_CPUID,		cpuid),
-	FEAT_OPP(HEADER_TOTAL_MEM,	total_mem),
-	FEAT_OPP(HEADER_EVENT_DESC,	event_desc),
-	FEAT_OPP(HEADER_CMDLINE,	cmdline),
-	FEAT_OPF(HEADER_CPU_TOPOLOGY,	cpu_topology),
-	FEAT_OPF(HEADER_NUMA_TOPOLOGY,	numa_topology),
-	FEAT_OPA(HEADER_BRANCH_STACK,	branch_stack),
-	FEAT_OPP(HEADER_PMU_MAPPINGS,	pmu_mappings),
-	FEAT_OPP(HEADER_GROUP_DESC,	group_desc),
-	FEAT_OPP(HEADER_AUXTRACE,	auxtrace),
-	FEAT_OPA(HEADER_STAT,		stat),
-	FEAT_OPF(HEADER_CACHE,		cache),
+	FEAT_OPN(TRACING_DATA,	tracing_data,	false),
+	FEAT_OPN(BUILD_ID,	build_id,	false),
+	FEAT_OPR(HOSTNAME,	hostname,	false),
+	FEAT_OPR(OSRELEASE,	osrelease,	false),
+	FEAT_OPR(VERSION,	version,	false),
+	FEAT_OPR(ARCH,		arch,		false),
+	FEAT_OPR(NRCPUS,	nrcpus,		false),
+	FEAT_OPR(CPUDESC,	cpudesc,	false),
+	FEAT_OPR(CPUID,		cpuid,		false),
+	FEAT_OPR(TOTAL_MEM,	total_mem,	false),
+	FEAT_OPR(EVENT_DESC,	event_desc,	false),
+	FEAT_OPR(CMDLINE,	cmdline,	false),
+	FEAT_OPR(CPU_TOPOLOGY,	cpu_topology,	true),
+	FEAT_OPR(NUMA_TOPOLOGY,	numa_topology,	true),
+	FEAT_OPN(BRANCH_STACK,	branch_stack,	false),
+	FEAT_OPR(PMU_MAPPINGS,	pmu_mappings,	false),
+	FEAT_OPN(GROUP_DESC,	group_desc,	false),
+	FEAT_OPN(AUXTRACE,	auxtrace,	false),
+	FEAT_OPN(STAT,		stat,		false),
+	FEAT_OPN(CACHE,		cache,		true),
 };
 
 struct header_print_data {
Signature · 1 line
2.13.0.219.gdb65acc882-goog
#27 lore raw

Re: [PATCH v2 11/13] perf header: change FEAT_OP* macros

On Tue, May 23, 2017 at 12:48:51AM -0700, David Carrillo-Cisneros wrote:

SNIP

Quoted 16 lines
tools/perf/util/header.c +8 -8
-#define FEAT_OPA(n, func) \
-	[n] = { .name = #n, .write = write_##func, .print = print_##func }
-#define FEAT_OPP(n, func) \
-	[n] = { .name = #n, .write = write_##func, .print = print_##func, \
-		.process = process_##func }
-#define FEAT_OPF(n, func) \
-	[n] = { .name = #n, .write = write_##func, .print = print_##func, \
-		.process = process_##func, .full_only = true }
+#define FEAT_OPR(n, func, __full_only) \
+	[HEADER_##n] = {					\
+		.name = __stringify(HEADER_##n),		\
+		.write = write_##func,				\
+		.print = print_##func,				\
+		.full_only = __full_only,			\
+		.process = process_##func,			\
+		.has_record = true				\

tools/perf/util/header.c struct feature_ops { · line 2130

Quoted 10 lines
tools/perf/util/header.c +10 -0
+	}
+
+#define FEAT_OPN(n, func, __full_only) \
+	[HEADER_##n] = {					\
+		.name = __stringify(HEADER_##n),		\
+		.write = write_##func,				\
+		.print = print_##func,				\
+		.full_only = __full_only,			\
+		.process = process_##func			\
+	}

could you please align this (and in other places of the patchset), like:

	.name		= __stringify(HEADER_##n),	\
	.write		= write_##func,			\
	.print		= print_##func,			\
	.full_only	= __full_only,			\
	.process	= process_##func		\

much more readable..

thanks,
jirka

#28 lore raw

Re: [PATCH v2 11/13] perf header: change FEAT_OP* macros

In reply to David Carrillo-Cisneros
Quoted 8 lines · David Carrillo-Cisneros

There are three FEAT_OP* macros:
- FEAT_OPA: for features without process record.
- FEAT_OPP: for features with process record.
- FEAT_OPF: like FEAT_OPP but to show only if show_full_info flags is set.

To add pipe-mode headers we need to specify whether a feature generates a corresponding sample record. This is an extra variation to the existing

nit, the sample word here is confusing, it's an aux event

Quoted 28 lines

macros. To simplify, redefine macros instead so that:
- show_full_info is specified as an argument (to remove the FEAT_OPF variation) and,
- it always sets "process" handler (to remove the FEAT_OPA variation). Individual process handlers can be NULLed individually.

This allows to define two variations only:
- FEAT_OPR: has sample record.
- FEAT_OPN: doesn't has sample record.

---
 tools/perf/util/header.c | 72 +++++++++++++++++++++++++++++-------------------
 1 file changed, 44 insertions(+), 28 deletions(-)
tools/perf/util/header.c +1 -0
index b7704b30ed52..7aa84a02b9bd 100644
@@ -11,6 +11,7 @@
 #include <linux/list.h>
 #include <linux/kernel.h>
 #include <linux/bitops.h>
+#include <linux/stringify.h>
 #include <sys/stat.h>
 #include <sys/types.h>
 #include <sys/utsname.h>

tools/perf/util/header.c struct feature_ops { · line 2110

Quoted 4 lines
tools/perf/util/header.c +1 -0
@@ -2106,42 +2107,57 @@ struct feature_ops {
 	int (*process)(struct feat_fd *fd, void *data);
 	const char *name;
 	bool full_only;
+	bool has_record;

'synthesize' might be better choice

jirka

#29 lore raw

Re: [PATCH v2 11/13] perf header: change FEAT_OP* macros

Quoted 11 lines · Jiri Olsa
Quoted 8 lines · David Carrillo-Cisneros

There are three FEAT_OP* macros:
- FEAT_OPA: for features without process record.
- FEAT_OPP: for features with process record.
- FEAT_OPF: like FEAT_OPP but to show only if show_full_info flags is set.

To add pipe-mode headers we need to specify whether a feature generates a corresponding sample record. This is an extra variation to the existing

nit, the sample word here is confusing, it's an aux event

Will change.

Quoted 27 lines
Quoted 27 lines

macros. To simplify, redefine macros instead so that:
- show_full_info is specified as an argument (to remove the FEAT_OPF variation) and,
- it always sets "process" handler (to remove the FEAT_OPA variation). Individual process handlers can be NULLed individually.

This allows to define two variations only:
- FEAT_OPR: has sample record.
- FEAT_OPN: doesn't has sample record.

---
 tools/perf/util/header.c | 72 +++++++++++++++++++++++++++++-------------------
 1 file changed, 44 insertions(+), 28 deletions(-)
tools/perf/util/header.c +1 -0
index b7704b30ed52..7aa84a02b9bd 100644
@@ -11,6 +11,7 @@
 #include <linux/list.h>
 #include <linux/kernel.h>
 #include <linux/bitops.h>
+#include <linux/stringify.h>
 #include <sys/stat.h>
 #include <sys/types.h>
Quoted 8 lines

tools/perf/util/header.c struct feature_ops { · line 2110

Quoted 6 lines
tools/perf/util/header.c +1 -0
 #include <sys/utsname.h>
@@ -2106,42 +2107,57 @@ struct feature_ops {
      int (*process)(struct feat_fd *fd, void *data);
      const char *name;
      bool full_only;
+     bool has_record;

'synthesize' might be better choice

Yeah, it's better. Will change.

Quoted 2 lines

jirka

#30 lore raw

PATCH v2 12/13 perf tool: add show_feature_header to perf_tool

In reply to David Carrillo-Cisneros

Add show_feat_hdr to control level of printed information of feature headers.

---
 tools/perf/builtin-report.c | 11 ++++++++---
 tools/perf/builtin-script.c |  3 +++
 tools/perf/util/tool.h      |  7 +++++++
 3 files changed, 18 insertions(+), 3 deletions(-)
tools/perf/builtin-report.c +8 -3
index 22478ff2b706..7620d708c78b 100644
@@ -985,9 +985,14 @@ int cmd_report(int argc, const char **argv)
 		perf_hpp_list.need_collapse = true;
 	}
 
-	/* Force tty output for header output and per-thread stat. */
-	if (report.header || report.header_only || report.show_threads)
-		use_browser = 0;
+	if (report.header || report.header_only) {
+		report.tool.show_feat_hdr = SHOW_FEAT_HEADER;
+		/* Force tty output for header output and per-thread stat. */
+		if (report.show_threads)
+			use_browser = 0;
+	}
+	if (report.show_full_info)
+		report.tool.show_feat_hdr = SHOW_FEAT_HEADER_FULL_INFO;
 
 	if (strcmp(input_name, "-") != 0)
 		setup_browser(true);
tools/perf/builtin-script.c +3 -0
index d05aec491cff..0934d0885614 100644
@@ -2684,10 +2684,13 @@ int cmd_script(int argc, const char **argv)
 		return -1;
 
 	if (header || header_only) {
+		script.tool.show_feat_hdr = SHOW_FEAT_HEADER;
 		perf_session__fprintf_info(session, stdout, show_full_info);
 		if (header_only)
 			goto out_delete;
 	}
+	if (show_full_info)
+		script.tool.show_feat_hdr = SHOW_FEAT_HEADER_FULL_INFO;
 
 	if (symbol__init(&session->header.env) < 0)
 		goto out_delete;
tools/perf/util/tool.h +7 -0
index 829471a1c6d7..baeca808dfda 100644
@@ -34,6 +34,12 @@ typedef int (*event_oe)(struct perf_tool *tool, union perf_event *event,
 typedef s64 (*event_op3)(struct perf_tool *tool, union perf_event *event,
 			 struct perf_session *session);
 
+enum show_feature_header {
+	SHOW_FEAT_NO_HEADER = 0,
+	SHOW_FEAT_HEADER,
+	SHOW_FEAT_HEADER_FULL_INFO,
+};
+
 struct perf_tool {
 	event_sample	sample,
 			read;
@@ -68,6 +74,7 @@ struct perf_tool {
 	bool		ordered_events;
 	bool		ordering_requires_timestamps;
 	bool		namespace_events;
+	enum show_feature_header show_feat_hdr;
 };
 
 #endif /* __PERF_TOOL_H */
Signature · 1 line
2.13.0.219.gdb65acc882-goog
#31 lore raw

PATCH v2 13/13 perf tools: add feature header record to pipe-mode

In reply to David Carrillo-Cisneros

Add header record types to pipe-mode, reusing the functions used in file-mode and leveraging the new struct feat_fd.

Add the perf_event__synthesize_feature event call back to process the new header records.

Before this patch:

  $ perf record -o - -e cycles sleep 1 | perf report --stdio --header
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.000 MB - ]
  ...

After this patch:

  $ perf record -o - -e cycles sleep 1 | perf report --stdio --header
  # ========
  # captured on: Mon May 22 16:33:43 2017
  # ========
  #
  # hostname : my_hostname
  # os release : 4.11.0-dbx-up_perf
  # perf version : 4.11.rc6.g6277c80
  # arch : x86_64
  # nrcpus online : 72
  # nrcpus avail : 72
  # cpudesc : Intel(R) Xeon(R) CPU E5-2696 v3 @ 2.30GHz
  # cpuid : GenuineIntel,6,63,2
  # total memory : 263457192 kB
  # cmdline : /root/perf record -o - -e cycles -c 100000 sleep 1
  # HEADER_CPU_TOPOLOGY info available, use -I to display
  # HEADER_NUMA_TOPOLOGY info available, use -I to display
  # pmu mappings: intel_bts = 6, uncore_imc_4 = 22, uncore_sbox_1 = 47, uncore_cbox_5 = 33, uncore_ha_0 = 16, uncore_cbox
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.000 MB - ]
  ...

Support added for the subcommands: report, inject, annotate and script.

---
 tools/perf/Documentation/perf.data-file-format.txt |  10 +-
 tools/perf/builtin-annotate.c                      |   1 +
 tools/perf/builtin-inject.c                        |   1 +
 tools/perf/builtin-record.c                        |   7 ++
 tools/perf/builtin-report.c                        |   1 +
 tools/perf/builtin-script.c                        |   1 +
 tools/perf/util/event.c                            |   1 +
 tools/perf/util/event.h                            |   8 ++
 tools/perf/util/header.c                           | 109 ++++++++++++++++++++-
 tools/perf/util/header.h                           |   9 ++
 tools/perf/util/session.c                          |  12 +++
 tools/perf/util/tool.h                             |   3 +-
 12 files changed, 157 insertions(+), 6 deletions(-)
tools/perf/Documentation/perf.data-file-format.txt +8 -2
index fa2a9132f0a9..a1d5c4303592 100644
@@ -398,6 +398,11 @@ struct auxtrace_error_event {
 	char msg[MAX_AUXTRACE_ERROR_MSG];
 };
 
+	PERF_RECORD_HEADER_FEATURE		= 80,
+
+Describes a header feature. These are records used in pipe-mode that
+contain information that otherwise would be in perf.data file's header.
+
 Event types
 
 Define the event attributes with their IDs.
@@ -422,8 +427,9 @@ struct perf_pipe_file_header {
 };
 
 The information about attrs, data, and event_types is instead in the
-synthesized events PERF_RECORD_ATTR, PERF_RECORD_HEADER_TRACING_DATA and
-PERF_RECORD_HEADER_EVENT_TYPE that are generated by perf record in pipe-mode.
+synthesized events PERF_RECORD_ATTR, PERF_RECORD_HEADER_TRACING_DATA,
+PERF_RECORD_HEADER_EVENT_TYPE, and PERF_RECORD_HEADER_FEATURE
+that are generated by perf record in pipe-mode.
 
 
 References:
tools/perf/builtin-annotate.c +1 -0
index ce44edc30c71..ffe28002dc4f 100644
@@ -398,6 +398,7 @@ int cmd_annotate(int argc, const char **argv)
 			.attr	= perf_event__process_attr,
 			.build_id = perf_event__process_build_id,
 			.tracing_data   = perf_event__process_tracing_data,
+			.feature	= perf_event__process_feature,
 			.ordered_events = true,
 			.ordering_requires_timestamps = true,
 		},
tools/perf/builtin-inject.c +1 -0
index ea8db38eedd1..2b8032908fb2 100644
@@ -770,6 +770,7 @@ int cmd_inject(int argc, const char **argv)
 			.finished_round	= perf_event__repipe_oe_synth,
 			.build_id	= perf_event__repipe_op2_synth,
 			.id_index	= perf_event__repipe_op2_synth,
+			.feature	= perf_event__repipe_op2_synth,
 		},
 		.input_name  = "-",
 		.samples = LIST_HEAD_INIT(inject.samples),
tools/perf/builtin-record.c +7 -0
index ee7d0a82ccd0..a1bcb72b4195 100644
@@ -799,6 +799,13 @@ static int record__synthesize(struct record *rec, bool tail)
 		return 0;
 
 	if (file->is_pipe) {
+		err = perf_event__synthesize_features(
+			tool, session, rec->evlist, process_synthesized_event);
+		if (err < 0) {
+			pr_err("Couldn't synthesize features.\n");
+			return err;
+		}
+
 		err = perf_event__synthesize_attrs(tool, session,
 						   process_synthesized_event);
 		if (err < 0) {
tools/perf/builtin-report.c +1 -0
index 7620d708c78b..33881ca7dde6 100644
@@ -718,6 +718,7 @@ int cmd_report(int argc, const char **argv)
 			.id_index	 = perf_event__process_id_index,
 			.auxtrace_info	 = perf_event__process_auxtrace_info,
 			.auxtrace	 = perf_event__process_auxtrace,
+			.feature	 = perf_event__process_feature,
 			.ordered_events	 = true,
 			.ordering_requires_timestamps = true,
 		},
tools/perf/builtin-script.c +1 -0
index 0934d0885614..3f97eeba8105 100644
@@ -2397,6 +2397,7 @@ int cmd_script(int argc, const char **argv)
 			.attr		 = process_attr,
 			.event_update   = perf_event__process_event_update,
 			.tracing_data	 = perf_event__process_tracing_data,
+			.feature	 = perf_event__process_feature,
 			.build_id	 = perf_event__process_build_id,
 			.id_index	 = perf_event__process_id_index,
 			.auxtrace_info	 = perf_event__process_auxtrace_info,
tools/perf/util/event.c +1 -0
index 142835c0ca0a..b23b02e21c92 100644
@@ -56,6 +56,7 @@ static const char *perf_event__names[] = {
 	[PERF_RECORD_STAT_ROUND]		= "STAT_ROUND",
 	[PERF_RECORD_EVENT_UPDATE]		= "EVENT_UPDATE",
 	[PERF_RECORD_TIME_CONV]			= "TIME_CONV",
+	[PERF_RECORD_HEADER_FEATURE]		= "FEATURE",
 };
 
 static const char *perf_ns__names[] = {
tools/perf/util/event.h +8 -0
index db2de6413518..c09c7a6f7663 100644
@@ -244,6 +244,7 @@ enum perf_user_event_type { /* above any possible kernel type */
 	PERF_RECORD_STAT_ROUND			= 77,
 	PERF_RECORD_EVENT_UPDATE		= 78,
 	PERF_RECORD_TIME_CONV			= 79,
+	PERF_RECORD_HEADER_FEATURE		= 80,
 	PERF_RECORD_HEADER_MAX
 };
 
@@ -488,6 +489,12 @@ struct time_conv_event {
 	u64 time_zero;
 };
 
+struct feature_event {
+	struct perf_event_header header;
+	u64 header_id;
+	char data[]; /* size bytes of raw data specific to the feature */
+};
+
 union perf_event {
 	struct perf_event_header	header;
 	struct mmap_event		mmap;
@@ -518,6 +525,7 @@ union perf_event {
 	struct stat_event		stat;
 	struct stat_round_event		stat_round;
 	struct time_conv_event		time_conv;
+	struct feature_event		feat;
 };
 
 void perf_event__print_totals(void);
tools/perf/util/header.c +106 -3

139 lines not on this page · raw message

tools/perf/util/header.h +9 -0
index 9d8dcd5eb727..f7a16ee527b8 100644
@@ -101,6 +101,15 @@ int perf_header__process_sections(struct perf_header *header, int fd,
 
 int perf_header__fprintf_info(struct perf_session *s, FILE *fp, bool full);
 
+int perf_event__synthesize_features(struct perf_tool *tool,
+				    struct perf_session *session,
+				    struct perf_evlist *evlist,
+				    perf_event__handler_t process);
+
+int perf_event__process_feature(struct perf_tool *tool,
+				union perf_event *event,
+				struct perf_session *session);
+
 int perf_event__synthesize_attr(struct perf_tool *tool,
 				struct perf_event_attr *attr, u32 ids, u64 *id,
 				perf_event__handler_t process);
tools/perf/util/session.c +12 -0
index 3041c6b98191..ffde0dafed6f 100644
@@ -256,6 +256,14 @@ static int process_event_stub(struct perf_tool *tool __maybe_unused,
 	return 0;
 }
 
+static int process_feature_stub(struct perf_tool *tool __maybe_unused,
+				union perf_event *event __maybe_unused,
+				struct perf_session *session __maybe_unused)
+{
+	dump_printf(": unhandled!\n");
+	return 0;
+}
+
 static int process_finished_round_stub(struct perf_tool *tool __maybe_unused,
 				       union perf_event *event __maybe_unused,
 				       struct ordered_events *oe __maybe_unused)
@@ -427,6 +435,8 @@ void perf_tool__fill_defaults(struct perf_tool *tool)
 		tool->stat_round = process_stat_round_stub;
 	if (tool->time_conv == NULL)
 		tool->time_conv = process_event_op2_stub;
+	if (tool->feature == NULL)
+		tool->feature = process_feature_stub;
 }
 
 static void swap_sample_id_all(union perf_event *event, void *data)
@@ -1370,6 +1380,8 @@ static s64 perf_session__process_user_event(struct perf_session *session,
 	case PERF_RECORD_TIME_CONV:
 		session->time_conv = event->time_conv;
 		return tool->time_conv(tool, event, session);
+	case PERF_RECORD_HEADER_FEATURE:
+		return tool->feature(tool, event, session);
 	default:
 		return -EINVAL;
 	}
tools/perf/util/tool.h +2 -1
index baeca808dfda..d549e50db397 100644
@@ -69,7 +69,8 @@ struct perf_tool {
 			cpu_map,
 			stat_config,
 			stat,
-			stat_round;
+			stat_round,
+			feature;
 	event_op3	auxtrace;
 	bool		ordered_events;
 	bool		ordering_requires_timestamps;
Signature · 1 line
2.13.0.219.gdb65acc882-goog
maintainer
#32 lore raw

Re: [PATCH v2 13/13] perf tools: add feature header record to pipe-mode

Quoted 40 lines · David Carrillo-Cisneros

Add header record types to pipe-mode, reusing the functions used in file-mode and leveraging the new struct feat_fd.

Add the perf_event__synthesize_feature event call back to process the new header records.

Before this patch:

  $ perf record -o - -e cycles sleep 1 | perf report --stdio --header
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.000 MB - ]
  ...

After this patch:

  $ perf record -o - -e cycles sleep 1 | perf report --stdio --header
  # ========
  # captured on: Mon May 22 16:33:43 2017
  # ========
  #
  # hostname : my_hostname
  # os release : 4.11.0-dbx-up_perf
  # perf version : 4.11.rc6.g6277c80
  # arch : x86_64
  # nrcpus online : 72
  # nrcpus avail : 72
  # cpudesc : Intel(R) Xeon(R) CPU E5-2696 v3 @ 2.30GHz
  # cpuid : GenuineIntel,6,63,2
  # total memory : 263457192 kB
  # cmdline : /root/perf record -o - -e cycles -c 100000 sleep 1
  # HEADER_CPU_TOPOLOGY info available, use -I to display
  # HEADER_NUMA_TOPOLOGY info available, use -I to display
  # pmu mappings: intel_bts = 6, uncore_imc_4 = 22, uncore_sbox_1 = 47, uncore_cbox_5 = 33, uncore_ha_0 = 16, uncore_cbox
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.000 MB - ]
  ...

Support added for the subcommands: report, inject, annotate and script.

---

[SNIP]

tools/perf/util/event.h struct time_conv_event { · line 494

Quoted 3 lines
tools/perf/util/event.h +3 -0
+struct feature_event {
+	struct perf_event_header header;
+	u64 header_id;

s/header_id/feat_id/ ?

Quoted 6 lines
tools/perf/util/event.h +3 -0
+	char data[]; /* size bytes of raw data specific to the feature */
+};
+
 union perf_event {
 	struct perf_event_header	header;
 	struct mmap_event		mmap;

[SNIP]

Quoted 11 lines
tools/perf/util/header.c +11 -0
+int perf_event__process_feature(struct perf_tool *tool,
+				union perf_event *event,
+				struct perf_session *session __maybe_unused)
+{
+	struct feat_fd fd = { .fd = 0 };
+	struct feature_event *fe = (struct feature_event *)event;
+	int type = fe->header.type;
+	u64 feat = fe->header_id;
+
+	if (type < 0 || type >= PERF_RECORD_HEADER_MAX) {
+		pr_warning("invalid record type %d\n", type);

tools/perf/util/header.c int perf_event__synthesize_attr(struct perf_tool *tool, · line 3028

Quoted 10 lines
tools/perf/util/header.c +10 -0
+		return 0;
+	}
+	if (feat == HEADER_RESERVED)
+		return -1;
+
+	if (feat > HEADER_LAST_FEATURE)
+		return 0;
+
+	if (!feat_ops[feat].process)
+		return 0;

Checking here..

Quoted 7 lines
tools/perf/util/header.c +7 -0
+
+	/*
+	 * no print routine
+	 */
+	if (!feat_ops[feat].print)
+		return 0;
+

tools/perf/util/header.c int perf_event__synthesize_attr(struct perf_tool *tool, · line 3045

Quoted 10 lines
tools/perf/util/header.c +10 -0
+	fd.buf  = (void *)fe->data;
+	fd.size = event->header.size - sizeof(event->header);
+	fd.ph = &session->header;
+
+	if (!tool->show_feat_hdr)
+		return 0;
+
+	if (!feat_ops[feat].full_only ||
+	    tool->show_feat_hdr >= SHOW_FEAT_HEADER_FULL_INFO) {
+		if (feat_ops[feat].process) {

.. and here again.

Thanks,
Namhyung

Quoted 11 lines
tools/perf/util/header.c +11 -0
+			if (feat_ops[feat].process(&fd, NULL))
+				return -1;
+		}
+		feat_ops[feat].print(&fd, stdout);
+	} else {
+		fprintf(stdout, "# %s info available, use -I to display\n",
+			feat_ops[feat].name);
+	}
+
+	return 0;
+}
#33 lore raw

Re: [PATCH v2 13/13] perf tools: add feature header record to pipe-mode

In reply to David Carrillo-Cisneros

On Tue, May 23, 2017 at 12:48:53AM -0700, David Carrillo-Cisneros wrote:

SNIP

Quoted 20 lines
tools/perf/util/session.c +8 -0
index 3041c6b98191..ffde0dafed6f 100644
@@ -256,6 +256,14 @@ static int process_event_stub(struct perf_tool *tool __maybe_unused,
 	return 0;
 }
 
+static int process_feature_stub(struct perf_tool *tool __maybe_unused,
+				union perf_event *event __maybe_unused,
+				struct perf_session *session __maybe_unused)
+{
+	dump_printf(": unhandled!\n");
+	return 0;
+}
+
 static int process_finished_round_stub(struct perf_tool *tool __maybe_unused,
 				       union perf_event *event __maybe_unused,
 				       struct ordered_events *oe __maybe_unused)

tools/perf/util/session.c void perf_tool__fill_defaults(struct perf_tool *tool) · line 439

Quoted 5 lines
tools/perf/util/session.c +2 -0
@@ -427,6 +435,8 @@ void perf_tool__fill_defaults(struct perf_tool *tool)
 		tool->stat_round = process_stat_round_stub;
 	if (tool->time_conv == NULL)
 		tool->time_conv = process_event_op2_stub;
+	if (tool->feature == NULL)
+		tool->feature = process_feature_stub;

no need to define new one, you can use process_event_op2_stub

jirka

#34 lore raw

Re: [PATCH v2 13/13] perf tools: add feature header record to pipe-mode

In reply to David Carrillo-Cisneros

On Tue, May 23, 2017 at 12:48:53AM -0700, David Carrillo-Cisneros wrote:

SNIP

Quoted 8 lines
tools/perf/util/header.c +8 -0
+int perf_event__process_feature(struct perf_tool *tool,
+				union perf_event *event,
+				struct perf_session *session __maybe_unused)
+{
+	struct feat_fd fd = { .fd = 0 };
+	struct feature_event *fe = (struct feature_event *)event;
+	int type = fe->header.type;
+	u64 feat = fe->header_id;

tools/perf/util/header.c int perf_event__synthesize_attr(struct perf_tool *tool, · line 3025

Quoted 10 lines
tools/perf/util/header.c +10 -0
+
+	if (type < 0 || type >= PERF_RECORD_HEADER_MAX) {
+		pr_warning("invalid record type %d\n", type);
+		return 0;
+	}
+	if (feat == HEADER_RESERVED)
+		return -1;
+
+	if (feat > HEADER_LAST_FEATURE)
+		return 0;

I think we should warn in here

tools/perf/util/header.c int perf_event__synthesize_attr(struct perf_tool *tool, · line 3032

Quoted 7 lines
tools/perf/util/header.c +7 -0
+
+	if (!feat_ops[feat].process)
+		return 0;
+
+	/*
+	 * no print routine
+	 */

superfluous comment

tools/perf/util/header.c int perf_event__synthesize_attr(struct perf_tool *tool, · line 3041

Quoted 9 lines
tools/perf/util/header.c +9 -0
+	if (!feat_ops[feat].print)
+		return 0;
+
+	fd.buf  = (void *)fe->data;
+	fd.size = event->header.size - sizeof(event->header);
+	fd.ph = &session->header;
+
+	if (!tool->show_feat_hdr)
+		return 0;

some of the features could provide data for processing, should we call process unconditionaly and check this just before calling print?

thanks,
jirka

#35 lore raw

Re: [PATCH v2 13/13] perf tools: add feature header record to pipe-mode

In reply to David Carrillo-Cisneros

On Tue, May 23, 2017 at 12:48:53AM -0700, David Carrillo-Cisneros wrote:

SNIP

tools/perf/util/event.h struct time_conv_event { · line 496

Quoted 11 lines

};

tools/perf/util/event.h +5 -0
@@ -488,6 +489,12 @@ struct time_conv_event {
 	u64 time_zero;
 };
 
+struct feature_event {
+	struct perf_event_header header;
+	u64 header_id;
+	char data[]; /* size bytes of raw data specific to the feature */
+};

please align the members, like it's in structs around

struct feature_event {
	struct perf_event_header	header;
	u64				header_id;
	char				data[];
}

thanks,
jirka

#36 lore raw

Re: [PATCH v2 13/13] perf tools: add feature header record to pipe-mode

In reply to David Carrillo-Cisneros

On Tue, May 23, 2017 at 12:48:53AM -0700, David Carrillo-Cisneros wrote:

SNIP

Quoted 29 lines
tools/perf/util/header.c +29 -0
+int perf_event__synthesize_features(struct perf_tool *tool,
+				    struct perf_session *session,
+				    struct perf_evlist *evlist,
+				    perf_event__handler_t process)
+{
+	struct perf_header *header = &session->header;
+	struct feat_fd fdd;
+	struct feature_event *fe;
+	size_t sz, sz_hdr;
+	int feat, ret;
+
+	sz_hdr = sizeof(fe->header);
+	sz = sizeof(union perf_event);
+	/* get a nice alignment */
+	sz = PERF_ALIGN(sz, getpagesize());
+
+	memset(&fdd, 0, sizeof(fdd));
+
+	fdd.buf = malloc(sz);
+	if (!fdd.buf)
+		return -ENOMEM;
+
+	fdd.size = sz - sz_hdr;
+
+	for_each_set_bit(feat, header->adds_features, HEADER_FEAT_BITS) {
+		if (!feat_ops[feat].has_record) {
+			pr_debug("No record header feature for header :%d\n", feat);
+			continue;
+		}

tools/perf/util/header.c int perf_event__synthesize_attr(struct perf_tool *tool, · line 2994

Quoted 10 lines
tools/perf/util/header.c +10 -0
+
+		fdd.offset = sizeof(*fe);
+
+		ret = feat_ops[feat].write(&fdd, evlist);
+		if (ret || fdd.offset <= (ssize_t)sizeof(*fe)) {
+			pr_debug("Error writing feature\n");
+			continue;
+		}
+
+		/* fdd.buf may have changed due to realloc in do_write() */

right, so how's ensured the data never cross the maximum event size (0xffff) ?

I think do_write should have some check on that

jirka

#37 lore raw

Re: [PATCH v2 13/13] perf tools: add feature header record to pipe-mode

Quoted 46 lines · Jiri Olsa

On Tue, May 23, 2017 at 12:48:53AM -0700, David Carrillo-Cisneros wrote:

SNIP

Quoted 29 lines
tools/perf/util/header.c +29 -0
+int perf_event__synthesize_features(struct perf_tool *tool,
+                                 struct perf_session *session,
+                                 struct perf_evlist *evlist,
+                                 perf_event__handler_t process)
+{
+     struct perf_header *header = &session->header;
+     struct feat_fd fdd;
+     struct feature_event *fe;
+     size_t sz, sz_hdr;
+     int feat, ret;
+
+     sz_hdr = sizeof(fe->header);
+     sz = sizeof(union perf_event);
+     /* get a nice alignment */
+     sz = PERF_ALIGN(sz, getpagesize());
+
+     memset(&fdd, 0, sizeof(fdd));
+
+     fdd.buf = malloc(sz);
+     if (!fdd.buf)
+             return -ENOMEM;
+
+     fdd.size = sz - sz_hdr;
+
+     for_each_set_bit(feat, header->adds_features, HEADER_FEAT_BITS) {
+             if (!feat_ops[feat].has_record) {
+                     pr_debug("No record header feature for header :%d\n", feat);
+                     continue;
+             }

tools/perf/util/header.c int perf_event__synthesize_attr(struct perf_tool *tool, · line 2994

Quoted 10 lines
tools/perf/util/header.c +10 -0
+
+             fdd.offset = sizeof(*fe);
+
+             ret = feat_ops[feat].write(&fdd, evlist);
+             if (ret || fdd.offset <= (ssize_t)sizeof(*fe)) {
+                     pr_debug("Error writing feature\n");
+                     continue;
+             }
+
+             /* fdd.buf may have changed due to realloc in do_write() */

right, so how's ensured the data never cross the maximum event size (0xffff) ?

Quoted 1 line

I think do_write should have some check on that

do_write reallocates ff->buff when it's not large enough.

Quoted 2 lines

jirka

#38 lore raw

Re: [PATCH v2 13/13] perf tools: add feature header record to pipe-mode

Quoted 47 lines · David Carrillo-Cisneros
Quoted 46 lines · Jiri Olsa

On Tue, May 23, 2017 at 12:48:53AM -0700, David Carrillo-Cisneros wrote:

SNIP

Quoted 29 lines
tools/perf/util/header.c +29 -0
+int perf_event__synthesize_features(struct perf_tool *tool,
+                                 struct perf_session *session,
+                                 struct perf_evlist *evlist,
+                                 perf_event__handler_t process)
+{
+     struct perf_header *header = &session->header;
+     struct feat_fd fdd;
+     struct feature_event *fe;
+     size_t sz, sz_hdr;
+     int feat, ret;
+
+     sz_hdr = sizeof(fe->header);
+     sz = sizeof(union perf_event);
+     /* get a nice alignment */
+     sz = PERF_ALIGN(sz, getpagesize());
+
+     memset(&fdd, 0, sizeof(fdd));
+
+     fdd.buf = malloc(sz);
+     if (!fdd.buf)
+             return -ENOMEM;
+
+     fdd.size = sz - sz_hdr;
+
+     for_each_set_bit(feat, header->adds_features, HEADER_FEAT_BITS) {
+             if (!feat_ops[feat].has_record) {
+                     pr_debug("No record header feature for header :%d\n", feat);
+                     continue;
+             }

tools/perf/util/header.c int perf_event__synthesize_attr(struct perf_tool *tool, · line 2994

Quoted 10 lines
tools/perf/util/header.c +10 -0
+
+             fdd.offset = sizeof(*fe);
+
+             ret = feat_ops[feat].write(&fdd, evlist);
+             if (ret || fdd.offset <= (ssize_t)sizeof(*fe)) {
+                     pr_debug("Error writing feature\n");
+                     continue;
+             }
+
+             /* fdd.buf may have changed due to realloc in do_write() */

right, so how's ensured the data never cross the maximum event size (0xffff) ?

Quoted 3 lines
Quoted 1 line

I think do_write should have some check on that

do_write reallocates ff->buff when it's not large enough.

and what if it's bigger than 0xffff?

jirka

#39 lore raw

Re: [PATCH v2 13/13] perf tools: add feature header record to pipe-mode

Quoted 48 lines · Jiri Olsa
Quoted 47 lines · David Carrillo-Cisneros
Quoted 46 lines · Jiri Olsa

On Tue, May 23, 2017 at 12:48:53AM -0700, David Carrillo-Cisneros wrote:

SNIP

Quoted 29 lines
tools/perf/util/header.c +29 -0
+int perf_event__synthesize_features(struct perf_tool *tool,
+                                 struct perf_session *session,
+                                 struct perf_evlist *evlist,
+                                 perf_event__handler_t process)
+{
+     struct perf_header *header = &session->header;
+     struct feat_fd fdd;
+     struct feature_event *fe;
+     size_t sz, sz_hdr;
+     int feat, ret;
+
+     sz_hdr = sizeof(fe->header);
+     sz = sizeof(union perf_event);
+     /* get a nice alignment */
+     sz = PERF_ALIGN(sz, getpagesize());
+
+     memset(&fdd, 0, sizeof(fdd));
+
+     fdd.buf = malloc(sz);
+     if (!fdd.buf)
+             return -ENOMEM;
+
+     fdd.size = sz - sz_hdr;
+
+     for_each_set_bit(feat, header->adds_features, HEADER_FEAT_BITS) {
+             if (!feat_ops[feat].has_record) {
+                     pr_debug("No record header feature for header :%d\n", feat);
+                     continue;
+             }

tools/perf/util/header.c int perf_event__synthesize_attr(struct perf_tool *tool, · line 2994

Quoted 10 lines
tools/perf/util/header.c +10 -0
+
+             fdd.offset = sizeof(*fe);
+
+             ret = feat_ops[feat].write(&fdd, evlist);
+             if (ret || fdd.offset <= (ssize_t)sizeof(*fe)) {
+                     pr_debug("Error writing feature\n");
+                     continue;
+             }
+
+             /* fdd.buf may have changed due to realloc in do_write() */

right, so how's ensured the data never cross the maximum event size (0xffff) ?

Quoted 6 lines
Quoted 3 lines
Quoted 1 line

I think do_write should have some check on that

do_write reallocates ff->buff when it's not large enough.

and what if it's bigger than 0xffff?

Oh yeah, I'll add that check.

#40 lore raw

Re: [PATCH v2 13/13] perf tools: add feature header record to pipe-mode

In reply to David Carrillo-Cisneros

On Tue, May 23, 2017 at 12:48:53AM -0700, David Carrillo-Cisneros wrote:

SNIP

Quoted 7 lines
tools/perf/util/header.c +7 -0
+	for_each_set_bit(feat, header->adds_features, HEADER_FEAT_BITS) {
+		if (!feat_ops[feat].has_record) {
+			pr_debug("No record header feature for header :%d\n", feat);
+			continue;
+		}
+
+		fdd.offset = sizeof(*fe);

tools/perf/util/header.c int perf_event__synthesize_attr(struct perf_tool *tool, · line 2996

Quoted 10 lines
tools/perf/util/header.c +10 -0
+
+		ret = feat_ops[feat].write(&fdd, evlist);
+		if (ret || fdd.offset <= (ssize_t)sizeof(*fe)) {
+			pr_debug("Error writing feature\n");
+			continue;
+		}
+
+		/* fdd.buf may have changed due to realloc in do_write() */
+		fe = fdd.buf;
+		memset(fe, 0, sizeof(*fe));

no need to wipe it out if you set all the members just below (apart from header.misc, which is not used)

jirka

A message that is not in the archive (sent off-list or not yet received): CALcN6mhWWDbnGkDP5unmbB3GPi8+LRoKW8DFAse-KMUy85Fpew@mail.gmail.com

#42 lore raw

Re: [PATCH v2 13/13] perf tools: add feature header record to pipe-mode

Quoted 29 lines · David Carrillo-Cisneros
Quoted 17 lines · Jiri Olsa

On Tue, May 23, 2017 at 12:48:53AM -0700, David Carrillo-Cisneros wrote:

SNIP

tools/perf/util/header.c int perf_event__synthesize_attr(struct perf_tool *tool,

Quoted 3 lines
tools/perf/util/header.c +3 -0
+     for_each_set_bit(feat, header->adds_features, HEADER_FEAT_BITS) {
+             if (!feat_ops[feat].has_record) {
+                     pr_debug("No record header feature for header

:%d\n", feat);

Quoted 9 lines
tools/perf/util/header.c +9 -0
+                     continue;
+             }
+
+             fdd.offset = sizeof(*fe);
+
+             ret = feat_ops[feat].write(&fdd, evlist);
+             if (ret || fdd.offset <= (ssize_t)sizeof(*fe)) {
+                     pr_debug("Error writing feature\n");
+                     continue;
Quoted 9 lines

tools/perf/util/header.c int perf_event__synthesize_attr(struct perf_tool *tool, · line 2996

Quoted 5 lines
tools/perf/util/header.c +5 -0
+             }
+
+             /* fdd.buf may have changed due to realloc in do_write() */
+             fe = fdd.buf;
+             memset(fe, 0, sizeof(*fe));

no need to wipe it out if you set all the members just below (apart from header.misc, which is not used)

Quoted 2 lines

what if that changes one day? isn't it better to zero to avoid hard to find bugs?

ok

jirka

#43 lore raw

Re: [PATCH v2 13/13] perf tools: add feature header record to pipe-mode

In reply to David Carrillo-Cisneros

On Tue, May 23, 2017 at 12:48:53AM -0700, David Carrillo-Cisneros wrote:

SNIP

Quoted 5 lines
tools/perf/util/header.c +5 -0
+int perf_event__synthesize_features(struct perf_tool *tool,
+				    struct perf_session *session,
+				    struct perf_evlist *evlist,
+				    perf_event__handler_t process)
+{

tools/perf/util/header.c int perf_event__synthesize_attr(struct perf_tool *tool, · line 2970

Quoted 10 lines
tools/perf/util/header.c +10 -0
+	struct perf_header *header = &session->header;
+	struct feat_fd fdd;
+	struct feature_event *fe;
+	size_t sz, sz_hdr;
+	int feat, ret;
+
+	sz_hdr = sizeof(fe->header);
+	sz = sizeof(union perf_event);
+	/* get a nice alignment */
+	sz = PERF_ALIGN(sz, getpagesize());

we have page_size global variable for this

jirka