Skip to contentSkip to conversation

perf: Support for SDT markers

RFC PATCH In review Hemant Kumar 3 participants 13 messages Feb 24, 2014 – Feb 26, 2014 perf events

lore
Share & tools
lore
#0 lore raw

cover perf: Support for SDT markers

This patchset helps in listing dtrace style markers(SDT) present in user space applications through perf.
Notes/markers are placed at important places by the
developers. They have a negligible overhead when not enabled.
We can enable them and probe at these places and find some important information like the arguments' values, etc.

We have lots of applications which use SDT markers today, like:
Postgresql, MySql, Mozilla, Perl, Python, Java, Ruby, libvirt, QEMU, glib

To add SDT markers into user applications:
We need to have this header sys/sdt.h present.
sys/sdt.h used is version 3.
If not present, install systemtap-sdt-devel package (for fedora-18).

Please refer to the Documentation patch to see how the SDT markers are added into a program.

With this patchset,
- Use perf to list the markers in the app:

# perf list sdt ./user_app

./user_app :
%user_app:foo_start
%user_app:fun_start

- Also, we can see the SDT markers present in our system in the usual binaries. These usual binaries are libraries (dsos) listed by ldconfig --print-cache and some binaries present in PATH environment variable.

First, scan the binaries using :

# perf list sdt --scan

Creating a cache of SDT markers...
perf sdt cache created!
Use : "perf list sdt"
to see the SDT markers

After the sdt cache file is created, use perf list to view the markers :

# perf list sdt

%rtld : init_start
%rtld : init_complete
%rtld : map_failed
%rtld : map_start
%rtld : lll_futex_wake
...
...
%libgcc : unwind
%libvirt : rpc_server_client_auth_allow
%libvirt : rpc_server_client_auth_fail
%libvirt : rpc_server_client_auth_deny

Alternatively, one can view the /var/cache/perf/perf-sdt.cache directly.

This link shows an example of marker probing with Systemtap:
https://sourceware.org/systemtap/wiki/AddingUserSpaceProbingToApps

Also, this link provides important info regarding SDT notes:
http://sourceware.org/systemtap/wiki/UserSpaceProbeImplementation

- Markers in binaries :
These SDT markers are present in the ELF in the section named ".note.stapsdt".
Here, the name of the marker, its provider, type, location, base address, semaphore address.
We can retrieve these values using the members name_off and desc_off in Nhdr structure. If these are not enabled, they are present in the ELF as nop.

Changes since last series :
- Made the SDT markers present in a system more visible through perf list as suggested by Pekka Enberg.
- Less complex command line interface to access the SDT markers.
- Added a cache file to store most of the SDT markers of a system.

TODO:
- Add support to probe these SDT markers and integrate with a previous patch (support to perf to probe SDT markers) posted in lkml.
https://lkml.org/lkml/2013/10/23/10

- Recognizing arguments and support to probe on them.

---

Hemant Kumar (2):
perf/sdt : Listing of SDT markers by perf
perf/sdt : Documentation

 tools/perf/Documentation/SDT-markers.txt |  122 +++++++
 tools/perf/Documentation/perf-list.txt   |    8 
 tools/perf/Makefile.perf                 |    1 
 tools/perf/builtin-list.c                |    4 
 tools/perf/util/parse-events.h           |    2 
 tools/perf/util/sdt.c                    |  489 ++++++++++++++++++++++++++++++
 tools/perf/util/symbol-elf.c             |  227 ++++++++++++++
 tools/perf/util/symbol.h                 |   19 +
 8 files changed, 869 insertions(+), 3 deletions(-)
 create mode 100644 tools/perf/Documentation/SDT-markers.txt
 create mode 100644 tools/perf/util/sdt.c
#1 lore raw

RFC PATCH 1/2 perf/sdt : Listing of SDT markers by perf

This patch enables perf to list the SDT markers present in a system. It looks in dsos given by ldconfig --print-cache and for other binaries, it looks into the PATH environment variable. After preparing a list of the binaries, then it starts searching for SDT markers in them.
To find the SDT markers, first an elf section named .note.stapsdt is searched for. And then the SDT notes are retreived one by one from that section. To counter the effect of prelinking, the section ".stapsdt.base" is searched. If its found, then the location of the SDT marker is adjusted.

All these markers' info is written into a cache file "/var/cache/perf/perf-sdt.cache".
Since, the presence of SDT markers is quite common these days, hence, its better to make them visible to a user easily. Also, creating a cache file will help a user to probe (to be implemented) these markers without much hussle. This cache file will hold most of the SDT markers.
The format of each entry is -

%provider : marker : file_path : build_id : location : semaphore_loc

% - marks the beginning of each entry.
provider - The provider name of the SDT marker.
marker - The marker name of the SDT marker.
file_path - Full/absolute path of the file in which this marker is present. location : Adjusted location of the SDT marker inside the program. semaphore_loc : The semaphore address if present otherwise 0x0.

This format should help when probing will be implemented. The adjusted address from the required entry can be directly used in probing if the build_id matches.

To use this feature, invoke :

# perf list sdt --scan

"--scan" should be used for the first time and whenever there is any change in the files containing the SDT markers.

And then use :

# perf list sdt

This displays a list of SDT markers (read from the cache file) present in most of the dsos and other binaries.

However, not all the SDT markers are present in the cache file. Only the ELFs present in the directories in PATH variable are looked into.
An individual file argument can be given to "perf list" to find out the SDT markers present in that file. Usage is as below :

# perf list sdt /home/hemant/tmp

/home/hemant/tmp:
%user : foo
%user : bar

Signed-off-by [email protected]
---
 tools/perf/Makefile.perf       |    1 
 tools/perf/builtin-list.c      |    4 
 tools/perf/util/parse-events.h |    2 
 tools/perf/util/sdt.c          |  489 ++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/symbol-elf.c   |  227 +++++++++++++++++++
 tools/perf/util/symbol.h       |   19 ++
 6 files changed, 741 insertions(+), 1 deletion(-)
 create mode 100644 tools/perf/util/sdt.c
tools/perf/Makefile.perf +1 -0
index 77b153f..25a6a21 100644
@@ -369,6 +369,7 @@ LIB_OBJS += $(OUTPUT)util/stat.o
 LIB_OBJS += $(OUTPUT)util/record.o
 LIB_OBJS += $(OUTPUT)util/srcline.o
 LIB_OBJS += $(OUTPUT)util/data.o
+LIB_OBJS += $(OUTPUT)util/sdt.o
 
 LIB_OBJS += $(OUTPUT)ui/setup.o
 LIB_OBJS += $(OUTPUT)ui/helpline.o
tools/perf/builtin-list.c +3 -1
index 011195e..da6e68c 100644
@@ -23,7 +23,7 @@ int cmd_list(int argc, const char **argv, const char *prefix __maybe_unused)
 		OPT_END()
 	};
 	const char * const list_usage[] = {
-		"perf list [hw|sw|cache|tracepoint|pmu|event_glob]",
+		"perf list [hw|sw|cache|tracepoint|pmu|event_glob|sdt]",
 		NULL
 	};
 
@@ -55,6 +55,8 @@ int cmd_list(int argc, const char **argv, const char *prefix __maybe_unused)
 			print_pmu_events(NULL, false);
 		else if (strcmp(argv[i], "--raw-dump") == 0)
 			print_events(NULL, true);
+		else if (strcmp(argv[i], "sdt") == 0)
+			print_sdt_events(argv[++i]);
 		else {
 			char *sep = strchr(argv[i], ':'), *s;
 			int sep_idx;
tools/perf/util/parse-events.h +2 -0
index f1cb4c4..33abbd0 100644
@@ -110,4 +110,6 @@ extern int is_valid_tracepoint(const char *event_string);
 
 extern int valid_debugfs_mount(const char *debugfs);
 
+void print_sdt_events(const char *arg);
+
 #endif /* __PERF_PARSE_EVENTS_H */
tools/perf/util/sdt.c new file +489 -0

489 lines not on this page · raw message

tools/perf/util/symbol-elf.c +227 -0

233 lines not on this page · raw message

tools/perf/util/symbol.h +19 -0
index 2553ae0..a919776 100644
@@ -291,4 +291,23 @@ int compare_proc_modules(const char *from, const char *to);
 int setup_list(struct strlist **list, const char *list_str,
 	       const char *list_name);
 
+struct sdt_note {
+	char *name;
+	char *provider;
+	bool bit32;
+	union {
+		Elf64_Addr a64[3];
+		Elf32_Addr a32[3];
+	} addr;
+	struct list_head note_list;
+};
+
+int get_sdt_note_list(struct list_head *head, const char *target);
+bool is_an_elf(char *file);
+
+#define SDT_BASE_SCN ".stapsdt.base"
+#define SDT_NOTE_SCN  ".note.stapsdt"
+#define SDT_NOTE_TYPE 3
+#define SDT_NOTE_NAME "stapsdt"
+
 #endif /* __PERF_SYMBOL */
maintainer
#2 lore raw

Re: [RFC PATCH v1 1/2] perf/sdt : Listing of SDT markers by perf

Hi Hemant,

Quoted 32 lines · Hemant Kumar

This patch enables perf to list the SDT markers present in a system. It looks in dsos given by ldconfig --print-cache and for other binaries, it looks into the PATH environment variable. After preparing a list of the binaries, then it starts searching for SDT markers in them.
To find the SDT markers, first an elf section named .note.stapsdt is searched for. And then the SDT notes are retreived one by one from that section. To counter the effect of prelinking, the section ".stapsdt.base" is searched. If its found, then the location of the SDT marker is adjusted.

All these markers' info is written into a cache file "/var/cache/perf/perf-sdt.cache".
Since, the presence of SDT markers is quite common these days, hence, its better to make them visible to a user easily. Also, creating a cache file will help a user to probe (to be implemented) these markers without much hussle. This cache file will hold most of the SDT markers.
The format of each entry is -

%provider : marker : file_path : build_id : location : semaphore_loc

% - marks the beginning of each entry.
provider - The provider name of the SDT marker.
marker - The marker name of the SDT marker.
file_path - Full/absolute path of the file in which this marker is present. location : Adjusted location of the SDT marker inside the program. semaphore_loc : The semaphore address if present otherwise 0x0.

This format should help when probing will be implemented. The adjusted address from the required entry can be directly used in probing if the build_id matches.

To use this feature, invoke :

# perf list sdt --scan
Quoted 8 lines

"--scan" should be used for the first time and whenever there is any change in the files containing the SDT markers.

And then use :

# perf list sdt

This displays a list of SDT markers (read from the cache file) present in most of the dsos and other binaries.

The default action of perf list (with no argument) prints all available events on the system - so it should also print SDT markers IMHO.

Quoted 14 lines

However, not all the SDT markers are present in the cache file. Only the ELFs present in the directories in PATH variable are looked into.
An individual file argument can be given to "perf list" to find out the SDT markers present in that file. Usage is as below :

# perf list sdt /home/hemant/tmp

/home/hemant/tmp:
%user : foo
%user : bar

Signed-off-by [email protected]

---

[SNIP]

Quoted 4 lines
tools/perf/util/sdt.c new file +4 -0
+/*
+ * Finds out the libraries present in a system as shown by the command
+ * "ldconfig --print-cache". Uses "=>" and '/' to find out the start of a
+ * dso path.

tools/perf/util/sdt.c line 159

Quoted 10 lines
tools/perf/util/sdt.c new file +10 -0
+ */
+static void parse_lib_name(char *str, char *path)
+{
+	char *ptr, *q, *r;
+	char c = '=';
+
+	while (str != NULL) {
+		/* look for "=>" and then '/' */
+		ptr = strchr(str, c);
+		if (ptr && (*(ptr + 1) == '>')) {

Hmm.. why not searching "=>" by strstr() then?

Quoted 16 lines
tools/perf/util/sdt.c new file +16 -0
+			ptr++;
+			q = strchr(ptr, '/');
+			if (!q)
+				return;
+			r = strchr(ptr, '\n');
+			*r = '\0';
+			strcpy(path, q);
+			return;
+		} else if (ptr == NULL) {
+			return;
+		} else {
+			str = ptr + 1;
+			continue;
+		}
+	}
+}

[SNIP]

tools/perf/util/sdt.c line 267

Quoted 8 lines
tools/perf/util/sdt.c new file +8 -0
+	/*
+	 * Take a line from the o/p of ldconfig and
+	 * parse it to find a library path
+	 */
+	while (getline(&line, &len, fp) != -1) {
+		memset(path, '\0', PATH_MAX);
+		parse_lib_name(line, path);
+		if (strcmp(path, ""))

I think it'd better changing the parse_lib_name to return a value instead of checking whether the path is an empty string.

Quoted 12 lines
tools/perf/util/sdt.c new file +12 -0
+			append_path(path, &lib_list.list);
+	}
+	/* line and fp no more required */
+	free(line);
+	pclose(fp);
+
+	/* Find and write the SDT markers in the perf-sdt cache*/
+	flush_sdt_list(&lib_list.list, cache);
+
+	cleanup_path_list(&lib_list.list);
+	return;
+}

[SNIP]

Quoted 35 lines
tools/perf/util/symbol-elf.c +35 -0
+static int populate_sdt_note(Elf **elf, const char *data, size_t len, int type,
+			     struct sdt_note **note)
+{
+	const char *provider, *name;
+	struct sdt_note *tmp = NULL;
+	GElf_Ehdr ehdr;
+	GElf_Addr base_off = 0;
+	GElf_Shdr shdr;
+	int ret = -1;
+	int i;
+
+	union {
+		Elf64_Addr a64[3];
+		Elf32_Addr a32[3];
+	} buf;
+
+	Elf_Data dst = {
+		.d_buf = &buf, .d_type = ELF_T_ADDR, .d_version = EV_CURRENT,
+		.d_size = gelf_fsize((*elf), ELF_T_ADDR, 3, EV_CURRENT),
+		.d_off = 0, .d_align = 0
+	};
+	Elf_Data src = {
+		.d_buf = (void *) data, .d_type = ELF_T_ADDR,
+		.d_version = EV_CURRENT, .d_size = dst.d_size, .d_off = 0,
+		.d_align = 0
+	};
+
+	/* Check the type of each of the notes */
+	if (type != SDT_NOTE_TYPE)
+		goto out_err;
+
+	tmp = (struct sdt_note *)calloc(1, sizeof(struct sdt_note));
+	if (!tmp) {
+		ret = -ENOMEM;
+		goto out_err;

tools/perf/util/symbol-elf.c void kcore_extract__delete(struct kcore_extract *kce) · line 1662

Quoted 10 lines
tools/perf/util/symbol-elf.c +10 -0
+	}
+
+	INIT_LIST_HEAD(&tmp->note_list);
+
+	if (len < dst.d_size + 3)
+		goto out_free_note;
+
+	/* Translation from file representation to memory representation */
+	if (gelf_xlatetom(*elf, &dst, &src,
+			  elf_getident(*elf, NULL)[EI_DATA]) == NULL)

Do we really need this xlate function? It seems elf_getdata() already did necessary conversions so only thing we need to do is checking its class and read out the addresses in a proper length, no?

Quoted 38 lines
tools/perf/util/symbol-elf.c +38 -0
+		printf("gelf_xlatetom : %s\n", elf_errmsg(-1));
+
+	/* Populate the fields of sdt_note */
+	provider = data + dst.d_size;
+
+	name = (const char *)memchr(provider, '\0', data + len - provider);
+	if (name++ == NULL)
+		goto out_free_note;
+
+	tmp->provider = strdup(provider);
+	if (!tmp->provider) {
+		ret = -ENOMEM;
+		goto out_free_note;
+	}
+	tmp->name = strdup(name);
+	if (!tmp->name) {
+		ret = -ENOMEM;
+		goto out_free_prov;
+	}
+
+	/* Obtain the addresses */
+	if (gelf_getclass(*elf) == ELFCLASS32) {
+		for (i = 0; i < 3; i++)
+			tmp->addr.a32[i] = buf.a32[i];
+		tmp->bit32 = true;
+	} else {
+		for (i = 0; i < 3; i++)
+			tmp->addr.a64[i] = buf.a64[i];
+		tmp->bit32 = false;
+	}
+
+	/* Now Adjust the prelink effect */
+	if (!gelf_getehdr(*elf, &ehdr)) {
+		pr_debug("%s : cannot get elf header.\n", __func__);
+		ret = -EBADF;
+		goto out_free_name;
+	}
+

tools/perf/util/symbol-elf.c void kcore_extract__delete(struct kcore_extract *kce) · line 1710

Quoted 10 lines
tools/perf/util/symbol-elf.c +10 -0
+	/*
+	 * Find out the .stapsdt.base section.
+	 * This scn will help us to handle prelinking (if present).
+	 * Compare the retrieved file offset of the base section with the
+	 * base address in the description of the SDT note. If its different,
+	 * then accordingly, adjust the note location.
+	 */
+	if (elf_section_by_name(*elf, &ehdr, &shdr, SDT_BASE_SCN, NULL))
+		base_off = shdr.sh_offset;
+	if (base_off) {

Why not moving this block under the above if block?

Quoted 64 lines
tools/perf/util/symbol-elf.c +64 -0
+		if (tmp->bit32)
+			tmp->addr.a32[0] = tmp->addr.a32[0] + base_off -
+				tmp->addr.a32[1];
+		else
+			tmp->addr.a64[0] = tmp->addr.a64[0] + base_off -
+				tmp->addr.a64[1];
+	}
+
+	*note = tmp;
+	return 0;
+
+out_free_name:
+	free(tmp->name);
+out_free_prov:
+	free(tmp->provider);
+out_free_note:
+	free(tmp);
+out_err:
+	return ret;
+}
+
+static int construct_sdt_notes_list(Elf *elf, struct list_head *sdt_notes)
+{
+	GElf_Ehdr ehdr;
+	Elf_Scn *scn = NULL;
+	Elf_Data *data;
+	GElf_Shdr shdr;
+	size_t shstrndx, next;
+	GElf_Nhdr nhdr;
+	size_t name_off, desc_off, offset;
+	struct sdt_note *tmp = NULL;
+	int ret = 0, val = 0;
+
+	if (gelf_getehdr(elf, &ehdr) == NULL) {
+		ret = -EBADF;
+		goto out_ret;
+	}
+	if (elf_getshdrstrndx(elf, &shstrndx) != 0) {
+		ret = -EBADF;
+		goto out_ret;
+	}
+
+	/* Look for the required section */
+	scn = elf_section_by_name(elf, &ehdr, &shdr, SDT_NOTE_SCN, NULL);
+	if (!scn) {
+		ret = -ENOENT;
+		goto out_ret;
+	}
+
+	if (!(shdr.sh_type == SHT_NOTE) || (shdr.sh_flags & SHF_ALLOC)) {
+		ret = -ENOENT;
+		goto out_ret;
+	}
+
+	data = elf_getdata(scn, NULL);
+
+	/* Get the SDT notes */
+	for (offset = 0; (next = gelf_getnote(data, offset, &nhdr, &name_off,
+					      &desc_off)) > 0; offset = next) {
+		if (nhdr.n_namesz == sizeof(SDT_NOTE_NAME) &&
+		    !memcmp(data->d_buf + name_off, SDT_NOTE_NAME,
+			    sizeof(SDT_NOTE_NAME))) {
+			val = populate_sdt_note(&elf, ((data->d_buf) + desc_off),
+						nhdr.n_descsz, nhdr.n_type,

tools/perf/util/symbol-elf.c void kcore_extract__delete(struct kcore_extract *kce) · line 1784

Quoted 10 lines
tools/perf/util/symbol-elf.c +10 -0
+						&tmp);
+			if (!val)
+				list_add_tail(&tmp->note_list, sdt_notes);
+			if (val == -ENOMEM) {
+				ret = val;
+				goto out_ret;
+			}
+		}
+	}
+	if (!sdt_notes)

Did you mean list_empty(sdt_notes) ? :)

Thanks,
Namhyung

Quoted 5 lines
tools/perf/util/symbol-elf.c +5 -0
+		Ret = -ENOENT;
+
+out_ret:
+	return ret;
+}
#3 lore raw

Re: [RFC PATCH v1 1/2] perf/sdt : Listing of SDT markers by perf

Quoted 38 lines · Namhyung Kim

Hi Hemant,

Quoted 35 lines · Hemant Kumar

This patch enables perf to list the SDT markers present in a system. It looks in dsos given by ldconfig --print-cache and for other binaries, it looks into the PATH environment variable. After preparing a list of the binaries, then it starts searching for SDT markers in them.
To find the SDT markers, first an elf section named .note.stapsdt is searched for. And then the SDT notes are retreived one by one from that section. To counter the effect of prelinking, the section ".stapsdt.base" is searched. If its found, then the location of the SDT marker is adjusted.

All these markers' info is written into a cache file "/var/cache/perf/perf-sdt.cache".
Since, the presence of SDT markers is quite common these days, hence, its better to make them visible to a user easily. Also, creating a cache file will help a user to probe (to be implemented) these markers without much hussle. This cache file will hold most of the SDT markers.
The format of each entry is -

%provider : marker : file_path : build_id : location : semaphore_loc

% - marks the beginning of each entry.
provider - The provider name of the SDT marker.
marker - The marker name of the SDT marker.
file_path - Full/absolute path of the file in which this marker is present. location : Adjusted location of the SDT marker inside the program. semaphore_loc : The semaphore address if present otherwise 0x0.

This format should help when probing will be implemented. The adjusted address from the required entry can be directly used in probing if the build_id matches.

To use this feature, invoke :

# perf list sdt --scan

"--scan" should be used for the first time and whenever there is any change in the files containing the SDT markers.

Quoted 7 lines
Quoted 5 lines

And then use :

# perf list sdt

This displays a list of SDT markers (read from the cache file) present in most of the dsos and other binaries.

The default action of perf list (with no argument) prints all available events on the system - so it should also print SDT markers IMHO.

Right, but won't that list be huge?

Quoted 14 lines
Quoted 13 lines

However, not all the SDT markers are present in the cache file. Only the ELFs present in the directories in PATH variable are looked into.
An individual file argument can be given to "perf list" to find out the SDT markers present in that file. Usage is as below :

# perf list sdt /home/hemant/tmp

/home/hemant/tmp:
%user : foo
%user : bar

Signed-off-by [email protected]

---

[SNIP]

Quoted 15 lines
Quoted 4 lines
tools/perf/util/sdt.c new file +4 -0
+/*
+ * Finds out the libraries present in a system as shown by the command
+ * "ldconfig --print-cache". Uses "=>" and '/' to find out the start of a
+ * dso path.

tools/perf/util/sdt.c line 159

Quoted 10 lines
tools/perf/util/sdt.c new file +10 -0
+ */
+static void parse_lib_name(char *str, char *path)
+{
+	char *ptr, *q, *r;
+	char c = '=';
+
+	while (str != NULL) {
+		/* look for "=>" and then '/' */
+		ptr = strchr(str, c);
+		if (ptr && (*(ptr + 1) == '>')) {

Hmm.. why not searching "=>" by strstr() then?

Right! will change it to strstr.

Quoted 18 lines
Quoted 17 lines
tools/perf/util/sdt.c new file +16 -0
+			ptr++;
+			q = strchr(ptr, '/');
+			if (!q)
+				return;
+			r = strchr(ptr, '\n');
+			*r = '\0';
+			strcpy(path, q);
+			return;
+		} else if (ptr == NULL) {
+			return;
+		} else {
+			str = ptr + 1;
+			continue;
+		}
+	}
+}

[SNIP]

Quoted 10 lines

tools/perf/util/sdt.c line 267

Quoted 8 lines
tools/perf/util/sdt.c new file +8 -0
+	/*
+	 * Take a line from the o/p of ldconfig and
+	 * parse it to find a library path
+	 */
+	while (getline(&line, &len, fp) != -1) {
+		memset(path, '\0', PATH_MAX);
+		parse_lib_name(line, path);
+		if (strcmp(path, ""))

I think it'd better changing the parse_lib_name to return a value instead of checking whether the path is an empty string.

Ok. Seems better!

Quoted 55 lines
Quoted 14 lines
tools/perf/util/sdt.c new file +12 -0
+			append_path(path, &lib_list.list);
+	}
+	/* line and fp no more required */
+	free(line);
+	pclose(fp);
+
+	/* Find and write the SDT markers in the perf-sdt cache*/
+	flush_sdt_list(&lib_list.list, cache);
+
+	cleanup_path_list(&lib_list.list);
+	return;
+}

[SNIP]

Quoted 40 lines
tools/perf/util/symbol-elf.c +40 -0
+static int populate_sdt_note(Elf **elf, const char *data, size_t len, int type,
+			     struct sdt_note **note)
+{
+	const char *provider, *name;
+	struct sdt_note *tmp = NULL;
+	GElf_Ehdr ehdr;
+	GElf_Addr base_off = 0;
+	GElf_Shdr shdr;
+	int ret = -1;
+	int i;
+
+	union {
+		Elf64_Addr a64[3];
+		Elf32_Addr a32[3];
+	} buf;
+
+	Elf_Data dst = {
+		.d_buf = &buf, .d_type = ELF_T_ADDR, .d_version = EV_CURRENT,
+		.d_size = gelf_fsize((*elf), ELF_T_ADDR, 3, EV_CURRENT),
+		.d_off = 0, .d_align = 0
+	};
+	Elf_Data src = {
+		.d_buf = (void *) data, .d_type = ELF_T_ADDR,
+		.d_version = EV_CURRENT, .d_size = dst.d_size, .d_off = 0,
+		.d_align = 0
+	};
+
+	/* Check the type of each of the notes */
+	if (type != SDT_NOTE_TYPE)
+		goto out_err;
+
+	tmp = (struct sdt_note *)calloc(1, sizeof(struct sdt_note));
+	if (!tmp) {
+		ret = -ENOMEM;
+		goto out_err;
+	}
+
+	INIT_LIST_HEAD(&tmp->note_list);
+
+	if (len < dst.d_size + 3)
Quoted 9 lines

tools/perf/util/symbol-elf.c void kcore_extract__delete(struct kcore_extract *kce) · line 1662

Quoted 5 lines
tools/perf/util/symbol-elf.c +5 -0
+		goto out_free_note;
+
+	/* Translation from file representation to memory representation */
+	if (gelf_xlatetom(*elf, &dst, &src,
+			  elf_getident(*elf, NULL)[EI_DATA]) == NULL)

Do we really need this xlate function? It seems elf_getdata() already did necessary conversions so only thing we need to do is checking its class and read out the addresses in a proper length, no?

Hmm, alright. I thought the conversion was necessary for cross developed binaries. But I guess elf_getdata() should do all these conversions. Will remove that.

Quoted 41 lines
Quoted 41 lines
tools/perf/util/symbol-elf.c +41 -0
+		printf("gelf_xlatetom : %s\n", elf_errmsg(-1));
+
+	/* Populate the fields of sdt_note */
+	provider = data + dst.d_size;
+
+	name = (const char *)memchr(provider, '\0', data + len - provider);
+	if (name++ == NULL)
+		goto out_free_note;
+
+	tmp->provider = strdup(provider);
+	if (!tmp->provider) {
+		ret = -ENOMEM;
+		goto out_free_note;
+	}
+	tmp->name = strdup(name);
+	if (!tmp->name) {
+		ret = -ENOMEM;
+		goto out_free_prov;
+	}
+
+	/* Obtain the addresses */
+	if (gelf_getclass(*elf) == ELFCLASS32) {
+		for (i = 0; i < 3; i++)
+			tmp->addr.a32[i] = buf.a32[i];
+		tmp->bit32 = true;
+	} else {
+		for (i = 0; i < 3; i++)
+			tmp->addr.a64[i] = buf.a64[i];
+		tmp->bit32 = false;
+	}
+
+	/* Now Adjust the prelink effect */
+	if (!gelf_getehdr(*elf, &ehdr)) {
+		pr_debug("%s : cannot get elf header.\n", __func__);
+		ret = -EBADF;
+		goto out_free_name;
+	}
+
+	/*
+	 * Find out the .stapsdt.base section.
+	 * This scn will help us to handle prelinking (if present).
Quoted 9 lines

tools/perf/util/symbol-elf.c void kcore_extract__delete(struct kcore_extract *kce) · line 1710

Quoted 7 lines
tools/perf/util/symbol-elf.c +7 -0
+	 * Compare the retrieved file offset of the base section with the
+	 * base address in the description of the SDT note. If its different,
+	 * then accordingly, adjust the note location.
+	 */
+	if (elf_section_by_name(*elf, &ehdr, &shdr, SDT_BASE_SCN, NULL))
+		base_off = shdr.sh_offset;
+	if (base_off) {

Why not moving this block under the above if block?

Yeah, better! Will do it.

Quoted 67 lines
Quoted 67 lines
tools/perf/util/symbol-elf.c +67 -0
+		if (tmp->bit32)
+			tmp->addr.a32[0] = tmp->addr.a32[0] + base_off -
+				tmp->addr.a32[1];
+		else
+			tmp->addr.a64[0] = tmp->addr.a64[0] + base_off -
+				tmp->addr.a64[1];
+	}
+
+	*note = tmp;
+	return 0;
+
+out_free_name:
+	free(tmp->name);
+out_free_prov:
+	free(tmp->provider);
+out_free_note:
+	free(tmp);
+out_err:
+	return ret;
+}
+
+static int construct_sdt_notes_list(Elf *elf, struct list_head *sdt_notes)
+{
+	GElf_Ehdr ehdr;
+	Elf_Scn *scn = NULL;
+	Elf_Data *data;
+	GElf_Shdr shdr;
+	size_t shstrndx, next;
+	GElf_Nhdr nhdr;
+	size_t name_off, desc_off, offset;
+	struct sdt_note *tmp = NULL;
+	int ret = 0, val = 0;
+
+	if (gelf_getehdr(elf, &ehdr) == NULL) {
+		ret = -EBADF;
+		goto out_ret;
+	}
+	if (elf_getshdrstrndx(elf, &shstrndx) != 0) {
+		ret = -EBADF;
+		goto out_ret;
+	}
+
+	/* Look for the required section */
+	scn = elf_section_by_name(elf, &ehdr, &shdr, SDT_NOTE_SCN, NULL);
+	if (!scn) {
+		ret = -ENOENT;
+		goto out_ret;
+	}
+
+	if (!(shdr.sh_type == SHT_NOTE) || (shdr.sh_flags & SHF_ALLOC)) {
+		ret = -ENOENT;
+		goto out_ret;
+	}
+
+	data = elf_getdata(scn, NULL);
+
+	/* Get the SDT notes */
+	for (offset = 0; (next = gelf_getnote(data, offset, &nhdr, &name_off,
+					      &desc_off)) > 0; offset = next) {
+		if (nhdr.n_namesz == sizeof(SDT_NOTE_NAME) &&
+		    !memcmp(data->d_buf + name_off, SDT_NOTE_NAME,
+			    sizeof(SDT_NOTE_NAME))) {
+			val = populate_sdt_note(&elf, ((data->d_buf) + desc_off),
+						nhdr.n_descsz, nhdr.n_type,
+						&tmp);
+			if (!val)
+				list_add_tail(&tmp->note_list, sdt_notes);
Quoted 8 lines

tools/perf/util/symbol-elf.c void kcore_extract__delete(struct kcore_extract *kce) · line 1784

Quoted 7 lines
tools/perf/util/symbol-elf.c +7 -0
+			if (val == -ENOMEM) {
+				ret = val;
+				goto out_ret;
+			}
+		}
+	}
+	if (!sdt_notes)

Did you mean list_empty(sdt_notes) ? :)

Oops! Yes, will fix it.

[SNIP]

Thanks a lot for reviewing the patch.

Signature · 2 lines
Thanks
Hemant Kumar
maintainer
#4 lore raw

Re: [RFC PATCH v1 1/2] perf/sdt : Listing of SDT markers by perf

Hi Hemant,

Quoted 9 lines · Hemant Kumar
Quoted 7 lines · Namhyung Kim

tools/perf/util/symbol-elf.c void kcore_extract__delete(struct kcore_extract *kce) · line 1662

Quoted 3 lines
tools/perf/util/symbol-elf.c +3 -0
+	/* Translation from file representation to memory representation */
+	if (gelf_xlatetom(*elf, &dst, &src,
+			  elf_getident(*elf, NULL)[EI_DATA]) == NULL)

Do we really need this xlate function? It seems elf_getdata() already did necessary conversions so only thing we need to do is checking its class and read out the addresses in a proper length, no?

Quoted 3 lines

Hmm, alright. I thought the conversion was necessary for cross developed binaries.
But I guess elf_getdata() should do all these conversions. Will remove that.

Looking at the source, it seems we still need to xlate() anyway. The conversion function (elf_cvt_note) only handles the header part since it cannot know what the content is.

But obviously we cannot enable/disable the marker as we cannot run the cross-built binary - it only can be used to show the list of SDT markers in the binary. Or else, we can simply deny to do it..

Thanks,
Namhyung

#5 lore raw

RFC PATCH 2/2 perf/sdt : Documentation

In reply to Hemant Kumar

Documentation for perf listing of SDT markers.

---
 tools/perf/Documentation/SDT-markers.txt |  122 ++++++++++++++++++++++++++++++
 tools/perf/Documentation/perf-list.txt   |    8 +-
 2 files changed, 128 insertions(+), 2 deletions(-)
 create mode 100644 tools/perf/Documentation/SDT-markers.txt
tools/perf/Documentation/SDT-markers.txt new file +122 -0
new file mode 100644
index 0000000..9d12c224
@@ -0,0 +1,122 @@
+Support to perf for listing the SDT markers :
+
+This helps in listing dtrace style markers(SDT) present in user space
+applications through perf. Notes/markers are placed at important places by the
+developers. They have a negligible overhead when not enabled.
+We can enable them and probe at these places and find some important information
+like the arguments' values, etc.
+
+How to add SDT markers into user applications:
+We need to have this header sys/sdt.h present.
+sys/sdt.h used is version 3.
+If not present, install systemtap-sdt-devel package (for fedora-18).
+
+A very simple example:
+
+$ cat user_app.c
+
+#include <sys/sdt.h>
+
+void main () {
+       /* ... */
+       /* 
+        * user_app is the provider name
+	* test_probe is the marker name
+	*/
+       STAP_PROBE(user_app, test_mark);
+       /* ... */
+}
+
+$ gcc user_app.c
+$ perf list sdt ./a.out
+./a.out:
+%user_app:test_mark
+
+A different example to show the same:
+- Create a file with .d extension and mention the probe names in it with
+provider name and marker name.
+
+$ cat probes.d
+provider user_app {
+         probe foo_start();
+         probe fun_start();
+};
+
+- Now create the probes.h and probes.o file :
+$ dtrace -C -h -s probes.d -o probes.h
+$ dtrace -C -G -s probes.d -o probes.o
+
+- A program using the markers:
+
+$ cat user_app.c
+
+#include <stdio.h>
+#include "probes.h"
+
+void foo(void)
+{
+        USER_APP_FOO_START();
+        printf("This is foo\n");
+}
+
+void fun(void)
+{
+        USER_APP_FUN_START();
+        printf("Inside fun\n");
+}
+int main(void)
+{
+        printf("In main\n");
+        foo();
+        fun();
+        return 0;
+}
+- Compile it and also provide probes.o file to linker:
+$ gcc user_app.c probes.o -o user_app
+
+- Now use perf to list the markers in the app:
+# perf list sdt ./user_app
+
+./user_app :
+%user_app:foo_start
+%user_app:fun_start
+
+Also, we can see the SDT markers present in our system in the usual binaries.
+First, scan the binaries using :
+# perf list sdt --scan
+
+Creating a cache of SDT markers...
+perf sdt cache created!
+ Use : "perf list sdt"
+ to see the SDT markers
+
+After the sdt cache file is created, use perf list to view the markers :
+# perf list sdt
+
+%rtld : init_start
+%rtld : init_complete
+%rtld : map_failed
+%rtld : map_start
+%rtld : lll_futex_wake
+...
+...
+%libgcc : unwind
+%libvirt : rpc_server_client_auth_allow
+%libvirt : rpc_server_client_auth_fail
+%libvirt : rpc_server_client_auth_deny
+
+Alternatively, one can view the /var/cache/perf-sdt.cache directly.
+
+Also, this link provides important info regarding SDT notes:
+http://sourceware.org/systemtap/wiki/UserSpaceProbeImplementation
+
+This link shows an example of marker probing with Systemtap:
+https://sourceware.org/systemtap/wiki/AddingUserSpaceProbingToApps
+
+- Markers in binaries :
+These SDT markers are present in the ELF in the section named
+".note.stapsdt".
+Here, the name of the marker, its provider, type, location, base
+address, semaphore address.
+We can retrieve these values using the members name_off and desc_off in
+Nhdr structure. If these are not enabled, they are present in the ELF as nop.
tools/perf/Documentation/perf-list.txt +6 -2
index 6fce6a6..7a5552c 100644
@@ -8,7 +8,7 @@ perf-list - List all symbolic event types
 SYNOPSIS
 --------
 [verse]
-'perf list' [hw|sw|cache|tracepoint|pmu|event_glob]
+'perf list' [hw|sw|cache|tracepoint|pmu|event_glob|sdt]
 
 DESCRIPTION
 -----------
@@ -92,7 +92,7 @@ details. Some of them are referenced in the SEE ALSO section below.
 OPTIONS
 -------
 
-Without options all known events will be listed.
+Without options all known events will be listed except the SDT markers.
 
 To limit the list use:
 
@@ -108,6 +108,10 @@ To limit the list use:
 
 . 'pmu' to print the kernel supplied PMU events.
 
+. 'sdt' to print the SDT markers present in dsos and binaries. An additional
+  argument of filename will instruct perf to look for SDT markers only in that
+  file.
+
 . If none of the above is matched, it will apply the supplied glob to all
   events, printing the ones that match.
 
#6 lore raw
In reply to Hemant Kumar
Quoted 30 lines · Hemant Kumar

This patchset helps in listing dtrace style markers(SDT) present in user space applications through perf.
Notes/markers are placed at important places by the
developers. They have a negligible overhead when not enabled.
We can enable them and probe at these places and find some important information like the arguments' values, etc.

We have lots of applications which use SDT markers today, like:
Postgresql, MySql, Mozilla, Perl, Python, Java, Ruby, libvirt, QEMU, glib

To add SDT markers into user applications:
We need to have this header sys/sdt.h present.
sys/sdt.h used is version 3.
If not present, install systemtap-sdt-devel package (for fedora-18).

Please refer to the Documentation patch to see how the SDT markers are added into a program.

With this patchset,
- Use perf to list the markers in the app:

# perf list sdt ./user_app

./user_app :
%user_app:foo_start
%user_app:fun_start

- Also, we can see the SDT markers present in our system in the usual binaries. These usual binaries are libraries (dsos) listed by ldconfig --print-cache and some binaries present in PATH environment variable.

Quoted 7 lines

First, scan the binaries using :

# perf list sdt --scan

Creating a cache of SDT markers...
perf sdt cache created!
Use : "perf list sdt"
to see the SDT markers

Hmm, in that case, I think you'd better introduce perf-sdt for scanning. e.g.

 # perf sdt --scan app

then you can add app to sdt cache, without app,

 # perf sdt --scan

will just scans all binaries on the PATH and the libraries which listed
by ldconfig --print-caceh

And perf-list shows only the SDTs in the cache.

If there is no SDTs in the cache, perf-list sdt will warn you to run the perf sdt --scan.

Quoted 38 lines

After the sdt cache file is created, use perf list to view the markers :

# perf list sdt

%rtld : init_start
%rtld : init_complete
%rtld : map_failed
%rtld : map_start
%rtld : lll_futex_wake
...
...
%libgcc : unwind
%libvirt : rpc_server_client_auth_allow
%libvirt : rpc_server_client_auth_fail
%libvirt : rpc_server_client_auth_deny

Alternatively, one can view the /var/cache/perf/perf-sdt.cache directly.

This link shows an example of marker probing with Systemtap:
https://sourceware.org/systemtap/wiki/AddingUserSpaceProbingToApps

Also, this link provides important info regarding SDT notes:
http://sourceware.org/systemtap/wiki/UserSpaceProbeImplementation

- Markers in binaries :
These SDT markers are present in the ELF in the section named ".note.stapsdt".
Here, the name of the marker, its provider, type, location, base address, semaphore address.
We can retrieve these values using the members name_off and desc_off in Nhdr structure. If these are not enabled, they are present in the ELF as nop.

Changes since last series :
- Made the SDT markers present in a system more visible through perf list as suggested by Pekka Enberg.
- Less complex command line interface to access the SDT markers.
- Added a cache file to store most of the SDT markers of a system.

Quoted 4 lines

TODO:
- Add support to probe these SDT markers and integrate with a previous patch (support to perf to probe SDT markers) posted in lkml.
https://lkml.org/lkml/2013/10/23/10

Yeah, but I think we'd better choose another way to integrate it. Since SDT is like markers(static events), setting each of them via perf-probe is not intuitive. :) I'd like to use it as an event, e.g.

 # perf top -e "%libgcc:unwind"

And perf top internally calls perf-probe to add new uprobe event, and clean the new event at exit.

Thank you,

Signature · 4 lines
Masami HIRAMATSU
IT Management Research Dept. Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: [email protected]
#7 lore raw
Quoted 34 lines · Masami Hiramatsu
Quoted 33 lines · Hemant Kumar

This patchset helps in listing dtrace style markers(SDT) present in user space applications through perf.
Notes/markers are placed at important places by the
developers. They have a negligible overhead when not enabled.
We can enable them and probe at these places and find some important information like the arguments' values, etc.

We have lots of applications which use SDT markers today, like:
Postgresql, MySql, Mozilla, Perl, Python, Java, Ruby, libvirt, QEMU, glib

To add SDT markers into user applications:
We need to have this header sys/sdt.h present.
sys/sdt.h used is version 3.
If not present, install systemtap-sdt-devel package (for fedora-18).

Please refer to the Documentation patch to see how the SDT markers are added into a program.

With this patchset,
- Use perf to list the markers in the app:

# perf list sdt ./user_app

./user_app :
%user_app:foo_start
%user_app:fun_start

- Also, we can see the SDT markers present in our system in the usual binaries. These usual binaries are libraries (dsos) listed by ldconfig --print-cache and some binaries present in PATH environment variable.

First, scan the binaries using :

# perf list sdt --scan
Quoted 8 lines
Quoted 4 lines

Creating a cache of SDT markers...
perf sdt cache created!
Use : "perf list sdt"
to see the SDT markers

Hmm, in that case, I think you'd better introduce perf-sdt for scanning. e.g.

  # perf sdt --scan app

Hmm, this seems a better idea :)

Quoted 8 lines

then you can add app to sdt cache, without app,

  # perf sdt --scan

will just scans all binaries on the PATH and the libraries which listed
by ldconfig --print-caceh

And perf-list shows only the SDTs in the cache.

Well, what will be better? perf-list or perf-sdt or perf-list sdt?? If perf-list, then wouldn't it be a huge list!!

Quoted 3 lines

If there is no SDTs in the cache, perf-list sdt will warn you to run the perf sdt --scan.

Right.

Quoted 42 lines
Quoted 38 lines

After the sdt cache file is created, use perf list to view the markers :

# perf list sdt

%rtld : init_start
%rtld : init_complete
%rtld : map_failed
%rtld : map_start
%rtld : lll_futex_wake
...
...
%libgcc : unwind
%libvirt : rpc_server_client_auth_allow
%libvirt : rpc_server_client_auth_fail
%libvirt : rpc_server_client_auth_deny

Alternatively, one can view the /var/cache/perf/perf-sdt.cache directly.

This link shows an example of marker probing with Systemtap:
https://sourceware.org/systemtap/wiki/AddingUserSpaceProbingToApps

Also, this link provides important info regarding SDT notes:
http://sourceware.org/systemtap/wiki/UserSpaceProbeImplementation

- Markers in binaries :
These SDT markers are present in the ELF in the section named ".note.stapsdt".
Here, the name of the marker, its provider, type, location, base address, semaphore address.
We can retrieve these values using the members name_off and desc_off in Nhdr structure. If these are not enabled, they are present in the ELF as nop.

Changes since last series :
- Made the SDT markers present in a system more visible through perf list as suggested by Pekka Enberg.
- Less complex command line interface to access the SDT markers.
- Added a cache file to store most of the SDT markers of a system.

Quoted 4 lines

TODO:
- Add support to probe these SDT markers and integrate with a previous patch (support to perf to probe SDT markers) posted in lkml.
https://lkml.org/lkml/2013/10/23/10

Quoted 11 lines

Yeah, but I think we'd better choose another way to integrate it. Since SDT is like markers(static events), setting each of them via perf-probe is not intuitive. :) I'd like to use it as an event, e.g.

  # perf top -e "%libgcc:unwind"

And perf top internally calls perf-probe to add new uprobe event, and clean the new event at exit.

Yeah! Right :) Makes sense.

Will implement the suggestions in the next version asap!

Signature · 2 lines
Thanks
Hemant Kumar
maintainer
#8 lore raw

Hi Masami and Hemant,

Quoted 9 lines · Hemant Kumar
Quoted 8 lines · Masami Hiramatsu
Quoted 7 lines · Hemant Kumar

First, scan the binaries using :

# perf list sdt --scan

Creating a cache of SDT markers...
perf sdt cache created!
Use : "perf list sdt"
to see the SDT markers

Quoted 13 lines
Quoted 4 lines

Hmm, in that case, I think you'd better introduce perf-sdt for scanning. e.g.

  # perf sdt --scan app

Hmm, this seems a better idea :)

Quoted 6 lines

then you can add app to sdt cache, without app,

  # perf sdt --scan

will just scans all binaries on the PATH and the libraries which listed
by ldconfig --print-caceh

What should be done with the new perf sdt command? If it's only intended to list the markers, I'd just suggest to add "perf list sdt" as this patch did.

Plus I think it'd be better if event_glob pattern also looks for sdt markers so that user can find out a specific markers easily, e.g.:

  # perf list rtld:*

or

  # perf list %rtld:*
Quoted 5 lines
Quoted 2 lines

And perf-list shows only the SDTs in the cache.

Well, what will be better? perf-list or perf-sdt or perf-list sdt?? If perf-list, then wouldn't it be a huge list!!

The output of perf list is already a huge list and we paginate it. So I don't think it's gonna be a problem. :)

Quoted 7 lines
Quoted 7 lines
Quoted 3 lines

- Add support to probe these SDT markers and integrate with a previous patch (support to perf to probe SDT markers) posted in lkml.
https://lkml.org/lkml/2013/10/23/10

Yeah, but I think we'd better choose another way to integrate it. Since SDT is like markers(static events), setting each of them via perf-probe is not intuitive. :) I'd like to use it as an event, e.g.

Quoted 8 lines
Quoted 4 lines
  # perf top -e "%libgcc:unwind"

And perf top internally calls perf-probe to add new uprobe event, and clean the new event at exit.

Yeah! Right :) Makes sense.

Will implement the suggestions in the next version asap!

That would be great!

Thanks,
Namhyung

#9 lore raw
Quoted 20 lines · Namhyung Kim

Hi Masami and Hemant,

Quoted 17 lines · Hemant Kumar
Quoted 4 lines · Masami Hiramatsu
Quoted 3 lines · Hemant Kumar

First, scan the binaries using :

# perf list sdt --scan
Quoted 8 lines
Quoted 4 lines

Creating a cache of SDT markers...
perf sdt cache created!
Use : "perf list sdt"
to see the SDT markers

Hmm, in that case, I think you'd better introduce perf-sdt for scanning. e.g.

   # perf sdt --scan app

Hmm, this seems a better idea :)

Quoted 2 lines

then you can add app to sdt cache, without app,

Quoted 7 lines
Quoted 4 lines
Quoted 4 lines
   # perf sdt --scan

will just scans all binaries on the PATH and the libraries which listed
by ldconfig --print-caceh

What should be done with the new perf sdt command? If it's only intended to list the markers, I'd just suggest to add "perf list sdt" as this patch did.

If we display the SDT markers along with the other events in perf list, then I think we can go with perf list sdt. I am not too sure though! :)

For me, the main issue was that the markers are not events. They become events after we place them in the uprobe_events file just like functions. But we use perf list to display all the "events" available on a system. Isn't it?

Quoted 8 lines

Plus I think it'd be better if event_glob pattern also looks for sdt markers so that user can find out a specific markers easily, e.g.:

   # perf list rtld:*

or

   # perf list %rtld:*

Good idea! Will surely include support for this in event_glob pattern.

Quoted 5 lines
Quoted 3 lines
Quoted 1 line

And perf-list shows only the SDTs in the cache.

Well, what will be better? perf-list or perf-sdt or perf-list sdt?? If perf-list, then wouldn't it be a huge list!!

The output of perf list is already a huge list and we paginate it. So I don't think it's gonna be a problem. :)

Ok! Then we can use perf list. :)

Quoted 16 lines
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines

- Add support to probe these SDT markers and integrate with a previous patch (support to perf to probe SDT markers) posted in lkml.
https://lkml.org/lkml/2013/10/23/10

Yeah, but I think we'd better choose another way to integrate it. Since SDT is like markers(static events), setting each of them via perf-probe is not intuitive. :) I'd like to use it as an event, e.g.

Quoted 7 lines
Quoted 4 lines
   # perf top -e "%libgcc:unwind"

And perf top internally calls perf-probe to add new uprobe event, and clean the new event at exit.

Yeah! Right :) Makes sense.

Will implement the suggestions in the next version asap!

That would be great!

Signature · 2 lines
Thanks
Hemant Kumar
#10 lore raw
Quoted 21 lines · Hemant Kumar
Quoted 20 lines · Namhyung Kim

Hi Masami and Hemant,

Quoted 17 lines · Hemant Kumar
Quoted 4 lines · Masami Hiramatsu
Quoted 3 lines · Hemant Kumar

First, scan the binaries using :

# perf list sdt --scan
Quoted 8 lines
Quoted 4 lines

Creating a cache of SDT markers...
perf sdt cache created!
Use : "perf list sdt"
to see the SDT markers

Hmm, in that case, I think you'd better introduce perf-sdt for scanning. e.g.

   # perf sdt --scan app

Hmm, this seems a better idea :)

Quoted 2 lines

then you can add app to sdt cache, without app,

Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
Quoted 4 lines
   # perf sdt --scan

will just scans all binaries on the PATH and the libraries which listed
by ldconfig --print-caceh

What should be done with the new perf sdt command? If it's only intended to list the markers, I'd just suggest to add "perf list sdt" as this patch did.

No, here what I said is, the "perf sdt" is only for managing SDT cache as like as "perf buildid-cache". Thus, "perf sdt-cache" might be better.

BTW, the SDT markers can be changed if the application is updated. To ensure the correctness of SDT markers, we should store buildid in the cache file and check it when listing and using them.

Quoted 9 lines

If we display the SDT markers along with the other events in perf list, then I think we can go with
perf list sdt. I am not too sure though! :)

For me, the main issue was that the markers are not events. They become events after
we place them in the uprobe_events file just like functions. But we use perf list to
display all the "events" available on a system. Isn't it?

As I said, if perf accepts -e "%app:sdt" option, showing SDT events as fixed events does not matter, since it is transparent to users. :)

Thank you,

Signature · 4 lines
Masami HIRAMATSU
IT Management Research Dept. Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: [email protected]
#11 lore raw
Quoted 24 lines · Masami Hiramatsu
Quoted 23 lines · Hemant Kumar
Quoted 22 lines · Namhyung Kim

Hi Masami and Hemant,

Quoted 19 lines · Hemant Kumar
Quoted 4 lines · Masami Hiramatsu
Quoted 3 lines · Hemant Kumar

First, scan the binaries using :

# perf list sdt --scan
Quoted 8 lines
Quoted 4 lines

Creating a cache of SDT markers...
perf sdt cache created!
Use : "perf list sdt"
to see the SDT markers

Hmm, in that case, I think you'd better introduce perf-sdt for scanning. e.g.

    # perf sdt --scan app

Hmm, this seems a better idea :)

Quoted 4 lines

then you can add app to sdt cache, without app,

# perf sdt --scan

Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines

will just scans all binaries on the PATH and the libraries which listed
by ldconfig --print-caceh

What should be done with the new perf sdt command? If it's only intended to list the markers, I'd just suggest to add "perf list sdt" as this patch did.

No, here what I said is, the "perf sdt" is only for managing SDT cache as like as "perf buildid-cache". Thus, "perf sdt-cache" might be better.

Ah! ok.

Quoted 3 lines

BTW, the SDT markers can be changed if the application is updated. To ensure the correctness of SDT markers, we should store buildid in the cache file and check it when listing and using them.

Yeah! That's why perf list sdt --scan is storing the buildid too in the cache file.

Quoted 9 lines
Quoted 9 lines

If we display the SDT markers along with the other events in perf list, then I think we can go with
perf list sdt. I am not too sure though! :)

For me, the main issue was that the markers are not events. They become events after
we place them in the uprobe_events file just like functions. But we use perf list to
display all the "events" available on a system. Isn't it?

Quoted 5 lines

As I said, if perf accepts -e "%app:sdt" option, showing SDT events as fixed events does not matter, since it is transparent to users. :)

Thank you,

Alright. Got it! :)

Signature · 2 lines
Thanks
Hemant Kumar
#12 lore raw
In reply to Namhyung Kim
Quoted 30 lines · Namhyung Kim

Hi Masami and Hemant,

Quoted 9 lines · Hemant Kumar
Quoted 8 lines · Masami Hiramatsu
Quoted 7 lines · Hemant Kumar

First, scan the binaries using :

# perf list sdt --scan

Creating a cache of SDT markers...
perf sdt cache created!
Use : "perf list sdt"
to see the SDT markers

Quoted 13 lines
Quoted 4 lines

Hmm, in that case, I think you'd better introduce perf-sdt for scanning. e.g.

  # perf sdt --scan app

Hmm, this seems a better idea :)

Quoted 6 lines

then you can add app to sdt cache, without app,

  # perf sdt --scan

will just scans all binaries on the PATH and the libraries which listed
by ldconfig --print-caceh

What should be done with the new perf sdt command? If it's only intended to list the markers, I'd just suggest to add "perf list sdt" as this patch did.

Quoted 8 lines

Plus I think it'd be better if event_glob pattern also looks for sdt markers so that user can find out a specific markers easily, e.g.:

  # perf list rtld:*

or

  # perf list %rtld:*

Ah, that's a good idea :)

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

And perf-list shows only the SDTs in the cache.

Well, what will be better? perf-list or perf-sdt or perf-list sdt?? If perf-list, then wouldn't it be a huge list!!

The output of perf list is already a huge list and we paginate it. So I don't think it's gonna be a problem. :)

Agreed.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines

- Add support to probe these SDT markers and integrate with a previous patch (support to perf to probe SDT markers) posted in lkml.
https://lkml.org/lkml/2013/10/23/10

Yeah, but I think we'd better choose another way to integrate it. Since SDT is like markers(static events), setting each of them via perf-probe is not intuitive. :) I'd like to use it as an event, e.g.

Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
  # perf top -e "%libgcc:unwind"

And perf top internally calls perf-probe to add new uprobe event, and clean the new event at exit.

Yeah! Right :) Makes sense.

Note that we will need an option to make it permanent (not clear the event on exit) for people who want to use SDT event from ftrace. :)

Quoted 3 lines
Quoted 1 line

Will implement the suggestions in the next version asap!

That would be great!

Thank you!

Signature · 4 lines
Masami HIRAMATSU
IT Management Research Dept. Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: [email protected]