Skip to contentSkip to conversation

perf tools: Fix JIT profiling on heap

PATCH In review Namhyung Kim 6 participants 14 messages Jan 16, 2014 – Jan 31, 2014 perf events

lore
Share & tools
lore

Namhyung Kim

author maintainer
#0 lore raw

PATCH 1/1 perf tools: Fix JIT profiling on heap

Gaurav reported that perf cannot profile JIT program if it executes the code on heap. This was because current map__new() only handle JIT on anon mappings - extends it to handle no_dso (heap, stack) case too.

This patch assumes JIT profiling only provides dynamic function symbols so check the mapping type to distinguish the case. It'd provide no symbols for data mapping - if we need to support symbols on data mappings later it should be changed.

---
 tools/perf/util/map.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/util/map.c +2 -2
index 9b9bd719aa19..ee1dd687a262 100644
@@ -69,7 +69,7 @@ struct map *map__new(struct list_head *dsos__list, u64 start, u64 len,
 		map->ino = ino;
 		map->ino_generation = ino_gen;
 
-		if (anon) {
+		if ((anon || no_dso) && type == MAP__FUNCTION) {
 			snprintf(newfilename, sizeof(newfilename), "/tmp/perf-%d.map", pid);
 			filename = newfilename;
 		}
@@ -93,7 +93,7 @@ struct map *map__new(struct list_head *dsos__list, u64 start, u64 len,
 			 * functions still return NULL, and we avoid the
 			 * unnecessary map__load warning.
 			 */
-			if (no_dso)
+			if (type != MAP__FUNCTION)
 				dso__set_loaded(dso, map->type);
 		}
 	}
Signature · 1 line
1.7.11.7
#1 lore raw

Em Thu, Jan 16, 2014 at 10:49:31AM +0900, Namhyung Kim escreveu:

Quoted 8 lines

Gaurav reported that perf cannot profile JIT program if it executes the code on heap. This was because current map__new() only handle JIT on anon mappings - extends it to handle no_dso (heap, stack) case too.

This patch assumes JIT profiling only provides dynamic function symbols so check the mapping type to distinguish the case. It'd provide no symbols for data mapping - if we need to support symbols on data mappings later it should be changed.

But we do support symbols in data mappings, that is why we have MAP__VARIABLE, etc, can you elaborate?

Gaurav, do this fixes the problem you reported? Can I have your
Tested-by: tag?

- Arnaldo

Quoted 30 lines
---
 tools/perf/util/map.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/util/map.c +2 -2
index 9b9bd719aa19..ee1dd687a262 100644
@@ -69,7 +69,7 @@ struct map *map__new(struct list_head *dsos__list, u64 start, u64 len,
 		map->ino = ino;
 		map->ino_generation = ino_gen;
 
-		if (anon) {
+		if ((anon || no_dso) && type == MAP__FUNCTION) {
 			snprintf(newfilename, sizeof(newfilename), "/tmp/perf-%d.map", pid);
 			filename = newfilename;
 		}
@@ -93,7 +93,7 @@ struct map *map__new(struct list_head *dsos__list, u64 start, u64 len,
 			 * functions still return NULL, and we avoid the
 			 * unnecessary map__load warning.
 			 */
-			if (no_dso)
+			if (type != MAP__FUNCTION)
 				dso__set_loaded(dso, map->type);
 		}
 	}
Signature · 1 line
1.7.11.7
#2 lore raw
Quoted 5 lines · Arnaldo Carvalho de Melo

Em Thu, Jan 16, 2014 at 10:49:31AM +0900, Namhyung Kim escreveu:

Quoted 4 lines

Gaurav reported that perf cannot profile JIT program if it executes the code on heap. This was because current map__new() only handle JIT on anon mappings - extends it to handle no_dso (heap, stack) case too.

Quoted 7 lines
Quoted 4 lines

This patch assumes JIT profiling only provides dynamic function symbols so check the mapping type to distinguish the case. It'd provide no symbols for data mapping - if we need to support symbols on data mappings later it should be changed.

But we do support symbols in data mappings, that is why we have MAP__VARIABLE, etc, can you elaborate?

Does perf support data mappings from perf map files? Could you please share an example of how I may be able to use this.

Quoted 2 lines

Gaurav, do this fixes the problem you reported? Can I have your
Tested-by: tag?

Yep, I¹ve tested this version of the patch and it works great for me.

Thanks,

Gaurav

author maintainer
#3 lore raw

Hi Gaurav,

Quoted 13 lines · Gaurav Jain
Quoted 10 lines · Arnaldo Carvalho de Melo

Em Thu, Jan 16, 2014 at 10:49:31AM +0900, Namhyung Kim escreveu:

Quoted 8 lines

Gaurav reported that perf cannot profile JIT program if it executes the code on heap. This was because current map__new() only handle JIT on anon mappings - extends it to handle no_dso (heap, stack) case too.

This patch assumes JIT profiling only provides dynamic function symbols so check the mapping type to distinguish the case. It'd provide no symbols for data mapping - if we need to support symbols on data mappings later it should be changed.

Quoted 5 lines
Quoted 2 lines

But we do support symbols in data mappings, that is why we have MAP__VARIABLE, etc, can you elaborate?

Does perf support data mappings from perf map files? Could you please share an example of how I may be able to use this.

IIUC there's no difference between function and data mapping. So you can use same perf map file for both - in fact there's no way to use different map file in a single task. I guess perf will use it to find only function symbols in function mappings and variables in data mapping based on the address it accesses.

What I wasn't sure is whether JIT program also produces some dynamic data. And I think only perf mem command cares about data mappings, no?

Thanks,
Namhyung

#4 lore raw

Em Fri, Jan 17, 2014 at 04:44:04PM +0900, Namhyung Kim escreveu:

Quoted 17 lines · Gaurav Jain
Quoted 15 lines
Quoted 12 lines · Arnaldo Carvalho de Melo

Em Thu, Jan 16, 2014 at 10:49:31AM +0900, Namhyung Kim escreveu:

Quoted 8 lines

Gaurav reported that perf cannot profile JIT program if it executes the code on heap. This was because current map__new() only handle JIT on anon mappings - extends it to handle no_dso (heap, stack) case too.

This patch assumes JIT profiling only provides dynamic function symbols so check the mapping type to distinguish the case. It'd provide no symbols for data mapping - if we need to support symbols on data mappings later it should be changed.

But we do support symbols in data mappings, that is why we have MAP__VARIABLE, etc, can you elaborate?

Quoted 6 lines
Quoted 2 lines

Does perf support data mappings from perf map files? Could you please share an example of how I may be able to use this.

IIUC there's no difference between function and data mapping. So you can use same perf map file for both - in fact there's no way to use different map file in a single task. I guess perf will use it to find

Do the /tmp/perf mapping has any per entry indication on the type of symbol it is (data, text) like ELF and kallsyms symtabs have?

It is possible for a function and a variable to have the same virtual address in some architectures (SPARC, iirc), that is why we have different MAP_ types (FUNCTION, VARIABLE) (which should really be renamed to TEXT, DATA).

So a 'struct map' for a data mmap should possibly point to a different 'dso' of the JIT /tmp/perf-... style if those maps don't have per entry indication of text/data.

Quoted 2 lines

only function symbols in function mappings and variables in data mapping based on the address it accesses.

Well, the lookup should figure out if the IP refers to TEXT or DATA and use MAP__{FUNCTION, VARIABLE} accordingly when asking for symbol resolution.

Quoted 2 lines

What I wasn't sure is whether JIT program also produces some dynamic data. And I think only perf mem command cares about data mappings, no?

Well, I think it would be great to do that kind of data resolution for JITs the same way it is interesting to do for ELF ones :-)

I need to stare harder at that patch, but with the above in mind, do we really have to check if the map is MAP__FUNCTION as IIRC this patch does?

- Arnaldo

author maintainer
#5 lore raw

Hi Arnaldo,

Quoted 21 lines · Arnaldo Carvalho de Melo

Em Fri, Jan 17, 2014 at 04:44:04PM +0900, Namhyung Kim escreveu:

Quoted 19 lines · Gaurav Jain
Quoted 18 lines
Quoted 5 lines · Arnaldo Carvalho de Melo

Em Thu, Jan 16, 2014 at 10:49:31AM +0900, Namhyung Kim escreveu:

Quoted 4 lines

Gaurav reported that perf cannot profile JIT program if it executes the code on heap. This was because current map__new() only handle JIT on anon mappings - extends it to handle no_dso (heap, stack) case too.

Quoted 7 lines
Quoted 4 lines

This patch assumes JIT profiling only provides dynamic function symbols so check the mapping type to distinguish the case. It'd provide no symbols for data mapping - if we need to support symbols on data mappings later it should be changed.

But we do support symbols in data mappings, that is why we have MAP__VARIABLE, etc, can you elaborate?

Does perf support data mappings from perf map files? Could you please share an example of how I may be able to use this.

Quoted 6 lines
Quoted 3 lines

IIUC there's no difference between function and data mapping. So you can use same perf map file for both - in fact there's no way to use different map file in a single task. I guess perf will use it to find

Do the /tmp/perf mapping has any per entry indication on the type of symbol it is (data, text) like ELF and kallsyms symtabs have?

Quoting Documentation/jit-interface.txt:

Each line has the following format, fields separated with spaces:

START SIZE symbolname

START and SIZE are hex numbers without 0x.
symbolname is the rest of the line, so it could contain special characters.

Quoted 5 lines

It is possible for a function and a variable to have the same virtual address in some architectures (SPARC, iirc), that is why we have different MAP_ types (FUNCTION, VARIABLE) (which should really be renamed to TEXT, DATA).

Hmm.. didn't know that, interesting..

Quoted 4 lines

So a 'struct map' for a data mmap should possibly point to a different 'dso' of the JIT /tmp/perf-... style if those maps don't have per entry indication of text/data.

Yes, but there's no way to do it currently.

Quoted 7 lines
Quoted 3 lines

only function symbols in function mappings and variables in data mapping based on the address it accesses.

Well, the lookup should figure out if the IP refers to TEXT or DATA and use MAP__{FUNCTION, VARIABLE} accordingly when asking for symbol resolution.

Right. But in this case we cannot determine whether a symbol in the /tmp/perf-... file is a function or variable.

Quoted 10 lines
Quoted 3 lines

What I wasn't sure is whether JIT program also produces some dynamic data. And I think only perf mem command cares about data mappings, no?

Well, I think it would be great to do that kind of data resolution for JITs the same way it is interesting to do for ELF ones :-)

I need to stare harder at that patch, but with the above in mind, do we really have to check if the map is MAP__FUNCTION as IIRC this patch does?

Not sure. For a JIT case, I guess the mapping is always executable and we don't support data mapping yet, so it seems okay for now.

Thanks,
Namhyung

#6 lore raw

Em Tue, Jan 21, 2014 at 02:54:32PM +0900, Namhyung Kim escreveu:

Quoted 20 lines · Arnaldo Carvalho de Melo
Quoted 8 lines

Em Fri, Jan 17, 2014 at 04:44:04PM +0900, Namhyung Kim escreveu:

Quoted 4 lines · Gaurav Jain
Quoted 1 line

Does perf support data mappings from perf map files?

IIUC there's no difference between function and data mapping. So

Do the /tmp/perf mapping has any per entry indication on the type of symbol it is (data, text) like ELF and kallsyms symtabs have?

Quoting Documentation/jit-interface.txt:

Each line has the following format, fields separated with spaces:

START SIZE symbolname

Quoted 3 lines

It is possible for a function and a variable to have the same virt addrin some arches (SPARC, iirc), that is why we have different MAP_ types (FUNCTION, VARIABLE)

Quoted 7 lines

Hmm.. didn't know that, interesting..

Quoted 3 lines

So a 'struct map' for a data mmap should point to a different 'dso' of the JIT /tmp/perf-... style if those maps don't have per entry indication of text/data.

Yes, but there's no way to do it currently.

Why not? Its just a matter of having a /tmp/perf-...-data file that has the same structure as the current files for text mmaps :-)

It would then be used as the map->dso for resolving data addresses.

Quoted 9 lines
Quoted 6 lines
Quoted 2 lines

only function symbols in function mappings and variables in data mapping based on the address it accesses.

Well, the lookup should figure out if the IP refers to TEXT or DATA and use MAP__{FUNCTION, VARIABLE} accordingly when asking for symbol resolution.

Right. But in this case we cannot determine whether a symbol in the /tmp/perf-... file is a function or variable.

That is why we would then need to have separate /tmp/perf-... files, disambiguated by an extension, one for text addresses, another for data ones.

Or change the format to match /proc/kallsyms, which probably is what we should've done from day one...

Detecting the format change would be trivial, as we would find 3 tokens in the new format, instead of the current 2.

Quoted 6 lines
Quoted 6 lines
Quoted 2 lines

What I wasn't sure is whether JIT program also produces some dynamic data. And I think only perf mem command cares about data mappings, no?

Well, I think it would be great to do that kind of data resolution for JITs the same way it is interesting to do for ELF ones :-)

Quoted 6 lines
Quoted 3 lines

I need to stare harder at that patch, but with the above in mind, do we really have to check if the map is MAP__FUNCTION as IIRC this patch does?

Not sure. For a JIT case, I guess the mapping is always executable and we don't support data mapping yet, so it seems okay for now.

Ok, agreed, applying the patch, thanks,

- Arnaldo

#7 lore raw
Quoted 9 lines · Arnaldo Carvalho de Melo
Quoted 2 lines

Right. But in this case we cannot determine whether a symbol in the /tmp/perf-... file is a function or variable.

That is why we would then need to have separate /tmp/perf-... files, disambiguated by an extension, one for text addresses, another for data ones.

Or change the format to match /proc/kallsyms, which probably is what we should've done from day one...

Yeah. Using the same format would be lovely. The parser could try to detect the new one and could fall back to the old format?

Quoted 2 lines

Detecting the format change would be trivial, as we would find 3 tokens in the new format, instead of the current 2.

Yeah!

Thanks,

Ingo

#8 lore raw
In reply to Namhyung Kim
Quoted 10 lines · Namhyung Kim

Hi Gaurav,

Quoted 5 lines · Gaurav Jain

On 1/16/14, 9:37 AM, "Arnaldo Carvalho de Melo" <[email protected]> wrote:
Does perf support data mappings from perf map files? Could you please share an example of how I may be able to use this.

Quoted 8 lines

IIUC there's no difference between function and data mapping. So you can use same perf map file for both - in fact there's no way to use different map file in a single task. I guess perf will use it to find only function symbols in function mappings and variables in data mapping based on the address it accesses.

What I wasn't sure is whether JIT program also produces some dynamic data. And I think only perf mem command cares about data mappings, no?

I don't think we generate symbols for dynamic data at the moment. There may be uses in the future. I don't know about other JIT programs.

If we can include this behaviour it would be great, but I don't have a way to test it. I'd guess if somebody had a valid use case, they would report the issue. Otherwise, the patch as you've written it works great for me.

Gaurav

author maintainer
#9 lore raw

Hi Gaurav,

Quoted 7 lines · Gaurav Jain
Quoted 3 lines · Namhyung Kim

What I wasn't sure is whether JIT program also produces some dynamic data. And I think only perf mem command cares about data mappings, no?

I don't think we generate symbols for dynamic data at the moment. There may be uses in the future. I don't know about other JIT programs.

If we can include this behaviour it would be great, but I don't have a way to test it. I'd guess if somebody had a valid use case, they would report the issue. Otherwise, the patch as you've written it works great for me.

Thanks for confirming it and testing the patch!

Thanks,
Namhyung

#10 lore raw

[tip:perf/urgent] perf symbols: Fix JIT symbol resolution on heap

In reply to Namhyung Kim
perf symbols: Fix JIT symbol resolution on heap
Commit-ID:  578c03c86fadcc6fd7319ddf41dd4d1d88aab77a
Gitweb:     http://git.kernel.org/tip/578c03c86fadcc6fd7319ddf41dd4d1d88aab77a
Author:     Namhyung Kim <[email protected]>
AuthorDate: Thu, 16 Jan 2014 10:49:31 +0900
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Tue, 21 Jan 2014 10:56:05 -0300

perf symbols: Fix JIT symbol resolution on heap

Gaurav reported that perf cannot profile JIT program if it executes the code on heap. This was because current map__new() only handle JIT on anon mappings - extends it to handle no_dso (heap, stack) case too.

This patch assumes JIT profiling only provides dynamic function symbols so check the mapping type to distinguish the case. It'd provide no symbols for data mapping - if we need to support symbols on data mappings later it should be changed.

---
 tools/perf/util/map.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/util/map.c +2 -2
index 9b9bd71..ee1dd68 100644
@@ -69,7 +69,7 @@ struct map *map__new(struct list_head *dsos__list, u64 start, u64 len,
 		map->ino = ino;
 		map->ino_generation = ino_gen;
 
-		if (anon) {
+		if ((anon || no_dso) && type == MAP__FUNCTION) {
 			snprintf(newfilename, sizeof(newfilename), "/tmp/perf-%d.map", pid);
 			filename = newfilename;
 		}
@@ -93,7 +93,7 @@ struct map *map__new(struct list_head *dsos__list, u64 start, u64 len,
 			 * functions still return NULL, and we avoid the
 			 * unnecessary map__load warning.
 			 */
-			if (no_dso)
+			if (type != MAP__FUNCTION)
 				dso__set_loaded(dso, map->type);
 		}
 	}
#11 lore raw
In reply to Namhyung Kim
Quoted 11 lines · Namhyung Kim

Gaurav reported that perf cannot profile JIT program if it executes the code on heap. This was because current map__new() only handle JIT on anon mappings - extends it to handle no_dso (heap, stack) case too.

This patch assumes JIT profiling only provides dynamic function symbols so check the mapping type to distinguish the case. It'd provide no symbols for data mapping - if we need to support symbols on data mappings later it should be changed.

Looks OK to me. Gaurav, does this fix your problem?

Quoted 26 lines
---
  tools/perf/util/map.c | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/util/map.c +2 -2
index 9b9bd719aa19..ee1dd687a262 100644
@@ -69,7 +69,7 @@ struct map *map__new(struct list_head *dsos__list, u64 start, u64 len,
  		map->ino = ino;
  		map->ino_generation = ino_gen;
 
-		if (anon) {
+		if ((anon || no_dso) && type == MAP__FUNCTION) {
  			snprintf(newfilename, sizeof(newfilename), "/tmp/perf-%d.map", pid);
  			filename = newfilename;
  		}
@@ -93,7 +93,7 @@ struct map *map__new(struct list_head *dsos__list, u64 start, u64 len,
  			 * functions still return NULL, and we avoid the
  			 * unnecessary map__load warning.
  			 */
-			if (no_dso)
+			if (type != MAP__FUNCTION)
  				dso__set_loaded(dso, map->type);
  		}
  	}
#12 lore raw

Em Fri, Jan 31, 2014 at 01:50:27PM +0200, Pekka Enberg escreveu:

Quoted 10 lines · Namhyung Kim
Quoted 9 lines

Gaurav reported that perf cannot profile JIT program if it executes the code on heap. This was because current map__new() only handle JIT on anon mappings - extends it to handle no_dso (heap, stack) case too.

This patch assumes JIT profiling only provides dynamic function symbols so check the mapping type to distinguish the case. It'd provide no symbols for data mapping - if we need to support symbols on data mappings later it should be changed.

Quoted 4 lines
Quoted 2 lines

Looks OK to me. Gaurav, does this fix your problem?

Yes, he tested it, etc, its already in Linus' tree:

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/tools/perf/util/map.c?id=578c03c86fadcc6fd7319ddf41dd4d1d88aab77a

- Arnaldo

Quoted 25 lines
Quoted 25 lines
---
 tools/perf/util/map.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/util/map.c +2 -2
index 9b9bd719aa19..ee1dd687a262 100644
@@ -69,7 +69,7 @@ struct map *map__new(struct list_head *dsos__list, u64 start, u64 len,
 		map->ino = ino;
 		map->ino_generation = ino_gen;
-		if (anon) {
+		if ((anon || no_dso) && type == MAP__FUNCTION) {
 			snprintf(newfilename, sizeof(newfilename), "/tmp/perf-%d.map", pid);
 			filename = newfilename;
 		}
@@ -93,7 +93,7 @@ struct map *map__new(struct list_head *dsos__list, u64 start, u64 len,
 			 * functions still return NULL, and we avoid the
 			 * unnecessary map__load warning.
 			 */
-			if (no_dso)
+			if (type != MAP__FUNCTION)
 				dso__set_loaded(dso, map->type);
 		}
 	}
#13 lore raw
Quoted 11 lines · Arnaldo Carvalho de Melo

Em Fri, Jan 31, 2014 at 01:50:27PM +0200, Pekka Enberg escreveu:

Quoted 10 lines · Namhyung Kim
Quoted 9 lines

Gaurav reported that perf cannot profile JIT program if it executes the code on heap. This was because current map__new() only handle JIT on anon mappings - extends it to handle no_dso (heap, stack) case too.

This patch assumes JIT profiling only provides dynamic function symbols so check the mapping type to distinguish the case. It'd provide no symbols for data mapping - if we need to support symbols on data mappings later it should be changed.

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

Looks OK to me. Gaurav, does this fix your problem?

Yes, he tested it, etc, its already in Linus' tree:

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/tools/perf/util/map.c?id=578c03c86fadcc6fd7319ddf41dd4d1d88aab77a

Oh, great! I was going through my inbox and didn't notice it because I got dropped from the CC list. :-)

Thanks Arnaldo!

Pekka