perf tools: Fix JIT profiling on heap
Namhyung Kim
maintainerPATCH 1/1 perf tools: Fix JIT profiling on heap
Gaurav reported that perf cannot profile JIT program if it executes the code on heap. This was because current map__new() only handle JIT on anon mappings - extends it to handle no_dso (heap, stack) case too.
This patch assumes JIT profiling only provides dynamic function symbols so check the mapping type to distinguish the case. It'd provide no symbols for data mapping - if we need to support symbols on data mappings later it should be changed.
--- tools/perf/util/map.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/util/map.c +2 -2
@@ -69,7 +69,7 @@ struct map *map__new(struct list_head *dsos__list, u64 start, u64 len, map->ino = ino; map->ino_generation = ino_gen; -if (anon) { +if ((anon || no_dso) && type ==MAP__FUNCTION ) {snprintf (newfilename,sizeof (newfilename),"/tmp/perf-%d.map" , pid); filename = newfilename; }
@@ -93,7 +93,7 @@ struct map *map__new(struct list_head *dsos__list, u64 start, u64 len,* functions still return NULL, and we avoid the * unnecessary map__load warning. */ -if (no_dso) +if (type !=MAP__FUNCTION )dso__set_loaded (dso, map->type); } }
Signature · 1 line
1.7.11.7
Em Thu, Jan 16, 2014 at 10:49:31AM +0900, Namhyung Kim escreveu:
Quoted 8 lines
Gaurav reported that perf cannot profile JIT program if it executes the code on heap. This was because current
map__new()only handle JIT on anon mappings - extends it to handleno_dso(heap, stack) case too.This patch assumes JIT profiling only provides dynamic function symbols so check the mapping type to distinguish the case. It'd provide no symbols for data mapping - if we need to support symbols on data mappings later it should be changed.
But we do support symbols in data mappings, that is why we have MAP__VARIABLE, etc, can you elaborate?
Gaurav, do this fixes the problem you reported? Can I have your
Tested-by: tag?
- Arnaldo
Quoted 30 lines
Reported-by Gaurav Jain <[email protected]>Signed-off-by Namhyung Kim <[email protected]>--- tools/perf/util/map.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)tools/perf/util/map.c +2
-2@@ -69,7 +69,7 @@ struct map *map__new(struct list_head *dsos__list, u64 start, u64 len, map->ino = ino; map->ino_generation = ino_gen; -if (anon) { +if ((anon || no_dso) && type ==MAP__FUNCTION ) {snprintf (newfilename,sizeof (newfilename),"/tmp/perf-%d.map" , pid); filename = newfilename; }@@ -93,7 +93,7 @@ struct map *map__new(struct list_head *dsos__list, u64 start, u64 len,* functions still return NULL, and we avoid the * unnecessary map__load warning. */ -if (no_dso) +if (type !=MAP__FUNCTION )dso__set_loaded (dso, map->type); } }Signature · 1 line
1.7.11.7
Quoted 5 lines · Arnaldo Carvalho de Melo
Em Thu, Jan 16, 2014 at 10:49:31AM +0900, Namhyung Kim escreveu:
Quoted 4 lines
Gaurav reported that perf cannot profile JIT program if it executes the code on heap. This was because current
map__new()only handle JIT on anon mappings - extends it to handleno_dso(heap, stack) case too.
Quoted 7 lines
Quoted 4 lines
This patch assumes JIT profiling only provides dynamic function symbols so check the mapping type to distinguish the case. It'd provide no symbols for data mapping - if we need to support symbols on data mappings later it should be changed.
But we do support symbols in data mappings, that is why we have
MAP__VARIABLE, etc, can you elaborate?
Does perf support data mappings from perf map files? Could you please share an example of how I may be able to use this.
Quoted 2 lines
Gaurav, do this fixes the problem you reported? Can I have your
Tested-by: tag?
Yep, I¹ve tested this version of the patch and it works great for me.
Thanks,
Gaurav
Namhyung Kim
maintainerHi Gaurav,
Quoted 13 lines · Gaurav Jain
Quoted 10 lines · Arnaldo Carvalho de Melo
Em Thu, Jan 16, 2014 at 10:49:31AM +0900, Namhyung Kim escreveu:
Quoted 8 lines
Gaurav reported that perf cannot profile JIT program if it executes the code on heap. This was because current
map__new()only handle JIT on anon mappings - extends it to handleno_dso(heap, stack) case too.This patch assumes JIT profiling only provides dynamic function symbols so check the mapping type to distinguish the case. It'd provide no symbols for data mapping - if we need to support symbols on data mappings later it should be changed.
Quoted 5 lines
Quoted 2 lines
But we do support symbols in data mappings, that is why we have
MAP__VARIABLE, etc, can you elaborate?Does perf support data mappings from perf map files? Could you please share an example of how I may be able to use this.
IIUC there's no difference between function and data mapping. So you can use same perf map file for both - in fact there's no way to use different map file in a single task. I guess perf will use it to find only function symbols in function mappings and variables in data mapping based on the address it accesses.
What I wasn't sure is whether JIT program also produces some dynamic data. And I think only perf mem command cares about data mappings, no?
Thanks,
Namhyung
Em Fri, Jan 17, 2014 at 04:44:04PM +0900, Namhyung Kim escreveu:
Quoted 17 lines · Gaurav Jain
Quoted 15 lines
Quoted 12 lines · Arnaldo Carvalho de Melo
Em Thu, Jan 16, 2014 at 10:49:31AM +0900, Namhyung Kim escreveu:
Quoted 8 lines
Gaurav reported that perf cannot profile JIT program if it executes the code on heap. This was because current
map__new()only handle JIT on anon mappings - extends it to handleno_dso(heap, stack) case too.This patch assumes JIT profiling only provides dynamic function symbols so check the mapping type to distinguish the case. It'd provide no symbols for data mapping - if we need to support symbols on data mappings later it should be changed.
But we do support symbols in data mappings, that is why we have
MAP__VARIABLE, etc, can you elaborate?
Quoted 6 lines
Quoted 2 lines
Does perf support data mappings from perf map files? Could you please share an example of how I may be able to use this.
IIUC there's no difference between function and data mapping. So you can use same perf map file for both - in fact there's no way to use different map file in a single task. I guess perf will use it to find
Do the /tmp/perf mapping has any per entry indication on the type of symbol it is (data, text) like ELF and kallsyms symtabs have?
It is possible for a function and a variable to have the same virtual address in some architectures (SPARC, iirc), that is why we have different MAP_ types (FUNCTION, VARIABLE) (which should really be renamed to TEXT, DATA).
So a 'struct map' for a data mmap should possibly point to a different 'dso' of the JIT /tmp/perf-... style if those maps don't have per entry indication of text/data.
Quoted 2 lines
only function symbols in function mappings and variables in data mapping based on the address it accesses.
Well, the lookup should figure out if the IP refers to TEXT or DATA and use MAP__{FUNCTION, VARIABLE} accordingly when asking for symbol resolution.
Quoted 2 lines
What I wasn't sure is whether JIT program also produces some dynamic data. And I think only perf mem command cares about data mappings, no?
Well, I think it would be great to do that kind of data resolution for JITs the same way it is interesting to do for ELF ones :-)
I need to stare harder at that patch, but with the above in mind, do we really have to check if the map is MAP__FUNCTION as IIRC this patch does?
- Arnaldo
Namhyung Kim
maintainerHi Arnaldo,
Quoted 21 lines · Arnaldo Carvalho de Melo
Em Fri, Jan 17, 2014 at 04:44:04PM +0900, Namhyung Kim escreveu:
Quoted 19 lines · Gaurav Jain
Quoted 18 lines
Quoted 5 lines · Arnaldo Carvalho de Melo
Em Thu, Jan 16, 2014 at 10:49:31AM +0900, Namhyung Kim escreveu:
Quoted 4 lines
Gaurav reported that perf cannot profile JIT program if it executes the code on heap. This was because current
map__new()only handle JIT on anon mappings - extends it to handleno_dso(heap, stack) case too.Quoted 7 lines
Quoted 4 lines
This patch assumes JIT profiling only provides dynamic function symbols so check the mapping type to distinguish the case. It'd provide no symbols for data mapping - if we need to support symbols on data mappings later it should be changed.
But we do support symbols in data mappings, that is why we have
MAP__VARIABLE, etc, can you elaborate?Does perf support data mappings from perf map files? Could you please share an example of how I may be able to use this.
Quoted 6 lines
Quoted 3 lines
IIUC there's no difference between function and data mapping. So you can use same perf map file for both - in fact there's no way to use different map file in a single task. I guess perf will use it to find
Do the
/tmp/perfmapping has any per entry indication on the type of symbol it is (data, text) like ELF and kallsyms symtabs have?
Quoting Documentation/jit-interface.txt:
Each line has the following format, fields separated with spaces:
START SIZE symbolname
START and SIZE are hex numbers without 0x.
symbolname is the rest of the line, so it could contain special characters.
Quoted 5 lines
It is possible for a function and a variable to have the same virtual address in some architectures (SPARC, iirc), that is why we have different
MAP_types (FUNCTION, VARIABLE) (which should really be renamed to TEXT, DATA).
Hmm.. didn't know that, interesting..
Quoted 4 lines
So a 'struct map' for a data mmap should possibly point to a different 'dso' of the JIT
/tmp/perf-... style if those maps don't have per entry indication of text/data.
Yes, but there's no way to do it currently.
Quoted 7 lines
Quoted 3 lines
only function symbols in function mappings and variables in data mapping based on the address it accesses.
Well, the lookup should figure out if the IP refers to TEXT or DATA and use MAP__{FUNCTION, VARIABLE} accordingly when asking for symbol resolution.
Right. But in this case we cannot determine whether a symbol in the /tmp/perf-... file is a function or variable.
Quoted 10 lines
Quoted 3 lines
What I wasn't sure is whether JIT program also produces some dynamic data. And I think only perf mem command cares about data mappings, no?
Well, I think it would be great to do that kind of data resolution for JITs the same way it is interesting to do for ELF ones :-)
I need to stare harder at that patch, but with the above in mind, do we really have to check if the map is
MAP__FUNCTIONas IIRC this patch does?
Not sure. For a JIT case, I guess the mapping is always executable and we don't support data mapping yet, so it seems okay for now.
Thanks,
Namhyung
Em Tue, Jan 21, 2014 at 02:54:32PM +0900, Namhyung Kim escreveu:
Quoted 20 lines · Arnaldo Carvalho de Melo
Quoted 8 lines
Em Fri, Jan 17, 2014 at 04:44:04PM +0900, Namhyung Kim escreveu:
Quoted 4 lines · Gaurav Jain
Quoted 1 line
Does perf support data mappings from perf map files?
IIUC there's no difference between function and data mapping. So
Do the
/tmp/perfmapping has any per entry indication on the type of symbol it is (data, text) like ELF and kallsyms symtabs have?Quoting
Documentation/jit-interface.txt:Each line has the following format, fields separated with spaces:
START SIZE symbolname
Quoted 3 lines
It is possible for a function and a variable to have the same virt addrin some arches (SPARC, iirc), that is why we have different
MAP_types (FUNCTION, VARIABLE)
Quoted 7 lines
Hmm.. didn't know that, interesting..
Quoted 3 lines
So a 'struct map' for a data mmap should point to a different 'dso' of the JIT
/tmp/perf-... style if those maps don't have per entry indication of text/data.Yes, but there's no way to do it currently.
Why not? Its just a matter of having a /tmp/perf-...-data file that has the same structure as the current files for text mmaps :-)
It would then be used as the map->dso for resolving data addresses.
Quoted 9 lines
Quoted 6 lines
Quoted 2 lines
only function symbols in function mappings and variables in data mapping based on the address it accesses.
Well, the lookup should figure out if the IP refers to TEXT or DATA and use MAP__{FUNCTION, VARIABLE} accordingly when asking for symbol resolution.
Right. But in this case we cannot determine whether a symbol in the
/tmp/perf-... file is a function or variable.
That is why we would then need to have separate /tmp/perf-... files, disambiguated by an extension, one for text addresses, another for data ones.
Or change the format to match /proc/kallsyms, which probably is what we should've done from day one...
Detecting the format change would be trivial, as we would find 3 tokens in the new format, instead of the current 2.
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines
What I wasn't sure is whether JIT program also produces some dynamic data. And I think only perf mem command cares about data mappings, no?
Well, I think it would be great to do that kind of data resolution for JITs the same way it is interesting to do for ELF ones :-)
Quoted 6 lines
Quoted 3 lines
I need to stare harder at that patch, but with the above in mind, do we really have to check if the map is
MAP__FUNCTIONas IIRC this patch does?Not sure. For a JIT case, I guess the mapping is always executable and we don't support data mapping yet, so it seems okay for now.
Ok, agreed, applying the patch, thanks,
- Arnaldo
Quoted 9 lines · Arnaldo Carvalho de Melo
Quoted 2 lines
Right. But in this case we cannot determine whether a symbol in the
/tmp/perf-... file is a function or variable.That is why we would then need to have separate
/tmp/perf-... files, disambiguated by an extension, one for text addresses, another for data ones.Or change the format to match
/proc/kallsyms, which probably is what we should've done from day one...
Yeah. Using the same format would be lovely. The parser could try to detect the new one and could fall back to the old format?
Quoted 2 lines
Detecting the format change would be trivial, as we would find 3 tokens in the new format, instead of the current 2.
Yeah!
Thanks,
Ingo
Quoted 10 lines · Namhyung Kim
Hi Gaurav,
Quoted 5 lines · Gaurav Jain
On 1/16/14, 9:37 AM, "Arnaldo Carvalho de Melo" <[email protected]> wrote:
Does perf support data mappings from perf map files? Could you please share an example of how I may be able to use this.
Quoted 8 lines
IIUC there's no difference between function and data mapping. So you can use same perf map file for both - in fact there's no way to use different map file in a single task. I guess perf will use it to find only function symbols in function mappings and variables in data mapping based on the address it accesses.
What I wasn't sure is whether JIT program also produces some dynamic data. And I think only perf mem command cares about data mappings, no?
I don't think we generate symbols for dynamic data at the moment. There may be uses in the future. I don't know about other JIT programs.
If we can include this behaviour it would be great, but I don't have a way to test it. I'd guess if somebody had a valid use case, they would report the issue. Otherwise, the patch as you've written it works great for me.
Gaurav
Namhyung Kim
maintainerHi Gaurav,
Quoted 7 lines · Gaurav Jain
Quoted 3 lines · Namhyung Kim
What I wasn't sure is whether JIT program also produces some dynamic data. And I think only perf mem command cares about data mappings, no?
I don't think we generate symbols for dynamic data at the moment. There may be uses in the future. I don't know about other JIT programs.
If we can include this behaviour it would be great, but I don't have a way to test it. I'd guess if somebody had a valid use case, they would report the issue. Otherwise, the patch as you've written it works great for me.
Thanks for confirming it and testing the patch!
Thanks,
Namhyung
tip-bot for Namhyung Kim
merge bot[tip:perf/urgent] perf symbols: Fix JIT symbol resolution on heap
In reply to Namhyung Kimperf symbols: Fix JIT symbol resolution on heap
Commit-ID: 578c03c86fadcc6fd7319ddf41dd4d1d88aab77a Gitweb: http://git.kernel.org/tip/578c03c86fadcc6fd7319ddf41dd4d1d88aab77a Author: Namhyung Kim <[email protected]> AuthorDate: Thu, 16 Jan 2014 10:49:31 +0900 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Tue, 21 Jan 2014 10:56:05 -0300
perf symbols: Fix JIT symbol resolution on heap
Gaurav reported that perf cannot profile JIT program if it executes the code on heap. This was because current map__new() only handle JIT on anon mappings - extends it to handle no_dso (heap, stack) case too.
This patch assumes JIT profiling only provides dynamic function symbols so check the mapping type to distinguish the case. It'd provide no symbols for data mapping - if we need to support symbols on data mappings later it should be changed.
--- tools/perf/util/map.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/util/map.c +2 -2
@@ -69,7 +69,7 @@ struct map *map__new(struct list_head *dsos__list, u64 start, u64 len, map->ino = ino; map->ino_generation = ino_gen; -if (anon) { +if ((anon || no_dso) && type ==MAP__FUNCTION ) {snprintf (newfilename,sizeof (newfilename),"/tmp/perf-%d.map" , pid); filename = newfilename; }
@@ -93,7 +93,7 @@ struct map *map__new(struct list_head *dsos__list, u64 start, u64 len,* functions still return NULL, and we avoid the * unnecessary map__load warning. */ -if (no_dso) +if (type !=MAP__FUNCTION )dso__set_loaded (dso, map->type); } }
Quoted 11 lines · Namhyung Kim
Gaurav reported that perf cannot profile JIT program if it executes the code on heap. This was because current
map__new()only handle JIT on anon mappings - extends it to handleno_dso(heap, stack) case too.This patch assumes JIT profiling only provides dynamic function symbols so check the mapping type to distinguish the case. It'd provide no symbols for data mapping - if we need to support symbols on data mappings later it should be changed.
Reported-by Gaurav Jain <[email protected]>Signed-off-by Namhyung Kim <[email protected]>
Looks OK to me. Gaurav, does this fix your problem?
Quoted 26 lines
--- tools/perf/util/map.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)tools/perf/util/map.c +2
-2@@ -69,7 +69,7 @@ struct map *map__new(struct list_head *dsos__list, u64 start, u64 len, map->ino = ino; map->ino_generation = ino_gen; -if (anon) { +if ((anon || no_dso) && type ==MAP__FUNCTION ) {snprintf (newfilename,sizeof (newfilename),"/tmp/perf-%d.map" , pid); filename = newfilename; }@@ -93,7 +93,7 @@ struct map *map__new(struct list_head *dsos__list, u64 start, u64 len,* functions still return NULL, and we avoid the * unnecessary map__load warning. */ -if (no_dso) +if (type !=MAP__FUNCTION )dso__set_loaded (dso, map->type); } }
Em Fri, Jan 31, 2014 at 01:50:27PM +0200, Pekka Enberg escreveu:
Quoted 10 lines · Namhyung Kim
Quoted 9 lines
Gaurav reported that perf cannot profile JIT program if it executes the code on heap. This was because current
map__new()only handle JIT on anon mappings - extends it to handleno_dso(heap, stack) case too.This patch assumes JIT profiling only provides dynamic function symbols so check the mapping type to distinguish the case. It'd provide no symbols for data mapping - if we need to support symbols on data mappings later it should be changed.
Quoted 4 lines
Quoted 2 lines
Reported-by Gaurav Jain <[email protected]>Signed-off-by Namhyung Kim <[email protected]>Looks OK to me. Gaurav, does this fix your problem?
Yes, he tested it, etc, its already in Linus' tree:
- Arnaldo
Quoted 25 lines
Quoted 25 lines
--- tools/perf/util/map.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)tools/perf/util/map.c +2
-2@@ -69,7 +69,7 @@ struct map *map__new(struct list_head *dsos__list, u64 start, u64 len, map->ino = ino; map->ino_generation = ino_gen; -if (anon) { +if ((anon || no_dso) && type ==MAP__FUNCTION ) {snprintf (newfilename,sizeof (newfilename),"/tmp/perf-%d.map" , pid); filename = newfilename; }@@ -93,7 +93,7 @@ struct map *map__new(struct list_head *dsos__list, u64 start, u64 len,* functions still return NULL, and we avoid the * unnecessary map__load warning. */ -if (no_dso) +if (type !=MAP__FUNCTION )dso__set_loaded (dso, map->type); } }
Quoted 11 lines · Arnaldo Carvalho de Melo
Em Fri, Jan 31, 2014 at 01:50:27PM +0200, Pekka Enberg escreveu:
Quoted 10 lines · Namhyung Kim
Quoted 9 lines
Gaurav reported that perf cannot profile JIT program if it executes the code on heap. This was because current
map__new()only handle JIT on anon mappings - extends it to handleno_dso(heap, stack) case too.This patch assumes JIT profiling only provides dynamic function symbols so check the mapping type to distinguish the case. It'd provide no symbols for data mapping - if we need to support symbols on data mappings later it should be changed.
Quoted 6 lines
Quoted 3 lines
Quoted 2 lines
Reported-by Gaurav Jain <[email protected]>Signed-off-by Namhyung Kim <[email protected]>Looks OK to me. Gaurav, does this fix your problem?
Yes, he tested it, etc, its already in Linus' tree:
Oh, great! I was going through my inbox and didn't notice it because I got dropped from the CC list. :-)
Thanks Arnaldo!
Pekka