perf confused by modules being loaded in between addresses in /proc/kallsyms
Hi,
On arm, probably since "ARM: use linker magic for vectors and vector stubs" (although I haven't checked this), perf report tends to allocate all kernel time to the module loaded at the highest address.
This turns out to be because the "perf_event__synthesize_modules" generates a PERF_RECORD_MMAP for the last module thats runs from its start to the end of the address space.
This in turn turns out be because the first symbol in /proc/kallsyms is now at 0:
# head -n 2 /proc/kallsyms 00000000 t __vectors_start c00081c0 T asm_do_IRQ
/This/ means that the kallsyms entry in machine->kmaps[MAP__FUNCTION] is now the first entry rather than the last in the map, so the last module is the last in the map and so its "end" stays as ~0ULL.
I'm told that there is no particularly good reason for __vectors_start to be in kallsyms at 0, but in any case this seems like a bug in the user space tool. I notice that there is code to split the kallsyms symbols around symbols from modules (dso__split_kallsyms) but this doesn't seem to be called for record, only report. Is the fix as simple as making sure this happens for record as well? I don't feel qualified to answer that.
Cheers,
mwh
Hi Michael,
Quoted 10 lines · Michael Hudson-Doyle
Hi,
On arm, probably since "ARM: use linker magic for vectors and vector stubs" (although I haven't checked this), perf report tends to allocate all kernel time to the module loaded at the highest address.
This turns out to be because the "
perf_event__synthesize_modules" generates aPERF_RECORD_MMAPfor the last module thats runs from its start to the end of the address space.
Quoted 5 lines
This in turn turns out be because the first symbol in
/proc/kallsymsis now at 0:# head -n 2 /proc/kallsyms 00000000 t __vectors_start
I think it is the root cause, so I posted the below patch to remove these symbols from /proc/kallsyms days ago:
http://marc.info/?l=linux-kernel&m=138297540711321&w=2
On ARM, this symbol of zero address is only for generating code, and won't be run really by CPU, so it shouldn't be in /proc/kallsyms.
Quoted 9 lines
c00081c0Tasm_do_IRQ/This/ means that the kallsyms entry in machine->kmaps[MAP__FUNCTION] is now the first entry rather than the last in the map, so the last module is the last in the map and so its "end" stays as ~0ULL.
I'm told that there is no particularly good reason for
__vectors_startto be in kallsyms at 0, but in any case this seems like a bug in the user space tool. I notice that there is code to split the kallsyms
I am wondering if it is bug in perf utility, since perf may use /proc/kallsyms to figure out the start address of symbols in kernel address space, but with the zero address, looks not easy to figure out the real start address of symbols inside kernel if vmlinux file is missed, then can't parse kernel IP any more.
Thanks,
Signature · 1 line
Ming Lei