ftrace: Proposal for an Alternative RecordMcount framework
Hi Steven,
Current recordmcount framework collects the mcount call-sites by grep'ing the relocation info in each *.o file right after it is compiled, and then puts them into the __mcount_loc_start array. This works fine in many architectures, but as mentioned in this riscv/ftrace patch[1], aggressive relaxing optimizations corrupt the collected offsets, resulting in panics due to wrong call-site patching in runtime.
Meanwhile, some architectures, such as RISC-V and the on-going nds32, highly rely on linker relaxation as a link-time optimization to reduce code size and improve performance. It would be very undesirable to sacrifice them for ftrace only. But, why can't we collect the call-sites after all of them are fixed?
We propose an alternative framework, for architectures that cannot properly record call-sites because of relaxing. Here is the rough procedure:
1. During the final linking stages, do "objdump vmlinux.o | grep ..." [2]
2. Form the output as an ELF objecj
3. Link the object to __mcount_loc_start symbol
4. Done
With the similar reason as the patch [3], we should mark _mcount to be a weak symbol to prevent it from being relaxed later.
We would like to know your opinion and comments on this. Thanks!
Alan Kao
[1] riscv/ftrace dynamic support [patch v4 1/6]:
https://lkml.org/lkml/2018/2/13/12
[2] This used to not collect some call-sites since their jumps has no target symbol hint. It becomes possible after the fix in 2.30 release. See https://github.com/riscv/riscv-binutils-gdb/issues/129 for more details.
[3] https://lkml.org/lkml/2016/5/14/101
Quoted 1 line · On Tue, 27 Feb 2018 18:04:26 +0800 Alan Kao <[email protected]> wrote:
1. During the final linking stages, do "objdump vmlinux.o | grep ..." [2]
Note, doing it at that stage takes the longest time. It makes small changes much longer to compile. That said...
Quoted 9 lines
2. Form the output as an ELF objecj
3. Link the object to__mcount_loc_startsymbol
4. DoneWith the similar reason as the patch [3], we should mark
_mcountto be a weak symbol to prevent it from being relaxed later.We would like to know your opinion and comments on this. Thanks!
What about just having your arch use recordmcount.c instead? It doesn't do any grepping. It is an elf reader and modifier and modifies the .o file directly.
Note, I will be rewriting that code in the near future too, to consolidate it with objtool.
-- Steve
Quoted 8 lines · Steven Rostedt
Quoted 1 line · On Tue, 27 Feb 2018 18:04:26 +0800 Alan Kao <[email protected]> wrote:
1. During the final linking stages, do "objdump vmlinux.o | grep ..." [2]
Note, doing it at that stage takes the longest time. It makes small changes much longer to compile. That said...
What if we can have some option to *disable* all the recordmcount.pl lauches after every .o? There will be only an oneshot grep for a near-complete vmlinux binary.
Quoted 7 lines
Quoted 7 lines
2. Form the output as an ELF objecj
3. Link the object to__mcount_loc_startsymbol
4. DoneWith the similar reason as the patch [3], we should mark
_mcountto be a weak symbol to prevent it from being relaxed later.
Quoted 6 lines
Quoted 2 lines
We would like to know your opinion and comments on this. Thanks!
What about just having your arch use
recordmcount.cinstead? It doesn't do any grepping. It is an elf reader and modifier and modifies the .o file directly.
Thanks for the hint. But after a quick scan, it seems that recordmcount.c processes .o files in a per-file basis, which means that we will still suffer from the linker relaxation problem.
Quoted 5 lines
Note, I will be rewriting that code in the near future too, to consolidate it with objtool.
-- Steve
Please allow me to state the problem more clearly here. I hope this helps.
1. locations of mcount are recorded in a per-file basis.
2. to optimize the binary, the linker turns on some aggressive options, including relaxation.
3. the optimizations changes the original offset.
4. already recorded mcount call-sites no longer point to their real positions.
5. still a linked vmlinux is made.
6. dynamic ftrace breaks the real logic in the kernel space, panics happen.
Thanks,
Alan
Quoted 50 lines · Alan Kao
Quoted 8 lines · Steven Rostedt
Quoted 1 line · On Tue, 27 Feb 2018 18:04:26 +0800 Alan Kao <[email protected]> wrote:
1. During the final linking stages, do "objdump vmlinux.o | grep ..." [2]
Note, doing it at that stage takes the longest time. It makes small changes much longer to compile. That said...
What if we can have some option to *disable* all the recordmcount.pl lauches after every .o? There will be only an oneshot grep for a near-complete vmlinux binary.
Quoted 7 lines
Quoted 7 lines
2. Form the output as an ELF objecj
3. Link the object to__mcount_loc_startsymbol
4. DoneWith the similar reason as the patch [3], we should mark
_mcountto be a weak symbol to prevent it from being relaxed later.Quoted 6 lines
Quoted 2 lines
We would like to know your opinion and comments on this. Thanks!
What about just having your arch use
recordmcount.cinstead? It doesn't do any grepping. It is an elf reader and modifier and modifies the .o file directly.Thanks for the hint. But after a quick scan, it seems that
recordmcount.cprocesses .o files in a per-file basis, which means that we will still suffer from the linker relaxation problem.Quoted 5 lines
Note, I will be rewriting that code in the near future too, to consolidate it with objtool.
-- Steve
Please allow me to state the problem more clearly here. I hope this helps.
1. locations of mcount are recorded in a per-file basis.
2. to optimize the binary, the linker turns on some aggressive options, including relaxation.
3. the optimizations changes the original offset.
4. already recorded mcount call-sites no longer point to their real positions.
5. still a linked vmlinux is made.
6. dynamic ftrace breaks the real logic in the kernel space, panics happen.
Quoted 2 lines
Thanks,
Alan
Any comments on this?
BTW, the introduced framework has no effects on any other architectures that works fine. This feature should be configurable and turned on only when the arch has aggressive link-time optimizations.
If you consider this appropriate, I will send the patch to this once it gets ready. Currently this is targeting at RISC-V and upcoming NDS32.
Thanks,
Alan
Quoted 12 lines · On Wed, 7 Mar 2018 09:47:47 +0800 Alan Kao <[email protected]> wrote:
Quoted 12 lines
Please allow me to state the problem more clearly here. I hope this helps.
1. locations of mcount are recorded in a per-file basis.
2. to optimize the binary, the linker turns on some aggressive options, including relaxation.
3. the optimizations changes the original offset.
4. already recorded mcount call-sites no longer point to their real positions.
5. still a linked vmlinux is made.
6. dynamic ftrace breaks the real logic in the kernel space, panics happen.
Quoted 4 lines
Quoted 2 lines
Thanks,
AlanAny comments on this?
Sorry, I've been traveling and falling behind in this. I did read this when I was offline but forgot to reply when I was back online.
Quoted 7 lines
BTW, the introduced framework has no effects on any other architectures that works fine. This feature should be configurable and turned on only when the arch has aggressive link-time optimizations.
If you consider this appropriate, I will send the patch to this once it gets ready. Currently this is targeting at RISC-V and upcoming NDS32.
I see the issue you explained above and it makes sense. Perhaps make it an option that can be enabled by anyone, and archs that require aggressive link time optimizations would just select it.
-- Steve