Skip to contentSkip to conversation

recordmcount commutes with "ld -r"

DISCUSSION John Reiser 2 participants 3 messages Aug 10, 2009 – Aug 17, 2009

lore
Share & tools
lore
#0 lore raw

Executing recordmcount.pl for each *.o is adding minutes to the duration of my full kernel builds. Here is a way to recoup most of those minutes.

recordmcount commutes with "ld -r". Run "ld -r" on the outputs from running recordmcount on each *.o, or run recordmcount on the output from aggregating the original *.o using "ld -r". Either way, the final __mcount_loc section contains a list of locations of calls to mcount. The ELF32_R_SYM (ELF64_R_SYM) of the relocations may be different, but they will be equivalent. Subsequent static binding (ld without -r) will produce identical results. Instead of running recordmcount on each *.o input file that is part of built-in.o or <module>.ko, then just run recordmcount on built-in.o or <module>.ko that is constructed from the original compiler-generated *.o.

There is a special case for building vmlinux, namely the archive libraries lib/lib.a and arch/$ARCH/lib/lib.a. recordmcount must be run on each member individually. Alternately, recordmcount could be run on vmlinux.o (exactly once per build; not on any built-in.o) if vmlinux.o is then used to build vmlinux.

I noticed another property. Logically, recordmcount could modify a .o file in place. Both /bin/ld and the kernel module loader ignore bytes that are not designated by the ElfXX_Shdr[]. The __mcount_loc section and its relocations can be appended to the original file, then "activated" by rewriting the ElfXX_Ehdr fields .e_shnum and .e_shoff. This avoids some file operations as well as several fork+exec that are performed by recordmcount.pl. recordmcount becomes very fast. The bytes for the old ElfXX_Shdr[] remain as uncollected "garbage", typically a few kilobytes in each built-in.o or <module>.ko. If desired then the garbage may be excised quickly by running "ld -r".

I have written recordmcount.c which does such modify-in-place for all architectures supported by recordcmount, and tested it successfully on i686, x86_64, and 32-bit PowerPC, including cross-platform processing of *.o from any architecture. The differing data structures between Elf32 and Elf64 require parallel code in many places, so the C file is 900 lines. That might be too long for a mailing list, so I will defer posting the file.

#1 lore raw

Hi John,

Sorry for the late reply, but I just got back from vacation.

Quoted 13 lines · John Reiser

Executing recordmcount.pl for each *.o is adding minutes to the duration of my full kernel builds. Here is a way to recoup most of those minutes.

recordmcount commutes with "ld -r". Run "ld -r" on the outputs from running recordmcount on each *.o, or run recordmcount on the output from aggregating the original *.o using "ld -r". Either way, the final __mcount_loc section contains a list of locations of calls to mcount. The ELF32_R_SYM (ELF64_R_SYM) of the relocations may be different, but they will be equivalent. Subsequent static binding (ld without -r) will produce identical results. Instead of running recordmcount on each *.o input file that is part of built-in.o or <module>.ko, then just run recordmcount on built-in.o or <module>.ko that is constructed from the original compiler-generated *.o.

Running the linking on the built-in.o and *.ko files is a good idea.

Quoted 6 lines

There is a special case for building vmlinux, namely the archive libraries lib/lib.a and arch/$ARCH/lib/lib.a. recordmcount must be run on each member individually. Alternately, recordmcount could be run on vmlinux.o (exactly once per build; not on any built-in.o) if vmlinux.o is then used to build vmlinux.

I avoided performing on the vmlinux.o. Because myself (and a lot of others) use distcc to compile our kernels. The longest part of the average compile is the final linking stage of vmlinux.o. Most of my compiles are done when I only modify a couple of files, and the work on individual .o files is not that big of a deal. But to do the work on vmlinux, will increase the compile time for all compiles even if you only modified a single file.

Quoted 12 lines

I noticed another property. Logically, recordmcount could modify a .o file in place. Both /bin/ld and the kernel module loader ignore bytes that are not designated by the ElfXX_Shdr[]. The __mcount_loc section and its relocations can be appended to the original file, then "activated" by rewriting the ElfXX_Ehdr fields .e_shnum and .e_shoff. This avoids some file operations as well as several fork+exec that are performed by recordmcount.pl. recordmcount becomes very fast. The bytes for the old ElfXX_Shdr[] remain as uncollected "garbage", typically a few kilobytes in each built-in.o or <module>.ko. If desired then the garbage may be excised quickly by running "ld -r".

Quoted 7 lines

I have written recordmcount.c which does such modify-in-place for all architectures supported by recordcmount, and tested it successfully on i686, x86_64, and 32-bit PowerPC, including cross-platform processing of *.o from any architecture. The differing data structures between Elf32 and Elf64 require parallel code in many places, so the C file is 900 lines. That might be too long for a mailing list, so I will defer posting the file.

I would be interested in seeing it. Do you require any of the elf libraries to use this? If so, that would make the kernel build dependent on having the development elf libraries installed.

I've thought about converting recordmcount into a C file before, but I was a bit hesitant on rewriting elf routines (although I've done it before and they are quite trivial) but even more concerned on breaking other archs.

-- Steve

#2 lore raw
Quoted 5 lines · Steven Rostedt
Quoted 1 line · John Reiser

recordmcount commutes with "ld -r". ...

Running the linking on the built-in.o and *.ko files is a good idea.

I'll see what it takes to patch scripts/Makefile.build and any other build-control file that is required.

Quoted 7 lines
Quoted 3 lines

I noticed another property. Logically, recordmcount could modify a .o file in place. ...
I have written recordmcount.c which does such modify-in-place ...

I would be interested in seeing it. Do you require any of the elf libraries to use this? If so, that would make the kernel build dependent on having the development elf libraries installed.

No, my recordmcount.c is completely standalone except for libc.so. I'll sent it to you (and anyone else who asks) separately.

Quoted 4 lines

I've thought about converting recordmcount into a C file before, but I was a bit hesitant on rewriting elf routines (although I've done it before and they are quite trivial) but even more concerned on breaking other archs.

Yes, the major question is whether R_ARM_ABS32, R_IA64_IMM64, R_PPC64_ADDR64, R_SH_DIR32, R_SPARC_64, R_390_64, and R_390_32 are the correct relocation enums for a plain address. That will require testers...