Skip to contentSkip to conversation

perf symbols: Fix module symbol resolution for non-zero .text sh_addr

PATCH Applied Chuck Lever 4 participants 4 messages Mar 23 – Apr 2 perf events

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 perf symbols: Fix module symbol resolution for non-zero .text sh_addr

From: Chuck Lever <[email protected]>

When perf resolves symbols from kernel module ELF files (ET_REL), it converts symbol addresses to file offsets so that sample IPs can be matched to the correct symbol. The conversion adjusts each symbol's st_value:

  sym->st_value -= shdr->sh_addr - shdr->sh_offset;

For vmlinux (ET_EXEC), st_value is a virtual address and sh_addr is the section's virtual base, so subtracting sh_addr and adding sh_offset correctly yields a file offset.

For kernel modules (ET_REL), st_value is a section-relative offset. The module loader ignores sh_addr entirely and places symbols at module_base + st_value. Converting to file offset requires only adding sh_offset; subtracting sh_addr introduces an error equal to sh_addr bytes.

When .text has sh_addr == 0 -- the historical norm for simple modules -- both formulas produce the same result and the bug is latent. As modules gain more metadata sections before .text (.note, .static_call.text, etc.), the linker assigns .text a non-zero sh_addr, exposing the defect. For example, nfsd.ko on this kernel has sh_addr=0xa80, kvm-intel.ko has sh_addr=0x1e90.

The effect is that all .text symbols in affected modules shift by sh_addr bytes relative to sample IPs, causing perf report to attribute samples to incorrect, nearby symbols. This was observed as 13% of LLC-load-miss samples misattributed to nfsd_file_get_dio_attrs when the actual hot function was nfsd_cache_lookup, approximately 0xa80 bytes away in the symbol table.

Use the existing dso__rel() flag (already set for ET_REL modules) to select the correct adjustment: add sh_offset for ET_REL, subtract (sh_addr - sh_offset) for ET_EXEC/ET_DYN.

Fixes 0131c4ec794a ("perf tools: Make it possible to read object code from kernel modules")
---
 tools/perf/util/symbol-elf.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
tools/perf/util/symbol-elf.c +6 -2
index 76912c62b6a0..968e269d9be1 100644
@@ -1356,8 +1356,12 @@ static int dso__process_kernel_symbol(struct dso *dso, struct map *map,
 	char dso_name[PATH_MAX];
 
 	/* Adjust symbol to map to file offset */
-	if (adjust_kernel_syms)
-		sym->st_value -= shdr->sh_addr - shdr->sh_offset;
+	if (adjust_kernel_syms) {
+		if (dso__rel(dso))
+			sym->st_value += shdr->sh_offset;
+		else
+			sym->st_value -= shdr->sh_addr - shdr->sh_offset;
+	}
 
 	if (strcmp(section_name, (dso__short_name(curr_dso) + dso__short_name_len(dso))) == 0)
 		return 0;
Signature · 1 line
2.53.0
Tested-by on v1 1/1
#1 lore raw
Quoted 53 lines · Chuck Lever

From: Chuck Lever <[email protected]>

When perf resolves symbols from kernel module ELF files (ET_REL), it converts symbol addresses to file offsets so that sample IPs can be matched to the correct symbol. The conversion adjusts each symbol's st_value:

  sym->st_value -= shdr->sh_addr - shdr->sh_offset;

For vmlinux (ET_EXEC), st_value is a virtual address and sh_addr is the section's virtual base, so subtracting sh_addr and adding sh_offset correctly yields a file offset.

For kernel modules (ET_REL), st_value is a section-relative offset. The module loader ignores sh_addr entirely and places symbols at module_base + st_value. Converting to file offset requires only adding sh_offset; subtracting sh_addr introduces an error equal to sh_addr bytes.

When .text has sh_addr == 0 -- the historical norm for simple modules -- both formulas produce the same result and the bug is latent. As modules gain more metadata sections before .text (.note, .static_call.text, etc.), the linker assigns .text a non-zero sh_addr, exposing the defect. For example, nfsd.ko on this kernel has sh_addr=0xa80, kvm-intel.ko has sh_addr=0x1e90.

The effect is that all .text symbols in affected modules shift by sh_addr bytes relative to sample IPs, causing perf report to attribute samples to incorrect, nearby symbols. This was observed as 13% of LLC-load-miss samples misattributed to nfsd_file_get_dio_attrs when the actual hot function was nfsd_cache_lookup, approximately 0xa80 bytes away in the symbol table.

Use the existing dso__rel() flag (already set for ET_REL modules) to select the correct adjustment: add sh_offset for ET_REL, subtract (sh_addr - sh_offset) for ET_EXEC/ET_DYN.

Fixes 0131c4ec794a ("perf tools: Make it possible to read object code from kernel modules")
---
 tools/perf/util/symbol-elf.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
tools/perf/util/symbol-elf.c +0 -1
index 76912c62b6a0..968e269d9be1 100644
@@ -1356,8 +1356,12 @@ static int dso__process_kernel_symbol(struct dso *dso, struct map *map,
 	char dso_name[PATH_MAX];
 
 	/* Adjust symbol to map to file offset */
-	if (adjust_kernel_syms)

tools/perf/util/symbol-elf.c static int dso__process_kernel_symbol(struct dso *dso, struct map *map, · line 1367

Quoted 10 lines
tools/perf/util/symbol-elf.c +6 -1
-		sym->st_value -= shdr->sh_addr - shdr->sh_offset;
+	if (adjust_kernel_syms) {
+		if (dso__rel(dso))
+			sym->st_value += shdr->sh_offset;
+		else
+			sym->st_value -= shdr->sh_addr - shdr->sh_offset;
+	}
 
 	if (strcmp(section_name, (dso__short_name(curr_dso) + dso__short_name_len(dso))) == 0)
 		return 0;

For s390

Signature · 9 lines
Thomas Richter, Dept 3303, IBM s390 Linux Development, Boeblingen, Germany
--
IBM Deutschland Research & Development GmbH

Vorsitzender des Aufsichtsrats: Wolfgang Wendt

Geschäftsführung: David Faller

Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart, HRB 243294
reviewer Reviewed-by
#2 lore raw
Quoted 63 lines · Thomas Richter
Quoted 61 lines · Chuck Lever

From: Chuck Lever <[email protected]>

When perf resolves symbols from kernel module ELF files (ET_REL), it converts symbol addresses to file offsets so that sample IPs can be matched to the correct symbol. The conversion adjusts each symbol's st_value:

  sym->st_value -= shdr->sh_addr - shdr->sh_offset;

For vmlinux (ET_EXEC), st_value is a virtual address and sh_addr is the section's virtual base, so subtracting sh_addr and adding sh_offset correctly yields a file offset.

For kernel modules (ET_REL), st_value is a section-relative offset. The module loader ignores sh_addr entirely and places symbols at module_base + st_value. Converting to file offset requires only adding sh_offset; subtracting sh_addr introduces an error equal to sh_addr bytes.

When .text has sh_addr == 0 -- the historical norm for simple modules -- both formulas produce the same result and the bug is latent. As modules gain more metadata sections before .text (.note, .static_call.text, etc.), the linker assigns .text a non-zero sh_addr, exposing the defect. For example, nfsd.ko on this kernel has sh_addr=0xa80, kvm-intel.ko has sh_addr=0x1e90.

The effect is that all .text symbols in affected modules shift by sh_addr bytes relative to sample IPs, causing perf report to attribute samples to incorrect, nearby symbols. This was observed as 13% of LLC-load-miss samples misattributed to nfsd_file_get_dio_attrs when the actual hot function was nfsd_cache_lookup, approximately 0xa80 bytes away in the symbol table.

Use the existing dso__rel() flag (already set for ET_REL modules) to select the correct adjustment: add sh_offset for ET_REL, subtract (sh_addr - sh_offset) for ET_EXEC/ET_DYN.

Fixes 0131c4ec794a ("perf tools: Make it possible to read object code from kernel modules")
---
 tools/perf/util/symbol-elf.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
tools/perf/util/symbol-elf.c +6 -2
index 76912c62b6a0..968e269d9be1 100644
@@ -1356,8 +1356,12 @@ static int dso__process_kernel_symbol(struct dso *dso, struct map *map,
      char dso_name[PATH_MAX];
 
      /* Adjust symbol to map to file offset */
-     if (adjust_kernel_syms)
-             sym->st_value -= shdr->sh_addr - shdr->sh_offset;
+     if (adjust_kernel_syms) {
+             if (dso__rel(dso))
+                     sym->st_value += shdr->sh_offset;
+             else
+                     sym->st_value -= shdr->sh_addr - shdr->sh_offset;
+     }
 
Quoted 6 lines

tools/perf/util/symbol-elf.c static int dso__process_kernel_symbol(struct dso *dso, struct map *map, · line 1367

Quoted 2 lines
tools/perf/util/symbol-elf.c +0 -0
      if (strcmp(section_name, (dso__short_name(curr_dso) + dso__short_name_len(dso))) == 0)
              return 0;

For s390

Thanks,
Ian

Quoted 11 lines

--
Thomas Richter, Dept 3303, IBM s390 Linux Development, Boeblingen, Germany --
IBM Deutschland Research & Development GmbH

Vorsitzender des Aufsichtsrats: Wolfgang Wendt

Geschäftsführung: David Faller

Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart, HRB 243294

maintainer Applied
#3 lore raw
In reply to Chuck Lever
Quoted 8 lines · Chuck Lever

When perf resolves symbols from kernel module ELF files (ET_REL), it converts symbol addresses to file offsets so that sample IPs can be matched to the correct symbol. The conversion adjusts each symbol's st_value:

  sym->st_value -= shdr->sh_addr - shdr->sh_offset;

[...]

Applied to perf-tools-next, thanks!

Best regards,
Namhyung