Skip to contentSkip to conversation

klp-build: Update demangle_name for LTO

PATCH In review Song Liu 2 participants 5 messages Feb 3 – Feb 4

Subsystems (2)
lore
Share & tools
lore

Song Liu

author maintainer
#0 lore raw

PATCH 1/1 klp-build: Update demangle_name for LTO

With CONFIG_LTO_CLANG_THIN, __UNIQUE_ID_* can be global. Therefore, it is necessary to demangle global symbols.

Also, LTO may generate symbols like:

__UNIQUE_ID_addressable___UNIQUE_ID_pci_invalid_bar_694_695

Remove trailing '_' together with numbers and '.' so that both numbers added to the end of the symbol are removed. For example, the above s ymbol will be demangled as

__UNIQUE_ID_addressable___UNIQUE_ID_pci_invalid_bar

---
 tools/objtool/elf.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
tools/objtool/elf.c +7 -4
index 2c02c7b49265..b4a7ea4720e1 100644
@@ -445,9 +445,6 @@ static const char *demangle_name(struct symbol *sym)
 {
 	char *str;
 
-	if (!is_local_sym(sym))
-		return sym->name;
-
 	if (!is_func_sym(sym) && !is_object_sym(sym))
 		return sym->name;
 
@@ -463,7 +460,13 @@ static const char *demangle_name(struct symbol *sym)
 	for (int i = strlen(str) - 1; i >= 0; i--) {
 		char c = str[i];
 
-		if (!isdigit(c) && c != '.') {
+		/*
+		 * With CONFIG_LTO_CLANG_THIN, the UNIQUE_ID field could
+		 * be like:
+		 *   __UNIQUE_ID_addressable___UNIQUE_ID_<name>_628_629
+		 * Remove all the trailing number, '.', and '_'.
+		 */
+		if (!isdigit(c) && c != '.' && c != '_') {
 			str[i + 1] = '\0';
 			break;
 		}
Signature · 1 line
2.47.3
maintainer
#1 lore raw
Quoted 2 lines · Song Liu

With CONFIG_LTO_CLANG_THIN, __UNIQUE_ID_* can be global. Therefore, it is necessary to demangle global symbols.

Ouch, so LTO is changing symbol bindings :-/

If a patch causes a symbol to change from LOCAL to GLOBAL between the original and patched builds, that will break some fundamental assumptions in the correlation logic.

Also, notice sym->demangled_name isn't used for correlating global symbols in correlate_symbols(). That code currently assumes all global symbols are uniquely named (and don't change between orig and patched). So this first fix seems incomplete.

Quoted 1 line

Also, LTO may generate symbols like:

The "also" is a clue that this should probably be two separate patches.

Also, for objtool patches, please prefix the subject with "objtool:", or in this case, for klp-specific code, "objtool/klp:".

Quoted 7 lines

__UNIQUE_ID_addressable___UNIQUE_ID_pci_invalid_bar_694_695

Remove trailing '_' together with numbers and '.' so that both numbers added to the end of the symbol are removed. For example, the above s ymbol will be demangled as

__UNIQUE_ID_addressable___UNIQUE_ID_pci_invalid_bar

This is indeed a bug in demangle_name(), but not specific to LTO. The unique number is added by the __UNIQUE_ID() macro.

I guess in this case LTO is doing some kind of nested __UNIQUE_ID() to get two "__UNIQUE_ID" strings and two numbers? But the bug still exists for the non-nested case.

Quoted 20 lines
---
 tools/objtool/elf.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
tools/objtool/elf.c +0 -3
index 2c02c7b49265..b4a7ea4720e1 100644
@@ -445,9 +445,6 @@ static const char *demangle_name(struct symbol *sym)
 {
 	char *str;
 
-	if (!is_local_sym(sym))
-		return sym->name;
-
 	if (!is_func_sym(sym) && !is_object_sym(sym))
 		return sym->name;
 

tools/objtool/elf.c static const char *demangle_name(struct symbol *sym) · line 468

Quoted 10 lines
tools/objtool/elf.c +6 -1
@@ -463,7 +460,13 @@ static const char *demangle_name(struct symbol *sym)
 	for (int i = strlen(str) - 1; i >= 0; i--) {
 		char c = str[i];
 
-		if (!isdigit(c) && c != '.') {
+		/*
+		 * With CONFIG_LTO_CLANG_THIN, the UNIQUE_ID field could
+		 * be like:
+		 *   __UNIQUE_ID_addressable___UNIQUE_ID_<name>_628_629
+		 * Remove all the trailing number, '.', and '_'.
+		 */

A comment is indeed probably warranted, though I'm thinking it should instead go above the function, with examples of both __UNIQUE_ID and "." symbols.

tools/objtool/elf.c static const char *demangle_name(struct symbol *sym) · line 469

Quoted 1 line
tools/objtool/elf.c +1 -0
+		if (!isdigit(c) && c != '.' && c != '_') {

Ack.

Signature · 1 line
Josh
author maintainer
#2 lore raw
Quoted 10 lines · Josh Poimboeuf
Quoted 2 lines · Song Liu

With CONFIG_LTO_CLANG_THIN, __UNIQUE_ID_* can be global. Therefore, it is necessary to demangle global symbols.

Ouch, so LTO is changing symbol bindings :-/

If a patch causes a symbol to change from LOCAL to GLOBAL between the original and patched builds, that will break some fundamental assumptions in the correlation logic.

This can indeed happen. A function can be "LOCAL DEFAULT" XXXX in original, and "GLOBAL HIDDEN" XXXX.llvm.<hash> in patched.

I am trying to fix this with incremental steps.

Quoted 4 lines

Also, notice sym->demangled_name isn't used for correlating global symbols in correlate_symbols(). That code currently assumes all global symbols are uniquely named (and don't change between orig and patched). So this first fix seems incomplete.

We still need to fix correlate_symbols(). I am not 100% sure how to do that part yet.

OTOH, this part still helps. This is because checksum_update_insn() uses demangled_name. After the fix, if a function is renamed from XXXX to XXXX.llvm.<hash> after the patch, functions that call the function are not considered changed.

Quoted 13 lines
Quoted 1 line

Also, LTO may generate symbols like:

The "also" is a clue that this should probably be two separate patches.

Also, for objtool patches, please prefix the subject with "objtool:", or in this case, for klp-specific code, "objtool/klp:".

Quoted 6 lines

__UNIQUE_ID_addressable___UNIQUE_ID_pci_invalid_bar_694_695

Remove trailing '_' together with numbers and '.' so that both numbers added to the end of the symbol are removed. For example, the above s ymbol will be demangled as

Quoted 8 lines
Quoted 1 line

__UNIQUE_ID_addressable___UNIQUE_ID_pci_invalid_bar

This is indeed a bug in demangle_name(), but not specific to LTO. The unique number is added by the __UNIQUE_ID() macro.

I guess in this case LTO is doing some kind of nested __UNIQUE_ID() to get two "__UNIQUE_ID" strings and two numbers? But the bug still exists for the non-nested case.

I don't see nested __UNIQUE_ID() without LTO. Both gcc and clang without LTO only sees one level of __UNIQUE_ID.

With one level of __UNIQUE_ID(), existing code works fine. We just get one extra "_" at the end of the demanged_name.

Quoted 24 lines
Quoted 24 lines
---
 tools/objtool/elf.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
tools/objtool/elf.c +0 -3
index 2c02c7b49265..b4a7ea4720e1 100644
@@ -445,9 +445,6 @@ static const char *demangle_name(struct symbol *sym)
 {
      char *str;
 
-     if (!is_local_sym(sym))
-             return sym->name;
-
      if (!is_func_sym(sym) && !is_object_sym(sym))
              return sym->name;
 
@@ -463,7 +460,13 @@ static const char *demangle_name(struct symbol *sym)
      for (int i = strlen(str) - 1; i >= 0; i--) {
              char c = str[i];
 
Quoted 11 lines

tools/objtool/elf.c static const char *demangle_name(struct symbol *sym) · line 468

Quoted 7 lines
tools/objtool/elf.c +6 -1
-             if (!isdigit(c) && c != '.') {
+             /*
+              * With CONFIG_LTO_CLANG_THIN, the UNIQUE_ID field could
+              * be like:
+              *   __UNIQUE_ID_addressable___UNIQUE_ID_<name>_628_629
+              * Remove all the trailing number, '.', and '_'.
+              */

A comment is indeed probably warranted, though I'm thinking it should instead go above the function, with examples of both __UNIQUE_ID and "." symbols.

I will split the patch into two, and fix the comment and commit log.

Thanks,
Song

maintainer
#3 lore raw
Quoted 22 lines · Song Liu
Quoted 10 lines · Josh Poimboeuf
Quoted 2 lines · Song Liu

With CONFIG_LTO_CLANG_THIN, __UNIQUE_ID_* can be global. Therefore, it is necessary to demangle global symbols.

Ouch, so LTO is changing symbol bindings :-/

If a patch causes a symbol to change from LOCAL to GLOBAL between the original and patched builds, that will break some fundamental assumptions in the correlation logic.

This can indeed happen. A function can be "LOCAL DEFAULT" XXXX in original, and "GLOBAL HIDDEN" XXXX.llvm.<hash> in patched.

I am trying to fix this with incremental steps.

Quoted 4 lines

Also, notice sym->demangled_name isn't used for correlating global symbols in correlate_symbols(). That code currently assumes all global symbols are uniquely named (and don't change between orig and patched). So this first fix seems incomplete.

Quoted 7 lines

We still need to fix correlate_symbols(). I am not 100% sure how to do that part yet.

OTOH, this part still helps. This is because checksum_update_insn() uses demangled_name. After the fix, if a function is renamed from XXXX to XXXX.llvm.<hash> after the patch, functions that call the function are not considered changed.

Hm, wouldn't that still leave the .llvm at the end?

Quoted 23 lines
Quoted 14 lines
Quoted 2 lines

Also, LTO may generate symbols like:

The "also" is a clue that this should probably be two separate patches.

Also, for objtool patches, please prefix the subject with "objtool:", or in this case, for klp-specific code, "objtool/klp:".

Quoted 6 lines

__UNIQUE_ID_addressable___UNIQUE_ID_pci_invalid_bar_694_695

Remove trailing '_' together with numbers and '.' so that both numbers added to the end of the symbol are removed. For example, the above s ymbol will be demangled as

Quoted 8 lines
Quoted 1 line

__UNIQUE_ID_addressable___UNIQUE_ID_pci_invalid_bar

This is indeed a bug in demangle_name(), but not specific to LTO. The unique number is added by the __UNIQUE_ID() macro.

I guess in this case LTO is doing some kind of nested __UNIQUE_ID() to get two "__UNIQUE_ID" strings and two numbers? But the bug still exists for the non-nested case.

Quoted 5 lines

I don't see nested __UNIQUE_ID() without LTO. Both gcc and clang without LTO only sees one level of __UNIQUE_ID.

With one level of __UNIQUE_ID(), existing code works fine. We just get one extra "_" at the end of the demanged_name.

Ah, I see.

Signature · 1 line
Josh
author maintainer
#4 lore raw
Quoted 27 lines · Josh Poimboeuf
Quoted 25 lines · Song Liu
Quoted 10 lines · Josh Poimboeuf
Quoted 2 lines · Song Liu

With CONFIG_LTO_CLANG_THIN, __UNIQUE_ID_* can be global. Therefore, it is necessary to demangle global symbols.

Ouch, so LTO is changing symbol bindings :-/

If a patch causes a symbol to change from LOCAL to GLOBAL between the original and patched builds, that will break some fundamental assumptions in the correlation logic.

This can indeed happen. A function can be "LOCAL DEFAULT" XXXX in original, and "GLOBAL HIDDEN" XXXX.llvm.<hash> in patched.

I am trying to fix this with incremental steps.

Quoted 4 lines

Also, notice sym->demangled_name isn't used for correlating global symbols in correlate_symbols(). That code currently assumes all global symbols are uniquely named (and don't change between orig and patched). So this first fix seems incomplete.

We still need to fix correlate_symbols(). I am not 100% sure how to do that part yet.

Quoted 6 lines
Quoted 4 lines

OTOH, this part still helps. This is because checksum_update_insn() uses demangled_name. After the fix, if a function is renamed from XXXX to XXXX.llvm.<hash> after the patch, functions that call the function are not considered changed.

Hm, wouldn't that still leave the .llvm at the end?

E.. Right. I described a different case.

The actual case being fixed here is when foo.llvm.<hash_1> got changed to foo.llvm.<hash_2>.

Let me think more about all the cases and have a better solution.

Thanks,
Song