Skip to contentSkip to conversation

build: fix linker warnings about undefined symbols

PATCH Applied Bruce Richardson 6 participants 11 messages Jan 10, 2024 – Feb 20, 2024 Arm/Amlogic Meson SoC support

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 build: fix linker warnings about undefined symbols

The default behaviour of "ld.lld" has changed, so it now prints out warnings about entries in the version.map file which don't exist in the current build. Since we use our version.map file simply to filter out the functions we don't want made public, we include in it all functions across all OS's and builds that we want public if present. This causes these ld warnings to be emitted, e.g. on BSD, which is missing functionality found on Linux. For example:

* hpet functions in EAL
* regexdev enqueue and dequeue burst
* eventdev event_timer functions

Easiest solution, without major rework of how we use our version.map files, and without dynamically generating them per-build, is to pass the --undefined-version flag to the linker, to restore the old behaviour.

---
 config/meson.build | 3 +++
 1 file changed, 3 insertions(+)
config/meson.build +3 -0
index 73737921c3..cfc29ba757 100644
@@ -187,6 +187,9 @@ dpdk_conf.set('RTE_ARCH_32', cc.sizeof('void *') == 4)
 
 if not is_windows
     add_project_link_arguments('-Wl,--no-as-needed', language: 'c')
+    if cc.has_link_argument('-Wl,--undefined-version')
+        add_project_link_arguments('-Wl,--undefined-version', language: 'c')
+    endif
 endif
 
 # use pthreads if available for the platform
Signature · 1 line
2.40.1
Acked-by on v1 1/1
#1 lore raw
Quoted 12 lines · Bruce Richardson

The default behaviour of "ld.lld" has changed, so it now prints out warnings about entries in the version.map file which don't exist in the current build. Since we use our version.map file simply to filter out the functions we don't want made public, we include in it all functions across all OS's and builds that we want public if present. This causes these ld warnings to be emitted, e.g. on BSD, which is missing functionality found on Linux. For example:

* hpet functions in EAL
* regexdev enqueue and dequeue burst
* eventdev event_timer functions

Quoted 7 lines

Easiest solution, without major rework of how we use our version.map files, and without dynamically generating them per-build, is to pass the --undefined-version flag to the linker, to restore the old behaviour.

---

i don't know if has ever been discussed but a way to achieve a similar outcome would be to introduce a visibility macro allowing the data and function symbols to be explicitly made visible while making the build default hidden.

https://gcc.gnu.org/wiki/Visibility

on windows symbols are hidden by default (opposite of gnu toolchain) and allowing the symbols that are intended to be public be annotated in code with a macro instead of a separate map/def file would arguably appear more consistent. additionally, the current process of converting a map file to a def file for windows has limitations with respect to things like marking the type of symbol being exported.

ty

#2 lore raw
Quoted 24 lines

From: Tyler Retzlaff [mailto:[email protected]]
Sent: Wednesday, 10 January 2024 17.58

Quoted 12 lines · Bruce Richardson

The default behaviour of "ld.lld" has changed, so it now prints out warnings about entries in the version.map file which don't exist in the current build. Since we use our version.map file simply to filter out the functions we don't want made public, we include in it all functions across all OS's and builds that we want public if present. This causes these ld warnings to be emitted, e.g. on BSD, which is missing functionality found on Linux. For example:

* hpet functions in EAL
* regexdev enqueue and dequeue burst
* eventdev event_timer functions

Quoted 7 lines

Easiest solution, without major rework of how we use our version.map files, and without dynamically generating them per-build, is to pass the --undefined-version flag to the linker, to restore the old behaviour.

---

Quoted 8 lines

i don't know if has ever been discussed but a way to achieve a similar outcome would be to introduce a visibility macro allowing the data and function symbols to be explicitly made visible while making the build default hidden.

https://gcc.gnu.org/wiki/Visibility

This looks interesting!
Declaring a function "public" directly in its header seems much simpler to manage than having to add it to the version.map file too.

I wonder if function versioning is still supported if using this instead of version.map files? Of if there are other relevant reasons for continuing to use the version.map files instead of this?

Quoted 11 lines

on windows symbols are hidden by default (opposite of gnu toolchain) and
allowing the symbols that are intended to be public be annotated in code
with a macro instead of a separate map/def file would arguably appear more consistent. additionally, the current process of converting a map file to a def file for windows has limitations with respect to things like marking the type of symbol being exported.

ty

#3 lore raw
Quoted 31 lines · Morten Brørup
Quoted 31 lines

From: Tyler Retzlaff [mailto:[email protected]]
Sent: Wednesday, 10 January 2024 17.58

Quoted 12 lines · Bruce Richardson

The default behaviour of "ld.lld" has changed, so it now prints out warnings about entries in the version.map file which don't exist in the current build. Since we use our version.map file simply to filter out the functions we don't want made public, we include in it all functions across all OS's and builds that we want public if present. This causes these ld warnings to be emitted, e.g. on BSD, which is missing functionality found on Linux. For example:

* hpet functions in EAL
* regexdev enqueue and dequeue burst
* eventdev event_timer functions

Quoted 7 lines

Easiest solution, without major rework of how we use our version.map files, and without dynamically generating them per-build, is to pass the --undefined-version flag to the linker, to restore the old behaviour.

---

i don't know if has ever been discussed but a way to achieve a similar outcome would be to introduce a visibility macro allowing the data and function symbols to be explicitly made visible while making the build default hidden.

Quoted 8 lines
Quoted 1 line

https://gcc.gnu.org/wiki/Visibility

This looks interesting!
Declaring a function "public" directly in its header seems much simpler to manage than having to add it to the version.map file too.

I wonder if function versioning is still supported if using this instead of version.map files? Of if there are other relevant reasons for continuing to use the version.map files instead of this?

I don't see in that wiki page and details of how to mark symbols with different ABI versions. For example, as well as listing what functions are public, our version.map files also identify the ABI version (e.g. DPDK_24) they belong to, or whether they are experimental or internal. Having them all in the version file also makes it easy to see how many experimental functions we have in each component.

/Bruce

#4 lore raw
Quoted 41 lines · Bruce Richardson
Quoted 31 lines · Morten Brørup
Quoted 31 lines

From: Tyler Retzlaff [mailto:[email protected]]
Sent: Wednesday, 10 January 2024 17.58

Quoted 12 lines · Bruce Richardson

The default behaviour of "ld.lld" has changed, so it now prints out warnings about entries in the version.map file which don't exist in the current build. Since we use our version.map file simply to filter out the functions we don't want made public, we include in it all functions across all OS's and builds that we want public if present. This causes these ld warnings to be emitted, e.g. on BSD, which is missing functionality found on Linux. For example:

* hpet functions in EAL
* regexdev enqueue and dequeue burst
* eventdev event_timer functions

Quoted 7 lines

Easiest solution, without major rework of how we use our version.map files, and without dynamically generating them per-build, is to pass the --undefined-version flag to the linker, to restore the old behaviour.

---

i don't know if has ever been discussed but a way to achieve a similar outcome would be to introduce a visibility macro allowing the data and function symbols to be explicitly made visible while making the build default hidden.

Quoted 8 lines
Quoted 1 line

https://gcc.gnu.org/wiki/Visibility

This looks interesting!
Declaring a function "public" directly in its header seems much simpler to manage than having to add it to the version.map file too.

I wonder if function versioning is still supported if using this instead of version.map files? Of if there are other relevant reasons for continuing to use the version.map files instead of this?

Quoted 6 lines

I don't see in that wiki page and details of how to mark symbols with different ABI versions. For example, as well as listing what functions are public, our version.map files also identify the ABI version (e.g. DPDK_24) they belong to, or whether they are experimental or internal. Having them all in the version file also makes it easy to see how many experimental functions we have in each component.

you can use symver in combination with visibility default

https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html

anyway just food for thought, it would get me out of having to hack & enhance the .def from .map generation and unfortunately even with that there are going to be cases where i still have to annotate the actual symbol export in code (for windows).

just thought a more unified approach for all might appeal.

ty

Quoted 2 lines

/Bruce

#5 lore raw
Quoted 72 lines

From: Tyler Retzlaff [mailto:[email protected]]
Sent: Friday, 12 January 2024 21.11

Quoted 6 lines · Bruce Richardson
Quoted 5 lines · Morten Brørup
Quoted 5 lines

From: Tyler Retzlaff [mailto:[email protected]]
Sent: Wednesday, 10 January 2024 17.58

Quoted 1 line · Bruce Richardson

The default behaviour of "ld.lld" has changed, so it now prints

out

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

warnings about entries in the version.map file which don't

exist in

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

the current build. Since we use our version.map file simply to

filter

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

out the functions we don't want made public, we include in it

all

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

functions across all OS's and builds that we want public if

present.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

This causes these ld warnings to be emitted, e.g. on BSD, which

is

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

missing functionality found on Linux. For example:

* hpet functions in EAL
* regexdev enqueue and dequeue burst
* eventdev event_timer functions

Easiest solution, without major rework of how we use our

version.map

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

files, and without dynamically generating them per-build, is to

pass

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 5 lines

the --undefined-version flag to the linker, to restore the old behaviour.

---

i don't know if has ever been discussed but a way to achieve a

similar

Quoted 1 line
Quoted 1 line
Quoted 1 line

outcome would be to introduce a visibility macro allowing the

data and

Quoted 1 line
Quoted 1 line
Quoted 1 line

function symbols to be explicitly made visible while making the

build

Quoted 6 lines
Quoted 6 lines
Quoted 3 lines

default hidden.

https://gcc.gnu.org/wiki/Visibility

This looks interesting!
Declaring a function "public" directly in its header seems much

simpler to manage than having to add it to the version.map file too.

Quoted 2 lines
Quoted 2 lines

I wonder if function versioning is still supported if using this

instead of version.map files?

Quoted 1 line
Quoted 1 line

Of if there are other relevant reasons for continuing to use the

version.map files instead of this?

Quoted 4 lines
Quoted 1 line

I don't see in that wiki page and details of how to mark symbols with different ABI versions. For example, as well as listing what

functions are

Quoted 1 line

public, our version.map files also identify the ABI version (e.g.

DPDK_24)

Quoted 1 line

they belong to, or whether they are experimental or internal. Having

them

Quoted 1 line

all in the version file also makes it easy to see how many

experimental

Quoted 1 line

functions we have in each component.

you can use symver in combination with visibility default

Quoted 8 lines

https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html

anyway just food for thought, it would get me out of having to hack & enhance the .def from .map generation and unfortunately even with that there are going to be cases where i still have to annotate the actual symbol export in code (for windows).

just thought a more unified approach for all might appeal.

Assuming that we truly want DPDK to support Windows, a more unified approach is a reasonable ask.

If we can eliminate the technical obstacles, we should pursue it.

We may have to sacrifice some "nice to have" advantages of the version.map files along the way, such as having easy access to the list of experimental functions in the version.map file.

#6 lore raw
Quoted 9 lines · On Fri, 12 Jan 2024 21:49:13 +0100 Morten Brørup <[email protected]>…
Quoted 9 lines

you can use symver in combination with visibility default

https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html

anyway just food for thought, it would get me out of having to hack & enhance the .def from .map generation and unfortunately even with that there are going to be cases where i still have to annotate the actual symbol export in code (for windows).

Quoted 7 lines
Quoted 1 line

just thought a more unified approach for all might appeal.

Assuming that we truly want DPDK to support Windows, a more unified approach is a reasonable ask.

If we can eliminate the technical obstacles, we should pursue it.

We may have to sacrifice some "nice to have" advantages of the version.map files along the way, such as having easy access to the list of experimental functions in the version.map file.

Ideally __rte_experimental macro would do the right thing to make it publicly visible in the right section.

#7 lore raw
In reply to Morten Brørup
Quoted 8 lines · Morten Brørup
Quoted 8 lines

you can use symver in combination with visibility default

https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html

anyway just food for thought, it would get me out of having to hack & enhance the .def from .map generation and unfortunately even with that there are going to be cases where i still have to annotate the actual symbol export in code (for windows).

Versioning a symbol means you are exporting it, is it not that simple? So I don't see the need for annotating about symbol visibility.

Quoted 6 lines
Quoted 2 lines

just thought a more unified approach for all might appeal.

Assuming that we truly want DPDK to support Windows, a more unified approach is a reasonable ask.

If we can eliminate the technical obstacles, we should pursue it.

IIUC, we still need a "default" version script, as the linker needs one to declare version nodes, and so we have a catch all for unversioned symbols.

$ cat symver.c
__attribute__ ((__symver__ ("api1@DPDK_23")))
int old_api1(void) { return 0; }

__attribute__ ((__symver__ ("api1@@DPDK_24")))
int api1(void) { return 0; }

__attribute__ ((__symver__ ("api2@@DPDK_24")))
int api2(void) { return 0; }

int api3(void) { return 0; }

$ cat symver.map
DPDK_23 { };
DPDK_24 { };
EXPERIMENTAL { };
INTERNAL {
    local: *;
};
$ gcc -o symver.o -fPIC -Wall -Werror -c symver.c && gcc -o
libsymver.so -shared -fPIC -Wl,--version-script symver.map symver.o &&
readelf -s libsymver.so | grep api
     5: 0000000000001104    11 FUNC    GLOBAL DEFAULT   13 api1@@DPDK_24
     7: 00000000000010f9    11 FUNC    GLOBAL DEFAULT   13 api1@DPDK_23
     9: 000000000000110f    11 FUNC    GLOBAL DEFAULT   13 api2@@DPDK_24
    13: 00000000000010f9    11 FUNC    LOCAL  DEFAULT   13 old_api1
    15: 0000000000001104    11 FUNC    LOCAL  DEFAULT   13 api1
    16: 000000000000111a    11 FUNC    LOCAL  DEFAULT   13 api3
    19: 000000000000110f    11 FUNC    LOCAL  DEFAULT   13 api2
    24: 0000000000001104    11 FUNC    GLOBAL DEFAULT   13 api1@@DPDK_24
    29: 00000000000010f9    11 FUNC    GLOBAL DEFAULT   13 api1@DPDK_23
    31: 000000000000110f    11 FUNC    GLOBAL DEFAULT   13 api2@@DPDK_24
Quoted 2 lines

We may have to sacrifice some "nice to have" advantages of the version.map files along the way, such as having easy access to the list of experimental functions in the version.map file.

Developers lose the single location for versioning information, but we already have some scripts to help list symbols from a given version. We may have to enhance them.
But I don't think we would lose features.

The devil is probably in the details, though :-).

Signature · 1 line
David Marchand
#8 lore raw
Quoted 61 lines · David Marchand
Quoted 8 lines · Morten Brørup
Quoted 8 lines

you can use symver in combination with visibility default

https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html

anyway just food for thought, it would get me out of having to hack & enhance the .def from .map generation and unfortunately even with that there are going to be cases where i still have to annotate the actual symbol export in code (for windows).

Versioning a symbol means you are exporting it, is it not that simple? So I don't see the need for annotating about symbol visibility.

Quoted 6 lines
Quoted 2 lines

just thought a more unified approach for all might appeal.

Assuming that we truly want DPDK to support Windows, a more unified approach is a reasonable ask.

If we can eliminate the technical obstacles, we should pursue it.

IIUC, we still need a "default" version script, as the linker needs one to declare version nodes, and so we have a catch all for unversioned symbols.

$ cat symver.c
__attribute__ ((__symver__ ("api1@DPDK_23")))
int old_api1(void) { return 0; }

__attribute__ ((__symver__ ("api1@@DPDK_24")))
int api1(void) { return 0; }

__attribute__ ((__symver__ ("api2@@DPDK_24")))
int api2(void) { return 0; }

int api3(void) { return 0; }

$ cat symver.map
DPDK_23 { };
DPDK_24 { };
EXPERIMENTAL { };
INTERNAL {
    local: *;
};
$ gcc -o symver.o -fPIC -Wall -Werror -c symver.c && gcc -o
libsymver.so -shared -fPIC -Wl,--version-script symver.map symver.o &&
readelf -s libsymver.so | grep api
     5: 0000000000001104    11 FUNC    GLOBAL DEFAULT   13 api1@@DPDK_24
     7: 00000000000010f9    11 FUNC    GLOBAL DEFAULT   13 api1@DPDK_23
     9: 000000000000110f    11 FUNC    GLOBAL DEFAULT   13 api2@@DPDK_24
    13: 00000000000010f9    11 FUNC    LOCAL  DEFAULT   13 old_api1
    15: 0000000000001104    11 FUNC    LOCAL  DEFAULT   13 api1
    16: 000000000000111a    11 FUNC    LOCAL  DEFAULT   13 api3
    19: 000000000000110f    11 FUNC    LOCAL  DEFAULT   13 api2
    24: 0000000000001104    11 FUNC    GLOBAL DEFAULT   13 api1@@DPDK_24
    29: 00000000000010f9    11 FUNC    GLOBAL DEFAULT   13 api1@DPDK_23
    31: 000000000000110f    11 FUNC    GLOBAL DEFAULT   13 api2@@DPDK_24
Quoted 1 line
Quoted 8 lines
Quoted 1 line

We may have to sacrifice some "nice to have" advantages of the version.map files along the way, such as having easy access to the list of experimental functions in the version.map file.

Developers lose the single location for versioning information, but we already have some scripts to help list symbols from a given version. We may have to enhance them.
But I don't think we would lose features.

The devil is probably in the details, though :-).

so my mention of using visibility was more about changes that need to be made to handle windows dso/dll and wondering if a macro based approach would result in something unified for both windows and unix targets.

today as you know we use a .map as an authoritative source for the exported symbols and translate the contents of the .map to a .def for the windows linker.

going forward to fix dso/dll on windows i'll need to do 2 things.

* exclusive export of a subset of symbols for windows only
* annotation of certain data symbols with type information

since the .map is authoritative i can't just plant either of the above into the .map in order to pass it through to the generated .def file.

* we could maintain a mostly duplicated copy of .def and stop generating it allowing the above additions for windows only.

* we could go hybrid approach where some of the windows symbols are listed in the .map and some of them are macros in the code.

* we could move to some single authoritative source that isn't just defacto default of a toolchain and generate both .map and .def

* we could move to visibility and symbol marking with macros in the code for all symbols, of course the expansion is still conditional but you could reasonably grep the tree to generate a full set of exported names.

if the suggestion to use visibility and macros for both isn't popular i'm most likely to look at the hybrid approach since i don't have a lot of interest in trying to mangle things into comments of the .map for .def generation and don't think inventing a new common/authoritative format and maintaining a .map and .def generator is interesting.

i'm open to input, so just fishing for others who have stronger opinions.

ty

Quoted 3 lines
Signature · 2 lines
-- 
David Marchand
#9 lore raw
In reply to Tyler Retzlaff

10/01/2024 17:58, Tyler Retzlaff:

Quoted 13 lines · Bruce Richardson
Quoted 12 lines

The default behaviour of "ld.lld" has changed, so it now prints out warnings about entries in the version.map file which don't exist in the current build. Since we use our version.map file simply to filter out the functions we don't want made public, we include in it all functions across all OS's and builds that we want public if present. This causes these ld warnings to be emitted, e.g. on BSD, which is missing functionality found on Linux. For example:

* hpet functions in EAL
* regexdev enqueue and dequeue burst
* eventdev event_timer functions

Quoted 8 lines
Quoted 6 lines

Easiest solution, without major rework of how we use our version.map files, and without dynamically generating them per-build, is to pass the --undefined-version flag to the linker, to restore the old behaviour.

+Cc stable

Applied as easy solution for this linker.

The discussion about Windows linker and use of macros deserve to continue, maybe in a new dedicated thread?

Quoted 13 lines

i don't know if has ever been discussed but a way to achieve a similar outcome would be to introduce a visibility macro allowing the data and function symbols to be explicitly made visible while making the build default hidden.

https://gcc.gnu.org/wiki/Visibility

on windows symbols are hidden by default (opposite of gnu toolchain) and allowing the symbols that are intended to be public be annotated in code with a macro instead of a separate map/def file would arguably appear more consistent. additionally, the current process of converting a map file to a def file for windows has limitations with respect to things like marking the type of symbol being exported.

#10 lore raw
Quoted 21 lines · Thomas Monjalon

10/01/2024 17:58, Tyler Retzlaff:

Quoted 20 lines · Bruce Richardson
Quoted 18 lines

The default behaviour of "ld.lld" has changed, so it now prints out warnings about entries in the version.map file which don't exist in the current build. Since we use our version.map file simply to filter out the functions we don't want made public, we include in it all functions across all OS's and builds that we want public if present. This causes these ld warnings to be emitted, e.g. on BSD, which is missing functionality found on Linux. For example:

* hpet functions in EAL
* regexdev enqueue and dequeue burst
* eventdev event_timer functions

Easiest solution, without major rework of how we use our version.map files, and without dynamically generating them per-build, is to pass the --undefined-version flag to the linker, to restore the old behaviour.

Quoted 8 lines
Quoted 1 line

+Cc stable

Applied as easy solution for this linker.

The discussion about Windows linker and use of macros deserve to continue, maybe in a new dedicated thread?

yes, i will start a new discussion on the topic after clearing other windows / msvc changes as soon as i have time.

thanks!

Quoted 16 lines
Quoted 14 lines

i don't know if has ever been discussed but a way to achieve a similar outcome would be to introduce a visibility macro allowing the data and function symbols to be explicitly made visible while making the build default hidden.

https://gcc.gnu.org/wiki/Visibility

on windows symbols are hidden by default (opposite of gnu toolchain) and allowing the symbols that are intended to be public be annotated in code with a macro instead of a separate map/def file would arguably appear more consistent. additionally, the current process of converting a map file to a def file for windows has limitations with respect to things like marking the type of symbol being exported.