Skip to contentSkip to conversation

libbpf: Support symbol versioning for uprobe

PATCH Applied Hengqi Chen 3 participants 7 messages Sep 18, 2023 – Sep 25, 2023

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

v4 cover libbpf: Support symbol versioning for uprobe

Dynamic symbols in shared library may have the same name, for example:

    $ nm -D /lib/x86_64-linux-gnu/libc.so.6 | grep rwlock_wrlock
    000000000009b1a0 T __pthread_rwlock_wrlock@GLIBC_2.2.5
    000000000009b1a0 T pthread_rwlock_wrlock@@GLIBC_2.34
    000000000009b1a0 T pthread_rwlock_wrlock@GLIBC_2.2.5
    $ readelf -W --dyn-syms /lib/x86_64-linux-gnu/libc.so.6 | grep rwlock_wrlock
      706: 000000000009b1a0   878 FUNC    GLOBAL DEFAULT   15 __pthread_rwlock_wrlock@GLIBC_2.2.5
      2568: 000000000009b1a0   878 FUNC    GLOBAL DEFAULT   15 pthread_rwlock_wrlock@@GLIBC_2.34
      2571: 000000000009b1a0   878 FUNC    GLOBAL DEFAULT   15 pthread_rwlock_wrlock@GLIBC_2.2.5

There are two pthread_rwlock_wrlock symbols in libc.so .dynsym section. The one with @@ is the default version, the other is hidden. Note that the version info is stored in .gnu.version and .gnu.version_d sections of libc and the two symbols are at the _same_ offset.

Currently, specify pthread_rwlock_wrlock, pthread_rwlock_wrlock@@GLIBC_2.34 or pthread_rwlock_wrlock@GLIBC_2.2.5 in bpf_uprobe_opts::func_name won't work. Because there are two pthread_rwlock_wrlock in .dynsym sections without the version suffix and both are global bind.

We could solve this by introducing symbol versioning ([0]). So that users can specify func, func@LIB_VERSION or func@@LIB_VERSION to attach a uprobe.

This patchset resolves symbol conflicts and add symbol versioning for uprobe.
- Patch 1 resolves symbol conflicts at the same offset
- Patch 2 adds symbol versioning for dynsym
- Patch 3 adds selftests for the above changes

Changes from v3:
- Address comments from Andrii

Changes from v2:
- Add uretprobe selfttest (Alan)
- Check symbol exact match (Alan)
- Fix typo (Jiri)

Changes from v1:
- Address comments from Alan and Jiri
- Add selftests (Someone reminds me that there is an attempt at [1] and part of the selftest code from Andrii is taken from there)

[0]: https://refspecs.linuxfoundation.org/LSB_5.0.0/LSB-Core-generic/LSB-Core-generic/symversion.html
[1]: https://lore.kernel.org/lkml/CAEf4BzZTrjjyyOm3ak9JsssPSh6T_ZmGd677a2rt5e5rBLUrpQ@mail.gmail.com/

Hengqi Chen (3):
libbpf: Resolve symbol conflicts at the same offset for uprobe
libbpf: Support symbol versioning for uprobe selftests/bpf: Add tests for symbol versioning for uprobe

 tools/lib/bpf/elf.c                           | 139 ++++++++++++++++--
 tools/lib/bpf/libbpf.c                        |   2 +-
 tools/testing/selftests/bpf/Makefile          |   5 +-
 .../testing/selftests/bpf/liburandom_read.map |  15 ++
 .../testing/selftests/bpf/prog_tests/uprobe.c |  95 ++++++++++++
 .../testing/selftests/bpf/progs/test_uprobe.c |  61 ++++++++
 tools/testing/selftests/bpf/urandom_read.c    |  15 +-
 .../testing/selftests/bpf/urandom_read_lib1.c |  22 +++
 8 files changed, 337 insertions(+), 17 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/liburandom_read.map
 create mode 100644 tools/testing/selftests/bpf/prog_tests/uprobe.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_uprobe.c
Signature · 1 line
2.34.1
#1 lore raw

PATCH v4 1/3 libbpf: Resolve symbol conflicts at the same offset for uprobe

Dynamic symbols in shared library may have the same name, for example:

    $ nm -D /lib/x86_64-linux-gnu/libc.so.6 | grep rwlock_wrlock
    000000000009b1a0 T __pthread_rwlock_wrlock@GLIBC_2.2.5
    000000000009b1a0 T pthread_rwlock_wrlock@@GLIBC_2.34
    000000000009b1a0 T pthread_rwlock_wrlock@GLIBC_2.2.5
    $ readelf -W --dyn-syms /lib/x86_64-linux-gnu/libc.so.6 | grep rwlock_wrlock
     706: 000000000009b1a0   878 FUNC    GLOBAL DEFAULT   15 __pthread_rwlock_wrlock@GLIBC_2.2.5
    2568: 000000000009b1a0   878 FUNC    GLOBAL DEFAULT   15 pthread_rwlock_wrlock@@GLIBC_2.34
    2571: 000000000009b1a0   878 FUNC    GLOBAL DEFAULT   15 pthread_rwlock_wrlock@GLIBC_2.2.5

Currently, users can't attach a uprobe to pthread_rwlock_wrlock because there are two symbols named pthread_rwlock_wrlock and both are global bind. And libbpf considers it as a conflict.

Since both of them are at the same offset we could accept one of them harmlessly. Note that we already does this in elf_resolve_syms_offsets.

---
 tools/lib/bpf/elf.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
tools/lib/bpf/elf.c +4 -1
index 9d0296c1726a..5c9e588b17da 100644
@@ -214,7 +214,10 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
 
 			if (ret > 0) {
 				/* handle multiple matches */
-				if (last_bind != STB_WEAK && cur_bind != STB_WEAK) {
+				if (elf_sym_offset(sym) == ret) {
+					/* same offset, no problem */
+					continue;
+				} else if (last_bind != STB_WEAK && cur_bind != STB_WEAK) {
 					/* Only accept one non-weak bind. */
 					pr_warn("elf: ambiguous match for '%s', '%s' in '%s'\n",
 						sym->name, name, binary_path);
Signature · 1 line
2.34.1
#2 lore raw

PATCH v4 2/3 libbpf: Support symbol versioning for uprobe

In reply to Hengqi Chen

In current implementation, we assume that symbol found in .dynsym section would have a version suffix and use it to compare with symbol user supplied. According to the spec ([0]), this assumption is incorrect, the version info of dynamic symbols are stored in .gnu.version and .gnu.version_d sections of ELF objects. For example:

    $ nm -D /lib/x86_64-linux-gnu/libc.so.6 | grep rwlock_wrlock
    000000000009b1a0 T __pthread_rwlock_wrlock@GLIBC_2.2.5
    000000000009b1a0 T pthread_rwlock_wrlock@@GLIBC_2.34
    000000000009b1a0 T pthread_rwlock_wrlock@GLIBC_2.2.5
    $ readelf -W --dyn-syms /lib/x86_64-linux-gnu/libc.so.6 | grep rwlock_wrlock
      706: 000000000009b1a0   878 FUNC    GLOBAL DEFAULT   15 __pthread_rwlock_wrlock@GLIBC_2.2.5
      2568: 000000000009b1a0   878 FUNC    GLOBAL DEFAULT   15 pthread_rwlock_wrlock@@GLIBC_2.34
      2571: 000000000009b1a0   878 FUNC    GLOBAL DEFAULT   15 pthread_rwlock_wrlock@GLIBC_2.2.5

In this case, specify pthread_rwlock_wrlock@@GLIBC_2.34 or pthread_rwlock_wrlock@GLIBC_2.2.5 in bpf_uprobe_opts::func_name won't work. Because the qualified name does NOT match pthread_rwlock_wrlock (without version suffix) in .dynsym sections.

This commit implements the symbol versioning for dynsym and allows user to specify symbol in the following forms:
- func
- func@LIB_VERSION
- func@@LIB_VERSION

In case of symbol conflicts, error out and users should resolve it by specifying a qualified name.

[0]: https://refspecs.linuxfoundation.org/LSB_5.0.0/LSB-Core-generic/LSB-Core-generic/symversion.html

---
 tools/lib/bpf/elf.c    | 134 +++++++++++++++++++++++++++++++++++++----
 tools/lib/bpf/libbpf.c |   2 +-
 2 files changed, 124 insertions(+), 12 deletions(-)
tools/lib/bpf/elf.c +123 -11
index 5c9e588b17da..f7ad7a7acc29 100644
@@ -1,5 +1,8 @@
 // SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
 
+#ifndef _GNU_SOURCE
+#define _GNU_SOURCE
+#endif
 #include <libelf.h>
 #include <gelf.h>
 #include <fcntl.h>
@@ -10,6 +13,17 @@
 
 #define STRERR_BUFSIZE  128
 
+/* A SHT_GNU_versym section holds 16-bit words. This bit is set if
+ * the symbol is hidden and can only be seen when referenced using an
+ * explicit version number. This is a GNU extension.
+ */
+#define VERSYM_HIDDEN	0x8000
+
+/* This is the mask for the rest of the data in a word read from a
+ * SHT_GNU_versym section.
+ */
+#define VERSYM_VERSION	0x7fff
+
 int elf_open(const char *binary_path, struct elf_fd *elf_fd)
 {
 	char errmsg[STRERR_BUFSIZE];
@@ -64,13 +78,18 @@ struct elf_sym {
 	const char *name;
 	GElf_Sym sym;
 	GElf_Shdr sh;
+	int ver;
+	bool hidden;
 };
 
 struct elf_sym_iter {
 	Elf *elf;
 	Elf_Data *syms;
+	Elf_Data *versyms;
+	Elf_Data *verdefs;
 	size_t nr_syms;
 	size_t strtabidx;
+	size_t verdef_strtabidx;
 	size_t next_sym_idx;
 	struct elf_sym sym;
 	int st_type;
@@ -111,6 +130,29 @@ static int elf_sym_iter_new(struct elf_sym_iter *iter,
 	iter->nr_syms = iter->syms->d_size / sh.sh_entsize;
 	iter->elf = elf;
 	iter->st_type = st_type;
+
+	/* Version symbol table is meaningful to dynsym only */
+	if (sh_type != SHT_DYNSYM)
+		return 0;
+
+	scn = elf_find_next_scn_by_type(elf, SHT_GNU_versym, NULL);
+	if (!scn)
+		return 0;
+	if (!gelf_getshdr(scn, &sh))
+		return -EINVAL;
+	iter->versyms = elf_getdata(scn, 0);
+
+	scn = elf_find_next_scn_by_type(elf, SHT_GNU_verdef, NULL);
+	if (!scn) {
+		pr_debug("elf: failed to find verdef ELF sections in '%s'\n",
+			 binary_path);
+		return -ENOENT;
+	}
+	if (!gelf_getshdr(scn, &sh))
+		return -EINVAL;
+	iter->verdef_strtabidx = sh.sh_link;
+	iter->verdefs = elf_getdata(scn, 0);
+
 	return 0;
 }
 
@@ -119,6 +161,7 @@ static struct elf_sym *elf_sym_iter_next(struct elf_sym_iter *iter)
 	struct elf_sym *ret = &iter->sym;
 	GElf_Sym *sym = &ret->sym;
 	const char *name = NULL;
+	GElf_Versym versym;
 	Elf_Scn *sym_scn;
 	size_t idx;
 
@@ -138,12 +181,80 @@ static struct elf_sym *elf_sym_iter_next(struct elf_sym_iter *iter)
 
 		iter->next_sym_idx = idx + 1;
 		ret->name = name;
+		ret->ver = 0;
+		ret->hidden = false;
+
+		if (iter->versyms) {
+			if (!gelf_getversym(iter->versyms, idx, &versym))
+				continue;
+			ret->ver = versym & VERSYM_VERSION;
+			ret->hidden = versym & VERSYM_HIDDEN;
+		}
 		return ret;
 	}
 
 	return NULL;
 }
 
+static const char *elf_get_vername(struct elf_sym_iter *iter, int ver)
+{
+	GElf_Verdaux verdaux;
+	GElf_Verdef verdef;
+	int offset;
+
+	offset = 0;
+	while (gelf_getverdef(iter->verdefs, offset, &verdef)) {
+		if (verdef.vd_ndx != ver) {
+			if (!verdef.vd_next)
+				break;
+
+			offset += verdef.vd_next;
+			continue;
+		}
+
+		if (!gelf_getverdaux(iter->verdefs, offset + verdef.vd_aux, &verdaux))
+			break;
+
+		return elf_strptr(iter->elf, iter->verdef_strtabidx, verdaux.vda_name);
+
+	}
+	return NULL;
+}
+
+static bool symbol_match(struct elf_sym_iter *iter, int sh_type, struct elf_sym *sym,
+			 const char *name, size_t name_len, const char *lib_ver)
+{
+	const char *ver_name;
+
+	/* Symbols are in forms of func, func@LIB_VER or func@@LIB_VER
+	 * make sure the func part matches the user specified name
+	 */
+	if (strncmp(sym->name, name, name_len) != 0)
+		return false;
+
+	/* ...but we don't want a search for "foo" to match 'foo2" also, so any
+	 * additional characters in sname should be of the form "@@LIB".
+	 */
+	if (sym->name[name_len] != '\0' && sym->name[name_len] != '@')
+		return false;
+
+	/* If user does not specify symbol version, then we got a match */
+	if (!lib_ver)
+		return true;
+
+	/* If user specifies symbol version, for dynamic symbols,
+	 * get version name from ELF verdef section for comparison.
+	 */
+	if (sh_type == SHT_DYNSYM) {
+		ver_name = elf_get_vername(iter, sym->ver);
+		if (!ver_name)
+			return false;
+		return !strcmp(ver_name, lib_ver);
+	}
+
+	/* For normal symbols, it is already in form of func@LIB_VER */
+	return !strcmp(sym->name, name);
+}
 
 /* Transform symbol's virtual address (absolute for binaries and relative
  * for shared libs) into file offset, which is what kernel is expecting
@@ -166,7 +277,8 @@ static unsigned long elf_sym_offset(struct elf_sym *sym)
 long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
 {
 	int i, sh_types[2] = { SHT_DYNSYM, SHT_SYMTAB };
-	bool is_shared_lib, is_name_qualified;
+	const char *at_symbol, *lib_ver;
+	bool is_shared_lib;
 	long ret = -ENOENT;
 	size_t name_len;
 	GElf_Ehdr ehdr;
@@ -179,9 +291,15 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
 	/* for shared lib case, we do not need to calculate relative offset */
 	is_shared_lib = ehdr.e_type == ET_DYN;
 
-	name_len = strlen(name);
-	/* Does name specify "@@LIB"? */
-	is_name_qualified = strstr(name, "@@") != NULL;
+	/* Does name specify "@@LIB_VER" or "@LIB_VER" ? */
+	at_symbol = strchr(name, '@');
+	if (at_symbol) {
+		name_len = at_symbol - name;
+		lib_ver = strrchr(name, '@') + 1;
+	} else {
+		name_len = strlen(name);
+		lib_ver = NULL;
+	}
 
 	/* Search SHT_DYNSYM, SHT_SYMTAB for symbol. This search order is used because if
 	 * a binary is stripped, it may only have SHT_DYNSYM, and a fully-statically
@@ -201,13 +319,7 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
 			goto out;
 
 		while ((sym = elf_sym_iter_next(&iter))) {
-			/* User can specify func, func@@LIB or func@@LIB_VERSION. */
-			if (strncmp(sym->name, name, name_len) != 0)
-				continue;
-			/* ...but we don't want a search for "foo" to match 'foo2" also, so any
-			 * additional characters in sname should be of the form "@@LIB".
-			 */
-			if (!is_name_qualified && sym->name[name_len] != '\0' && sym->name[name_len] != '@')
+			if (!symbol_match(&iter, sh_types[i], sym, name, name_len, lib_ver))
 				continue;
 
 			cur_bind = GELF_ST_BIND(sym->sym.st_info);
tools/lib/bpf/libbpf.c +1 -1
index 3a6108e3238b..b4758e54a815 100644
@@ -11630,7 +11630,7 @@ static int attach_uprobe(const struct bpf_program *prog, long cookie, struct bpf
 
 	*link = NULL;
 
-	n = sscanf(prog->sec_name, "%m[^/]/%m[^:]:%m[a-zA-Z0-9_.]+%li",
+	n = sscanf(prog->sec_name, "%m[^/]/%m[^:]:%m[a-zA-Z0-9_.@]+%li",
 		   &probe_type, &binary_path, &func_name, &offset);
 	switch (n) {
 	case 1:
Signature · 1 line
2.34.1
#3 lore raw
Quoted 95 lines · Hengqi Chen

In current implementation, we assume that symbol found in .dynsym section would have a version suffix and use it to compare with symbol user supplied. According to the spec ([0]), this assumption is incorrect, the version info of dynamic symbols are stored in .gnu.version and .gnu.version_d sections of ELF objects. For example:

    $ nm -D /lib/x86_64-linux-gnu/libc.so.6 | grep rwlock_wrlock
    000000000009b1a0 T __pthread_rwlock_wrlock@GLIBC_2.2.5
    000000000009b1a0 T pthread_rwlock_wrlock@@GLIBC_2.34
    000000000009b1a0 T pthread_rwlock_wrlock@GLIBC_2.2.5
    $ readelf -W --dyn-syms /lib/x86_64-linux-gnu/libc.so.6 | grep rwlock_wrlock
      706: 000000000009b1a0   878 FUNC    GLOBAL DEFAULT   15 __pthread_rwlock_wrlock@GLIBC_2.2.5
      2568: 000000000009b1a0   878 FUNC    GLOBAL DEFAULT   15 pthread_rwlock_wrlock@@GLIBC_2.34
      2571: 000000000009b1a0   878 FUNC    GLOBAL DEFAULT   15 pthread_rwlock_wrlock@GLIBC_2.2.5

In this case, specify pthread_rwlock_wrlock@@GLIBC_2.34 or pthread_rwlock_wrlock@GLIBC_2.2.5 in bpf_uprobe_opts::func_name won't work. Because the qualified name does NOT match pthread_rwlock_wrlock (without version suffix) in .dynsym sections.

This commit implements the symbol versioning for dynsym and allows user to specify symbol in the following forms:
- func
- func@LIB_VERSION
- func@@LIB_VERSION

In case of symbol conflicts, error out and users should resolve it by specifying a qualified name.

[0]: https://refspecs.linuxfoundation.org/LSB_5.0.0/LSB-Core-generic/LSB-Core-generic/symversion.html

---
 tools/lib/bpf/elf.c    | 134 +++++++++++++++++++++++++++++++++++++----
 tools/lib/bpf/libbpf.c |   2 +-
 2 files changed, 124 insertions(+), 12 deletions(-)
tools/lib/bpf/elf.c +19 -0
index 5c9e588b17da..f7ad7a7acc29 100644
@@ -1,5 +1,8 @@
 // SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
 
+#ifndef _GNU_SOURCE
+#define _GNU_SOURCE
+#endif
 #include <libelf.h>
 #include <gelf.h>
 #include <fcntl.h>
@@ -10,6 +13,17 @@
 
 #define STRERR_BUFSIZE  128
 
+/* A SHT_GNU_versym section holds 16-bit words. This bit is set if
+ * the symbol is hidden and can only be seen when referenced using an
+ * explicit version number. This is a GNU extension.
+ */
+#define VERSYM_HIDDEN  0x8000
+
+/* This is the mask for the rest of the data in a word read from a
+ * SHT_GNU_versym section.
+ */
+#define VERSYM_VERSION 0x7fff
+
 int elf_open(const char *binary_path, struct elf_fd *elf_fd)
 {
        char errmsg[STRERR_BUFSIZE];
@@ -64,13 +78,18 @@ struct elf_sym {
        const char *name;
        GElf_Sym sym;
        GElf_Shdr sh;
+       int ver;
+       bool hidden;
 };
 
 struct elf_sym_iter {
        Elf *elf;
        Elf_Data *syms;
+       Elf_Data *versyms;
+       Elf_Data *verdefs;
        size_t nr_syms;
        size_t strtabidx;
+       size_t verdef_strtabidx;
        size_t next_sym_idx;
        struct elf_sym sym;
        int st_type;
@@ -111,6 +130,29 @@ static int elf_sym_iter_new(struct elf_sym_iter *iter,
        iter->nr_syms = iter->syms->d_size / sh.sh_entsize;
        iter->elf = elf;
        iter->st_type = st_type;

tools/lib/bpf/elf.c static int elf_sym_iter_new(struct elf_sym_iter *iter, · line 142

Quoted 10 lines
tools/lib/bpf/elf.c +10 -0
+
+       /* Version symbol table is meaningful to dynsym only */
+       if (sh_type != SHT_DYNSYM)
+               return 0;
+
+       scn = elf_find_next_scn_by_type(elf, SHT_GNU_versym, NULL);
+       if (!scn)
+               return 0;
+       if (!gelf_getshdr(scn, &sh))
+               return -EINVAL;

we don't seem to use sh, why are we calling gelf_getshdr? sanity check or something? I dropped it for now, but let me know if I'm missing something

Quoted 16 lines
tools/lib/bpf/elf.c +13 -0
+       iter->versyms = elf_getdata(scn, 0);
+
+       scn = elf_find_next_scn_by_type(elf, SHT_GNU_verdef, NULL);
+       if (!scn) {
+               pr_debug("elf: failed to find verdef ELF sections in '%s'\n",
+                        binary_path);
+               return -ENOENT;
+       }
+       if (!gelf_getshdr(scn, &sh))
+               return -EINVAL;
+       iter->verdef_strtabidx = sh.sh_link;
+       iter->verdefs = elf_getdata(scn, 0);
+
        return 0;
 }
 

[...]

Quoted 15 lines

/* Transform symbol's virtual address (absolute for binaries and relative
* for shared libs) into file offset, which is what kernel is expecting

tools/lib/bpf/elf.c +2 -1
@@ -166,7 +277,8 @@ static unsigned long elf_sym_offset(struct elf_sym *sym)
 long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
 {
        int i, sh_types[2] = { SHT_DYNSYM, SHT_SYMTAB };
-       bool is_shared_lib, is_name_qualified;
+       const char *at_symbol, *lib_ver;
+       bool is_shared_lib;
        long ret = -ENOENT;
        size_t name_len;
        GElf_Ehdr ehdr;
@@ -179,9 +291,15 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
        /* for shared lib case, we do not need to calculate relative offset */

tools/lib/bpf/elf.c long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name) · line 298

Quoted 10 lines
tools/lib/bpf/elf.c +5 -3
        is_shared_lib = ehdr.e_type == ET_DYN;
 
-       name_len = strlen(name);
-       /* Does name specify "@@LIB"? */
-       is_name_qualified = strstr(name, "@@") != NULL;
+       /* Does name specify "@@LIB_VER" or "@LIB_VER" ? */
+       at_symbol = strchr(name, '@');
+       if (at_symbol) {
+               name_len = at_symbol - name;
+               lib_ver = strrchr(name, '@') + 1;

hm... this makes me a bit uneasy. We basically just need to skip extra @ if it's there. Reverse searching from the end seems both unnecessary and potentially error prone.

So why not

if (at_symbol[1] == '@')

    at_symbol++;

lib_ver = at_symbol + 1;

?

This is what I changed it to while applying, but please do let me know if it's problematic.

Quoted 38 lines
tools/lib/bpf/elf.c +5 -7
+       } else {
+               name_len = strlen(name);
+               lib_ver = NULL;
+       }
 
        /* Search SHT_DYNSYM, SHT_SYMTAB for symbol. This search order is used because if
         * a binary is stripped, it may only have SHT_DYNSYM, and a fully-statically
@@ -201,13 +319,7 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
                        goto out;
 
                while ((sym = elf_sym_iter_next(&iter))) {
-                       /* User can specify func, func@@LIB or func@@LIB_VERSION. */
-                       if (strncmp(sym->name, name, name_len) != 0)
-                               continue;
-                       /* ...but we don't want a search for "foo" to match 'foo2" also, so any
-                        * additional characters in sname should be of the form "@@LIB".
-                        */
-                       if (!is_name_qualified && sym->name[name_len] != '\0' && sym->name[name_len] != '@')
+                       if (!symbol_match(&iter, sh_types[i], sym, name, name_len, lib_ver))
                                continue;
 
                        cur_bind = GELF_ST_BIND(sym->sym.st_info);
tools/lib/bpf/libbpf.c +1 -2
index 3a6108e3238b..b4758e54a815 100644
@@ -11630,7 +11630,7 @@ static int attach_uprobe(const struct bpf_program *prog, long cookie, struct bpf
 
        *link = NULL;
 
-       n = sscanf(prog->sec_name, "%m[^/]/%m[^:]:%m[a-zA-Z0-9_.]+%li",
+       n = sscanf(prog->sec_name, "%m[^/]/%m[^:]:%m[a-zA-Z0-9_.@]+%li",
                   &probe_type, &binary_path, &func_name, &offset);
        switch (n) {
        case 1:
--

2.34.1

#4 lore raw
Quoted 103 lines · Andrii Nakryiko
Quoted 101 lines · Hengqi Chen

In current implementation, we assume that symbol found in .dynsym section would have a version suffix and use it to compare with symbol user supplied. According to the spec ([0]), this assumption is incorrect, the version info of dynamic symbols are stored in .gnu.version and .gnu.version_d sections of ELF objects. For example:

    $ nm -D /lib/x86_64-linux-gnu/libc.so.6 | grep rwlock_wrlock
    000000000009b1a0 T __pthread_rwlock_wrlock@GLIBC_2.2.5
    000000000009b1a0 T pthread_rwlock_wrlock@@GLIBC_2.34
    000000000009b1a0 T pthread_rwlock_wrlock@GLIBC_2.2.5
    $ readelf -W --dyn-syms /lib/x86_64-linux-gnu/libc.so.6 | grep rwlock_wrlock
      706: 000000000009b1a0   878 FUNC    GLOBAL DEFAULT   15 __pthread_rwlock_wrlock@GLIBC_2.2.5
      2568: 000000000009b1a0   878 FUNC    GLOBAL DEFAULT   15 pthread_rwlock_wrlock@@GLIBC_2.34
      2571: 000000000009b1a0   878 FUNC    GLOBAL DEFAULT   15 pthread_rwlock_wrlock@GLIBC_2.2.5

In this case, specify pthread_rwlock_wrlock@@GLIBC_2.34 or pthread_rwlock_wrlock@GLIBC_2.2.5 in bpf_uprobe_opts::func_name won't work. Because the qualified name does NOT match pthread_rwlock_wrlock (without version suffix) in .dynsym sections.

This commit implements the symbol versioning for dynsym and allows user to specify symbol in the following forms:
- func
- func@LIB_VERSION
- func@@LIB_VERSION

In case of symbol conflicts, error out and users should resolve it by specifying a qualified name.

[0]: https://refspecs.linuxfoundation.org/LSB_5.0.0/LSB-Core-generic/LSB-Core-generic/symversion.html

---
 tools/lib/bpf/elf.c    | 134 +++++++++++++++++++++++++++++++++++++----
 tools/lib/bpf/libbpf.c |   2 +-
 2 files changed, 124 insertions(+), 12 deletions(-)
tools/lib/bpf/elf.c +25 -0
index 5c9e588b17da..f7ad7a7acc29 100644
@@ -1,5 +1,8 @@
 // SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
 
+#ifndef _GNU_SOURCE
+#define _GNU_SOURCE
+#endif
 #include <libelf.h>
 #include <gelf.h>
 #include <fcntl.h>
@@ -10,6 +13,17 @@
 
 #define STRERR_BUFSIZE  128
 
+/* A SHT_GNU_versym section holds 16-bit words. This bit is set if
+ * the symbol is hidden and can only be seen when referenced using an
+ * explicit version number. This is a GNU extension.
+ */
+#define VERSYM_HIDDEN  0x8000
+
+/* This is the mask for the rest of the data in a word read from a
+ * SHT_GNU_versym section.
+ */
+#define VERSYM_VERSION 0x7fff
+
 int elf_open(const char *binary_path, struct elf_fd *elf_fd)
 {
        char errmsg[STRERR_BUFSIZE];
@@ -64,13 +78,18 @@ struct elf_sym {
        const char *name;
        GElf_Sym sym;
        GElf_Shdr sh;
+       int ver;
+       bool hidden;
 };
 
 struct elf_sym_iter {
        Elf *elf;
        Elf_Data *syms;
+       Elf_Data *versyms;
+       Elf_Data *verdefs;
        size_t nr_syms;
        size_t strtabidx;
+       size_t verdef_strtabidx;
        size_t next_sym_idx;
        struct elf_sym sym;
        int st_type;
@@ -111,6 +130,29 @@ static int elf_sym_iter_new(struct elf_sym_iter *iter,
        iter->nr_syms = iter->syms->d_size / sh.sh_entsize;
        iter->elf = elf;
        iter->st_type = st_type;
+
+       /* Version symbol table is meaningful to dynsym only */
+       if (sh_type != SHT_DYNSYM)
+               return 0;
+
+       scn = elf_find_next_scn_by_type(elf, SHT_GNU_versym, NULL);
Quoted 9 lines

tools/lib/bpf/elf.c static int elf_sym_iter_new(struct elf_sym_iter *iter, · line 142

Quoted 4 lines
tools/lib/bpf/elf.c +4 -0
+       if (!scn)
+               return 0;
+       if (!gelf_getshdr(scn, &sh))
+               return -EINVAL;

we don't seem to use sh, why are we calling gelf_getshdr? sanity check or something? I dropped it for now, but let me know if I'm missing something

OK, It's just a sanity check.

Quoted 51 lines
Quoted 16 lines
tools/lib/bpf/elf.c +13 -0
+       iter->versyms = elf_getdata(scn, 0);
+
+       scn = elf_find_next_scn_by_type(elf, SHT_GNU_verdef, NULL);
+       if (!scn) {
+               pr_debug("elf: failed to find verdef ELF sections in '%s'\n",
+                        binary_path);
+               return -ENOENT;
+       }
+       if (!gelf_getshdr(scn, &sh))
+               return -EINVAL;
+       iter->verdef_strtabidx = sh.sh_link;
+       iter->verdefs = elf_getdata(scn, 0);
+
        return 0;
 }
 

[...]

Quoted 15 lines

/* Transform symbol's virtual address (absolute for binaries and relative
* for shared libs) into file offset, which is what kernel is expecting

tools/lib/bpf/elf.c +2 -1
@@ -166,7 +277,8 @@ static unsigned long elf_sym_offset(struct elf_sym *sym)
 long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
 {
        int i, sh_types[2] = { SHT_DYNSYM, SHT_SYMTAB };
-       bool is_shared_lib, is_name_qualified;
+       const char *at_symbol, *lib_ver;
+       bool is_shared_lib;
        long ret = -ENOENT;
        size_t name_len;
        GElf_Ehdr ehdr;
@@ -179,9 +291,15 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
        /* for shared lib case, we do not need to calculate relative offset */

tools/lib/bpf/elf.c long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name) · line 298

Quoted 10 lines
tools/lib/bpf/elf.c +5 -3
        is_shared_lib = ehdr.e_type == ET_DYN;
 
-       name_len = strlen(name);
-       /* Does name specify "@@LIB"? */
-       is_name_qualified = strstr(name, "@@") != NULL;
+       /* Does name specify "@@LIB_VER" or "@LIB_VER" ? */
+       at_symbol = strchr(name, '@');
+       if (at_symbol) {
+               name_len = at_symbol - name;
+               lib_ver = strrchr(name, '@') + 1;

hm... this makes me a bit uneasy. We basically just need to skip extra @ if it's there. Reverse searching from the end seems both unnecessary and potentially error prone.

So why not

Quoted 10 lines

if (at_symbol[1] == '@')

    at_symbol++;

lib_ver = at_symbol + 1;

?

This is what I changed it to while applying, but please do let me know if it's problematic.

That's better. Thanks.

Quoted 38 lines
Quoted 38 lines
tools/lib/bpf/elf.c +5 -7
+       } else {
+               name_len = strlen(name);
+               lib_ver = NULL;
+       }
 
        /* Search SHT_DYNSYM, SHT_SYMTAB for symbol. This search order is used because if
         * a binary is stripped, it may only have SHT_DYNSYM, and a fully-statically
@@ -201,13 +319,7 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
                        goto out;
 
                while ((sym = elf_sym_iter_next(&iter))) {
-                       /* User can specify func, func@@LIB or func@@LIB_VERSION. */
-                       if (strncmp(sym->name, name, name_len) != 0)
-                               continue;
-                       /* ...but we don't want a search for "foo" to match 'foo2" also, so any
-                        * additional characters in sname should be of the form "@@LIB".
-                        */
-                       if (!is_name_qualified && sym->name[name_len] != '\0' && sym->name[name_len] != '@')
+                       if (!symbol_match(&iter, sh_types[i], sym, name, name_len, lib_ver))
                                continue;
 
                        cur_bind = GELF_ST_BIND(sym->sym.st_info);
tools/lib/bpf/libbpf.c +1 -2
index 3a6108e3238b..b4758e54a815 100644
@@ -11630,7 +11630,7 @@ static int attach_uprobe(const struct bpf_program *prog, long cookie, struct bpf
 
        *link = NULL;
 
-       n = sscanf(prog->sec_name, "%m[^/]/%m[^:]:%m[a-zA-Z0-9_.]+%li",
+       n = sscanf(prog->sec_name, "%m[^/]/%m[^:]:%m[a-zA-Z0-9_.@]+%li",
                   &probe_type, &binary_path, &func_name, &offset);
        switch (n) {
        case 1:
--

2.34.1

#5 lore raw

PATCH v4 3/3 selftests/bpf: Add tests for symbol versioning for uprobe

In reply to Hengqi Chen

This exercises the newly added dynsym symbol versioning logics. Now we accept symbols in form of func, func@LIB_VERSION or func@@LIB_VERSION.

The test rely on liburandom_read.so. For liburandom_read.so, we have:

    $ nm -D liburandom_read.so
                     w __cxa_finalize@GLIBC_2.17
                     w __gmon_start__
                     w _ITM_deregisterTMCloneTable
                     w _ITM_registerTMCloneTable
    0000000000000000 A LIBURANDOM_READ_1.0.0
    0000000000000000 A LIBURANDOM_READ_2.0.0
    000000000000081c T urandlib_api@@LIBURANDOM_READ_2.0.0
    0000000000000814 T urandlib_api@LIBURANDOM_READ_1.0.0
    0000000000000824 T urandlib_api_sameoffset@LIBURANDOM_READ_1.0.0
    0000000000000824 T urandlib_api_sameoffset@@LIBURANDOM_READ_2.0.0
    000000000000082c T urandlib_read_without_sema@@LIBURANDOM_READ_1.0.0
    00000000000007c4 T urandlib_read_with_sema@@LIBURANDOM_READ_1.0.0
    0000000000011018 D urandlib_read_with_sema_semaphore@@LIBURANDOM_READ_1.0.0

For urandlib_api, specifying urandlib_api will cause a conflict because there are two symbols named urandlib_api and both are global bind. For urandlib_api_sameoffset, there are also two symbols in the .so, but both are at the same offset and essentially they refer to the same function so no conflict.

---
 tools/testing/selftests/bpf/Makefile          |  5 +-
 .../testing/selftests/bpf/liburandom_read.map | 15 +++
 .../testing/selftests/bpf/prog_tests/uprobe.c | 95 +++++++++++++++++++
 .../testing/selftests/bpf/progs/test_uprobe.c | 61 ++++++++++++
 tools/testing/selftests/bpf/urandom_read.c    | 15 ++-
 .../testing/selftests/bpf/urandom_read_lib1.c | 22 +++++
 6 files changed, 209 insertions(+), 4 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/liburandom_read.map
 create mode 100644 tools/testing/selftests/bpf/prog_tests/uprobe.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_uprobe.c
tools/testing/selftests/bpf/Makefile +3 -2
index caede9b574cb..47365161b6fc 100644
@@ -196,11 +196,12 @@ endif
 
 # Filter out -static for liburandom_read.so and its dependent targets so that static builds
 # do not fail. Static builds leave urandom_read relying on system-wide shared libraries.
-$(OUTPUT)/liburandom_read.so: urandom_read_lib1.c urandom_read_lib2.c
+$(OUTPUT)/liburandom_read.so: urandom_read_lib1.c urandom_read_lib2.c liburandom_read.map
 	$(call msg,LIB,,$@)
 	$(Q)$(CLANG) $(filter-out -static,$(CFLAGS) $(LDFLAGS))   \
-		     $^ $(filter-out -static,$(LDLIBS))	     \
+		     $(filter %.c,$^) $(filter-out -static,$(LDLIBS)) \
 		     -fuse-ld=$(LLD) -Wl,-znoseparate-code -Wl,--build-id=sha1 \
+		     -Wl,--version-script=liburandom_read.map \
 		     -fPIC -shared -o $@
 
 $(OUTPUT)/urandom_read: urandom_read.c urandom_read_aux.c $(OUTPUT)/liburandom_read.so
tools/testing/selftests/bpf/liburandom_read.map new file +15 -0
new file mode 100644
index 000000000000..38a97a419a04
@@ -0,0 +1,15 @@
+LIBURANDOM_READ_1.0.0 {
+	global:
+		urandlib_api;
+		urandlib_api_sameoffset;
+		urandlib_read_without_sema;
+		urandlib_read_with_sema;
+		urandlib_read_with_sema_semaphore;
+	local:
+		*;
+};
+
+LIBURANDOM_READ_2.0.0 {
+	global:
+		urandlib_api;
+} LIBURANDOM_READ_1.0.0;
tools/testing/selftests/bpf/prog_tests/uprobe.c new file +95 -0

95 lines not on this page · raw message

tools/testing/selftests/bpf/progs/test_uprobe.c new file +61 -0

61 lines not on this page · raw message

tools/testing/selftests/bpf/urandom_read.c +13 -2
index e92644d0fa75..4ed795655b9f 100644
@@ -11,6 +11,9 @@
 #define _SDT_HAS_SEMAPHORES 1
 #include "sdt.h"
 
+#define SHARED 1
+#include "bpf/libbpf_internal.h"
+
 #define SEC(name) __attribute__((section(name), used))
 
 #define BUF_SIZE 256
@@ -21,10 +24,14 @@ void urand_read_without_sema(int iter_num, int iter_cnt, int read_sz);
 void urandlib_read_with_sema(int iter_num, int iter_cnt, int read_sz);
 void urandlib_read_without_sema(int iter_num, int iter_cnt, int read_sz);
 
+int urandlib_api(void);
+COMPAT_VERSION(urandlib_api_old, urandlib_api, LIBURANDOM_READ_1.0.0)
+int urandlib_api_old(void);
+int urandlib_api_sameoffset(void);
+
 unsigned short urand_read_with_sema_semaphore SEC(".probes");
 
-static __attribute__((noinline))
-void urandom_read(int fd, int count)
+static noinline void urandom_read(int fd, int count)
 {
 	char buf[BUF_SIZE];
 	int i;
@@ -83,6 +90,10 @@ int main(int argc, char *argv[])
 
 	urandom_read(fd, count);
 
+	urandlib_api();
+	urandlib_api_old();
+	urandlib_api_sameoffset();
+
 	close(fd);
 	return 0;
 }
tools/testing/selftests/bpf/urandom_read_lib1.c +22 -0
index 86186e24b740..8c1356d8b4ee 100644
@@ -3,6 +3,9 @@
 #define _SDT_HAS_SEMAPHORES 1
 #include "sdt.h"
 
+#define SHARED 1
+#include "bpf/libbpf_internal.h"
+
 #define SEC(name) __attribute__((section(name), used))
 
 unsigned short urandlib_read_with_sema_semaphore SEC(".probes");
@@ -11,3 +14,22 @@ void urandlib_read_with_sema(int iter_num, int iter_cnt, int read_sz)
 {
 	STAP_PROBE3(urandlib, read_with_sema, iter_num, iter_cnt, read_sz);
 }
+
+COMPAT_VERSION(urandlib_api_v1, urandlib_api, LIBURANDOM_READ_1.0.0)
+int urandlib_api_v1(void)
+{
+	return 1;
+}
+
+DEFAULT_VERSION(urandlib_api_v2, urandlib_api, LIBURANDOM_READ_2.0.0)
+int urandlib_api_v2(void)
+{
+	return 2;
+}
+
+COMPAT_VERSION(urandlib_api_sameoffset, urandlib_api_sameoffset, LIBURANDOM_READ_1.0.0)
+DEFAULT_VERSION(urandlib_api_sameoffset, urandlib_api_sameoffset, LIBURANDOM_READ_2.0.0)
+int urandlib_api_sameoffset(void)
+{
+	return 3;
+}
Signature · 1 line
2.34.1
#6 lore raw
In reply to Hengqi Chen
This series was applied to bpf/bpf-next.git (master) by Andrii Nakryiko <[email protected]>:

Hello:

This series was applied to bpf/bpf-next.git (master) by Andrii Nakryiko <[email protected]>:

Quoted 8 lines · On Mon, 18 Sep 2023 02:48:10 +0000 you wrote:

Dynamic symbols in shared library may have the same name, for example:

    $ nm -D /lib/x86_64-linux-gnu/libc.so.6 | grep rwlock_wrlock
    000000000009b1a0 T __pthread_rwlock_wrlock@GLIBC_2.2.5
    000000000009b1a0 T pthread_rwlock_wrlock@@GLIBC_2.34
    000000000009b1a0 T pthread_rwlock_wrlock@GLIBC_2.2.5

[...]

Here is the summary with links:
- [bpf-next,v4,1/3] libbpf: Resolve symbol conflicts at the same offset for uprobe

    https://git.kernel.org/bpf/bpf-next/c/7257cee65269
  - [bpf-next,v4,2/3] libbpf: Support symbol versioning for uprobe
    https://git.kernel.org/bpf/bpf-next/c/bb7fa09399b9
  - [bpf-next,v4,3/3] selftests/bpf: Add tests for symbol versioning for uprobe
    https://git.kernel.org/bpf/bpf-next/c/7089f85a9eb9

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html